Exams / AIF-C01 / Evaluating foundation models / Question 2
AIF-C01 practice question: Evaluating foundation models (2 of 3)
Applications of Foundation Models A free question from our original AIF-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.
A support team has 5,000 responses from its chatbot and wants each one scored for correctness and completeness, with an explanation for every score. It wants results quickly, without assembling a large human review team. Which Amazon Bedrock evaluation approach fits BEST?
Choose one answer.
- AA. A human-based evaluation job with a large in-house workforce
- BB. An LLM-as-a-judge evaluation job that uses a second model as evaluator
- CC. A built-in benchmark dataset for general text generation
- DD. A Provisioned Throughput purchase for the chatbot model
Show the answer and explanation
Answer: B
- A. Wrong: Human evaluation suits subjective judgements, but the team wants to avoid assembling a large review workforce.
- B. Right: Correct. A judge model scores each response against selected metrics and explains each score, at scale.
- C. Wrong: Benchmark datasets test a model on standard prompts. They do not score the team's own 5,000 responses.
- D. Wrong: Provisioned Throughput adds dedicated capacity. It does not evaluate response quality.
Explanation
Amazon Bedrock evaluations include programmatic metrics with built-in or custom datasets, human evaluation with your own workforce, and LLM-as-a-judge, where an evaluator model scores responses on built-in or custom metrics and explains each score. You can also bring your own responses to be judged. LLM-as-a-judge gives fast, scalable, human-like quality scoring.
Study
Keep practising
Answer questions with a score at the end, or practise just this topic with the explanation after each one. No sign-up needed.