ExamPark
Course menuAIF-C01

Exams / AIF-C01 / Evaluating foundation models / Question 1

AIF-C01 practice question: Evaluating foundation models (1 of 3)

Applications of Foundation Models A free question from our original AIF-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.

Last verified against current AWS documentation ·How we write our questions

A publisher is choosing between two foundation models to translate 40,000 book blurbs from English into German.

It has human-approved reference translations for 1,000 of them and budget for staff to review only a small sample of new output.

Which evaluation approach BEST measures translation quality at scale?

Choose one answer.

  1. A. Score both models against the 1,000 reference translations, then have staff review a random sample of new output.
  2. B. Ask each model to rate its own German translations, then choose the model with the higher self-rating.
  3. C. Have staff read every one of the 40,000 translations before choosing a model, so that every error is found.
  4. D. Count the words in each translation and choose the model whose output length is closest to the English original.
Show the answer and explanation

Answer: A

  • A. Right: Correct. Comparing with reference translations gives an automatic score for every output, which scales to all 40,000 blurbs. A small human sample checks quality within the review budget. Learn more: Use metrics to understand model performance - Amazon Bedrock (opens in a new tab)
  • B. Wrong: A model judging its own output is not an independent check. It also ignores the reference translations the publisher already has, so the result does not show which model is more accurate.
  • C. Wrong: Full human review is thorough but far beyond the review budget, which covers only a small sample. It is the wrong approach for a choice at this scale.
  • D. Wrong: Word count says nothing about whether the German keeps the meaning of the English. Two translations of the same length can differ greatly in quality.

Explanation

Automatic evaluation against reference data gives a repeatable score for every output, and a sample of human review checks what automated scores can miss. Both fit the scale and the budget in this scenario. This question tests choosing an evaluation method that matches the volume and the review resources.

Study

Keep practicing

Answer questions with a score at the end, or practice just this topic with the explanation after each one. No sign-up needed.

Take the free AIF-C01 practice examPractice Evaluating foundation modelsAll Evaluating foundation models questions

Want more on Evaluating foundation models? The 5 full-length practice sets have 21 more Evaluating foundation models questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets