Exams / AIF-C01 / Evaluating foundation models / Question 1
AIF-C01 practice question: Evaluating foundation models (1 of 3)
Applications of Foundation Models A free question from our original AIF-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.
A publisher is comparing two models for translating book blurbs from English into German. It has professional reference translations for 1,000 blurbs and wants an automatic score of how closely each model's output matches them. Which metric was designed for this task?
Choose one answer.
- AA. BLEU
- BB. Recall
- CC. Root mean squared error
- DD. Area under the curve
Show the answer and explanation
Answer: A
- A. Right: Correct. BLEU was designed to score machine translation by comparing the output with reference translations.
- B. Wrong: Recall measures how many actual positives a classifier finds. It does not compare generated text with a reference.
- C. Wrong: RMSE measures error in numeric regression predictions, not text quality.
- D. Wrong: AUC evaluates binary classifiers across thresholds. It does not score translations.
Explanation
Common automatic metrics for generated text compare output with human references. BLEU was designed for machine translation. ROUGE measures n-gram overlap and is widely used for summarisation. BERTScore compares meaning using embeddings, so it credits paraphrases that use different words.
Study
Keep practising
Answer questions with a score at the end, or practise just this topic with the explanation after each one. No sign-up needed.