ExamPark
Course menuAIF-C01

Exams / AIF-C01 / Evaluating foundation models / Question 3

AIF-C01 practice question: Evaluating foundation models (3 of 3)

Applications of Foundation Models A free question from our original AIF-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.

Last verified against current AWS documentation ·How we write our questions

An online retailer launched a generative AI shopping assistant three months ago. The board asks whether it is delivering business value and wants one measure that links the assistant to sales.

Which measure BEST answers that question?

Choose one answer.

  1. A. Conversion rate of chat users to orders, compared with a control group of shoppers who never used the assistant.
  2. B. Perplexity score of the model on a held-out set of product questions, which measures how well it predicts text.
  3. C. Average number of tokens the assistant generates per reply, which drives the model's per-request usage cost.
  4. D. Number of prompt revisions the team made to the assistant's wording during the three months after launch.
Show the answer and explanation

Answer: A

  • A. Right: Correct. Comparing orders from chat users with a control group links the assistant to revenue, which is the business value the board asks about.
  • B. Wrong: Perplexity measures how well a language model predicts text. It says nothing about whether shoppers buy, so it does not answer the board's question.
  • C. Wrong: Tokens per reply affect cost and speed, which are operational concerns. They do not show whether the assistant brings in sales.
  • D. Wrong: Prompt revisions measure team effort, not results. Heavy editing could still leave the assistant with no effect on sales.

Explanation

Business value for a generative AI assistant is judged by outcomes the business cares about, such as sales and conversion, not by model internals or team activity. A control group helps show that the assistant, rather than other factors, caused any change. This question tests choosing a business measure over technical ones.

Study

Keep practicing

Answer questions with a score at the end, or practice just this topic with the explanation after each one. No sign-up needed.

Take the free AIF-C01 practice examPractice Evaluating foundation modelsAll Evaluating foundation models questions

Want more on Evaluating foundation models? The 5 full-length practice sets have 21 more Evaluating foundation models questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets