AIP-C01 practice question: Transparency and explainability (citations, reasoning traces)

AI Safety, Security, and Governance A free question from our original AIP-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.

A construction firm runs an assistant that answers site safety questions for its field crews. The assistant uses the firm's own safety manuals as its source.

Crews often ask whether an answer is official before they act on it. The firm wants each answer to show the manual and section it came from, so that a supervisor can check it on the spot.

The assistant must still return one written answer per question, not just a list of text passages.

Which approach meets these requirements?

Choose one answer.

A. Prompt the model to end each answer with the phrase according to the safety manual, without giving the crew any link, document name or section reference.
B. Call the RetrieveAndGenerate API, which returns a generated answer with citations to its source passages, and show those citations next to each answer.
C. Call the Retrieve API only, and display the matching passages to the crew as raw text, leaving the crew to read the passages and decide the answer for themselves.
D. Turn on model invocation logging so that each prompt and answer is saved, and let supervisors look up the saved records after an incident has occurred.

Show the answer and explanation

Answer: B

A. Wrong: A phrase like this claims a source without pointing to one, so a supervisor still cannot verify the answer. The model can also invent a source name, which defeats the purpose of showing where an answer came from.
B. Right: RetrieveAndGenerate returns both the answer and the references it was built from, and the interface can display the document and section for each. The crew gets one written answer with a source for each statement. Learn more: Query a knowledge base and generate responses based off the retrieved data (opens in a new tab)
C. Wrong: Retrieve returns relevant passages with their locations, but it does not generate an answer, so the assistant would not give the one written answer the firm requires. Crews would have to interpret the manuals on their own. Learn more: Amazon Bedrock Knowledge Bases (opens in a new tab)
D. Wrong: Invocation logging preserves records for audits, but the crew does not see sources at the moment they read an answer. The requirement is to show the source to the crew while they work. Learn more: Monitor model invocation using CloudWatch Logs and Amazon S3 (opens in a new tab)

Explanation

RetrieveAndGenerate returns a generated answer with citations to the passages it used, so the interface can show the source next to each answer. Retrieve alone does not produce an answer, prompting for a phrase creates an unverifiable claim, and invocation logs only help after the fact.

AIP-C01 practice question: Transparency and explainability (citations, reasoning traces)

Explanation

Study

Keep practicing