AIP-C01 practice question: Choosing foundation models for a use case

Foundation Model Integration, Data Management, and Compliance A free question from our original AIP-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.

A legal publisher runs a research tool that sends each user's question, together with the court opinions that match it, to a foundation model on Amazon Bedrock.

Most opinions are 2,000 to 3,000 words long, and short questions should come back fast and at a low price. The large-context models the team tested cost several times more per token than its small default model.

A few full case files run to about 300,000 tokens, and an answer can depend on any part of such a file. The team will not split these files by hand.

Which TWO approaches meet these requirements? (Choose two.)

Choose two answers.

A. Use one large-context model for every request, so that any case file fits in a single prompt and the application code stays simple.
B. Check the token count of each request, send short questions to a low-cost model, and send the few very long case files to a model whose context window holds about 300,000 tokens.
C. Fine-tune a small model on the case files so that it memorizes their contents, and then send only the user's question at request time, with no case text attached.
D. Load the case files into a knowledge base that chunks them automatically, and retrieve only the passages that match each question for the low-cost model.
E. Truncate every case file to its first 4,000 tokens before sending it to a low-cost model, because the answer to most questions appears near the start of an opinion.

Show the answer and explanation

Answers: B, D

A. Wrong: A large-context model can hold the longest file, but the team measured a much higher price per token for it, and every short question would pay that price. That breaks the requirement for fast, low-cost answers to the common requests.
B. Right: Choosing the model by request length keeps most traffic on the cheap, fast model and reserves the long-context model for the rare large files. Each case file is still answered from its full text, so both requirements are met. Learn more: Carry out a conversation with the Converse API (opens in a new tab)
C. Wrong: Fine-tuning does not reliably store the full text of long opinions, so answers may be inaccurate or incomplete. Each new case file would also require another training job before it could be used.
D. Right: The knowledge base chunks the files during ingestion, so no one splits them by hand, and retrieval can reach a passage from any part of a long file. The small model sees only a few passages per request, which keeps latency and cost low. Learn more: Chunking strategies for knowledge bases (opens in a new tab)
E. Wrong: Truncation discards the later parts of long opinions, where holdings and dissents often appear, so answers would be wrong for many questions. It meets the cost goal but breaks the requirement that an answer can come from any part of the file.

Explanation

Routing by request length keeps routine traffic on the cheap, fast model while a long-context model handles the rare huge files. A knowledge base also works, because it chunks the files automatically and retrieval can reach content anywhere in a file. Fine-tuning and truncation both lose content, and a single large-context model is too costly for everyday questions.

AIP-C01 practice question: Choosing foundation models for a use case

Explanation

Study

Keep practicing