AIP-C01 practice question: Prompt caching and semantic caching

Operational Efficiency and Optimization for GenAI Applications A free question from our original AIP-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.

A commercial real estate brokerage runs an assistant that answers analysts' questions about a single 60,000-token master lease agreement. The system prompt and the agreement text are the same for every question about that lease, and analysts ask dozens of questions in a session.

Each call sends the full prefix again, so input costs and response times are high. Only the question at the end of each request changes, and the brokerage wants answer quality to stay exactly as it is now.

The change must cut the input cost and latency of the repeated prefix while keeping the full agreement in every request. It must not change the model.

Which change meets these requirements with the LEAST code?

Choose one answer.

A. Store each question and its answer in Amazon DynamoDB keyed by the question text, and return the stored answer whenever an analyst repeats the same question word for word.
B. Add a cache checkpoint after the system prompt and agreement text in the Converse request, so later calls reuse the cached prefix, with the analyst's question placed after it.
C. Lower the temperature to 0 so that the model returns the same answer for each repeated question, and keep sending the full agreement in every request, as the current design does.
D. Split the agreement into fifty smaller sections, and send each question with only the section it seems to need, so that each request carries far fewer input tokens than the full document.

Show the answer and explanation

Answer: B

A. Wrong: Exact-match storage helps only when a question is asked again word for word, and analysts rarely repeat questions exactly. Each new question still sends the full agreement, so the input cost stays high.
B. Right: Prompt caching stores the repeated prefix, so cached input tokens cost less and are processed faster. The prefix must match exactly, which is why the question goes after the checkpoint, and the model and answers stay the same. Learn more: Prompt caching for faster model inference (opens in a new tab)
C. Wrong: Temperature affects randomness in the output, not the cost of the input tokens sent with each request. The agreement is still billed in full on every call, so the cost and latency targets are not met. Learn more: Inference request parameters (opens in a new tab)
D. Wrong: Smaller prompts cut input tokens, but a question about a clause in another section can be answered wrongly, which changes answer quality. The option also breaks the requirement to keep the full agreement in every request.

Explanation

Prompt caching lets the repeated system prompt and agreement text be reused at lower cost and with faster processing, as long as the prefix matches exactly. Putting the varying question after the cache checkpoint keeps the prefix stable. Exact-match answer storage, lower temperature and section splitting either miss the input cost or change answer quality.

AIP-C01 practice question: Prompt caching and semantic caching

Explanation

Study

Keep practicing