Token efficiency (context pruning, prompt compression, response limits): AIP-C01 study notes and practice questions
What to know about Token efficiency (context pruning, prompt compression, response limits) for the AIP-C01 exam, from the Operational Efficiency and Optimization for GenAI Applications domain (12% of the exam guide), then 1 free question to test it. Part of the AIP-C01 study guide.
AIP-C01 tests whether you can cut the tokens a request uses, because input and output tokens drive both cost and latency. The right fix is usually a design change, such as less context or shorter output, rather than a bigger model.
Cap and shape the output
Set maxTokens in the inferenceConfig of a Converse request (or the model’s own field in InvokeModel) to limit how long a response can run. A response that hits the limit returns a stopReason of max_tokens, which means the answer was cut off. Raising the cap fixes truncation but costs more. Often the better fix is to ask for a short structured format, such as JSON with a few named fields, instead of long prose.
Keep the context small
A multi-turn chat resends every earlier turn on each call. Prune old turns, or summarize them into a short running summary that you carry forward. In RAG, retrieve fewer, better chunks: lower numberOfResults in the retrieval configuration, add metadata filters, and rerank so the top few chunks carry the answer. Remove repeated boilerplate from prompts, and compress long source documents before they enter the prompt.
Measure before you send
The Bedrock CountTokens API returns the input token count for supported models without running inference. Use it, or an estimate for other models, to enforce a budget per request. Reject or trim oversized input before it reaches the model. After each call, read the input and output token usage in the response and in CloudWatch metrics, so you can find the prompts that cost the most.
How to choose
Scenario wording
Choose
Responses run long and cost keeps rising
Set maxTokens and ask for a short, structured output
A SaaS company runs a ticket summarizer that writes a short summary of each support conversation for the agent's sidebar. The summary box holds about 150 words, and the summarizer calls a model on Amazon Bedrock through the Converse API.
The model often writes several paragraphs, and most of that text is cut off by the box or ignored by agents. Output tokens are billed, and the company handles about 400,000 conversations each day.
The summarizer must keep each summary within the box, reduce the output token charges, and avoid cutting a summary off in the middle of a sentence where possible.
Which change meets these requirements?
Choose one answer.
AA. Lower the temperature to 0.1 so that the model writes more concisely, and keep the existing prompt unchanged, because lower randomness usually makes answers shorter.
BB. Set maxTokens in the inference configuration close to the box's length, and add a prompt instruction for a summary of about 150 words, so replies stay short and cost less.
CC. Raise maxTokens to 4,000 so the model can finish its thoughts, and let the application cut the summary down to the box size after the reply arrives.
DD. Send the full conversation history with every request, plus a list of all earlier summaries, so that the model can keep each new summary consistent with the ones written before it.
Show the answer and explanation
Answer: B
A. Wrong: Temperature changes how the model chooses words, not how long its reply is, so the summaries may stay long. The option sets no limit on output, so the billed output tokens do not change. Learn more: Inference request parameters (opens in a new tab)
B. Right: The maxTokens limit caps the output tokens billed for each reply, and the prompt instruction keeps most replies close to the box size. A reply that reaches the cap can end mid-sentence, so the limit should leave some margin. Learn more: Inference request parameters (opens in a new tab)
C. Wrong: A higher limit allows longer replies, which increases the output tokens billed on every conversation. Cutting text in the application discards the model's work and does not reduce the tokens already billed.
D. Wrong: Extra history increases the input tokens on every call and does nothing to limit the length of the new summary. It moves the cost to a different place rather than reducing it.
Explanation
Setting maxTokens caps the output tokens billed for each reply, and a prompt instruction for about 150 words keeps most replies inside the box. Lowering temperature does not shorten replies, a higher limit raises cost, and extra history adds input tokens to every call.
Want more on Token efficiency (context pruning, prompt compression, response limits)? The 5 full-length practice sets have 5 more Token efficiency (context pruning, prompt compression, response limits) questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →
Study Token efficiency (context pruning, prompt compression, response limits)