Query expansion, decomposition and rewriting: AIP-C01 study notes and practice questions
What to know about Query expansion, decomposition and rewriting for the AIP-C01 exam, from the Foundation Model Integration, Data Management, and Compliance domain (31% of the exam guide), then 1 free question to test it. Part of the AIP-C01 study guide.
AIP-C01 asks how to make retrieval work for vague, multi-part or follow-up questions, and which transformation to add when the search fails for a given reason.
The main techniques
Technique
What it does
Decomposition
Splits a multi-part question into sub-questions, retrieves for each, then combines the answers.
Rewriting
Turns a follow-up such as “what about the EU one?” into a standalone question using conversation history.
Expansion
Adds synonyms and related terms with a foundation model, so keyword and semantic search cover more wording.
HyDE
Asks a model to write a hypothetical answer, then searches with that text. It helps when questions are short and documents are long.
Agentic retrieval
An agent chooses which knowledge base or tool to query, and can query several and merge results.
Knowledge bases offer built-in query decomposition as a setting of RetrieveAndGenerate, so you do not need to build it yourself. Each technique adds at least one model call before retrieval, which adds latency and cost. Use the smallest transformation that fixes the observed failure.
How to choose
Scenario wording
Choose
“Question asks two or three things at once”
Decomposition, then combine sub-answers
“User says ‘and the second one?’ in chat”
Rewrite using conversation history
“Users use different words from the documents”
Expansion with synonyms
“Short query, long technical documents, weak matches”
HyDE
“Answers live in several knowledge bases”
Agentic retrieval across knowledge bases
“Simple, direct question, latency is critical”
No transformation
Pipelines on AWS
A transformation pipeline can run as a Bedrock Flows chain with a prompt node that rewrites the query, followed by a knowledge base node. For more control, use an AWS Step Functions workflow that calls Bedrock, runs sub-queries in a Map state, and aggregates the results. Step Functions gives retries and error handling for each step.
Common traps
Rewriting every query when most are already clear. The extra call adds delay and cost for no gain.
Losing entities during rewrite. A rewrite that drops a product name or date sends retrieval to the wrong place. Keep the original question alongside the rewrite.
HyDE for exact lookups, where an invented answer can pull in plausible but wrong passages.
Decomposition without a combine step, so the answer covers only one part of the question.
Using history without a window. Pass recent turns that matter, not the entire chat.
Choosing agentic retrieval for one knowledge base. A single retrieve call is simpler and cheaper.
A travel company's assistant answers questions from its fare rules, baggage policy and pet travel documents, all stored in one knowledge base.
A customer asks: compare the cancellation terms and the baggage allowance for the Lisbon fare, and tell me whether my dog can ride in the cabin. The assistant returns a long answer that covers only the cancellation terms, because a single search for the whole question ranks those passages highest.
The team wants every part of a multi-part question answered from the right passages, with the answer still grounded in the retrieved text.
Which TWO approaches meet these requirements? (Choose two.)
Choose two answers.
AA. Retrieve the top 100 passages for the full question in one search, and send all of them to a larger generation model, which can sort out which passages answer each part.
BB. Ask the customer to send each part of the question in a separate message, and answer each message with its own search, so the assistant never needs to split a question.
CC. Turn on query decomposition in the knowledge base's RetrieveAndGenerate settings, so that the question is split into sub-queries that are each retrieved before the answer is written.
DD. Convert the question into a SQL query and run it against a structured data knowledge base connected to Amazon Redshift, so that each policy is looked up as a table row.
EE. Build an agent on AgentCore Runtime with a knowledge base search tool, and let the agent call the tool once for each part of the question before it writes the answer.
Show the answer and explanation
Answers: C, E
A. Wrong: The right passages may still be ranked below the cancellation text, and a larger model cannot read passages that were never retrieved. The requirement for per-part retrieval is not met.
B. Wrong: This works, but it moves the splitting work to the customer and changes the product experience. The requirement is for the assistant to answer multi-part questions by itself.
E. Right: The agent decides the parts, calls the search tool separately for each, and answers only from the returned passages. Each part gets its own retrieval, which meets both the completeness and grounding requirements. Learn more: Overview of Amazon Bedrock AgentCore (opens in a new tab)
Explanation
Both correct approaches split the question into parts and retrieve for each part, so the baggage and pet passages are not crowded out. Query decomposition does the splitting inside the knowledge base request, while the agent chooses its own searches, and both answer from the retrieved text. A single large search, a structured knowledge base over free text and pushing the work to the customer each fail a requirement.
Want more on Query expansion, decomposition and rewriting? The 5 full-length practice sets have 6 more Query expansion, decomposition and rewriting questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →
Study Query expansion, decomposition and rewriting