ExamPark
Course menuMLA-C02

Exams / MLA-C02 / Study guide / RAG architecture and retrieval tuning

RAG architecture and retrieval tuning: MLA-C02 study notes and practice questions

What to know about RAG architecture and retrieval tuning for the MLA-C02 exam, from the ML Model and Foundation Model (FM) Development domain (24% of the exam guide), then 2 free questions to test it. Part of the MLA-C02 study guide.

Last verified against current AWS documentation ·How we write our questions

What to know for the exam

RAG questions describe wrong or incomplete answers and ask which change fixes retrieval or generation.

The flow

  1. The user’s query is turned into an embedding.
  2. The vector store returns the most similar chunks.
  3. The chunks and the query go into the prompt.
  4. The model generates an answer, ideally with citations.

Use Retrieve to test step 2 on its own; RetrieveAndGenerate runs the whole flow in Bedrock knowledge bases.

Fixing retrieval

Symptom Fix
Exact codes, part numbers or names are missed Hybrid search (semantic plus keyword), supported with OpenSearch stores
The right passage is retrieved but ranks low A reranking model reorders results by relevance, with no re-indexing
Results come from the wrong product, customer or year Metadata filters (or implicit filtering, which infers filters from the query)
Multi-part questions get partial answers Query decomposition splits the question into sub-queries
Not enough context Raise the number of results, or use hierarchical chunking
Tables or layouts are garbled Better parsing at ingestion

Fixing generation

  • Tune the prompt template: tell the model to answer only from the context and say when it doesn’t know.
  • Turn on contextual grounding checks in Guardrails to block answers not supported by the sources.
  • Use a model with a larger context window if many chunks must fit.
  • Lower temperature for factual answers.

RAG or fine-tuning?

RAG suits private, changing or citable knowledge and needs no training. Fine-tuning suits style, format and task behavior. They combine well.

Beyond vectors

  • GraphRAG with Neptune Analytics follows relationships between entities.
  • Structured data retrieval in knowledge bases turns questions into SQL over Redshift.
  • Kendra can act as a retriever over enterprise content.

Common traps

  • Bigger chunks or a bigger embedding model to fix exact-ID lookups. Use hybrid search.
  • Re-indexing when reranking fixes the order.
  • Fine-tuning to add documents that change every week.

Try the 2 practice questions

Practice RAG architecture and retrieval tuningTake the free practice exam

Question 1

Medium ·Link to this question (question 1)

A public library runs a retrieval-augmented chatbot over its policy manual and catalog help pages, using an Amazon Bedrock knowledge base with an Amazon OpenSearch Serverless vector store. Patrons often ask about exact codes, such as a policy number like POL-2290, and the assistant often answers with a general policy instead.

The team wants retrieval to match exact identifiers as well as meaning, using only the knowledge base it already has, without adding a new search service.

Which change should the team make?

Choose one answer.

  1. A. Increase the number of retrieved chunks from 5 to 50 so that the exact code is more likely to appear somewhere in the results.
  2. B. Replace the embedding model with a larger one, since larger embedding models always match exact codes more precisely.
  3. C. Shorten the chatbot prompt so the model focuses only on the policy text and ignores the catalog help pages.
  4. D. Change the retrieval search type to hybrid, which combines semantic vector matching with keyword matching for the query.
Show the answer and explanation

Answer: D

  • A. Wrong: More chunks increase the amount of text sent to the model and do not guarantee that the exact code ranks high. The stem asks for matching of identifiers.
  • B. Wrong: Embedding models capture meaning, and a bigger model does not guarantee that exact strings match. The keyword side of hybrid search is what handles identifiers.
  • C. Wrong: The retrieval step is where the wrong passage is missed, and a shorter prompt does not fix that. The stem calls for changing how passages are found.
  • D. Right: Correct. Hybrid search combines semantic vector search with a search of the raw text, so exact codes such as POL-2290 can match even when meaning-based search misses them. It is supported for OpenSearch Serverless vector stores that have a filterable text field and is set in the retrieval configuration. Learn more: Configure and customize queries and response generation (opens in a new tab)

Explanation

Hybrid search combines semantic similarity with text matching, which helps when a query contains exact identifiers, and the knowledge base's OpenSearch Serverless store supports it. More chunks, larger embedding models or prompt edits do not change how exact codes are matched during retrieval.

Study

Question 2

Harder ·Link to this question (question 2)

A pet insurer built a retrieval-augmented assistant over 40,000 policy clauses in an Amazon Bedrock knowledge base. For questions about exclusions, the top five retrieved clauses are often topically similar but do not answer the question, and the answers are wrong.

The retrieved set must be reordered so that the clauses that actually answer the question come first. Latency must stay under three seconds for each answer, and the team does not want to change the embedding model or re-index the clauses.

Which change best meets these requirements?

Choose one answer.

  1. A. Add a reranking model to the retrieval step so that it scores the retrieved clauses against the question and returns them in that order.
  2. B. Increase the number of results returned from the vector search to 100, and pass all of them to the foundation model together with the customer's question.
  3. C. Re-index all 40,000 clauses with a larger chunk size, so that each chunk holds an entire policy section.
  4. D. Replace the foundation model that writes answers with a larger one, expecting it to ignore unrelated clauses.
Show the answer and explanation

Answer: A

  • A. Right: Correct. A reranking model rescores the candidates by relevance to the question, so answering clauses move to the top. The embeddings and index stay unchanged, and only a small number of passages is passed on. Learn more: Improve the relevance of query responses with a reranker model (opens in a new tab)
  • B. Wrong: Passing 100 clauses increases the model input and can push latency past the target, and the relevant clause may still sit low in the list. The stem asks for better ordering, not more text.
  • C. Wrong: Re-indexing is the change the team wants to avoid, and larger chunks do not reorder the candidates. Each section also brings more irrelevant text into the answer.
  • D. Wrong: The generation model is not the step that orders the clauses, and a bigger model adds cost and latency. The wrong clauses are reaching the model in the first place.

Explanation

A reranking model reorders a set of retrieved passages by relevance to the question, which fixes the case where the right clause ranks low. This needs no re-indexing and keeps the model input small. Larger result sets, bigger chunks or a larger generator each add cost or text without solving the ordering problem.

Study

Want more on RAG architecture and retrieval tuning? The 5 full-length practice sets have 9 more RAG architecture and retrieval tuning questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets

Study RAG architecture and retrieval tuning

More ML Model and Foundation Model (FM) Development topics