Document metadata and filtering for precise retrieval: AIP-C01 study notes and practice questions
What to know about Document metadata and filtering for precise retrieval for the AIP-C01 exam, from the Foundation Model Integration, Data Management, and Compliance domain (31% of the exam guide), then 1 free question to test it. Part of the AIP-C01 study guide.
Typical scenarios: a multi-tenant assistant shows another customer’s documents, a question about the current policy returns an old version, or one product line must be searched alone. Metadata and filters are the levers.
Attach metadata at ingestion
Place a sidecar file named <file>.metadata.json next to each document in an S3 data source, for example policy.pdf.metadata.json. Its metadataAttributes object holds key-value pairs such as department, product line, effective date or tenant ID.
Attributes are indexed at sync time. To change them, update the sidecar and re-sync.
A misspelled sidecar name means the document indexes without attributes, and filters quietly match nothing.
Explicit filters come from your application in the Retrieve request. They use operators such as equals, in and greater-than, combined with andAll and orAll.
Tenant and access filters come from the authenticated user’s identity claims, set by trusted code. The model never chooses them.
Implicit filtering lets the model read the user’s question and generate a filter, for example turning “2025 policies” into a year filter. It helps natural queries, but it is not a security control.
Access control rule
A security filter that only the model writes can be bypassed by a crafted prompt. Set tenant and group filters in code from verified identity, and scope access to the knowledge base with IAM as well. Defense in depth means the filter and the permission both hold.
Prefer current documents
Store an effective or published date and filter or sort on it. Otherwise a superseded version can outrank the current one on semantic similarity alone.
Mixed corpora
Tag each document with a domain class such as HR, finance or engineering at ingestion, so one assistant’s queries stay inside its domain. A classifier step can write these tags before the sidecar file is created.
How to choose
Scenario wording
Choose
“Each customer sees only their own documents”
Tenant filter from the verified token
“Only the 2025 policy version”
Equals or range filter on effective date
“User types ‘latest HR policy’ in free text”
Implicit filtering, with explicit filters for security
“Restrict documents to certain groups”
ACL-style metadata set by trusted code
“Search one product line in a mixed corpus”
Product line metadata with an explicit filter
Common traps
Letting the model build the security filter from the prompt text.
Missing or misnamed sidecar files, which silently produce empty filter results.
Forgetting to re-sync after metadata changes.
Putting secrets or personal data in metadata, which can surface in responses and logs.
Filtering only in the prompt (“answer only about X”) instead of in retrieval.
A regional water utility keeps about 9,000 operating procedures in one knowledge base. Each procedure belongs to a department, such as billing, field operations or customer care, and has an effective date.
Staff ask questions in a chat tool that already knows their department. Answers must come only from procedures for that department that are in force on the date of the question, and staff must never see another department's procedures.
The team wants one knowledge base to manage, and the filtering must be applied before the top results are chosen, so that enough matching results are returned.
Which solution meets these requirements?
Choose one answer.
AA. Write the department name and effective date at the top of each document's text so the embeddings capture them, and rely on similarity to rank that department first.
BB. Add a .metadata.json sidecar file with department and effective date attributes beside each procedure, and pass a filter on those attributes in every retrieval request.
CC. Create one knowledge base for each department, and have the application choose the knowledge base that matches the user's department before each query, so that the data is separated by design.
DD. Store the department and date of each document in a DynamoDB table, retrieve the top results first, and then drop any result whose metadata does not match the user's department and date.
Show the answer and explanation
Answer: B
A. Wrong: Similarity scores do not enforce a hard rule, so procedures from other departments can still be returned. Dates written as text are also not compared reliably, so the access and date rules are not guaranteed.
C. Wrong: Separate knowledge bases do isolate the data, but the team would sync, monitor and pay for nine thousand documents spread across many bases. The requirement for one knowledge base to manage is not met.
D. Wrong: Filtering after retrieval can leave too few results, because the top results may belong to other departments and are discarded. The requirement that filtering happen before the top results are chosen is not met.
Explanation
Sidecar metadata with department and effective date, combined with a retrieval filter, enforces both rules inside one knowledge base and before the top results are chosen. Text-only metadata is not a hard rule, separate knowledge bases add management overhead, and post-retrieval filtering can return too few results.
Want more on Document metadata and filtering for precise retrieval? The 5 full-length practice sets have 6 more Document metadata and filtering for precise retrieval questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →
Study Document metadata and filtering for precise retrieval