AIP-C01 practice question: Validating data quality before it reaches a model (Glue Data Quality, Lambda checks) (2 of 2)
Foundation Model Integration, Data Management, and Compliance A free question from our original AIP-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.
A digital bank answers customers' questions about its fee schedules and loan terms with a retrieval assistant. Its knowledge base syncs from a SharePoint library every hour.
Last quarter, a sync ran with a partial file listing and reported success, so the assistant kept quoting a fee schedule that had been replaced two weeks earlier. Nobody noticed until a customer complained.
The risk team wants the bank to detect a stale or incomplete knowledge base within one hour of a bad sync, using checks that compare the source with what was ingested, and it wants the alert to reach an on-call engineer.
Which solution meets these requirements?
Choose one answer.
AA. Treat a sync job with a Complete status as proof that every file was ingested, and add an EventBridge rule that pages on-call when a sync job fails.
BB. Re-embed the entire SharePoint library every hour with direct ingestion, so that the knowledge base always holds a full copy, and skip any comparison step.
CC. After each sync, run a Lambda function that compares SharePoint file counts and modified dates with the ingested documents, and alarm through CloudWatch and SNS on any gap.
DD. Add a contextual grounding check with a guardrail to the assistant, so that any answer not supported by the retrieved passages is blocked before it reaches the customer.
Show the answer and explanation
Answer: C
A. Wrong: The failure in this case was a sync that reported success while missing files, so a Complete status is not proof of completeness. The rule would never fire for this type of error, so the requirement is missed.
B. Wrong: Re-ingesting everything every hour is costly, and it still does not confirm that the source and index match. Without a comparison, the team cannot detect a stale or incomplete state, so the requirement is missed.
C. Right: The comparison catches both missing files and out-of-date files within one sync cycle, and the CloudWatch alarm notifies on-call through SNS. This matches the check and alert requirements without changing how the content is synced. Learn more: Invoke a Lambda function from an alarm (opens in a new tab)
D. Wrong: A grounding check tests answers against the passages that were retrieved, and an old fee schedule is still grounded in the index. The guardrail would not detect the stale or missing content, so the requirement is missed. Learn more: Contextual grounding checks (opens in a new tab)
Explanation
Comparing source files with the ingested documents after each sync detects both missing and outdated content, and a CloudWatch alarm reaches on-call. A sync's success status does not prove completeness, a full re-embed is costly and still unchecked, and a grounding check cannot tell that a retrieved passage is outdated.
Want more on Validating data quality before it reaches a model (Glue Data Quality, Lambda checks)? The 5 full-length practice sets have 5 more Validating data quality before it reaches a model (Glue Data Quality, Lambda checks) questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →