AIP-C01 practice question: CI/CD and GenAI gateway architectures (CodePipeline, central gateway)
Implementation and Integration A free question from our original AIP-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.
A healthcare claims processor runs a generative AI service that writes plain-language explanations of benefits for members. Prompt templates and model versions change every few weeks, and each change is deployed through AWS CodePipeline.
Before any candidate version reaches members, its replies to an 800-case labeled golden set must score no more than 2 points below the current production version, and no reply may be flagged as harmful. The pipeline must deploy automatically when a candidate passes and must stop before deployment when it fails.
The compliance team wants no manual step on the normal path, and each run must leave an audit record of the scores.
Which pipeline design meets these requirements?
Choose one answer.
AA. Deploy each candidate to production right after the build, then run the golden-set evaluation each night and roll back by hand if the scores drop.
BB. Add a manual approval action after the deploy stage, so that a reviewer checks the team's evaluation dashboard before the candidate stays in production.
CC. Run only unit tests on the prompt templates in an AWS CodeBuild action, because a full golden-set run costs too much to repeat for every candidate.
DD. Before deployment, add a CodeBuild action that scores the candidate and production on the golden set, saves the scores to S3 and fails below the threshold.
Show the answer and explanation
Answer: D
A. Wrong: Members would see a regression before anyone checks it, which breaks the rule that failing candidates never reach production. Detection after deployment is not a gate, and the rollback is manual.
B. Wrong: The candidate has already been deployed when the reviewer looks, and the approval is a manual step on the normal path. The compliance team wants a gate that runs automatically before deployment.
C. Wrong: Unit tests check template syntax, not whether explanations stay accurate and free of harmful content. The pipeline would pass candidates that score worse than production.
D. Right: A gate before deployment blocks a failing candidate automatically, so members never see it. Storing each run's scores gives the compliance team the audit record, and no step on the normal path is manual. Learn more: CodePipeline concepts (opens in a new tab)
Explanation
An automated evaluation stage placed before deployment compares the candidate with production on the golden set and stops the pipeline when it regresses, so bad versions never reach members. Storing each run's scores provides the audit trail. Deployment first, manual approval and unit tests alone each leave regressions reaching members or add a manual step.
Want more on CI/CD and GenAI gateway architectures (CodePipeline, central gateway)? The 5 full-length practice sets have 7 more CI/CD and GenAI gateway architectures (CodePipeline, central gateway) questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →