Exams / AIP-C01 / Study guide / CI/CD and GenAI gateway architectures (CodePipeline, central gateway)
CI/CD and GenAI gateway architectures (CodePipeline, central gateway): AIP-C01 study notes and practice questions
What to know about CI/CD and GenAI gateway architectures (CodePipeline, central gateway) for the AIP-C01 exam, from the Implementation and Integration domain (26% of the exam guide), then 1 free question to test it. Part of the AIP-C01 study guide.
AIP-C01 asks how to change prompts, models, retrieval settings and guardrails with the same discipline as application code. Questions usually name one failure, such as a prompt edit that lowered answer quality, a team that exceeded its quota, or a guardrail changed by hand, and ask for the control that prevents it.
Pipeline stages with a quality gate
An AWS CodePipeline pipeline for a GenAI feature typically runs source, build, unit tests, deployment to staging, evaluation, and then production. The evaluation stage runs a Bedrock evaluation job (automatic or LLM-as-a-judge) or a test harness against a golden dataset, and fails when metrics fall below thresholds. A failing action stops the pipeline before production. A scan stage can also check prompt files and agent tool permissions for secrets or overly broad access.
Infrastructure as code
Define knowledge bases, guardrails, flows, IAM roles and prompt references in CloudFormation or CDK, so every environment gets the same configuration. A guardrail edited in the console drifts from the template and makes results impossible to reproduce. Keep prompt text in version control, and reference a pinned prompt version in production while staging tests the next one.
Safe rollout and rollback
Shift traffic gradually. AWS AppConfig can switch a prompt version or model choice for a percentage of requests and roll back when a CloudWatch alarm fires. SageMaker AI deployment guardrails do the same for endpoints, with blue/green traffic shifting and automatic rollback. The alarm metric should reflect quality or safety, such as guardrail intervention rate or a scored evaluation sample, not only latency or errors.
Central GenAI gateway
A gateway is one endpoint, often API Gateway in front of a Lambda function or container, that every team calls instead of Bedrock directly. It enforces a model allowlist, per-team quotas and rate limits, attaches the approved guardrail, writes logs, and tags each call with the team ID. Owners change the gateway once, not in every application. Application inference profiles with cost tags offer a lighter way to track usage per team.
How to choose
Scenario wording
Choose
“Block a release when answer quality drops”
Evaluation stage in CodePipeline with thresholds
“Recreate the same knowledge base and guardrail in another account”
CloudFormation or CDK templates
“Send a new prompt to 10 percent of users and revert quickly”
AppConfig rollout with alarm-based rollback
“Quotas, logs and model access for many teams”
Central GenAI gateway
“Charge each team for model use”
Application inference profiles with cost tags
“Stop a bad change without waiting for a person”
CloudWatch alarm that triggers rollback
Common traps
Console edits to guardrails or knowledge bases, which cause environment drift.
Evaluating only after production deploy, when the question asks to prevent a bad release.
A manual approval as the only quality gate, when the scenario asks for automatic enforcement.
A gateway without per-team throttling, which lets one team starve the others.
Rolling back by hand after users complain, when an alarm-driven rollback would act sooner.
A healthcare claims processor runs a generative AI service that writes plain-language explanations of benefits for members. Prompt templates and model versions change every few weeks, and each change is deployed through AWS CodePipeline.
Before any candidate version reaches members, its replies to an 800-case labeled golden set must score no more than 2 points below the current production version, and no reply may be flagged as harmful. The pipeline must deploy automatically when a candidate passes and must stop before deployment when it fails.
The compliance team wants no manual step on the normal path, and each run must leave an audit record of the scores.
Which pipeline design meets these requirements?
Choose one answer.
AA. Deploy each candidate to production right after the build, then run the golden-set evaluation each night and roll back by hand if the scores drop.
BB. Add a manual approval action after the deploy stage, so that a reviewer checks the team's evaluation dashboard before the candidate stays in production.
CC. Run only unit tests on the prompt templates in an AWS CodeBuild action, because a full golden-set run costs too much to repeat for every candidate.
DD. Before deployment, add a CodeBuild action that scores the candidate and production on the golden set, saves the scores to S3 and fails below the threshold.
Show the answer and explanation
Answer: D
A. Wrong: Members would see a regression before anyone checks it, which breaks the rule that failing candidates never reach production. Detection after deployment is not a gate, and the rollback is manual.
B. Wrong: The candidate has already been deployed when the reviewer looks, and the approval is a manual step on the normal path. The compliance team wants a gate that runs automatically before deployment.
C. Wrong: Unit tests check template syntax, not whether explanations stay accurate and free of harmful content. The pipeline would pass candidates that score worse than production.
D. Right: A gate before deployment blocks a failing candidate automatically, so members never see it. Storing each run's scores gives the compliance team the audit record, and no step on the normal path is manual. Learn more: CodePipeline concepts (opens in a new tab)
Explanation
An automated evaluation stage placed before deployment compares the candidate with production on the golden set and stops the pipeline when it regresses, so bad versions never reach members. Storing each run's scores provides the audit trail. Deployment first, manual approval and unit tests alone each leave regressions reaching members or add a manual step.
Want more on CI/CD and GenAI gateway architectures (CodePipeline, central gateway)? The 5 full-length practice sets have 7 more CI/CD and GenAI gateway architectures (CodePipeline, central gateway) questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →
Study CI/CD and GenAI gateway architectures (CodePipeline, central gateway)