MLA-C02 practice question: Deployment guardrails and rollback
Deployment and Orchestration of ML and AI Workflows A free question from our original MLA-C02 practice exam. Try it, then open the answer to see why each option is right or wrong.
An esports league serves a match-prediction model from a production SageMaker AI endpoint. The team has a new model version that it expects to be more accurate, but a bad release would hurt the league's live betting partners within minutes.
The team wants the new version to receive a small share of live traffic first, and to shift the remaining traffic only if the endpoint's error and latency alarms stay quiet. If an alarm fires, traffic must return to the previous version automatically.
Which deployment approach meets these requirements?
Choose one answer.
AA. Replace the production endpoint configuration with the new model in a single all-at-once update, and roll back manually if partners report a problem.
BB. Create a second endpoint with the new model, and split traffic between the two endpoints with a weighted DNS change made by hand.
CC. Use a canary deployment guardrail on the endpoint to shift a small share of traffic first, with automatic rollback when CloudWatch alarms fire.
DD. Deploy the new model with a shadow variant that receives all traffic, so the old model only sees the requests that fail.
Show the answer and explanation
Answer: C
A. Wrong: An all-at-once replacement sends all traffic to the new version immediately, and rollback depends on people noticing the problem. The stem asks for a gradual shift with automatic rollback.
B. Wrong: Manual DNS changes give no automatic rollback on alarms and are slow to undo. The stem requires the rollback to happen without people acting.
C. Right: Correct. Deployment guardrails with canary traffic shifting send a small share of traffic to the new version, monitor the configured CloudWatch alarms, and roll back automatically on an alarm. The remaining traffic shifts only after the canary period passes. Learn more: Deployment guardrails for updating models in production (opens in a new tab)
D. Wrong: A shadow variant receives a copy of the traffic for comparison and does not serve responses to the callers. The stem asks for the new version to take live traffic with rollback.
Explanation
Deployment guardrails on a SageMaker AI endpoint use traffic shifting, such as a canary, with CloudWatch alarms that trigger an automatic rollback. All-at-once replacement and manual DNS changes lack automatic rollback, and a shadow variant does not serve the live responses.
The endpoint's canary guardrail sends a small share of predictions to the new model and rolls back if an alarm fires.
Want more on Deployment guardrails and rollback? The 5 full-length practice sets have 7 more Deployment guardrails and rollback questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →