MLA-C02 practice question: Model and data drift monitoring (3 of 3)
Operating, Monitoring, and Securing ML and AI Solutions A free question from our original MLA-C02 practice exam. Try it, then open the answer to see why each option is right or wrong.
A regional bank monitors a credit-decision model in production with SageMaker Model Monitor and SageMaker Clarify, which it already uses. Its pre-deployment analysis found no bias on the age facet, but the model-risk team now wants to detect whether approval rates for each age group drift over the coming months.
The team wants a scheduled check that compares the model's predictions on production traffic with the bias baseline recorded before deployment, and an alert when the bias moves beyond a set threshold.
Which SageMaker AI capability should the team use?
Choose one answer.
AA. A pre-training bias analysis of the original training data, run again each month with the same Clarify settings, facet and label values as before.
BB. Model Monitor bias drift monitoring, using a Clarify configuration with the age facet and a baseline from the pre-deployment analysis.
CC. Feature attribution drift monitoring, which reports when the importance of each feature changes over time.
DD. Data quality monitoring on the endpoint, with the age column added to the baseline statistics.
Show the answer and explanation
Answer: B
A. Wrong: Pre-training analysis checks the dataset and does not look at the model's predictions on production traffic. Running it again repeats the wrong measurement.
B. Right: Correct. Bias drift monitoring runs on a schedule, compares the production predictions for the chosen facet with the baseline, and reports violations. This matches the drift check the risk team wants. Learn more: Bias drift for models in production (opens in a new tab)
C. Wrong: Attribution drift tracks changes in how features influence predictions, which is useful, but it does not measure the approval rates across age groups. The stem asks about outcomes by group.
D. Wrong: Data quality monitoring compares input distributions and constraints, and it does not measure the model's predictions by group. The bias question needs bias drift monitoring.
Explanation
Model Monitor bias drift monitoring runs Clarify bias metrics on production predictions on a schedule and compares them with a baseline for the chosen facet. Pre-training analysis repeats a dataset check, and attribution or data quality monitoring measure different things than group-level approval rates.
Want more on Model and data drift monitoring? The 5 full-length practice sets have 11 more Model and data drift monitoring questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →