MLA-C02 practice question: Model and data drift monitoring (1 of 3)
Operating, Monitoring, and Securing ML and AI Solutions A free question from our original MLA-C02 practice exam. Try it, then open the answer to see why each option is right or wrong.
A logistics broker serves an ETA model from a real-time SageMaker AI endpoint, and its ML team already uses SageMaker Model Monitor on its other production endpoints. Last week a carrier changed how it reports dwell time, and missing values in one input feature rose from under 1 percent to about 20 percent without any endpoint error.
The team wants a daily automatic check that compares the endpoint's inference inputs with statistics and constraints from the training data, and an alert when they no longer match.
Which solution meets these requirements?
Choose one answer.
AA. Turn on endpoint logs in CloudWatch, and have an analyst search them each morning for the word "missing" in the feature payloads.
BB. Run a weekly SageMaker Processing job that retrains the model and compares the training accuracy with the last week's accuracy.
CC. Enable AWS Config rules on the endpoint to check that the model artifact has not changed since deployment.
DD. Enable data capture, and create a Model Monitor data quality schedule on the endpoint with a baseline from the training data and CloudWatch alarms on violations.
Show the answer and explanation
Answer: D
A. Wrong: Log text search depends on the payload containing that word, and it does not compare the data against training statistics. The option is manual and would miss subtle shifts.
B. Wrong: Retraining does not check the incoming data, and training accuracy does not show a change in input quality. The option does not detect the missing-value shift.
C. Wrong: AWS Config checks resource configuration, such as whether the artifact changed, and does not look at the data flowing into the model. The problem is in the inputs.
D. Right: Correct. Data quality monitoring compares captured inference data with a baseline of statistics and constraints from the training data, and it reports violations on a schedule. Alarms on the violation metrics notify the team. Learn more: Data quality monitoring with Model Monitor (opens in a new tab)
Explanation
Model Monitor data quality monitoring compares captured inference data with baseline statistics and constraints computed from the training data, and reports violations on a schedule. Alarms on those results tell the team when the inputs change. Log searches, retraining and configuration checks do not compare the data with the baseline.
Want more on Model and data drift monitoring? The 5 full-length practice sets have 11 more Model and data drift monitoring questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →