SAP-C03 practice question: Monitoring AI workloads and user experience (Synthetics, RUM) (1 of 2)
Operational Excellence and Automation A free question from our original SAP-C03 practice exam. Try it, then open the answer to see why each option is right or wrong.
A logistics firm runs a customer-support agent on Amazon Bedrock AgentCore. The agent calls tracking and claims tools, and it is released in several versions a month. Support managers report that some customers get slow answers and that tool calls sometimes fail without an error reaching the dashboard.
The platform team wants per-version visibility into response latency, token use and tool-call failures, with alarms that fire when answer quality drifts below an agreed level. It wants managed observability rather than a metrics pipeline it builds itself.
Which approach meets these requirements?
Choose one answer.
AA. Read the agent's chat transcripts once a month, count the complaints that mention slow answers or failed tools for each agent version, and share the counts with the support managers in a report.
BB. Turn on AgentCore Observability so latency, token use and tool-call errors appear in CloudWatch for each agent version, and run scheduled evaluations on a fixed question set that publish a quality metric with an alarm.
CC. Add a print statement to each tool function that writes its duration to the application log, and search the log for the word error once a day.
DD. Create a CloudWatch Synthetics canary that asks the agent one fixed question every hour, and alarm when the canary gets no answer.
Show the answer and explanation
Answer: B
A. Wrong: Monthly transcript reviews are slow and do not give per-version metrics or alarms. The option does not measure quality or latency as the agent runs.
B. Right: Correct. AgentCore Observability sends the agent's operational metrics and traces, including latency, token use and errors, to CloudWatch without a custom pipeline. Scheduled evaluations turn answer quality into a metric, so an alarm can fire when quality drifts below the agreed level.
C. Wrong: Print statements are a home-made measurement that needs code in every tool, and a daily search for the word error misses slow answers and quality drift. The option gives no per-version alarms.
D. Wrong: One fixed question shows only that the agent answers at all, not how fast or how well it answers across versions. Quality drift would stay hidden until a failure.
Explanation
AgentCore Observability provides latency, token and error metrics for the agent in CloudWatch, and scheduled quality evaluations published as metrics let alarms catch drift. Monthly reviews, daily error searches and a single canary question each miss the per-version or quality view.
Want more on Monitoring AI workloads and user experience (Synthetics, RUM)? The 5 full-length practice sets have 6 more Monitoring AI workloads and user experience (Synthetics, RUM) questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →