Exams / AIP-C01 / Study guide / Agent and tool observability (AgentCore Observability, tracing)
Agent and tool observability (AgentCore Observability, tracing): AIP-C01 study notes and practice questions
What to know about Agent and tool observability (AgentCore Observability, tracing) for the AIP-C01 exam, from the Operational Efficiency and Optimization for GenAI Applications domain (12% of the exam guide), then 1 free question to test it. Part of the AIP-C01 study guide.
Agents fail in ways a single model call does not. They pick the wrong tool, send bad arguments, loop, or wait on a slow dependency. The exam tests whether you can see each of those in traces and metrics rather than from the final answer alone.
Trace every step
Amazon Bedrock AgentCore Observability emits OpenTelemetry traces with spans for each model call, each tool call and each session. Traces and metrics appear in CloudWatch, with dashboards per agent. Agents built with a framework that emits OpenTelemetry, such as Strands Agents, feed the same traces. Pass trace context across every service the agent calls, so one user request can be followed from start to finish.
Metrics and baselines
Track tool error rate, tool latency, tool calls per session and session duration. Set baselines from normal traffic and alarm on deviations. Examples include a tool whose latency suddenly rises tenfold, or a session that makes far more calls than usual. A rising call count often signals a loop.
Multi-agent and cross-account work
In a multi-agent system, pass trace context from the coordinator to each specialist, so the calls form one tree. To view agents in several accounts from one place, use CloudWatch cross-account observability, which shares telemetry with a monitoring account without copying logs. Keep sensitive data out of trace attributes, or mask it.
Alerts and response
Alert on unusual tool-call patterns, such as a tool called outside its normal sequence, a sudden spike on one tool, or repeated failures for one user. Notify on-call through an alarm. If a tool is misbehaving, tighten the AgentCore Policy rule that governs it, rather than changing the agent code under pressure.
How to choose
Scenario wording
Choose
See each tool call inside a session
AgentCore Observability traces
A tool has become slow
Tool latency metric with a baseline alarm
An agent seems stuck in a loop
Alarm on tool calls per session
Trace work across a coordinator and specialists
Propagated trace context, one trace tree
View agents running in many accounts
CloudWatch cross-account observability
Vendor-neutral instrumentation
OpenTelemetry through AWS Distro for OpenTelemetry
Common traps
Judging the agent by its final answer. The failure may be an intermediate tool call that happened to succeed.
Logging tool inputs with personal data unmasked.
Using the X-Ray SDKs for new instrumentation. They are in maintenance, so use OpenTelemetry.
Static alarm thresholds with no baseline, so normal peaks page on-call.
Forgetting the monitoring account when agents run in several accounts.
An airline's rebooking team uses an AI agent hosted on Amazon Bedrock AgentCore. For each disruption, the agent calls a model several times and runs tools that check seats, fares and passenger records. Some runs take over a minute, and the team cannot see why.
The team needs to see the time spent in each model call and each tool call within one run, and to see which step returned an error. The agent is built with an open-source framework, and the team wants a standard format that other tools can read.
The telemetry must cover every agent run in production, use a vendor-neutral format, and appear in CloudWatch for the on-call engineers without the team writing its own tracing system.
Which approach meets these requirements?
Choose one answer.
AA. Add print statements with timestamps to each tool function, write them to a file on the runtime host, and search that file with a script whenever a slow run is reported.
BB. Turn on AgentCore Observability, which emits OpenTelemetry traces with a span for each model and tool call, and view the traces and their metrics in CloudWatch dashboards.
CC. Rely on Amazon Bedrock model invocation logs, which record every call the agent makes, including the time that the agent's tools took to run inside its own code.
DD. Ask Amazon Q Developer to review the agent's source code for slow loops and blocking calls, and fix the issues it reports before each new release of the agent.
Show the answer and explanation
Answer: B
A. Wrong: Print statements show timing only for the code that was instrumented, and log files on runtime hosts are hard to correlate across a run. Searching them after each report also gives the on-call engineers no live view.
D. Wrong: A code review can suggest likely causes, but it cannot show the actual timing of a live run or which step failed in production. The option does not provide the per-run telemetry the team needs.
Explanation
AgentCore Observability emits OpenTelemetry traces with spans for each model call and tool call, so a slow run can be traced step by step in CloudWatch. Print statements and invocation logs miss tool execution, and a code review cannot show live timing or the failing step.
Want more on Agent and tool observability (AgentCore Observability, tracing)? The 5 full-length practice sets have 5 more Agent and tool observability (AgentCore Observability, tracing) questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →
Study Agent and tool observability (AgentCore Observability, tracing)