Handling throttling, retries and fallbacks: AIP-C01 study notes and practice questions
What to know about Handling throttling, retries and fallbacks for the AIP-C01 exam, from the Implementation and Integration domain (26% of the exam guide), then 1 free question to test it. Part of the AIP-C01 study guide.
AIP-C01 describes a throttling error, a traffic spike, a failing dependency, or a tool call that ran twice, and asks for the fix that restores service without overspending.
Throttling from Bedrock
A ThrottlingException (HTTP 429) means the account has used up a runtime quota in that Region for that model. Choose the fix that matches the cause:
SDK retries with exponential backoff and jitter. The AWS SDKs retry throttling errors by default, so tune the attempt count rather than writing a loop that retries immediately.
Queues such as SQS smooth short bursts.
Cross-Region inference profiles let a request use capacity in other Regions when the local Region is busy.
A quota increase through Service Quotas, when sustained demand is legitimate and expected to continue.
Provisioned Throughput for steady, predictable high volume, or the Priority tier when latency-critical requests must go first.
A quota increase raises the limit in one Region, while cross-Region inference spreads traffic.
Fallbacks and circuit breakers
Define a fallback order: a primary model, then a second model from the same family, then a smaller model or a cached answer. A circuit breaker stops calls to a dependency that keeps failing, so the application fails fast and recovers when the dependency returns. A fallback must keep the same guardrail and output schema. A fallback that skips the guardrail creates a safety gap.
Limits at the front door
API Gateway usage plans and API keys apply request quotas and throttling per client for REST APIs. They stop one noisy client from consuming the backend capacity that others need.
Tracing
Trace each request across API Gateway, Lambda and model calls with AWS X-Ray or OpenTelemetry, so you can see where retries and latency occur. The X-Ray SDKs are in maintenance, so instrument new code with OpenTelemetry through the AWS Distro for OpenTelemetry.
Idempotent retries
Retrying a model call to regenerate text is safe. Retrying a tool call that creates an order, sends an email or charges a card is not. Pass an idempotency key with the tool request, and use a conditional write in DynamoDB so the second attempt does nothing when the first one already succeeded.
How to choose
Scenario wording
Choose
“ThrottlingException at peak, spare capacity elsewhere”
Cross-Region inference profile plus SDK retries
“Steady traffic that permanently exceeds the quota”
Service Quotas increase request
“Bursty batch calls fail during a spike”
SQS queue with limited consumer concurrency
“Primary model returns server errors”
Fallback to a second model with the same guardrail
“One client floods the API”
API Gateway usage plan with throttling
“Agent tool call ran twice after a timeout”
Idempotency key and conditional write
Common traps
Immediate retries without backoff, which deepen the throttling.
A quota increase for a short spike that a queue would absorb.
Global cross-Region inference for data bound to one geography. A geographic profile is the correct fix there.
A fallback model without the guardrail.
New X-Ray SDK instrumentation when OpenTelemetry is the supported path.
A ride-sharing company's support assistant calls a foundation model on Amazon Bedrock from an AWS Lambda function. During the evening peak, some calls fail with a ThrottlingException, which means the account briefly exceeded its request quota for that model in the Region.
The team wants most throttled calls to succeed within a few seconds, without dropping requests and without sending a burst of retries that makes throttling worse. The team does not want to request a higher quota, and the change should be small.
Which approach meets these requirements with the LEAST code change?
Choose one answer.
AA. Configure the AWS SDK retry strategy with exponential backoff, jitter and a capped number of attempts, so retries are spread out over a few seconds.
BB. Catch the throttling error and retry immediately in a loop of up to 100 attempts with no wait, so that a quota slot is likely to free up on the next try.
CC. Catch the throttling error and return a fixed apology to the rider without retrying, which keeps the code the simplest and avoids any extra calls.
DD. Request a higher quota for the model and launch the feature after the increase is approved, so that throttling does not happen during the evening peak.
Show the answer and explanation
Answer: A
A. Right: Exponential backoff with jitter spreads retries out, so the bursts that cause more throttling are avoided. The SDK setting is a small change, and most throttled calls succeed within seconds.
B. Wrong: Immediate retries from every Lambda invocation create a retry storm that keeps the account over its quota. The loop also ties up each function for a long time.
C. Wrong: This avoids extra calls, but most throttled requests would fail for the rider, which misses the success requirement. Dropping requests is the outcome the team wants to prevent.
D. Wrong: A quota increase may help, but the team has ruled it out for this change. The feature would also wait on an approval, while the requirement is a fix that works now.
Explanation
Exponential backoff with jitter in the SDK retries throttled calls over a spread-out schedule, so most requests succeed within seconds without a retry storm. The change is a configuration setting. Immediate retries, dropping requests and a quota increase each fail a stated requirement.
Want more on Handling throttling, retries and fallbacks? The 5 full-length practice sets have 5 more Handling throttling, retries and fallbacks questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →