AIP-C01 practice question: Throughput, quotas and capacity planning
Operational Efficiency and Optimization for GenAI Applications A free question from our original AIP-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.
A marketing agency runs a campaign tool that generates product descriptions for about 60 retail clients. Each request includes a long brand guide, so most requests carry several thousand input tokens, and the tool calls a model on Amazon Bedrock in one Region.
In a load test a month before launch, the tool sends about 300 requests a minute in short bursts. Many calls fail with throttling errors, even though the request count is well below what the team expected to be the limit. The launch will last only a week, and the budget is tight.
The fix must remove most of the throttling during the launch, avoid paying for capacity the team will not need after the week, and keep the workload on the same model.
Which approach meets these requirements?
Choose one answer.
AA. Estimate tokens per minute from real prompt and response lengths at the peak rate, request a quota increase with headroom, and retry throttled calls with backoff.
BB. Move the requests to Provisioned Throughput with the largest number of model units for the whole month, so that capacity is reserved and throttling cannot occur during the launch.
CC. Set the client timeout to five minutes and retry throttled calls at a fixed interval, so that each request waits in line until quota frees up, without changing the quota.
DD. Split the tool into one Lambda function per client, because each function gets its own default quota, and send each client's requests from its own function during the launch.
Show the answer and explanation
Answer: A
A. Right: Bedrock throttles on tokens per minute as well as requests per minute, so long brand-guide prompts can hit the quota at modest request rates. Sizing tokens per minute from real lengths and requesting an increase a month ahead addresses the cause with no capacity commitment, and backoff smooths the bursts. Learn more: Request an increase for Amazon Bedrock runtime quotas (opens in a new tab)
B. Wrong: Reserved capacity removes throttling, but the hourly commitment would run for the whole month, far beyond the one-week launch. This conflicts with the budget and with the requirement to avoid paying for unneeded capacity. Learn more: Provisioned Throughput for Amazon Bedrock (opens in a new tab)
D. Wrong: Quotas apply to the account and Region, not to each function, so splitting the tool does not add quota. The option changes the architecture without solving the throttling. Learn more: Amazon Bedrock quotas (opens in a new tab)
Explanation
Bedrock throttles on tokens per minute as well as requests per minute, so large brand-guide prompts can exhaust the quota at modest request rates. Sizing tokens per minute from real lengths, requesting a quota increase well before launch and retrying with backoff address the cause at on-demand prices. Monthly reserved capacity, longer timeouts and per-function splitting do not fix the quota.
Want more on Throughput, quotas and capacity planning? The 5 full-length practice sets have 5 more Throughput, quotas and capacity planning questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →