Exams / AIP-C01 / Study guide / Choosing how to deploy models (on-demand, Provisioned Throughput, SageMaker AI endpoints)
Choosing how to deploy models (on-demand, Provisioned Throughput, SageMaker AI endpoints): AIP-C01 study notes and practice questions
What to know about Choosing how to deploy models (on-demand, Provisioned Throughput, SageMaker AI endpoints) for the AIP-C01 exam, from the Implementation and Integration domain (26% of the exam guide), then 1 free question to test it. Part of the AIP-C01 study guide.
AIP-C01 gives a traffic pattern, a latency need and a model type, then asks which way to serve the model. Start with the simplest managed option that meets the latency target, and move to committed or self-managed capacity only when traffic or control requires it.
Bedrock options
On-demand: pay per request with no commitment. It is the default for spiky or low-volume traffic.
Provisioned Throughput: buy model units with a commitment for steady, high-volume traffic, or for some custom models. The cost is paid whether or not you send traffic.
Batch inference: submit prompts as JSONL files in Amazon S3 and collect results from S3 later. It costs less than on-demand for supported models, and it fits large overnight jobs.
Service tiers: Priority tier gives lower latency at a higher price, Standard is the default, and Flex trades longer processing for a lower price. You choose the tier per request.
SageMaker AI options
Use a real-time endpoint when you host an open-weight model yourself and need control over the container, instance type and scaling. Asynchronous inference queues large payloads or long jobs and can scale to zero. Serverless inference suits intermittent traffic where cold starts are acceptable, but it runs on CPU only, so it does not fit GPU-bound LLMs. Batch transform scores a dataset offline.
How to choose
Scenario wording
Choose
“Spiky, low-volume chatbot traffic”
On-demand
“Steady, high volume, guaranteed capacity”
Provisioned Throughput
“Nightly job over millions of documents”
Bedrock batch inference
“Open-weight model, need container and scaling control”
Provisioned Throughput for spiky, low traffic. You pay for idle capacity.
Real-time calls for work that could run as batch, which costs more and adds no value.
Assuming every tier or option works for every model. Service tiers, batch inference and Provisioned Throughput are each offered for a subset of models.
A SageMaker AI endpoint for a model that Bedrock already serves, when no control is needed. This adds operations work.
Batch inference for interactive chat, which needs a real-time answer.
A recruiting platform is adding a feature that summarizes candidate resumes. The feature is new, so the team has no traffic history yet. Usage is spiky: recruiters upload batches on weekday mornings, and on most other days the feature handles only a few hundred summaries.
The team wants to pay only for the tokens it actually processes, and it does not want to commit to hourly capacity until real usage patterns are known. During a batch, each summary must return within a few seconds.
Which deployment option is the MOST cost-effective for these requirements?
Choose one answer.
AA. Call the foundation model on demand through the Amazon Bedrock Converse API, paying per token processed, and retry throttled requests with exponential backoff.
BB. Purchase Provisioned Throughput with a one-month commitment, sized for the weekday morning peak, so that every request has dedicated capacity.
CC. Deploy the model to a SageMaker AI real-time endpoint on GPU instances that run around the clock, with autoscaling to a minimum of one instance.
DD. Run the summaries with a SageMaker AI batch transform job each night over that day's uploads, so that each resume is processed at the lowest cost per job.
Show the answer and explanation
Answer: A
A. Right: On-demand pricing charges only for tokens processed, with no commitment and no idle capacity. Short responses during a batch meet the latency target, and the team can move to reserved capacity later if usage grows.
B. Wrong: Provisioned Throughput bills for reserved model units every hour, including the many days with little traffic. This conflicts with the pay-only-for-tokens requirement before any usage pattern is known.
C. Wrong: A real-time endpoint keeps GPU instances running at night and on quiet days, so the team pays for idle hours. It also adds endpoint and model operations the team does not need for a new feature.
D. Wrong: Batch transform is cheap per job, but results arrive hours later, which misses the requirement of a few seconds per summary during a batch. Recruiters need answers while they are working.
Explanation
On-demand inference suits a new feature with unknown and spiky traffic, because the team pays only for tokens processed and has no hourly commitment. Provisioned Throughput and always-on endpoints charge for capacity during quiet periods, and batch transform gives up the seconds-level response that recruiters need.
Want more on Choosing how to deploy models (on-demand, Provisioned Throughput, SageMaker AI endpoints)? The 5 full-length practice sets have 7 more Choosing how to deploy models (on-demand, Provisioned Throughput, SageMaker AI endpoints) questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →
Study Choosing how to deploy models (on-demand, Provisioned Throughput, SageMaker AI endpoints)