Cost-effective model choice and tiered usage: AIP-C01 study notes and practice questions

What to know for the exam

This topic tests how to lower the bill without missing quality or latency targets. Expect scenarios that ask you to choose a model, a service tier or a cost-tracking method for a specific workload.

Tier models by task difficulty

Send simple steps, such as classification, field extraction and routing, to a small model. Reserve the largest model for hard reasoning. Intelligent prompt routing in Amazon Bedrock sends each request to one of two models in the same family, chosen by predicted response quality and cost. Use it when prompts vary in difficulty and both models are acceptable. Write your own router, such as a classifier in Lambda, when you need rules the built-in router cannot express.

Measure price-performance on your own data

Public leaderboards do not show how a model handles your documents. Build a representative prompt dataset, run each candidate model with Bedrock evaluations, and compare quality, latency and cost per task side by side. Choose the cheapest model that passes your quality threshold. Repeat the test when the provider releases a new version.

Distill for narrow, high-volume tasks

Bedrock model distillation trains a smaller student model to imitate a larger teacher on your task. It fits a narrow, repeated task where the large model is overkill and you have enough examples. It does not fit a broad, open-ended task that needs general knowledge.

Choose a service tier and batch

Bedrock service tiers trade cost against latency per request. Flex suits latency-tolerant work at a lower price, Standard is the default, and Priority serves urgent traffic ahead of the others at a higher price. Batch inference reads prompts from JSONL files in S3 and writes results back to S3. It costs less than on-demand calls for large jobs that do not need an immediate answer.

Track cost per team

Application inference profiles let you tag requests, so you can allocate cost to an application or team with cost allocation tags. Use AWS Cost Anomaly Detection to alert on unexpected spend. A profile’s model is fixed when it is created, so moving a team to a cheaper model means creating a new profile and updating the ARN the app reads from configuration.

How to choose

Common traps

• Choosing a model from a public leaderboard instead of measuring on your own prompts.
• Using the largest model for every agent step, including simple formatting steps.
• Using batch for a live chat, which needs an immediate reply.
• Distilling a broad task where the smaller student lacks general knowledge.
• Tagging nothing, so the bill cannot be split by team.

See also: throughput and capacity and latency

Try the 1 practice question ↓