Model cascading and routing to the right model: AIP-C01 study notes and practice questions
What to know about Model cascading and routing to the right model for the AIP-C01 exam, from the Implementation and Integration domain (26% of the exam guide), then 1 free question to test it. Part of the AIP-C01 study guide.
AIP-C01 asks how to send a request to the right model. The choice depends on whether routing should be predictable and auditable, automatic within one model family, or driven by the model’s own result. A cascade can save money, but each escalation adds latency, so the design must limit how often it happens.
Routing methods
Intelligent prompt routing: a Bedrock router endpoint sends each request to one of two models in the same family, based on predicted response quality. You set the trade-off, and no routing code is needed. It works only within one family.
Static routing config: the application maps each task type to a model ID in config. It is predictable, easy to audit and easy to change. Store the mapping in AWS AppConfig so you can change which model a task uses without redeploying code.
Step Functions content-based routing: a Choice state inspects the input, such as its type or length, and branches to the model call that fits. Each branch is recorded in the execution history.
Escalation cascade: call the small model first. If its answer fails a check, such as a validator, a score or a judge, call the large model. Use this when most requests are easy.
API Gateway request transformation: map a header, path or request field to a model ID before the request reaches the backend, so clients do not need to know model names.
Metrics-based routing: use per-model latency and cost metrics from CloudWatch to shift traffic away from a slow or expensive model.
How to choose
Scenario wording
Choose
“Mixed prompts within one model family, let AWS route by quality”
Intelligent prompt routing
“Each feature always uses the same model, changes must be auditable”
Static routing config, switched with AppConfig
“Workflow branches on request type and records each branch”
Step Functions Choice state
“Start cheap, use the large model only when needed”
Escalation cascade
“Clients send a tier header and should not know model names”
API Gateway request transformation
“Shift traffic when a model slows down or costs rise”
Metrics-based routing
Common traps
Intelligent prompt routing across families or to models outside the family it supports.
Escalation without a reliable signal. A confidence check that is not tested on real data lets wrong answers from the small model through.
Hard-coded model IDs in many places, so each change needs a redeploy.
Serial cascades on latency-critical paths. An escalated request waits for two model calls.
Routing by prompt length alone. Short prompts can be hard, and long prompts can be simple.
A customer support software company sends every chat message from its product to a foundation model on Amazon Bedrock. Most messages are simple, such as password resets and order status questions. A smaller share are complex, such as multi-step troubleshooting that needs careful reasoning.
The company wants to lower its model spend without a noticeable drop in answer quality on hard questions. It does not want to build or maintain its own classifier to decide which messages are hard.
The application currently calls one model ID, and the team wants to change as little application code as possible.
Which change meets these requirements with the LEAST engineering effort?
Choose one answer.
AA. Send all messages to the smaller model, and have a Lambda function resend a message to the larger model whenever the reply is shorter than a fixed number of words.
BB. Train a custom classifier in Amazon SageMaker AI that labels each message as simple or complex, and have the application call one of two model IDs based on the label.
CC. Keep the larger model for all requests, and turn on prompt caching so that repeated system prompts cost less per call.
DD. Point the application at an intelligent prompt routing endpoint, which predicts response quality and sends each request to one of two models in a family.
Show the answer and explanation
Answer: D
A. Wrong: Reply length is a weak signal of quality, so hard questions may get short wrong answers that never reach the larger model. The team also has to build and tune the fallback logic itself.
B. Wrong: A classifier can route well, but the team must label data, train, host and retrain it, which is the work the company wants to avoid. It also adds a call before every model request.
C. Wrong: Prompt caching lowers the cost of repeated prefixes, but the company would still pay the larger model's rate for every simple message. Simple requests are the main source of spend, so this misses the goal.
Intelligent prompt routing predicts response quality for each request and picks one of two models in the same family, so cost falls without a custom classifier. The application only changes the model ID it calls. Length-based fallbacks, custom classifiers and caching each either miss hard questions, add maintenance work, or leave simple requests on the expensive model.
Want more on Model cascading and routing to the right model? The 5 full-length practice sets have 5 more Model cascading and routing to the right model questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →
Study Model cascading and routing to the right model