AIP-C01 practice question: Model cascading and routing to the right model
Implementation and Integration A free question from our original AIP-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.
A customer support software company sends every chat message from its product to a foundation model on Amazon Bedrock. Most messages are simple, such as password resets and order status questions. A smaller share are complex, such as multi-step troubleshooting that needs careful reasoning.
The company wants to lower its model spend without a noticeable drop in answer quality on hard questions. It does not want to build or maintain its own classifier to decide which messages are hard.
The application currently calls one model ID, and the team wants to change as little application code as possible.
Which change meets these requirements with the LEAST engineering effort?
Choose one answer.
AA. Send all messages to the smaller model, and have a Lambda function resend a message to the larger model whenever the reply is shorter than a fixed number of words.
BB. Train a custom classifier in Amazon SageMaker AI that labels each message as simple or complex, and have the application call one of two model IDs based on the label.
CC. Keep the larger model for all requests, and turn on prompt caching so that repeated system prompts cost less per call.
DD. Point the application at an intelligent prompt routing endpoint, which predicts response quality and sends each request to one of two models in a family.
Show the answer and explanation
Answer: D
A. Wrong: Reply length is a weak signal of quality, so hard questions may get short wrong answers that never reach the larger model. The team also has to build and tune the fallback logic itself.
B. Wrong: A classifier can route well, but the team must label data, train, host and retrain it, which is the work the company wants to avoid. It also adds a call before every model request.
C. Wrong: Prompt caching lowers the cost of repeated prefixes, but the company would still pay the larger model's rate for every simple message. Simple requests are the main source of spend, so this misses the goal.
Intelligent prompt routing predicts response quality for each request and picks one of two models in the same family, so cost falls without a custom classifier. The application only changes the model ID it calls. Length-based fallbacks, custom classifiers and caching each either miss hard questions, add maintenance work, or leave simple requests on the expensive model.
Want more on Model cascading and routing to the right model? The 5 full-length practice sets have 5 more Model cascading and routing to the right model questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →