Exams / AIP-C01 / Study guide / Coordinating several models (specialists, ensembles, routing)
Coordinating several models (specialists, ensembles, routing): AIP-C01 study notes and practice questions
What to know about Coordinating several models (specialists, ensembles, routing) for the AIP-C01 exam, from the Implementation and Integration domain (26% of the exam guide), then 1 free question to test it. Part of the AIP-C01 study guide.
AIP-C01 presents a workload with mixed tasks, such as images, code, classification and summaries, and asks how to use more than one model. The right design depends on whether the models do different jobs, whether they must agree, or whether most requests are easy and only a few need the large model.
Patterns
Specialist models: each task type goes to the model best suited for it, such as a vision model for images, a code model for code and a small model for summaries. The application or a router decides which one to call. Choose each model by testing it on your own data, not by its size alone.
Ensemble voting: run several models on the same input and take the majority label. It improves reliability for classification when the models make different kinds of errors. It multiplies cost, and it needs a tie-breaker rule.
Triage, then escalate: a small, cheap model handles or classifies each request. Only requests it cannot handle, or flags as hard, go to the large model.
Judge model: a model scores several candidate answers against a rubric and picks the best. Judges can favor answers that look like their own output, so check the judge against human-scored samples.
Aggregation logic: when parallel model calls each return part of an answer, merge them with deterministic code, such as a Lambda function that combines lists, removes duplicates and validates the schema. Use another model only when code cannot do the merge.
Model selection framework
Match each task to a model using four inputs: quality on your evaluation set, latency target, cost per request and any data or Region rules. Re-run the comparison when a new model arrives or your task mix changes.
How to choose
Scenario wording
Choose
“Each request type has a different best model”
Specialist models with routing in the application
“Classification must agree across models”
Ensemble voting with a tie-breaker
“Most requests are simple, a few are hard, and cost matters”
Small-model triage with escalation
“Pick the best of several drafts”
Judge model with a rubric, checked by humans
“Combine results from parallel model calls”
Deterministic aggregation in Lambda
“Route between models of one family automatically”
Intelligent prompt routing
Common traps
Ensembles on every request. Cost multiplies by the number of models. Reserve voting for requests where accuracy justifies it.
Choosing the largest model by default for every step. Most steps need a smaller model.
Triage without monitoring. Track how often the small model escalates, and how often it is wrong on requests it kept.
Mixed output formats from different models. Normalize them to one schema before merging.
Trusting a judge without checking it against human judgment or position-swapped comparisons.
Using a second model to merge results that a simple code merge handles.
A media company runs a content platform where users upload posts that mix photos, code snippets and long articles. Each post goes through one processing pipeline that must produce three outputs: tags for each photo, a review of each code snippet, and a summary of each article.
The quality target differs by task. Image tagging, code review and summarization each need the strongest model that is available for that task, and the team has evaluated several candidates for each one. Summaries must appear in under three seconds, and the monthly model spend must stay within budget.
The team also wants to replace the model for one task without redeploying the other two tasks.
Which architecture meets these requirements?
Choose one answer.
AA. Send every post to one large general-purpose multimodal model with a long system prompt that covers tagging, code review and summarization, and parse the three outputs from one response.
BB. Use a routing step that detects each part's content type and calls the specialist model that scored best for that task, each with its own prompt and configuration.
CC. Send each post to all three candidate models for each task, and use a fourth model to choose the best output, so that the strongest answer is always selected.
DD. Fine-tune one small text model on examples of all three task types, and convert each photo to a text caption first so that the same model handles every part of a post.
Show the answer and explanation
Answer: B
A. Wrong: One model call per post is simple and cheap, but the single prompt cannot be tuned per task. Swapping that model would also change all three outputs at once, which breaks the independent replacement requirement.
B. Right: A router sends each part to the model that scored best in evaluation, and each specialist keeps its own prompt and configuration. Only one specialist changes when a better model appears, so the other tasks are not redeployed.
C. Wrong: Calling every candidate multiplies model spend and adds a second model call on the critical path, which makes the three-second summary target harder to meet. Choosing among outputs needs its own evaluation as well.
D. Wrong: A captioned photo loses detail that tagging depends on, so the tagging quality target is missed. One shared model also couples all three tasks, so replacing one of them means retraining and redeploying the rest.
Explanation
Specialist models chosen per task reach the quality target for each part of a post, and a router sends each part only to the model that handles it. Because each specialist has its own configuration, one can be replaced without touching the others. A single general model, an ensemble of every candidate and a single fine-tuned text model each fail at least one of the requirements.
Want more on Coordinating several models (specialists, ensembles, routing)? The 5 full-length practice sets have 5 more Coordinating several models (specialists, ensembles, routing) questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →
Study Coordinating several models (specialists, ensembles, routing)