Preparing data for fine-tuning and distillation: MLA-C02 study notes and practice questions
What to know about Preparing data for fine-tuning and distillation for the MLA-C02 exam, from the Data Preparation for ML and AI domain (28% of the exam guide), then 1 free question to test it. Part of the MLA-C02 study guide.
These questions ask what data a customization method needs and how to prepare it.
Methods and their data
Method
Data
Use when
Supervised fine-tuning
Labeled examples, usually JSON Lines with prompt and completion pairs, or multi-turn conversation records with system, user and assistant messages
You want a consistent format, tone or task behavior
Continued pre-training
Large amounts of unlabeled domain text
The model must learn a field’s vocabulary and knowledge
Distillation
Prompts only; a large teacher model generates responses that train a smaller student
You want a cheaper, faster model with near-teacher quality and have no human labels
Reinforcement fine-tuning (some models)
Prompts plus a reward function or grader
You can score answers but don’t have ideal answers to copy
Training and validation files go in S3; Bedrock reads them with a service role and writes metrics and the custom model output.
Preparing the data
Follow the exact format the model’s documentation describes; a wrong field name fails the job.
Keep examples consistent with how the model will be prompted in production.
Remove duplicates, errors and PII.
Include enough diverse examples; quality beats quantity.
Hold out a validation set to watch for overfitting.
For distillation, use prompts that reflect real traffic; Bedrock can also use invocation logs of real prompts.
Mix some general examples into fine-tuning data to limit catastrophic forgetting of broad skills.
SageMaker AI options
SageMaker JumpStart and training jobs fine-tune open models, including parameter-efficient methods such as LoRA, when you need more control than Bedrock offers.
Common traps
Fine-tuning to add frequently changing facts. Use RAG.
Labeling thousands of answers by hand when distillation generates them from a teacher.
Unlabeled text for supervised fine-tuning. That’s continued pre-training data.
A library system wants a small, inexpensive model to answer patron questions about borrowing rules, in the same style as a large foundation model the team already uses for these questions. The team has about 40,000 past questions but no human-written answers.
The new model must run at low cost in production, and the data must stay in the team's own Amazon S3 bucket.
Which approach meets these requirements?
Choose one answer.
AA. Hire annotators to write answers for all 40,000 questions, then fine-tune the small model on those human answers.
BB. Run the large model on every patron question in production and cache its answers, so that a smaller model is never needed.
CC. Train a small model from scratch on the 40,000 questions alone, with no answers, using an unsupervised clustering algorithm.
DD. Use Bedrock model distillation with the large model as teacher and prompts stored in S3, so the teacher generates training responses for the small model.
Show the answer and explanation
Answer: D
A. Wrong: Writing 40,000 answers by hand is slow and costly, and the stem gives the team a large model to learn from. The team does not need human answers to get useful training targets.
B. Wrong: Caching large-model answers does not produce a smaller model, and the large model still runs at its higher cost. The option does not meet the cost goal.
C. Wrong: Clustering groups questions but does not produce answers in the large model's style. The stem asks for the behavior of the large model to be transferred.
Bedrock model distillation takes prompts, has a large teacher model generate responses, and fine-tunes a smaller student model on them, so no human labels are needed. Caching answers or clustering questions does not produce a cheaper model that answers in the teacher's style.
Want more on Preparing data for fine-tuning and distillation? The 5 full-length practice sets have 7 more Preparing data for fine-tuning and distillation questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →
Study Preparing data for fine-tuning and distillation