Training and fine-tuning: AIF-C01 study notes and practice questions

Correct order: C, A, E, D, B

1. C. Pre-train the model on a very large, broad dataset. First: the lifecycle starts with a broad pre-training run on a very large dataset, which gives the model general language or visual knowledge.
2. A. Fine-tune the model on smaller, task-specific datasets. Second: fine-tuning adapts the broadly trained model to specific tasks, which happens after the large pre-training run.
3. E. Evaluate the model against benchmarks and the lab's requirements. Third: before release, the model is measured against benchmarks and the lab's own requirements to confirm it is fit for use.
4. D. Deploy the model so that applications can call it. Fourth: once the model has been tuned and checked, it is deployed so applications can send it requests.
5. B. Collect feedback from real use to guide the next improvements. Last: feedback from users shows where the deployed model falls short, and it guides the next round of work.

Explanation

A foundation model begins with broad pre-training, is then adapted to tasks through fine-tuning, is evaluated against requirements, and is deployed. Feedback from real use closes the loop and feeds the next cycle. This question tests the order of the foundation model lifecycle.

Study

• Customize your model in Amazon Bedrock

Correct matches

• Keep training an existing model on a large, unlabeled archive of the group's film scripts so it absorbs their vocabulary and background: A. Continued pre-training on a large unlabeled corpus
• Train an existing model on labeled pairs of creative briefs and approved scripts, so it learns to draft scripts from briefs: F. Supervised fine-tuning on labeled prompt-response pairs
• Train a smaller, cheaper model on responses produced by a larger model for one use case: B. Distillation from a larger teacher model
• Train a reward model from human rankings of responses, then tune the model to score well against it: E. RLHF, using human rankings to train a reward model
• Put a few worked examples in the prompt, with no change to the model's weights: C. In-context learning with examples placed in the prompt

About each answer

• A. Continued pre-training keeps training an existing model on large amounts of unlabeled domain text. It matches the first description, where the goal is domain knowledge rather than one task.
• B. Distillation trains a smaller student model on outputs from a larger teacher model, which lowers cost and latency. It matches the third description.
• C. In-context learning gives the model examples inside the prompt, with no training and no change to its weights. It matches the fifth description.
• D. Pre-training from scratch builds a new model from a very large dataset, far beyond these projects. No description starts from nothing, so this option is not needed.
• E. RLHF trains a reward model from human rankings of responses, then tunes the model to score well with it. It matches the fourth description.
• F. Supervised fine-tuning updates a model with labeled input and output pairs for one task. It matches the second description.

Explanation

Techniques differ in the data they need and the cost they carry. In-context learning needs no training, fine-tuning needs labeled examples, continued pre-training needs large unlabeled text, distillation uses a teacher model, and RLHF uses human preference data. Building a model from scratch is not needed to adapt one.

Study

• Customize your model
• Amazon Bedrock Model Distillation
• Prompt engineering concepts

Answer: D

• A. Wrong: Continued pre-training adds knowledge to the same large model, which stays just as slow and costly per request. It also needs large volumes of unlabeled text, not a task-specific target.
• B. Wrong: Temperature changes randomness in the output, not the speed or cost of running the model. Shorter answers would not make a classification task cheaper.
• C. Wrong: Training from scratch is the most expensive route and needs a very large dataset. A model trained on public email data would not match the retailer's accuracy on its own emails.
• D. Right: Correct. Amazon Bedrock Model Distillation takes prompts, lets the teacher model generate responses, and fine-tunes a smaller, cheaper student on them. The retailer's emails can serve as prompts, so no labeled answers are needed. Learn more: Amazon Bedrock Model Distillation (opens in a new tab)

Explanation

Amazon Bedrock Model Distillation transfers knowledge from a large teacher model to a smaller student model that is faster and cheaper to run. It can start from prompts alone, because the teacher generates the responses. The resulting distilled model is available only to the customer.

Study

• Amazon Bedrock Model Distillation

Answers: A, C

• A. Right: Correct. Fine-tuning learns from the examples it receives, so accurate, reviewed replies teach the house style and the right answers. The examples should reflect the complaints the bank actually handles. Learn more: Customize your model (opens in a new tab)
• B. Wrong: Unrelated text dilutes the signal the model needs. Relevance and quality matter more than raw volume, so a bigger dataset is not better here.
• C. Right: Correct. Personal data in training examples can leak into generated replies and creates privacy and compliance risk. Removing or masking it before training keeps customers' data out of the model.
• D. Wrong: Unchecked drafts may contain errors, and labeling them as ideal teaches the model to repeat those errors. Only reviewed examples should be used as targets.
• E. Wrong: A narrow sample is not representative of the complaints the bank receives. The model would handle other products poorly.

Explanation

Good fine-tuning data is accurate, relevant to the task, representative of real use, and free of sensitive personal data. Reviewed replies teach the house style, and masking personal data before training reduces privacy and leakage risk.

Study

• Customize your model