AIF-C01 practice question: Training and fine-tuning (2 of 4)
Applications of Foundation Models A free question from our original AIF-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.
A media group is comparing ways to adapt foundation models for five projects.
Match each project's description to the technique it describes. Each option is used at most once, and one option is not needed.
Match each item to one answer. An answer can be used more than once, and some may not be used.
Items
Keep training an existing model on a large, unlabeled archive of the group's film scripts so it absorbs their vocabulary and background
Train an existing model on labeled pairs of creative briefs and approved scripts, so it learns to draft scripts from briefs
Train a smaller, cheaper model on responses produced by a larger model for one use case
Train a reward model from human rankings of responses, then tune the model to score well against it
Put a few worked examples in the prompt, with no change to the model's weights
Answers
AA. Continued pre-training on a large unlabeled corpus
BB. Distillation from a larger teacher model
CC. In-context learning with examples placed in the prompt
DD. Pre-training a brand-new model from scratch
EE. RLHF, using human rankings to train a reward model
FF. Supervised fine-tuning on labeled prompt-response pairs
Show the answer and explanation
Correct matches
Keep training an existing model on a large, unlabeled archive of the group's film scripts so it absorbs their vocabulary and background: A. Continued pre-training on a large unlabeled corpus
Train an existing model on labeled pairs of creative briefs and approved scripts, so it learns to draft scripts from briefs: F. Supervised fine-tuning on labeled prompt-response pairs
Train a smaller, cheaper model on responses produced by a larger model for one use case: B. Distillation from a larger teacher model
Train a reward model from human rankings of responses, then tune the model to score well against it: E. RLHF, using human rankings to train a reward model
Put a few worked examples in the prompt, with no change to the model's weights: C. In-context learning with examples placed in the prompt
About each answer
A. Continued pre-training keeps training an existing model on large amounts of unlabeled domain text. It matches the first description, where the goal is domain knowledge rather than one task.
B. Distillation trains a smaller student model on outputs from a larger teacher model, which lowers cost and latency. It matches the third description.
C. In-context learning gives the model examples inside the prompt, with no training and no change to its weights. It matches the fifth description.
D. Pre-training from scratch builds a new model from a very large dataset, far beyond these projects. No description starts from nothing, so this option is not needed.
E. RLHF trains a reward model from human rankings of responses, then tunes the model to score well with it. It matches the fourth description.
F. Supervised fine-tuning updates a model with labeled input and output pairs for one task. It matches the second description.
Explanation
Techniques differ in the data they need and the cost they carry. In-context learning needs no training, fine-tuning needs labeled examples, continued pre-training needs large unlabeled text, distillation uses a teacher model, and RLHF uses human preference data. Building a model from scratch is not needed to adapt one.
Want more on Training and fine-tuning? The 5 full-length practice sets have 18 more Training and fine-tuning questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →