Exams / MLA-C02 / Study guide / Overfitting, underfitting and convergence

Overfitting, underfitting and convergence: MLA-C02 study notes and practice questions

What to know about Overfitting, underfitting and convergence for the MLA-C02 exam, from the ML Model and Foundation Model (FM) Development domain (24% of the exam guide), then 1 free question to test it. Part of the MLA-C02 study guide.

Last verified October 2026 against current AWS documentation ·How we write our questions

What to know for the exam

These questions show training and validation metrics, or a loss curve, and ask what’s wrong and how to fix it.

Read the symptoms

Training score	Validation score	Diagnosis
High	Much lower	Overfitting (high variance)
Low	Low	Underfitting (high bias)
Loss jumps around or grows		Learning rate too high, bad data or exploding gradients
Loss falls very slowly		Learning rate too low or the model is too simple

Fixing overfitting

Regularization: L1 (can zero out features), L2 / weight decay (shrinks weights).
Dropout in neural networks.
Early stopping at the best validation checkpoint.
More data or data augmentation (flips, crops, noise).
Simpler model: fewer layers, smaller trees (max_depth), fewer features.
Tree models: raise min_child_weight, use subsample and colsample_bytree in XGBoost.
Cross-validation to get a reliable estimate on small datasets.

Fixing underfitting

A more complex model or more features.
Less regularization.
Train longer or with a better learning rate.
Better feature engineering.

Convergence

Learning rate is usually the first lever; schedules and warm-up help large models.
Batch normalization and gradient clipping stabilize deep networks.
SageMaker Debugger rules detect vanishing or exploding gradients, overfitting and loss that isn’t decreasing.

Data problems that look like model problems

Leakage makes validation scores suspiciously high.
A validation set from a different distribution than production misleads.
Duplicates across splits inflate scores.

Common traps

Training longer to fix overfitting. It makes it worse.
Raising the learning rate when loss is already unstable.
Dropping the validation set to have more training data.

Try the 1 practice question

Practice Overfitting, underfitting and convergence Take the free practice exam

Question 1

Medium · choose 2 ·Link to this question

A fintech analytics team trains a neural network to predict late invoice payments. Training loss keeps falling, but validation loss starts rising after about 15 epochs, and the SageMaker Debugger overfit rule that the team already runs on its training jobs has fired.

The team wants to reduce overfitting without shrinking the training data or cutting the model's capacity. The final model must come from the epoch with the best validation performance.

Which TWO changes meet these requirements? (Choose two.)

Choose two answers.

A. Increase the number of epochs from 20 to 100, so the model has more time to fit the training data.
B. Add dropout or weight decay regularization to the network.
C. Raise the learning rate so that the model converges faster and reaches a lower training loss.
D. Use early stopping that ends training when validation loss stops improving, and keep the weights from the best validation epoch.
E. Remove the validation set so that all the data is used for training and the overfit rule stops firing.

Show the answer and explanation

Answers: B, D

A. Wrong: More epochs fit the training data further while validation loss keeps rising, which increases overfitting. The stem asks for less overfitting.
B. Right: Correct. Regularization such as dropout or weight decay discourages the network from fitting noise in the training data, which reduces the gap between training and validation loss. It keeps the same data and capacity. Learn more: List of Debugger built-in rules (opens in a new tab)
C. Wrong: A higher learning rate speeds up fitting but does not reduce the gap between training and validation loss. It can also make training unstable.
D. Right: Correct. Early stopping halts training before validation performance degrades, and keeping the best checkpoint returns the model from the epoch the stem asks for. It needs no change to the data or model size.
E. Wrong: Without a validation set, the team can no longer see overfitting, so the rule firing would be hidden rather than fixed. The model would also be chosen without validation evidence.

Explanation

Regularization like dropout or weight decay and early stopping with the best validation checkpoint both reduce overfitting while keeping the data and model capacity. Training longer, raising the learning rate or dropping validation data all make the gap between training and validation performance worse or hide it.

Study

List of Debugger built-in rules

Want more on Overfitting, underfitting and convergence? The 5 full-length practice sets have 7 more Overfitting, underfitting and convergence questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets

Study Overfitting, underfitting and convergence

What is overfitting?