MLA-C02 practice question: Distributed training and training time

ML Model and Foundation Model (FM) Development A free question from our original MLA-C02 practice exam. Try it, then open the answer to see why each option is right or wrong.

A ski resort group is training a 13-billion-parameter language model on SageMaker AI for resort-guest support. The model does not fit in memory on the GPU instances the team has budgeted for, and the training job fails with out-of-memory errors at the first batch.

The team wants to keep the same model architecture and the same training data, and to use the budgeted instance type without making the model smaller.

Which TWO changes meet these requirements? (Choose two.)

Choose two answers.

A. Turn on data parallelism, so that each GPU holds a full copy of the model and processes a different slice of the data.
B. Turn on model parallelism, so that the layers or parameters of the model are split across GPUs.
C. Increase the batch size so that more examples fit in each training step.
D. Turn on activation checkpointing, so that intermediate activations are recomputed during the backward pass instead of being kept in memory.
E. Switch the training data from text to compressed archives, which reduces the GPU memory the model needs.

Show the answer and explanation

Answers: B, D

A. Wrong: Data parallelism still gives each GPU a full copy of the model, so the out-of-memory error remains. It speeds up training when the model fits but does not solve this problem.
B. Right: Correct. Model parallelism splits the model across devices so that no single GPU must hold all the parameters. This lets the same architecture train on the budgeted instances. Learn more: SageMaker model parallelism library v2 (opens in a new tab)
C. Wrong: A larger batch needs more memory for activations, so it makes the out-of-memory error worse. The fix needs to reduce the memory the model uses.
D. Right: Correct. Activation checkpointing trades extra computation for lower memory use, because it stores fewer intermediate activations. Combined with model parallelism, it helps the large model fit without changing its architecture. Learn more: Activation checkpointing (opens in a new tab)
E. Wrong: Compressing the input files does not change the memory the model parameters and activations need on the GPU. The problem is the model, not the storage format.

Explanation

Model parallelism splits the parameters across GPUs, and activation checkpointing reduces the memory used for intermediate results, so the same model can train on the budgeted instances. Data parallelism and larger batches keep a full model copy or add memory, and the input format does not change GPU memory.

MLA-C02 practice question: Distributed training and training time

Explanation

Study

Keep practicing