Exams / MLA-C02 / Study guide / Experiments and reproducibility

Experiments and reproducibility: MLA-C02 study notes and practice questions

What to know about Experiments and reproducibility for the MLA-C02 exam, from the ML Model and Foundation Model (FM) Development domain (24% of the exam guide), then 1 free question to test it. Part of the MLA-C02 study guide.

Last verified October 2026 against current AWS documentation ·How we write our questions

What to know for the exam

These questions ask how a team compares runs, reproduces a past model, or proves how a model was built.

Managed MLflow on SageMaker AI

Tracking servers are managed by AWS; you don’t run your own server.
Log parameters, metrics, artifacts and models from notebooks, training jobs and pipelines with the MLflow SDK.
Compare runs side by side in the MLflow UI.
The MLflow model registry can sync with the SageMaker Model Registry.
Access is controlled with IAM.

SageMaker Experiments (the older SDK) has largely been superseded by MLflow.

What to record for reproducibility

Item	How
Code	Git commit; container image digest
Data	S3 version IDs, dataset snapshots or Feature Store point-in-time queries
Parameters	Hyperparameters and preprocessing settings
Environment	Framework and package versions (pinned)
Randomness	Fixed random seeds
Results	Metrics, plots, model artifacts

Lineage and audit

SageMaker ML Lineage Tracking links datasets, jobs, models and endpoints automatically.
SageMaker Pipelines record every step’s inputs and outputs.
Model Registry versions carry metrics and approval status.
Model cards document intended use, risks and evaluation results.

Working as a team

SageMaker Unified Studio and SageMaker Studio give shared projects and notebooks.
Store notebooks and code in Git; promote models through accounts with pipelines.

Common traps

Spreadsheets or shared folders of results with no comparison view.
Self-hosting MLflow on EC2 when a managed tracking server exists.
Unpinned package versions, so last month’s model can’t be rebuilt.
Logging metrics only at the end of a run, so a failed run leaves no record of how far it got or which settings it used.

Try the 1 practice question

Practice Experiments and reproducibility Take the free practice exam

Question 1

Medium ·Link to this question

A regional bank has three data science teams training credit models in SageMaker AI notebooks. Each team tracks its runs in spreadsheets, so no one can compare parameters, metrics and artifacts across teams or reproduce last quarter's best run.

The bank wants one shared place where every team logs runs and metrics, with a UI to compare them, and it wants the tracking server to be managed by AWS rather than run by the bank.

Which solution meets these requirements?

Choose one answer.

A. Ask each team to save its metrics in a shared Amazon S3 folder as CSV files, named by date and team.
B. Give each team its own Amazon EC2 instance running a self-hosted MLflow server, and have the teams share their URLs by email.
C. Create a managed MLflow tracking server in SageMaker AI, and have each team log runs to it with the MLflow client.
D. Store every trained model in the SageMaker Model Registry only, since the registry tracks metrics for each run.

Show the answer and explanation

Answer: C

A. Wrong: Shared files provide storage but no run comparison UI, and they rely on naming discipline for reproducibility. The bank also needs parameters and artifacts linked to each run.
B. Wrong: A self-hosted server puts the operation of the tracking server on the bank, which the stem rules out. Sharing by email does not give one place for all runs.
C. Right: Correct. SageMaker AI provides managed MLflow tracking servers, which give a shared place to log parameters, metrics and artifacts and to compare runs in a UI. AWS operates the server for the bank. Learn more: Managed MLflow on Amazon SageMaker AI (opens in a new tab)
D. Wrong: The Model Registry tracks model versions and approval status, not the parameters and metrics of every experiment run. It does not replace experiment tracking.

Explanation

SageMaker AI managed MLflow tracking servers let teams log runs and compare parameters, metrics and artifacts in one place, with AWS managing the server. Shared files lack a comparison interface, self-hosting adds operations, and the Model Registry tracks versions rather than every experiment.

Study

Want more on Experiments and reproducibility? The 5 full-length practice sets have 4 more Experiments and reproducibility questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets

Study Experiments and reproducibility

Managed MLflow on Amazon SageMaker AI