ExamPark
Course menuAIF-C01

Exams / AIF-C01 / Study guide / Bias, variance and datasets

Bias, variance and datasets: AIF-C01 study notes and practice questions

What to know about Bias, variance and datasets for the AIF-C01 exam, from the Guidelines for Responsible AI domain (14% of the exam guide), then 2 free questions to test it. Part of the AIF-C01 study guide.

Last verified against current AWS documentation ·How we write our questions

What to know for the exam

These questions ask where unfairness comes from, how to detect it, and how to recognize overfitting and underfitting.

Sources of bias

  • Unrepresentative data: training data doesn’t include all the groups the model will be used on.
  • Historical bias: the labels reflect past unfair decisions.
  • Sampling bias: data was collected in a skewed way.
  • Measurement bias: features are recorded differently for different groups.

The effects: some groups get worse results, which can be unfair, illegal and damaging to trust.

Characteristics of good datasets

Inclusive, diverse, balanced across groups, curated from reliable sources, accurately labeled, and large enough.

Bias and variance

Bias (underfitting) Variance (overfitting)
Symptom Poor on training and test data Great on training data, poor on test data
Cause Model too simple, too few features Model too complex, memorizes noise
Fixes More features, a more complex model, longer training More data, regularization, simpler model, early stopping, cross-validation

A big gap between training and test accuracy points to overfitting.

AWS tools

  • Amazon SageMaker Clarify: measures bias in data and models, and explains predictions.
  • Amazon SageMaker Model Monitor: watches models in production for drift, including bias drift.
  • Amazon Augmented AI (A2I): sends low-confidence or sensitive predictions for human review.
  • SageMaker Ground Truth: labels data, with human workers, to improve dataset quality.

Common traps

  • “More data fixes bias.” Only if the new data is representative.
  • Underfitting vs overfitting. If the model is bad on training data too, it’s underfitting.
  • Removing a sensitive feature (such as gender) may not remove bias if other features act as stand-ins.

Try the 2 practice questions

Practice Bias, variance and datasetsTake the free practice exam

Question 1

Harder ·Link to this question (question 1)

A logistics company's model predicts delivery delays. It scores 98 percent on its training data but 71 percent on a held-out test set.

The team cannot collect new data this quarter and must keep the model's size unchanged to meet a latency limit.

Which action is MOST likely to improve performance on new data?

Choose one answer.

  1. A. Apply regularization and early stopping, and tune them on a validation set kept separate from the test set.
  2. B. Train for many more epochs, so that the model fits the training data even more closely.
  3. C. Merge the validation and test sets, then retrain on all of the data so the model sees more examples.
  4. D. Add more hidden layers, so that the network can learn more complex patterns in the training data.
Show the answer and explanation

Answer: A

  • A. Right: Correct. Regularization and early stopping limit how closely the model fits noise in the training data. A separate validation set keeps the tuning honest, so the test set still measures new-data performance. Learn more: What is Overfitting? - AWS (opens in a new tab)
  • B. Wrong: More epochs usually widen the gap between training and new-data accuracy. The model memorizes the training examples even further.
  • C. Wrong: Merging the sets removes the only unbiased check on new-data accuracy. Retraining on the same data does not fix overfitting, which is the real problem.
  • D. Wrong: A larger network increases model size, which breaks the latency limit. More capacity also tends to overfit more, so it works against the goal.

Explanation

A large gap between training and test accuracy is a sign of overfitting, where a model learns the training data too closely to generalize. Regularization and early stopping address this without new data or a larger model, and a clean validation process keeps the evaluation honest. This question tests recognizing overfitting and choosing a fix that respects the constraints.

Study

Question 2

Medium · choose 2 ·Link to this question (question 2)

A lender is collecting data to train a credit risk model that must treat applicants fairly. Past loan decisions were made by staff, and applicants come from many regions.

Which TWO dataset characteristics are MOST important for fair treatment? (Choose two.)

Choose two answers.

  1. A. Enough labeled applications from each applicant group the model will score, so that no group is underrepresented.
  2. B. Outcome labels reviewed for past bias, so that historical unfair refusals are not learned as the correct answer.
  3. C. The largest possible number of records, even if most of them come from a single region.
  4. D. Keeping protected attributes such as gender as predictors, since they show real differences in default risk.
  5. E. Only applications from the last two years, so that the model reflects current lending rules.
Show the answer and explanation

Answers: A, B

  • A. Right: Correct. A model learns little about groups with few examples, so representation across groups must be checked before its scores can be judged fair.
  • B. Right: Correct. If past decisions were unfair, using them as labels teaches the model to repeat that unfairness. The labels need review before training.
  • C. Wrong: Size does not fix bias. A very large dataset that mostly comes from one region still fails applicants elsewhere.
  • D. Wrong: Using protected attributes as predictors can lead to unfair treatment of protected groups, which the lender must avoid. Apparent differences in risk do not justify it.
  • E. Wrong: Recent records help the model stay current, but they do not address fairness across groups. Two years of biased decisions can still carry unfair patterns.

Explanation

Fair treatment depends on whether the training data represents every group the model will score, and on whether its labels reflect fair decisions. Data volume and recency matter less than these two points. This question tests identifying the dataset properties behind bias.

Study

Want more on Bias, variance and datasets? The 5 full-length practice sets have 9 more Bias, variance and datasets questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets

Study Bias, variance and datasets

More Guidelines for Responsible AI topics