MLA-C02 practice question: Bias and class imbalance (2 of 3)
Data Preparation for ML and AI A free question from our original MLA-C02 practice exam. Try it, then open the answer to see why each option is right or wrong.
A pet insurer is training a claims-fraud model. Only about 0.3 percent of its historical claims are confirmed as fraud, and the team wants a model whose predicted probabilities are still meaningful for underwriting.
The evaluation sets must represent the real claim mix so that precision and recall reflect production. The team also wants no synthetic records in the training data, because the claims team has to audit every training example.
Which approach meets these requirements?
Choose one answer.
AA. Undersample the legitimate claims until the classes are balanced in the full dataset, then split the balanced data into training, validation and test sets.
BB. Generate synthetic fraud claims with SMOTE and add them to the training set, keeping the validation and test sets unchanged.
CC. Split into stratified training, validation and test sets at the real fraud rate, and set class weights only in the training set.
DD. Duplicate the fraud claims in the training set until the classes are equal, then split the data randomly into the three sets.
Show the answer and explanation
Answer: C
A. Wrong: Balancing before the split changes the evaluation sets, so precision and recall no longer reflect the real claim mix. Undersampling also discards many legitimate examples the team could learn from.
B. Wrong: SMOTE creates synthetic examples, which the claims team has ruled out for audit reasons. Synthetic records also distort the probabilities the underwriters rely on.
C. Right: Correct. A stratified split keeps the real 0.3 percent rate in every set, so evaluation matches production, and class weights in training change the loss without adding records. No synthetic examples are created.
D. Wrong: Duplicating before a random split places copies of the same claim in training and evaluation, which inflates the test scores. The option also changes the data the auditors review.
Explanation
Stratified splits keep the real fraud rate in every evaluation set, while class weights in the training loss handle the imbalance without creating records. Balancing or duplicating before the split distorts evaluation, and SMOTE adds synthetic records the audit rules out.
Want more on Bias and class imbalance? The 5 full-length practice sets have 5 more Bias and class imbalance questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →