Data quality: DEA-C01 study notes and practice questions
What to know about Data quality for the DEA-C01 exam, from the Data Operations and Support domain (22% of the exam guide), then 2 free questions to test it. Part of the DEA-C01 study guide.
Data quality questions ask how to stop bad records reaching reports, or how to measure quality with least effort.
Dimensions to check
Completeness: required columns aren’t null.
Uniqueness: keys aren’t duplicated.
Validity: values match types, ranges, formats or allowed lists.
Consistency: values agree across tables (referential integrity).
Timeliness / freshness: data arrived on time.
Accuracy: values reflect reality (often checked against a trusted source).
AWS Glue Data Quality
Rules written in DQDL (Data Quality Definition Language), for example IsComplete "order_id", IsUnique "order_id", ColumnValues "status" in ["NEW","SHIPPED"], RowCount > 1000, Completeness "email" > 0.95.
Recommendations analyze data and suggest a starting ruleset.
Run on Data Catalog tables or inside Glue ETL jobs (the Evaluate Data Quality transform).
In jobs, route failing rows to a quarantine location and keep good rows flowing, or stop the job.
Results publish to CloudWatch and EventBridge, so failures can alert or stop downstream steps.
Anomaly detection learns usual patterns, such as row counts, and flags unusual runs.
Other tools
Glue DataBrew profile jobs and data quality rules.
SageMaker Data Wrangler data quality and insights reports for ML data.
Deequ (open source, which Glue Data Quality builds on) in Spark on EMR.
SQL checks in Athena or Redshift scheduled by Step Functions or MWAA.
Practices
Check quality at each zone boundary (raw to cleaned to curated).
Keep bad records rather than dropping them, so they can be fixed and replayed.
Make the pipeline fail visibly on serious problems.
Common traps
Custom Lambda checks when Glue Data Quality rules cover them with less effort.
Silently dropping bad rows with no record.
Checking only at the end after bad data has reached many tables.
A media company is migrating an on-premises SQL Server database to Amazon Aurora PostgreSQL with an AWS DMS task that runs full load and ongoing change data capture.
Before the cutover, auditors want evidence that every migrated row matches the source, including rows changed during replication. The team wants the check to run automatically and to name any rows that differ, with little custom code.
Which solution meets these requirements?
Choose one answer.
AA. Run a nightly script that counts the rows in each source and target table, and compare the totals in a shared spreadsheet.
BB. Run an AWS Glue crawler against both databases, and compare the table schemas in the two catalogs after each replication run.
CC. Turn on data validation for the DMS task, and review its validation state and the awsdms_validation_failures_v1 table on the target.
DD. Add a Glue Data Quality ruleset with a RowCount rule to the target tables, and evaluate it after each change data capture batch.
Show the answer and explanation
Answer: C
A. Wrong: Matching counts do not prove that the values match, and the script cannot name the rows that differ. It is also custom code that the team would maintain.
B. Wrong: A crawler records table schemas, not row values. Two tables can have the same schema and still hold different data, so this gives no evidence about the rows.
C. Right: Correct. DMS data validation compares each source row with its target row after the full load and as changes replicate. Rows that fail are recorded with their keys and differing values in the validation failures table on the target. Learn more: AWS DMS data validation (opens in a new tab)
D. Wrong: The rule checks only the target and only how many rows it holds. It cannot compare values with the source, so mismatched rows would pass.
Explanation
A migration check must compare values, not just counts or schemas. DMS data validation compares source and target rows during full load and change data capture, and it records each failed row with its key and the values that differ. Count scripts, crawlers and a target-only RowCount rule cannot show that every row matches.
A logistics broker's ETL job writes shipment records to a curated table. The upstream system occasionally sends the same shipment twice, and it sometimes sends a status code that is not one of the three allowed values: PICKED_UP, IN_TRANSIT or DELIVERED.
The data team wants two Glue Data Quality rules that fail the run for either problem, using DQDL.
Which TWO rules meet these requirements? (Choose two.)
Choose two answers.
AA. Add the rule IsUnique "shipment_id" to the ruleset that the job evaluates before it writes to the curated table.
BB. Add the rule ColumnValues "status" in ["PICKED_UP", "IN_TRANSIT", "DELIVERED"] to the ruleset that the job evaluates before it writes to the curated table.
CC. Add the rule RowCount > 0 to the ruleset that the job evaluates before it writes to the curated table.
DD. Add the rule IsComplete "status" to the ruleset that the job evaluates before it writes to the curated table.
EE. Add the rule ColumnLength "status" <= 10 to the ruleset that the job evaluates before it writes to the curated table.
DQDL rules run inside the Glue job and fail the run on a bad batch. IsUnique catches duplicate shipment IDs, and ColumnValues with an allowed list catches status codes outside the three valid values. Row counts, null checks and length checks pass for those bad records, so they do not meet the requirement.
Want more on Data quality? The 5 full-length practice sets have 11 more Data quality questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →