Open table formats and Apache Iceberg: DEA-C01 study notes and practice questions
What to know about Open table formats and Apache Iceberg for the DEA-C01 exam, from the Data Store Management domain (26% of the exam guide), then 2 free questions to test it. Part of the DEA-C01 study guide.
A regional bank stores daily account-balance snapshots in an Apache Iceberg table on Amazon S3, queried with Amazon Athena. Auditors need to see the table exactly as it was at each month-end close, even after later corrections. Engineers also need to rename a column for a downstream system without rewriting the full history.
The bank wants to keep a single copy of the data and avoid building a separate monthly archive job.
Which approach meets these requirements?
Choose one answer.
AA. Keep a Hive table in Parquet, copy it to a new S3 prefix at each month-end with a Glue job, and rename the column in every copy.
BB. Use an Iceberg table with Athena time travel to query each month-end version, and rename the column without rewriting files.
CC. Enable S3 Versioning on the bucket, and restore each object version by hand when an auditor asks for a month-end view.
DD. Use a DynamoDB table for the balances, and rely on its TTL to keep the month-end values for the auditors.
Show the answer and explanation
Answer: B
A. Wrong: Monthly copies store the data several times and need a job to create and maintain them. Renaming the column in every copy also rewrites a lot of data, which the bank wants to avoid.
B. Right: Correct. Iceberg keeps table snapshots, so Athena can query the table as of a past time. Iceberg tracks columns by ID, so a rename changes metadata and does not rewrite the existing data files. Learn more: Query Apache Iceberg tables (opens in a new tab)
C. Wrong: S3 versions the objects, not the table's rows, so restoring a month-end view means rebuilding the table manually from object versions. The option does not give table-level time travel or metadata-only schema changes.
D. Wrong: TTL deletes expired items, which is the opposite of keeping historical values for audit. The option does not preserve table state and is not suited to analytic queries.
Explanation
Apache Iceberg keeps snapshots of a table, which lets Athena query a month-end state without separate copies. Schema changes such as renaming a column are metadata operations that leave the data files alone. Monthly copies, S3 object versions and DynamoDB TTL do not give table-level history or metadata-only changes.
A bank's Apache Iceberg table receives small batches of transaction files every five minutes, and each commit adds more small data files and more snapshots. Athena queries have slowed, and storage costs keep growing because old files are never removed.
The bank must keep the time-travel history it needs for audits, which covers the last 90 days, and it wants the table maintained with Athena rather than by hand.
Which TWO actions meet these requirements? (Choose two.)
Choose two answers.
AA. Run OPTIMIZE on the table on a schedule, which rewrites many small data files into fewer, larger files.
BB. Delete the old data files in the table's S3 prefix with a lifecycle rule that expires objects after 30 days, which removes the files without any Iceberg command.
CC. Turn off snapshots for the table, so that each commit replaces the previous state and the old files are removed immediately.
DD. Run VACUUM on the table on a schedule, which expires old snapshots past the retention window and removes files that no snapshot references.
EE. Convert the table to CSV files every week so that the snapshots are not needed, and drop the Iceberg metadata.
Show the answer and explanation
Answers: A, D
A. Right: Correct. OPTIMIZE rewrites small data files into larger ones, which reduces the file count that queries must open. It keeps the table's data and history intact while improving query speed. Learn more: Optimizing Iceberg tables (opens in a new tab)
B. Wrong: Deleting data files outside Iceberg removes files the table's snapshots still reference, which corrupts the table and breaks time travel. Lifecycle rules work on objects, not on Iceberg snapshots.
C. Wrong: Iceberg snapshots are the mechanism for time travel, and the audit requirement needs them. Removing them would break the 90-day history the bank must keep.
D. Right: Correct. VACUUM expires snapshots older than the table's vacuum_max_snapshot_age_seconds setting and removes files that are no longer part of the table state. Setting that age to 90 days keeps the audit history while storage stops growing. Learn more: Optimizing Iceberg tables (opens in a new tab)
E. Wrong: CSV drops the table format's time travel and schema history, which the audit requirement needs. The option trades the audit history for simpler files, so it fails a stated requirement.
Explanation
Iceberg tables need periodic maintenance. OPTIMIZE compacts small files so queries open fewer objects, and VACUUM expires old snapshots outside the retention window and deletes files that nothing references. Deleting files outside Iceberg, disabling snapshots and converting to CSV each break the audit history or the table itself.
Want more on Open table formats and Apache Iceberg? The 5 full-length practice sets have 10 more Open table formats and Apache Iceberg questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →