Transformation with AWS Glue: DEA-C01 study notes and practice questions
What to know about Transformation with AWS Glue for the DEA-C01 exam, from the Data Ingestion and Transformation domain (34% of the exam guide), then 3 free questions to test it. Part of the DEA-C01 study guide.
Glue questions ask how to transform data without managing clusters, how to process only new data, and how to make jobs cheaper or faster.
Key features
Feature
What it does
Job types
Spark (batch), Spark Streaming, Python shell (small jobs), Ray
DynamicFrame
A Spark DataFrame extension that handles messy schemas; a column with mixed types becomes a choice type; resolveChoice casts or splits it before writing to a typed format such as Parquet
Job bookmarks
Remember what was processed, so each run handles only new files or rows (incremental loads)
Worker types
G.1X, G.2X and larger for memory-heavy jobs; G.025X for small streaming jobs
Auto scaling
Adds and removes workers during a run
Flex execution
Runs non-urgent jobs on spare capacity at a lower price; may start later
Glue Studio
Visual job authoring; generates code
Glue Data Quality
Rules in DQDL to check data inside jobs
Connections
Reach JDBC databases in a VPC (subnet, security group with a self-referencing rule)
Pushdown predicates
Read only the partitions you need from the catalog
Cost and speed
Pay per DPU-hour; right-size workers, use auto scaling, and Flex for jobs without deadlines.
Write Parquet with sensible file sizes; avoid thousands of tiny files (use groupFiles when reading many small files).
Use partition pruning and pushdown predicates.
When not Glue
Very large, long-running or custom-framework jobs may suit EMR.
Tiny transformations on each event suit Lambda.
Analysts without code can use DataBrew.
Common traps
Reprocessing everything every night when bookmarks would process only new data.
Writing a choice-typed column to Parquet without resolving it.
A Glue connection without the self-referencing security group rule, so workers can’t talk to each other.
A logistics broker runs a nightly AWS Glue job that reads new carrier-invoice files from an S3 prefix and writes them to a curated table. Each night the job reprocesses every invoice file since launch, so the runtime keeps growing and some invoices are loaded twice.
The team wants each run to process only files that arrived since the previous successful run, and it wants to keep the same job script with minimal changes.
Which change meets these requirements?
Choose one answer.
AA. Add a step that deletes the curated table at the start of each run, and then reprocess every invoice file from the beginning.
BB. Move the job to an AWS Lambda function triggered once a day, and have the function read the whole prefix into memory before writing.
CC. Run an AWS Glue crawler before each job so that the job can see new partitions, and keep the job's source reads unchanged.
DD. Turn on job bookmarks, and give each source read a transformation_ctx name so that Glue tracks which files were processed.
Show the answer and explanation
Answer: D
A. Wrong: Rebuilding everything each night still reprocesses the full history, which is the problem the team wants to remove. The table would also be empty during every run.
B. Wrong: A daily function that reads the full prefix repeats the same work and keeps no record of what it has processed. Lambda's memory and runtime limits also make it a poor fit for a growing invoice history.
C. Wrong: A crawler updates catalog metadata, but it does not track which files the job has already read. The job would still process the whole prefix every night.
Job bookmarks let a Glue job remember what it has already processed, so each run handles only new data. Enabling them and naming each source with transformation_ctx keeps the existing script almost unchanged. Deleting and rebuilding, moving to a timed function, or running a crawler do not track processed files.
A public library system runs a Spark job in AWS Glue each night at 1:00 AM. The job joins circulation records with patron data and writes a summary table. It takes about 90 minutes on G.1X workers with the standard execution class, and nobody needs the output before the library's reporting starts at 7:00 AM.
The team wants to cut the nightly Glue cost without changing the summary logic. A run that starts a few minutes late is acceptable, but the output must still be ready before 7:00 AM.
Which TWO changes meet these requirements? (Choose two.)
Choose two answers.
AA. Change the job's execution class to Flex, which uses spare capacity at a lower price for non-urgent jobs that can wait.
BB. Move the job to G.8X workers and halve the worker count, so that each worker handles more data and the job finishes faster at about the same cost.
CC. Replace the Spark job with a Glue Python shell job so that it runs on a single small instance.
DD. Keep the standard workers and add a second identical job that runs at 3:00 AM, so that if the first run fails, the second run finishes the work.
EE. Turn on Glue auto scaling with a maximum worker count, so the job adds workers only while it has work to do.
Show the answer and explanation
Answers: A, E
A. Right: Correct. The Flex execution class runs on spare capacity at a lower price and is meant for non-urgent jobs that can tolerate a later start. It supports AWS Glue 3.0 and later jobs on G.1X or G.2X workers, and the 7:00 AM deadline leaves plenty of slack. Learn more: Configuring job properties for Spark jobs in AWS Glue (opens in a new tab)
B. Wrong: Larger workers cost more per hour, and halving the count does not keep the runtime or the cost the same. The option does not reduce cost and may slow the job down.
C. Wrong: A Python shell job runs single-process Python and does not provide the Spark engine that the join depends on. The job might not finish before 7:00 AM, and the summary logic would need a rewrite.
D. Wrong: A second run repeats the same compute when the first run succeeds, which raises the cost. The option adds spend instead of reducing it.
E. Right: Correct. With Auto Scaling, Glue removes idle executors and their workers and adds workers only when Spark requests them, up to the maximum you set. The job stops paying for workers that sit idle, such as during file listing or skewed stages. Learn more: Using Auto Scaling for AWS Glue (opens in a new tab)
Explanation
Non-urgent batch jobs can cut Glue cost in two ways without changing their logic. Flex execution uses spare capacity at a lower price for jobs that can wait for it, and auto scaling keeps only the workers that have work to do. Bigger workers, a Python shell rewrite and a duplicate run all increase cost or risk the deadline.
A healthcare-scheduling vendor's AWS Glue job reads CSV exports from its customers. In some files, the phone_extension column holds numbers, and in others it holds text such as "ext. 12". The DynamicFrame shows the column as a choice type, and the write to the Parquet target fails.
The data team must keep every value, the job must finish without manual steps, and the downstream tables should use one type for the column.
Which change meets these requirements?
Choose one answer.
AA. Call resolveChoice on the DynamicFrame to cast phone_extension to string before the write, so that every value is kept.
BB. Set the job to retry up to ten times, so that the mismatch clears itself on a later attempt when the file is read again.
CC. Delete the mixed files before the job runs, and add a note to the customers to send only numeric phone extensions.
DD. Run a Glue crawler before each job so that the column's type is set to integer, and let the job discard any value that is not a number.
Show the answer and explanation
Answer: A
A. Right: Correct. A DynamicFrame records a column with more than one type as a choice type, and resolveChoice with a cast action turns it into a single type. Casting to string keeps both the numbers and the text, so no value is lost. Learn more: ResolveChoice class (opens in a new tab)
B. Wrong: Retries repeat the same read of the same file, so the type mismatch comes back each time. The option does not change how the job handles the column.
C. Wrong: Deleting files drops real data, and it relies on customers changing their exports. The team must keep every value, so this option breaks a stated requirement.
D. Wrong: A crawler sets the catalog type from the data it sees, and the job would then discard the text values. The option loses the data the team needs to keep.
Explanation
A DynamicFrame can carry mixed types in one column as a choice type, but a typed target such as Parquet needs one type. Resolving the choice with a cast to string before the write keeps every value and needs no manual step. Retries, deleting data and forcing an integer type each fail one of the requirements.
Want more on Transformation with AWS Glue? The 5 full-length practice sets have 10 more Transformation with AWS Glue questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →