Processing with Amazon EMR: DEA-C01 study notes and practice questions
What to know about Processing with Amazon EMR for the DEA-C01 exam, from the Data Ingestion and Transformation domain (34% of the exam guide), then 1 free question to test it. Part of the DEA-C01 study guide.
EMR questions ask when to use it instead of Glue, and how to run it cheaply and reliably.
Deployment options
Option
Use
EMR on EC2
Full control of instance types, configuration and frameworks
EMR Serverless
Run Spark or Hive jobs without sizing or managing clusters
EMR on EKS
Run Spark on an existing Kubernetes cluster
EMR on EC2 nodes
Node
Role
Spot?
Primary
Manages the cluster
Avoid; losing it ends the cluster
Core
Runs tasks and stores HDFS data
Risky; losing core nodes can lose HDFS data
Task
Runs tasks only, no HDFS
Good fit for Spot
Instance fleets mix instance types and purchase options; managed scaling adjusts capacity with load.
EMRFS reads and writes S3 directly, so data survives the cluster. Prefer S3 over HDFS for durable data.
Transient vs long-running
Transient clusters start, run a job (steps) and terminate: no idle cost. Data lives in S3.
Long-running clusters serve interactive users; use auto-termination for idle periods.
When EMR over Glue
Frameworks Glue doesn’t offer (Presto/Trino, HBase, Flink on EMR), custom libraries or tuning, very large steady workloads where EC2 pricing and Spot save money, or existing Hadoop jobs.
Tuning
Partitioning and avoiding data skew in joins.
Columnar formats and compression.
Right-size executors; watch the Spark UI and YARN metrics.
Common traps
Spot on core nodes for data that lives in HDFS.
Keeping data in HDFS on a transient cluster.
A cluster running all day for a one-hour nightly job.
A long-running cluster for occasional ad hoc jobs. EMR Serverless runs Spark without managing nodes.
A media analytics company runs a nightly Apache Spark job on an Amazon EMR cluster. The job takes about six hours on the current on-demand instances, and the company wants to cut the cost of the cluster.
The cluster stores intermediate HDFS data on its core nodes during the run. Losing a core node would lose that data and force the job to restart. The company can tolerate an interrupted task, but not a lost core node.
Which configuration meets these requirements?
Choose one answer.
AA. Run the primary, core and task nodes all on Spot Instances, so that the whole cluster costs the least while the job runs.
BB. Keep the primary and core nodes On-Demand and run the task nodes on Spot Instances, so interrupted tasks rerun without losing HDFS data.
CC. Run all nodes On-Demand, and buy Reserved Instances for the cluster's instance types, so that each hour of the nightly run costs less.
DD. Run the primary node on Spot Instances and the core and task nodes On-Demand, so that the cluster's coordinator is the cheapest node to replace.
Show the answer and explanation
Answer: B
A. Wrong: Spot Instances can be reclaimed at any time, and that includes core nodes that hold HDFS data. The company cannot accept losing core nodes, so this option fails the stated constraint.
B. Right: Correct. Task nodes hold no HDFS data, so Spot interruptions only rerun tasks. Keeping the primary and core nodes On-Demand protects HDFS, and the Spot task capacity lowers the overall cost. Learn more: What is Amazon EMR? (opens in a new tab)
C. Wrong: Reserved Instances bill for every hour of their term, but this cluster runs for about six hours a night. The company would pay for reserved capacity that sits unused most of the day, and the option ignores the cheaper Spot capacity that interruptible task nodes can use.
D. Wrong: The primary node coordinates the cluster, and losing it ends the job, so it is a poor candidate for Spot. The option gets the Spot placement backward and does not meet the requirement.
Explanation
EMR cost savings come from Spot Instances on nodes that can be lost safely. Task nodes do not store HDFS data, so Spot reclamation only reruns their work. The primary and core nodes stay On-Demand to protect the cluster's coordination and HDFS data.
Want more on Processing with Amazon EMR? The 5 full-length practice sets have 7 more Processing with Amazon EMR questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →