DEA-C01 practice question: Orchestration with Step Functions, MWAA and Glue workflows (1 of 2)

Data Ingestion and Transformation A free question from our original DEA-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.

A logistics broker's nightly pipeline runs a Glue job that takes about two hours, followed by a Glue job that rebuilds the reporting tables. The pipeline must start each step only after the previous one finishes successfully, retry transient failures, and notify the on-call team if a step fails.

Each step writes to a shared table, so a duplicate execution would corrupt the data. The team wants AWS to manage the state of the run.

Which orchestration approach meets these requirements?

Choose one answer.

A. Use an AWS Step Functions Express workflow, which starts quickly and keeps each run's state in the workflow.
B. Use a Step Functions Standard workflow with a Glue task that waits for each job, retry rules, and a catch path that publishes to SNS.
C. Use an Amazon EventBridge rule that starts the second Glue job when the first Glue job's state changes, and an SNS topic for alerts.
D. Use an AWS Lambda function on a schedule that polls the Glue job status every minute and starts the next job when the first one is done.

Show the answer and explanation

Answer: B

A. Wrong: Express workflows suit short, high-volume runs and can run for up to five minutes, which is far shorter than the two-hour job. Their at-least-once behavior for asynchronous runs also risks duplicate writes. Learn more: Choosing between Standard and Express workflows (opens in a new tab)
B. Right: Correct. Standard workflows can run for up to one year with exactly-once execution, which suits long, non-idempotent steps. The Glue task waits for each job to finish, and the catch path sends the SNS alert when a step fails. Learn more: Choosing between Standard and Express workflows (opens in a new tab)
C. Wrong: EventBridge can react to a Glue job state change, but a rule does not track the run as a whole or retry a failed step. The team would have to build the sequencing and retry logic itself, which leaves AWS managing less of the run's state.
D. Wrong: Polling works, but the team would own the state, retries and failure handling in code. Lambda's timeout also limits how long one function can keep polling for a two-hour job. Learn more: Configure Lambda function timeout (opens in a new tab)

Explanation

Step Functions Standard workflows run for up to a year and execute each step exactly once, which suits a long Glue job that writes shared tables. Waiting tasks, retry rules and catch paths give the sequencing, retries and alerts the team needs, and AWS keeps the run's state. Express workflows, EventBridge rules and polling functions each miss one of these.

Study

Choosing between Standard and Express workflows

DEA-C01 practice question: Orchestration with Step Functions, MWAA and Glue workflows (1 of 2)

Explanation

Study

Keep practicing