DEA-C01 practice question: Orchestration with Step Functions, MWAA and Glue workflows (2 of 2)
Data Ingestion and Transformation A free question from our original DEA-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.
A data team runs its pipelines in an Amazon Managed Workflows for Apache Airflow environment. After the team uploads a new DAG file to the environment's dags folder in S3, the DAG never appears in the Airflow UI, and no DAG run is scheduled. The DAG imports Glue and Redshift operators from provider packages.
The team needs to find the cause without deleting or recreating the environment.
Which TWO steps help identify the cause? (Choose two.)
Choose two answers.
AA. Delete the environment and create a new one with the same DAG folder, so that Airflow starts with a clean state.
BB. Move the DAG's logic into an AWS Lambda function and trigger that function from an EventBridge rule, so that Airflow no longer needs to load the file.
CC. Check the Import Errors list in the Airflow UI and the scheduler logs in CloudWatch for parse errors raised when the DAG file is loaded.
DD. Update the environment to restart the Airflow components, and wait for the scheduler to rescan the DAG folder after the update.
EE. Confirm that every provider package the DAG imports is listed in requirements.txt, since a missing package stops parsing.
Show the answer and explanation
Answers: C, E
A. Wrong: Recreating the environment is slow and disruptive, and the new environment would have the same DAG file and the same problem. The team asked to avoid this step.
B. Wrong: Moving the logic out of Airflow replaces the pipeline design rather than diagnosing it. The team wants the existing DAG to run in MWAA, so this option does not answer the question.
C. Right: Correct. Airflow lists DAG files that fail to parse as import errors in its UI, and MWAA can publish DAG processing and scheduler logs to CloudWatch Logs. A syntax error or a failed import shows up there and points to the cause directly. Learn more: Accessing Airflow logs in Amazon CloudWatch (opens in a new tab)
D. Wrong: Restarting the components does not fix a DAG file that fails to parse, so the scheduler would fail the same way again. The option also takes the environment through an update without telling the team what is wrong.
E. Right: Correct. Packages that the DAG imports must be installed in the environment, and they are declared in requirements.txt. A missing or incompatible package causes an import error, which keeps the DAG from loading. Learn more: Installing Python dependencies (opens in a new tab)
Explanation
A DAG that does not appear usually failed to parse. The Airflow Import Errors view and the scheduler logs in CloudWatch show parse errors, and missing provider packages in requirements.txt are a common cause. Recreating the environment, moving the logic out of Airflow or restarting the web server does not diagnose the problem.
Want more on Orchestration with Step Functions, MWAA and Glue workflows? The 5 full-length practice sets have 11 more Orchestration with Step Functions, MWAA and Glue workflows questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →