Batch ingestion and migration with DMS and AppFlow: DEA-C01 study notes and practice questions
What to know about Batch ingestion and migration with DMS and AppFlow for the DEA-C01 exam, from the Data Ingestion and Transformation domain (34% of the exam guide), then 2 free questions to test it. Part of the DEA-C01 study guide.
Batch ingestion questions name a source (a database, a SaaS application, a file share, an SFTP partner) and ask for the managed way to bring its data in.
The services
Source
Service
Notes
Relational and NoSQL databases
AWS Database Migration Service (DMS)
Full load, CDC (ongoing changes from the database log), or full load + CDC; targets include S3, Redshift, RDS, Aurora, DynamoDB, Kinesis; DMS Serverless removes instance sizing
Different engines
DMS Schema Conversion / AWS SCT
Converts schemas and reports what needs manual work
SaaS apps (Salesforce, ServiceNow, Google Analytics, SAP and others)
Amazon AppFlow
Managed connectors; on schedule, on event or on demand; filters, mapping and validation; writes to S3, Redshift and more
NFS, SMB, HDFS, object stores, other clouds
AWS DataSync
Scheduled online transfers with verification and bandwidth control
Partners sending files over SFTP, FTPS, FTP or AS2
AWS Transfer Family
Managed endpoints that write to S3 or EFS
Huge offline transfers
AWS Snowball Edge
When the network would take too long
Ongoing hybrid file access
AWS Storage Gateway
Zero-ETL
Aurora/RDS/DynamoDB zero-ETL integrations with Redshift or SageMaker lakehouse
Replicates operational data for analytics without building pipelines
DMS details
CDC reads the source’s transaction log, so it needs logging turned on (for example, binary logging in MySQL).
Data validation compares source and target rows after migration.
Endpoints, replication instance (or serverless), and tasks with table mappings.
Run the replication instance in a VPC with access to both endpoints.
Common traps
Writing custom extraction code for a SaaS app AppFlow supports.
Repeated full loads when only changes are needed: use CDC.
Zero-ETL vs DMS. Zero-ETL is the lowest-effort route from Aurora, RDS or DynamoDB into Redshift for analytics.
A dental chain keeps patient-appointment records in an on-premises Oracle database. The analytics team needs a copy in Amazon S3 that reflects inserts, updates and deletes within about 15 minutes, and the full table must be loaded first.
The source is busy during clinic hours, so the team wants a migration service that reads the database's change log rather than running full-table queries every few minutes. The output should be files that a later pipeline can process.
Which solution meets these requirements?
Choose one answer.
AA. Run an AWS Glue crawler every 15 minutes against the Oracle database, and write the results to S3 as Parquet files.
BB. Create an AWS DMS task with full load and change data capture from Oracle, writing changes to an S3 target as files.
CC. Export the Oracle tables to CSV every night, then copy the files to S3 with AWS DataSync.
DD. Run an AWS Lambda function on a five-minute timer that runs SELECT * on each table and writes the rows to S3.
Show the answer and explanation
Answer: B
A. Wrong: A crawler reads schema metadata to build catalog tables and does not copy row data from the database. It would not capture updates or deletes, so it cannot meet the freshness requirement.
B. Right: Correct. A DMS full-load-and-CDC task loads the existing table, then reads the Oracle change log and applies inserts, updates and deletes to the target. The S3 target writes those changes as files that a later pipeline can process. Learn more: What is AWS Database Migration Service? (opens in a new tab)
C. Wrong: A nightly export misses the 15-minute freshness target. Full-table exports also add heavy read load to the source database.
D. Wrong: Full-table queries every few minutes add heavy load to the busy source, and they cannot see rows that were deleted. The approach fails both the change-tracking and the load requirements.
Explanation
DMS full-load-and-CDC tasks handle both the initial copy and the ongoing changes, reading the database's change log rather than querying tables. The S3 target delivers the changes as files for later processing. Crawlers, nightly exports and polling queries each miss the freshness goal or load the busy source.
A solar installer tracks installation jobs and customer accounts in a Salesforce org. The data team needs a nightly copy of changed records in an S3 bucket, with the data encrypted under a customer managed AWS KMS key.
The team has three engineers, no existing Salesforce integration code, and a goal of as little custom code as possible. Only records changed since the last run should be transferred, and the copy must finish in an overnight window.
Which solution meets these requirements?
Choose one answer.
AA. Write a Lambda function that pages through the Salesforce REST API with a stored OAuth token, keeps its own high-water mark in DynamoDB, and writes the results to S3.
BB. Use AWS DMS with a Salesforce source endpoint and an S3 target, running as a continuous change data capture task.
CC. Create an Amazon AppFlow flow from Salesforce to S3 on a nightly schedule, with incremental transfer and a customer managed KMS key for encryption.
DD. Create an AWS DataSync task with a Salesforce source location and an S3 destination, scheduled nightly with SSE-KMS on the bucket.
Show the answer and explanation
Answer: C
A. Wrong: This can work, but the team would have to build and maintain API paging, token refresh and change tracking. The team wants minimal custom code, so this option adds more operating work than needed.
B. Wrong: AWS DMS replicates from database engines, and Salesforce is not one of its sources. A continuous task would also run all day, which does not match a nightly window.
C. Right: Correct. AppFlow has a managed Salesforce connector, so the flow needs no integration code. A schedule-triggered flow with incremental transfer moves only records added or changed since the last successful run, and AppFlow can encrypt the S3 output with a customer managed KMS key. Learn more: Flow triggers (opens in a new tab)
D. Wrong: DataSync moves files and objects between storage systems such as NFS, SMB, HDFS and object stores. It has no Salesforce connector, so it cannot read records from the org or tell which ones changed. Learn more: What is AWS DataSync? (opens in a new tab)
Explanation
AppFlow is a managed integration service with built-in SaaS connectors, so the team avoids writing and maintaining Salesforce integration code. A schedule-triggered flow can run nightly, use incremental transfer to move only changed records, and encrypt the output with a customer managed KMS key. The custom Lambda approach, DMS and DataSync each fail one requirement.
Want more on Batch ingestion and migration with DMS and AppFlow? The 5 full-length practice sets have 10 more Batch ingestion and migration with DMS and AppFlow questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →
Study Batch ingestion and migration with DMS and AppFlow