Streaming data for ML: MLA-C02 study notes and practice questions
What to know about Streaming data for ML for the MLA-C02 exam, from the Data Preparation for ML and AI domain (28% of the exam guide), then 2 free questions to test it. Part of the MLA-C02 study guide.
Streaming questions describe events arriving continuously (clicks, sensor readings, transactions) and ask how to capture them for training or real-time features.
Services
Service
Use it when
Kinesis Data Streams
You need custom consumers, ordering per key, replay within the retention period and sub-second latency
Amazon Data Firehose
You want managed delivery to S3, Redshift, OpenSearch or Iceberg tables within minutes, with no consumers to run
Amazon MSK
Producers already use Apache Kafka; only the broker endpoints change, and topic retention allows replay
Managed Service for Apache Flink
You need stateful stream processing: windows, aggregations, joins, such as “transactions per card in the last 5 minutes”
Lambda
Light per-record processing triggered by a stream
Firehose features worth knowing
Buffering by size and time trades latency against file size.
Format conversion to Parquet or ORC using a Glue table schema.
Dynamic partitioning builds S3 prefixes from fields in each record, so queries and training jobs read only the partitions they need.
Lambda transformation for light record changes.
Streaming into ML
Real-time features: compute aggregates in Flink and write them to the Feature Store online store with PutRecord, so the model reads fresh values at inference.
Training data: land the stream in S3 (Firehose) and train on batches.
Streaming inference: a consumer calls an endpoint for each record or small batch.
Kinesis basics
Capacity scales with shards or on-demand mode; a good partition key spreads load.
Enhanced fan-out gives each consumer its own throughput.
Retention is 24 hours by default and can be extended up to 365 days.
Common traps
Rewriting Kafka producers for Kinesis when MSK keeps them working.
Writing custom consumers to land data in S3 when Firehose does it with no code.
Computing windowed aggregates in Lambda with external state, when Flink is built for it.
An esports league streams match telemetry from its game servers at roughly 30,000 events per second. A data engineering team wants the raw events in Amazon S3 as Parquet files, partitioned by league and by game ID, so that the ML team can query them with Amazon Athena.
Files should appear within a few minutes of each event. The team does not want to run or patch a Spark cluster, and the data must keep its schema when it is converted.
Which solution meets these requirements?
Choose one answer.
AA. Invoke an AWS Lambda function for every event that writes the event as its own small JSON object to Amazon S3 for Athena to query.
BB. Run an Amazon EMR cluster with a Spark Structured Streaming job that reads the events and writes partitioned Parquet files to Amazon S3.
CC. Send the events to Amazon Data Firehose with Parquet conversion from a Glue table schema, and dynamically partition on the league and game IDs.
DD. Send the events to Amazon Data Firehose with Parquet conversion from a Glue table schema, and deliver all records under the default time-based S3 prefixes.
Show the answer and explanation
Answer: C
A. Wrong: Writing one object per event creates an enormous number of small files, which slows Athena and drives up request costs. The function would also need its own logic to group events into Parquet.
B. Wrong: Spark produces the right output, but the team would size, patch and scale a cluster, which the requirement rules out. Operating the cluster adds effort the team does not want to take on.
Amazon Data Firehose is managed, delivers to Amazon S3 within minutes, and can convert records to Parquet using a Glue table schema. Dynamic partitioning writes each record under a prefix built from its league and game keys, so Athena can prune partitions. Per-event Lambda writes create too many objects, and a Spark cluster adds the operations the team wants to avoid.
A logistics broker runs its shipment-tracking system on Apache Kafka in an on-premises data center. Its ML team wants to stream these events into AWS for model training, and the operations team wants to keep the existing Kafka producers and their client libraries unchanged.
The stream must be durable, must support replay of old events for retraining, and must be managed by AWS with no Kafka brokers to patch.
Which solution meets these requirements?
Choose one answer.
AA. Rewrite each producer to send records to Amazon Kinesis Data Streams with the Kinesis Producer Library, and read the stream with AWS Lambda.
BB. Send the events to an Amazon SQS FIFO queue and have a Lambda function write each message to Amazon S3.
CC. Run a Kafka cluster on Amazon EC2 instances that the team builds from open-source packages and manages itself.
DD. Create an Amazon Managed Streaming for Apache Kafka cluster, point the existing producers at its brokers, and use an MSK Connect sink connector to deliver events to Amazon S3.
Show the answer and explanation
Answer: D
A. Wrong: Kinesis Data Streams is durable and supports replay within its retention period, but every producer would need code changes to use the Kinesis API. The operations team requires the existing Kafka producers and client libraries to stay unchanged. Learn more: What is Amazon Kinesis Data Streams? (opens in a new tab)
B. Wrong: SQS uses its own API rather than the Kafka protocol, so the producers would still change. Messages are removed after they are consumed, which makes replay for retraining difficult.
C. Wrong: This keeps the Kafka protocol, but the team would patch, scale and secure the brokers, which the requirement rules out. Managed Kafka on AWS gives the same protocol without that work.
Amazon MSK provides managed Apache Kafka, so existing producers and client libraries keep working and only change their broker endpoints. Topic retention lets the ML team replay old events. Kinesis and SQS require producer changes, and self-managed Kafka on EC2 leaves the broker operations with the team.
Want more on Streaming data for ML? The 5 full-length practice sets have 10 more Streaming data for ML questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →