Service comparisons / Kinesis Data Streams vs Amazon Data Firehose
Kinesis Data Streams vs Amazon Data Firehose
Kinesis Data Streams stores ordered records for your own real-time consumers to replay; Amazon Data Firehose loads streaming data into S3, Redshift and other destinations with no code. How to choose, and the exam traps.
Last checked against AWS documentation · How we write our content
| Question | Kinesis Data Streams | Amazon Data Firehose |
|---|---|---|
| Use it when | Your own applications must process records in real time, often several consumers at once | You just need streaming data delivered into a store, with little or no code |
| Consumers | You build them (Lambda, the KCL, Managed Service for Apache Flink) | None to build; it delivers to S3, Redshift, OpenSearch, Splunk, Iceberg tables and HTTP endpoints |
| Speed | Real time, typically well under a second | Near real time; buffers by size or time before delivering |
| Keeps data | 24 hours by default, up to 365 days, and consumers can replay it | No retention or replay; failed deliveries go to an S3 backup |
| Ordering | Ordered within a shard, by partition key | Not something you control |
| Capacity and cost | On-demand mode, or provisioned shards you size; pay per shard-hour or for throughput | Fully managed scaling; pay per GB ingested plus any format conversion |
Both ingest streaming data such as clickstreams, logs and IoT readings. The question is whether you need to process the stream yourself or just land it somewhere.
How to choose
- Real-time dashboards, fraud checks, or several apps reading the same stream: Kinesis Data Streams.
- “Load streaming data into S3 (or Redshift, or OpenSearch) with the least operational effort”: Firehose. It batches, compresses, can convert JSON to Parquet or ORC, and can call a Lambda function to transform records on the way.
- Need to reprocess yesterday’s data after fixing a bug: Kinesis Data Streams, because it keeps records and consumers can replay them. Firehose keeps nothing.
- Both: a common pipeline sends records into a stream for real-time consumers, and also attaches Firehose to that stream to archive everything in S3.
Exam traps
- “Least operational overhead” plus a supported destination is almost always Firehose.
- “Ordering per device or per customer” means a partition key in Kinesis Data Streams; records with the same key go to the same shard, in order.
- A
ProvisionedThroughputExceededExceptionmeans a hot shard or too few shards: spread partition keys, add shards, or switch to on-demand mode. - For an Apache Kafka workload that must keep using Kafka APIs, the answer is Amazon MSK, not Kinesis.