What Is Kinesis?
Think of a river. Data flows continuously — user clicks, sensor readings, financial transactions — and you need infrastructure to capture, route, and store that flow. AWS Kinesis is that infrastructure. But "Kinesis" is actually a brand name covering two very different services, and confusing them is the single most common mistake in streaming architecture interviews and real-world designs.
1. Kinesis Data Streams (KDS)
Think of KDS as a customizable DVR (digital video recorder). When you record a TV show, the content is stored on disk. You can pause it, rewind it, fast-forward it, or watch it multiple times. KDS works the same way: it ingests data and stores it (default 24 hours, configurable up to 365 days). You write custom consumer code to read from it, replay old data, or process it in real-time.
- Latency: Real-time (~70–200ms end-to-end with Enhanced Fan-Out).
- Use when: You need custom processing logic, multiple independent consumers, or the ability to replay data.
- You manage: Shard provisioning, consumer code, checkpointing.
2. Kinesis Data Firehose (Amazon Data Firehose)
Think of Firehose as a delivery truck. You load packages onto the truck (send records), the truck waits until it has a full load or a timer expires, then it drives to the destination and unloads. Firehose ingests data, buffers it into batches, and delivers it to a configured destination (S3, Redshift, OpenSearch, Splunk). You write no consumer code — you just configure a destination.
- Latency: Near real-time (60-second minimum buffer interval).
- Use when: You need to dump logs, events, or metrics into a data store without custom processing.
- You manage: Buffer settings and optional Lambda transformation.
The Decision Rule: If you need to store data, replay it, or run custom complex logic with multiple consumers, use Kinesis Data Streams. If you just need to reliably move data into S3, Redshift, or OpenSearch with minimal code, use Firehose.