DynamoDB Streams: The Change Log

What Streams Are

DynamoDB Streams is an ordered, time-stamped log of every item-level change (insert, update, delete) in your table. Think of it as a database transaction log made accessible to your application. Changes are retained for exactly 24 hours and are delivered in the order they occurred within each partition.

Streams enable a pattern called Change Data Capture (CDC): instead of polling the database for changes, downstream systems react to a stream of change events. This decouples producers from consumers and enables event-driven architectures.

Stream View Types

When enabling streams, you choose what data each stream record contains:

Choosing the right view: NEW_AND_OLD_IMAGES is the most flexible but doubles the stream record size, increasing Lambda invocation payload size and stream storage costs. Use KEYS_ONLY when the downstream system only needs to know that something changed (e.g., a cache invalidation service that will re-fetch the item itself).

The Lambda Trigger Pattern

The most common integration is a Lambda function triggered by the stream. DynamoDB invokes Lambda with a batch of stream records (configurable batch size: 1–10,000 records).

Architecture patterns:

Pattern 1: Welcome Email on Sign-Up
User writes to DynamoDB
  → Stream (NEW_IMAGE)
  → Lambda filters for INSERT events on SK = "PROFILE"
  → Lambda calls SES to send welcome email

Pattern 2: Search Index Synchronization
Any item change in DynamoDB
  → Stream (NEW_AND_OLD_IMAGES)
  → Lambda transforms item to OpenSearch document format
  → Lambda upserts/deletes document in OpenSearch cluster

Pattern 3: Cross-Region Replication
Item written in us-east-1
  → Stream (NEW_IMAGE)
  → Lambda writes identical item to DynamoDB table in eu-west-1
  → Provides a manually managed replica (DynamoDB Global Tables automates this)

Pattern 4: Aggregation / Counter Maintenance
Order item written with Status = "COMPLETED"
  → Stream (NEW_AND_OLD_IMAGES)
  → Lambda detects status transition from PENDING → COMPLETED
  → Lambda increments a counter item using atomic UpdateItem

Lambda error handling: If your Lambda function throws an error, DynamoDB retries the entire batch. This means your Lambda must be idempotent — processing the same stream record twice must produce the same result. Use the stream record's eventID as an idempotency key.

Ordering guarantee: DynamoDB Streams guarantees ordering within a single partition. Records from different partitions may be processed in parallel by different Lambda instances. If your downstream logic requires strict global ordering, you must serialize processing — which eliminates the parallelism benefit.