S3 with a Turbocharger
The natural question is: S3 has high latency (tens of milliseconds per object request). How can UltraWarm return search results in seconds when the data lives on S3?
The answer is a combination of three mechanisms:
1. Migration and Segment Optimization
When you move an index from Hot to Warm, OpenSearch first performs a force_merge — collapsing many small Lucene segment files into a single large segment per shard. Fewer segments means fewer S3 object reads per query. The merged segments are then uploaded to S3.
2. Local NVMe Cache
UltraWarm nodes are large EC2 instances (built on the AWS Nitro System) with substantial local NVMe storage. When a query arrives, the node fetches the required segment blocks from S3 and caches them locally. Subsequent queries against the same data hit the local cache at NVMe speeds, not S3 speeds.
3. Intelligent Prefetching
When you query a time range (e.g., "last 14 days"), the UltraWarm node analyzes the query and prefetches adjacent time-range blocks from S3 into the cache before you ask for them. This makes sequential dashboard exploration feel fast even though the underlying data is on object storage.
Key Constraint: UltraWarm is read-only. You cannot index new documents into a Warm index. The workflow is always: write to Hot → roll over → migrate to Warm. This is not a limitation of S3 — it is an architectural choice. Allowing writes to Warm would require invalidating cached segments constantly, destroying the performance model.
Operational implication: If you need to backfill data into a Warm index (e.g., you discovered a parsing bug and need to re-index), you must move the index back to Hot, re-index, then migrate to Warm again. Plan your data pipelines accordingly.