Why the Old Builder Was a Bottleneck
The original Docker build engine processed Dockerfile instructions sequentially, one at a time, in the order they appeared. Even if two stages were completely independent of each other, they ran one after the other. Cache invalidation was coarse-grained. There was no way to safely use secrets during a build without them ending up in a layer.
BuildKit, introduced as an experimental feature in Docker 18.09 and made the default in Docker 23.0, replaces this engine entirely. It treats the Dockerfile as a dependency graph rather than a sequential script.
Key BuildKit Features
1. Parallel Execution
BuildKit analyzes the dependency graph of your Dockerfile stages. Stages that don't depend on each other are built concurrently. In a multi-stage build with independent test and lint stages, both run at the same time.
2. Cache Mounts
This is one of the most impactful features for build speed. A cache mount (--mount=type=cache) tells BuildKit to persist a specific directory between builds, outside of the image layer system. Package manager caches are the primary use case — the same pattern works whether it's apt's package cache, Go's module cache, or npm's cache directory. The target directory persists on the build host between builds. The first build downloads packages normally. Subsequent builds reuse the cache. On a large project, this can reduce a 3-minute dependency installation to under 10 seconds.
3. Secret Mounts
A common mistake is baking secrets (API keys, SSH keys, private registry credentials) into image layers. Even if you delete the secret in a later RUN instruction, it remains permanently accessible in the earlier layer.
BuildKit's secret mounts (--mount=type=secret) solve this by providing the secret as a temporary in-memory file that is available only during that specific RUN instruction and is never written to any layer, passed in at build time rather than baked into the Dockerfile itself. The secret never appears in docker history or any image layer.
4. SSH Agent Forwarding
For cloning private Git repositories during a build, BuildKit can forward your SSH agent socket (--mount=type=ssh) rather than requiring you to copy SSH keys into the image.
Key Point: Cache mounts are not part of the image. They live on the build host. This means they are not portable — a fresh CI runner won't have the cache. For CI environments, combine BuildKit cache mounts with --cache-from (registry-based layer caching) for the best results. The two mechanisms are complementary: --cache-from handles layer-level caching across machines; cache mounts handle package manager caches on a single machine.