What is BuildKit and what advantages does it provide?

docs.questions.sections.docker~4 min read

BuildKit is the modern Docker build engine. The legacy builder is the one Docker shipped from 2014; BuildKit replaced it as default in Docker 23 (2023). The differences are visible immediately: faster builds, better caching, new Dockerfile features, no more secret leaks.

Theory

TL;DR

Default since Docker 23 (2023). Older Docker versions had it as opt-in via DOCKER_BUILDKIT=1.
Built on a different architecture: parallel stage execution, content-addressable graph of operations, pluggable frontends.
Five killer features over legacy:
1. Parallel multi-stage — independent stages build concurrently.
2. Cache mounts — RUN --mount=type=cache keeps caches across builds without baking into image.
3. Secret mounts — RUN --mount=type=secret for build-time secrets, never in image.
4. Smarter cache key — COPY invalidates only on real file changes, not on directory mtime.
5. # syntax= directive — pin Dockerfile frontend version, get new instructions without daemon upgrade.
Access via docker buildx (the BuildKit-aware CLI extension).

Architecture vs legacy builder

Legacy builder:                    BuildKit:
  Linear, sequential                Graph-based, parallel
  Built-in to dockerd               Separate engine (can run remote)
  No cache mounts                   First-class cache mounts
  ARG values leak in history        Secret mounts (no leak)
  Single Dockerfile parser          Pluggable frontend (#syntax=)
  Slow rebuilds                     Smart cache invalidation

BuildKit is essentially a new build daemon with a different mental model — operations form a DAG, BuildKit schedules them.

Killer feature 1: parallel stages

dockerfile

FROM golang:1.23 AS go-builder
RUN go build -o /out/server ./cmd/server

FROM rust:1.81 AS rust-builder
RUN cargo build --release --bin tool

FROM ubuntu:24.04
COPY --from=go-builder /out/server /usr/local/bin/
COPY --from=rust-builder /target/release/tool /usr/local/bin/

Legacy builder: builds Go stage, then Rust stage, then runtime. Sequential. BuildKit: Go and Rust stages build in parallel. Wall-clock time = max of both, not sum.

Killer feature 2: cache mounts

dockerfile

# syntax=docker/dockerfile:1.7
FROM python:3.13-slim
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "app.py"]

The pip wheel cache lives in a Docker-managed cache directory outside the image. Subsequent builds with the same requirements.txt reuse cached wheels. The image stays small (no pip cache baked in); the build stays fast.

Common targets:

pip: /root/.cache/pip
npm: /root/.npm
apt: /var/cache/apt and /var/lib/apt/lists (with sharing=locked)
Go: /go/pkg/mod
Cargo: /usr/local/cargo/registry

Killer feature 3: secret mounts

dockerfile

# syntax=docker/dockerfile:1.7
FROM node:22-alpine
WORKDIR /app
COPY package*.json ./
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm ci
COPY . .

bash

docker buildx build --secret id=npmrc,src=$HOME/.npmrc -t myapp .

The .npmrc is mounted into the build but never lands in any image layer. docker history shows no trace; pulled images do not contain it.

Compare with the bad pattern:

dockerfile

# WRONG: ARG appears in image history
ARG NPM_TOKEN
RUN echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > ~/.npmrc && npm ci

docker history --no-trunc shows the RUN line, including the token.

Killer feature 4: smarter cache key

Legacy builder invalidated COPY based on file timestamps (mtime). Touching a file without editing it triggered cache miss.

BuildKit: cache key for COPY is the content hash of the files. mtime changes do not invalidate. Only real content changes do.

Killer feature 5: `# syntax=` directive

dockerfile

# syntax=docker/dockerfile:1.7
# Now you have access to:
#   --mount=type=cache
#   --mount=type=secret
#   --mount=type=ssh
#   --mount=type=tmpfs
#   here-doc syntax (RUN <<EOF ... EOF)
#   Anonymous build stages with named output

The directive pins the Dockerfile frontend image. New features ship without daemon upgrade — just bump the syntax version.

Enabling BuildKit (when not default)

bash

# Per-shell
export DOCKER_BUILDKIT=1
docker build -t myapp .

# Per-build
DOCKER_BUILDKIT=1 docker build -t myapp .

# Daemon-wide (older Docker)
# /etc/docker/daemon.json
{ "features": { "buildkit": true } }

Docker 23+ has BuildKit on by default; older versions need explicit opt-in. Confirm:

bash

docker buildx version
# github.com/docker/buildx v0.17.0 ...

If buildx is installed and works, BuildKit is available.

`docker buildx` — the BuildKit-aware CLI

bash

docker buildx build -t myapp .                    # like docker build, but with BuildKit features
docker buildx build --platform linux/amd64,linux/arm64  # multi-arch in one go
docker buildx build --cache-to type=registry,ref=cache  # registry cache export
docker buildx build --cache-from type=registry,ref=cache  # registry cache import
docker buildx build --provenance=true --sbom=true       # SLSA provenance + SBOM

buildx is what you actually invoke for full BuildKit features. Plain docker build works too on Docker 23+ (uses BuildKit underneath).

Common mistakes

Forgetting # syntax= directive when using new features

dockerfile

# Won't work — needs syntax directive
RUN --mount=type=cache,target=/root/.cache/pip pip install ...

Add # syntax=docker/dockerfile:1.7 (or higher) at the top of the Dockerfile.

Building without a buildx builder for multi-arch

bash

$ docker buildx build --platform linux/amd64,linux/arm64 -t myapp .
ERROR: Multi-platform builds require a builder instance.

Fix: docker buildx create --use --name multi-builder first. Default builder may be a single-platform Docker driver.

Confusing docker build and docker buildx build

In Docker 23+, both invoke BuildKit. But some flags (--platform, --cache-from type=registry) work better on buildx. For complex CI, prefer buildx.

Treating BuildKit cache mounts as image content

dockerfile

RUN --mount=type=cache,target=/build-output \
    make && cp -r /build-output/* /app/
# After this RUN, /build-output is gone (it was a mount, not part of the image).

Cache mounts disappear after the RUN. To get content into the image, copy it to a regular path inside the same RUN.

Real-world impact

CI build times: typical Node/Python project from 5 minutes (legacy) → 90 seconds (BuildKit + cache mounts).
Image sizes: smaller, because cache directories are no longer baked in.
Secret hygiene: secret leaks via ARG history are largely eliminated.
Multi-arch: one CI step builds for amd64 + arm64. Critical for clusters mixing CPU types.
Supply chain: --provenance=true --sbom=true produces SLSA-attested builds for verification by admission controllers.

Follow-up questions

Q: Do I need BuildKit if my Dockerfile is simple?

A: It is the default in modern Docker — you are using BuildKit unless you went out of your way to disable it. Even simple Dockerfiles benefit from faster cache.

Q: What is buildx vs BuildKit?

A: BuildKit is the engine. buildx is the CLI plugin that talks to BuildKit (and supports multi-platform, multi-cache, etc.). docker buildx ... is how you use full BuildKit features.

Q: How do I migrate existing Dockerfiles to BuildKit?

A: They work as-is on BuildKit (it is backward-compatible). To get the new features, add # syntax=docker/dockerfile:1.7 at the top, then start using --mount=type=cache and --mount=type=secret where they help.

Q: Can BuildKit run remotely?

A: Yes. docker buildx create --driver kubernetes ... or --driver remote lets you run builds on a remote cluster. Useful for big builds that you do not want eating your laptop.

Q: (Senior) How does BuildKit's DAG architecture differ from the legacy builder's linear approach?

A: Legacy: each Dockerfile instruction created a layer in sequence; cache check was per-instruction. BuildKit: the build is a DAG of LLB (low-level builder) operations; BuildKit schedules them based on dependencies. Independent operations run in parallel; cache lookup happens at the operation level (often more granular than instructions). The DAG also enables novel features (mount points that exist only during a specific operation, frontend swaps for different Dockerfile syntax variants, distributed builds).

Examples

Maxed-out modern Dockerfile

dockerfile

# syntax=docker/dockerfile:1.7
FROM node:22-alpine AS deps
WORKDIR /app
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm ci

FROM deps AS build
COPY . .
RUN npm run build

FROM node:22-alpine AS runtime
WORKDIR /app
COPY --from=build /app/dist /app/dist
COPY --from=build /app/node_modules /app/node_modules
USER node
CMD ["node", "dist/server.js"]

Npm cache survives between builds; npmrc never lands in image; deps + build stages share work; runtime stage is slim.

CI registry-cache pattern

bash

docker buildx build \
    --cache-to type=registry,ref=ghcr.io/myorg/myapp:cache,mode=max \
    --cache-from type=registry,ref=ghcr.io/myorg/myapp:cache \
    --platform linux/amd64,linux/arm64 \
    --provenance=true \
    --sbom=true \
    --push \
    -t ghcr.io/myorg/myapp:1.0 .

Multi-arch + registry cache + provenance + SBOM in one command. CI runners on different machines reuse the same cache.

Building remotely

bash

$ docker buildx create --name remote --driver remote tcp://buildkit:1234 --use
$ docker buildx build -t myapp .
# Build runs on the remote BuildKit daemon, not your laptop.

Useful for shared builders, GPU-accelerated builds, or just keeping your laptop free.

Short Answer

Interview ready

Premium

A concise answer to help you respond confidently on this topic during an interview.

Comments

No comments yet