Runner Isolation: Handling Untrusted Builds

The Risk of Public Pull Requests

If you maintain an open-source project, anyone on the internet can submit a Pull Request (PR). If your CI pipeline runs automatically on every PR, a malicious contributor can submit a PR that modifies the build script to:

1. Print your AWS_SECRET_ACCESS_KEY to the build log.
2. Mine cryptocurrency on your infrastructure.
3. Exfiltrate the contents of ~/.ssh/ or other sensitive files on the runner.
4. Attempt a container escape to attack the host machine and any other jobs running on it.

The fundamental problem is trust boundary mismatch: you are running untrusted, externally-submitted code on infrastructure that has access to your secrets and production environment.

Standard Docker: A Thin Wall

Standard Docker containers share the host Linux kernel. Every container on a machine makes system calls directly to the same kernel. This is efficient, but it means a single kernel vulnerability can allow a container to "break out" and take over the host server — and every other container running on it.

This is acceptable for running your own trusted code. It is not acceptable for running code submitted by anonymous contributors.

Hard Isolation: gVisor and Firecracker

For untrusted workloads, you need a fundamentally stronger isolation boundary.

gVisor (Google)

gVisor interposes a userspace kernel between the container and the real host kernel. Instead of making system calls directly to the Linux kernel, the container's processes make system calls to gVisor's sentry component, which is written in Go and runs in userspace.

The sentry implements a large subset of the Linux system call interface. It only passes a small, carefully reviewed set of calls through to the real kernel. This dramatically reduces the attack surface. Even if a process inside the container exploits a bug in gVisor's sentry, it has not reached the real kernel — it has only compromised another userspace process.

Container Process
      ↓  syscall
  gVisor Sentry (userspace, Go)
      ↓  limited syscalls only
  Real Host Kernel

Firecracker (AWS)

Firecracker takes a different approach: hardware virtualization. Each workload runs inside a lightweight Micro-VM backed by KVM (Kernel-based Virtual Machine), which is a Linux kernel module that uses CPU hardware virtualization extensions (Intel VT-x, AMD-V).

Breaking out of a Firecracker VM requires exploiting the hypervisor itself — the same class of vulnerability that would allow escaping an EC2 instance. This is orders of magnitude harder than a container escape.

Firecracker VMs boot in under 125 milliseconds and use as little as 5 MB of memory overhead, making them practical for CI workloads. AWS Lambda and GitHub Actions both use Firecracker under the hood.

Choosing Between Them

Practical Mitigations for Self-Hosted Runners

If you run self-hosted GitHub Actions or GitLab runners, apply these controls: require manual approval before running CI on pull requests from outside collaborators, and structure your workflow so that only the jobs which don't touch untrusted PR code — like a deploy job gated on the branch being main and requiring a manual approval environment — ever have secrets injected into their environment. A test job that runs arbitrary PR code should have no credentials available to it at all.

The key principle: never expose secrets to jobs that run untrusted code. Separate the test job (which runs PR code) from the deploy job (which has credentials) and gate the deploy job on the branch being main.