Secrets in Kubernetes: External Secrets Operator (ESO)

The Kubernetes "Secret" Lie

Kubernetes has a native object called a Secret. The name implies security, but the default behavior is deceptive:

apiVersion: v1
kind: Secret
metadata:
  name: my-db-secret
data:
  username: YWRtaW4=      # base64 decode → "admin"
  password: cGFzc3dvcmQ=  # base64 decode → "password"

Base64 is not encryption. It is an encoding scheme — a way of representing binary data as text. Anyone who can read the YAML can decode it in seconds with a single base64 --decode call — no key, no password, nothing to crack.

By default, Kubernetes stores these Secret objects unencrypted in etcd (the cluster's key-value database). Anyone with etcd access — or anyone who can run kubectl get secret -o yaml — sees the plaintext values. Kubernetes does support Encryption at Rest for etcd, but it must be explicitly configured and is not enabled by default in most distributions.

This means you cannot safely store actual secrets in Kubernetes Secret manifests committed to Git. Yet your applications need to read credentials somehow. This is the Kubernetes secrets dilemma.

The Bridge: External Secrets Operator (ESO)

External Secrets Operator (ESO) is a Kubernetes controller that bridges the gap between your secure external secrets store (Vault, AWS Secrets Manager, GCP Secret Manager, Azure Key Vault) and the native Kubernetes Secret objects your applications expect.

The workflow:

1. You install ESO in your cluster (via Helm).
2. You configure a SecretStore or ClusterSecretStore object that tells ESO how to authenticate to your external secrets backend.
3. You create an ExternalSecret object that declares: "Fetch the secret at this path in AWS Secrets Manager and create a Kubernetes Secret named X with these keys."
4. ESO's controller watches for ExternalSecret objects, fetches the data from the external store, and creates or updates the corresponding native Kubernetes Secret.
5. ESO continuously polls (or uses push-based updates) to keep the Kubernetes Secret synchronized with the external store. When the external secret rotates, the Kubernetes Secret is automatically updated.

Concretely, this is two custom resources: a ClusterSecretStore that tells ESO which backend to talk to (AWS Secrets Manager, in a given region, authenticating via a bound ServiceAccount), and an ExternalSecret that maps specific keys — pulled from a named path and property in that backend — onto the keys of a target Kubernetes Secret, refreshed on an interval.

After ESO processes this, a standard Kubernetes Secret named app-db-creds exists in the my-app namespace. Your application's Deployment references it exactly as it would any other Kubernetes Secret — no AWS SDK calls, no Vault client libraries, no awareness of the external system.

ESO vs. Vault Agent Injector

Vault also provides its own Kubernetes integration called the Vault Agent Injector. It works differently: a mutating webhook intercepts Pod creation and injects a sidecar container (Vault Agent) into the Pod. The sidecar authenticates to Vault and writes secrets to a shared in-memory volume that the main container reads.

ESO is the better choice when you want simplicity and multi-backend support. Vault Agent Injector is better when you need secrets to never touch the Kubernetes API server and want live updates without Pod restarts.