Finalizers: Handling Deletion

The Soft Delete Problem

When a user runs kubectl delete mydb my-production-db, Kubernetes wants to remove the object from etcd immediately. But what if your Operator needs to do cleanup first?

• Delete an external AWS S3 bucket that the operator provisioned?
• Deregister the database from a service discovery system?
• Take a final snapshot before destroying the data?
• Release a license from an external license server?

Kubernetes Garbage Collection handles child Kubernetes objects automatically (via Owner References). But it has no knowledge of external resources — AWS, GCP, DNS providers, or any system outside the cluster. For those, you need Finalizers.

How Finalizers Work

A Finalizer is a string in the object's metadata — for example, "myoperator.example.com/cleanup". It acts as a lock that prevents deletion.

The complete lifecycle in your controller:

On Create/Update (add the finalizer):

const myFinalizer = "myoperator.example.com/cleanup"

// If the object doesn't have our finalizer, add it
if !controllerutil.ContainsFinalizer(&myResource, myFinalizer) {
    controllerutil.AddFinalizer(&myResource, myFinalizer)
    return ctrl.Result{}, r.Update(ctx, &myResource)
}

On every reconcile (check for deletion):

// Is the object being deleted?
if !myResource.DeletionTimestamp.IsZero() {
    // Yes — run cleanup
    if controllerutil.ContainsFinalizer(&myResource, myFinalizer) {
        if err := r.deleteExternalResources(&myResource); err != nil {
            // Cleanup failed — return error to retry
            return ctrl.Result{}, err
        }
        // Cleanup succeeded — remove the finalizer
        controllerutil.RemoveFinalizer(&myResource, myFinalizer)
        return ctrl.Result{}, r.Update(ctx, &myResource)
    }
    // Finalizer already removed — nothing to do
    return ctrl.Result{}, nil
}

// Normal reconcile logic continues here...

Critical rules for finalizer cleanup logic:

1. It must be idempotent. deleteExternalResources() might be called multiple times if it fails partway through. Deleting an already-deleted S3 bucket should return success, not an error.
2. It must be bounded. If cleanup is stuck (e.g., the external system is unreachable), the object will be stuck in a terminating state forever. Implement timeouts and consider a "force delete" escape hatch.
3. Never block on user action. Cleanup logic that requires a human to intervene will leave objects stuck in Terminating indefinitely, which is a terrible user experience.