Free — no signup required

The Cutover — The Moment of Truth

2 min read

The cutover is the most operationally sensitive moment of the entire migration. You have two tables: users (the original, still serving live traffic) and _users_gho (the ghost, fully backfilled and nearly in sync). You need to swap them atomically — and you need to do it without losing a single write.

The Atomic Rename

MySQL supports renaming multiple tables in a single atomic statement — RENAME TABLE users TO users_old, _users_gho TO users;.

From the database's perspective, this is instantaneous. But gh-ost has a problem: it's been applying binlog events asynchronously, so the ghost table might be a few hundred milliseconds behind the original. If you rename immediately, you lose those last few writes.

The gh-ost Cutover Protocol

gh-ost solves this with a carefully orchestrated lock-and-catch-up sequence:

  1. Lock: gh-ost acquires a write lock on the original table (LOCK TABLES users WRITE).
  2. Application hangs: Your application's write queries queue up, waiting for the lock to release. They do not fail — they wait.
  3. Drain: gh-ost processes the remaining binlog events (typically a few dozen, taking milliseconds).
  4. Rename: Once fully caught up, gh-ost executes the atomic RENAME TABLE.
  5. Unlock: The lock is released.
  6. Writes resume: The queued application writes now execute against the new table.

From the user's perspective, this appears as a brief latency spike — typically 200–500 milliseconds. No writes are lost. No errors are returned to the application.

Controlling the Cutover

gh-ost supports a --postpone-cut-over-flag-file option. When this file exists on disk, gh-ost will complete the backfill and reach "ready to cut over" state — but it will not perform the rename. It waits until you delete the file. This lets you:

  • Complete the migration during off-peak hours.
  • Coordinate the cutover with a deployment or feature flag change.
  • Validate the ghost table's data before committing.

Passing that flag file path to gh-ost at startup makes it complete the backfill and reach "ready to cut over" without performing the rename; deleting the file later is the signal that tells gh-ost to proceed.

Key insight for senior engineers: The cutover lock duration is bounded by binlog lag, not by table size. A 2 TB table with 50 ms of binlog lag will have a ~50 ms cutover lock — the same as a 10 GB table with the same lag. This makes gh-ost's cutover time predictable regardless of table size, which is critical for SLA planning.

This is one of 18 chapters

Every chapter is free — Kubernetes, Terraform, SRE, distributed systems, and more — with fast daily review built in.

Sign up free