Introduction: Terraform is a Plugin System

The Split Brain Architecture

Think of Terraform like a general contractor who manages a construction project. The contractor (Terraform Core) reads the blueprints, tracks what has been built, and coordinates the work — but they don't personally lay bricks or wire electricity. For that, they hire specialist subcontractors (Providers) who know how to work with specific materials and systems. The contractor and subcontractors communicate through a well-defined protocol: the contractor says "build this," and the subcontractor figures out exactly how.

When you run terraform apply, you aren't running one program. You are running a distributed system on your local machine. Terraform consists of two distinct parts:

1. Terraform Core: The binary you download from HashiCorp. It handles the CLI, reads .tf files, manages the state graph, and calculates the dependency tree. It knows nothing about AWS, Azure, or your internal API.
2. Terraform Plugins (Providers): Separate binaries (e.g., terraform-provider-aws) that know how to talk to specific APIs. Each provider is a standalone executable that Core spawns as a child process.

When you run terraform init, Core downloads these plugin binaries into the .terraform/providers/ directory. During execution, Core starts the plugins as background processes and communicates with them over a local network socket using a well-defined protocol.

Why Write a Custom Provider?

The public Terraform Registry hosts thousands of providers, but there are legitimate reasons to write your own:

• Internal APIs: Your company has a legacy provisioning system, a custom CMDB, or an internal service catalog with no public provider.
• SaaS vendors: You want customers to manage your service's resources (teams, projects, API keys) via Terraform, just like they manage AWS resources.
• Wrapping local tools: You need to model a complex script, a database migration runner, or a certificate authority as a standard Terraform resource with proper state tracking.
• Proprietary hardware: Network switches, storage arrays, and on-premises appliances often have REST APIs but no Terraform support.

Writing a custom provider turns any API into a first-class Terraform citizen — complete with plan/apply lifecycle, state management, drift detection, and dependency resolution.