Terraform on GCP: State, Modules, and the Patterns That Actually Scale

Terraform has become the practical default for provisioning GCP infrastructure, including among teams running entirely on Google Cloud who could use the native Deployment Manager instead — a signal worth taking seriously about which tool actually holds up as infrastructure complexity grows. Writing individual resource blocks is the easy part and well covered by the official documentation; the part that actually determines whether a Terraform setup stays maintainable at scale or becomes a source of dread is state management, module structure, and how you handle multiple environments — decisions that are hard to retrofit cleanly once made poorly and lived with for a year.

This is where real, hard-won practice diverges most from a getting-started tutorial, so that’s the focus here.

Remote State: The Decision That Matters Most Early

Terraform’s local state file (terraform.tfstate) works fine for a solo experiment and becomes a genuine liability the moment more than one person touches the same infrastructure — two people applying changes against their own local state files independently is a recipe for drift and conflicting changes neither person can see coming.

terraform {
  backend "gcs" {
    bucket = "my-company-terraform-state"
    prefix = "prod/networking"
  }
}

# Create the state bucket with versioning enabled — non-negotiable
gsutil mb -l us-central1 gs://my-company-terraform-state
gsutil versioning set on gs://my-company-terraform-state

Storing state in a Cloud Storage bucket, with versioning enabled, gives you both team-shared state (everyone applies against the same source of truth) and a recovery path if state ever gets corrupted or an apply goes badly wrong — versioning means you can restore a previous state file version, which has saved real infrastructure teams from what would otherwise be a genuinely bad day. Skipping remote state, or skipping bucket versioning specifically, is one of the few Terraform decisions that’s actively dangerous rather than just suboptimal.

State Locking: Preventing Concurrent Apply Disasters

Engineer A runs `terraform apply`
         │
         ▼
GCS backend acquires a lock (via Cloud Storage's
native object locking mechanism)
         │
Engineer B runs `terraform apply` at the same time
         │
         ▼
   Lock held — Engineer B's apply blocks
   until Engineer A's completes

The GCS backend handles state locking automatically — this isn’t optional configuration, it’s built into using the GCS backend, and it’s what prevents the genuinely bad scenario of two concurrent applies racing against the same state and corrupting it. This is one of several reasons the GCS backend (over local state, even local state shared via a network drive) is the correct default for any team, not a nice-to-have for larger organizations specifically.

Module Structure: Reuse Without Chaos

modules/
  ├── gke-cluster/
  │     ├── main.tf
  │     ├── variables.tf
  │     └── outputs.tf
  ├── vpc-network/
  │     ├── main.tf
  │     ├── variables.tf
  │     └── outputs.tf
  └── cloud-sql-instance/
        ├── main.tf
        ├── variables.tf
        └── outputs.tf

environments/
  ├── dev/
  │     └── main.tf   (calls modules with dev-specific variables)
  ├── staging/
  │     └── main.tf   (calls modules with staging-specific variables)
  └── prod/
        └── main.tf   (calls modules with prod-specific variables)

module "networking" {
  source       = "../../modules/vpc-network"
  network_name = "prod-vpc"
  subnet_cidr  = "10.10.0.0/16"
}

module "gke" {
  source       = "../../modules/gke-cluster"
  cluster_name = "prod-cluster"
  network      = module.networking.network_self_link
  node_count   = 5
}

This structure — reusable modules parameterized by variables, called from environment-specific configurations — is what actually lets a team avoid writing the same VPC or GKE cluster configuration three times with copy-pasted, subtly-diverging HCL for dev, staging, and prod. The module encapsulates the “how,” the environment configuration supplies the “what specifically for this environment,” and a change to the module’s internal logic (say, a security hardening update to the GKE module) propagates to every environment consistently once each environment’s main.tf is re-applied.

Module outputs, like module.networking.network_self_link in the example above, are what let modules compose cleanly — the GKE module doesn’t need to know how the VPC module constructed its network internally, only that it can reference the network’s identifier through a stable, documented output. This composability is what makes larger Terraform codebases genuinely maintainable rather than an increasingly tangled web of resources referencing each other’s internal implementation details directly.

Workspaces vs Separate Directories: The Multi-Environment Decision

┌────────────────────────┬────────────────────────────────────────┐
│ Approach                   │ Trade-off                                   │
├────────────────────────┼────────────────────────────────────────┤
│ Terraform Workspaces        │ One codebase, multiple named state         │
│                             │ instances (`terraform workspace select     │
│                             │ prod`) — less code duplication, but        │
│                             │ genuinely easy to apply against the wrong  │
│                             │ workspace by mistake                        │
│ Separate directories         │ Explicit directory per environment (as     │
│ (shown above)                │ in the module example) — more files, but   │
│                             │ you can never accidentally run `apply`      │
│                             │ against the wrong environment, because the │
│                             │ environment is the directory you're in      │
└────────────────────────┴────────────────────────────────────────┘

Workspaces are appealing for their reduced duplication, but the “which workspace am I currently in” ambiguity is a genuinely common source of real incidents — an engineer runs terraform apply believing they’re in the dev workspace, when a previous workspace select command actually left them in prod. Separate directories per environment trade some file duplication for an explicit, hard-to-mistake safety property: the environment you’re operating on is literally the directory you’re standing in, not an easily-forgotten prior command’s side effect. For anything genuinely production-critical, most practitioners consider this trade-off worth it.

The GCP Provider: Authentication in CI/CD

provider "google" {
  project = var.project_id
  region  = "us-central1"
}

For CI/CD pipelines applying Terraform automatically, the authentication method matters for security, not just convenience:

# Workload Identity Federation — no long-lived key needed
gcloud iam workload-identity-pools create github-pool \
  --location=global \
  --display-name="GitHub Actions pool"

Using Workload Identity Federation to let a CI/CD pipeline (GitHub Actions, GitLab CI) authenticate to GCP without a long-lived service account JSON key stored as a pipeline secret is the current best practice — a leaked long-lived key is a standing risk for as long as it remains valid, while federated authentication issues short-lived tokens per pipeline run with no persistent credential to leak in the first place. This is a genuinely important security upgrade that a surprising number of Terraform-on-GCP setups still haven’t made, often because the long-lived-key approach was what an early tutorial showed and nobody’s revisited it since.

A Safe CI/CD Apply Pipeline, End to End

Putting the pieces together — remote state, plan review, and federated authentication — into an actual pipeline flow clarifies how they combine in practice.

The plan-as-PR-comment step is worth calling out specifically — it turns an infrastructure change into something reviewable the same way application code is, with the actual computed diff (not just the source code diff) visible to whoever’s approving it. A reviewer can see “this will destroy and recreate a Cloud SQL instance” directly in the plan output before approving, which is a meaningfully stronger safety net than reviewing only the HCL source and trying to mentally simulate what Terraform will actually do with it.

Real-World Use Case: Onboarding a New Environment Safely

A team adding a new environment (say, a dedicated environment for a specific enterprise customer with data residency requirements) benefits directly from a well-structured module setup — creating the new environment is largely copying an existing environment’s main.tf, adjusting variables (region, network CIDR, sizing), and running terraform plan to review exactly what will be created before applying. Because the underlying modules are already tested and used in production for other environments, this new environment inherits the same security hardening, tagging conventions, and architecture patterns automatically, rather than requiring a from-scratch design and review process each time a new environment is needed.

Best Practices

• Always use a remote backend (GCS) with versioning enabled, from the very first environment, not retrofitted after the first multi-person incident.
• Prefer separate directories over workspaces for production-critical environments, accepting the file duplication trade-off for the safety of explicit environment selection.
• Use Workload Identity Federation for CI/CD authentication, retiring any long-lived service account keys currently used for Terraform automation.
• Run terraform plan and require human review before every apply against production, treating infrastructure changes with the same rigor as application code deployments — a plan output is your best defense against an unintended change slipping through.

Common Mistakes

Using local state, or state shared via a network drive, for anything beyond a solo experiment. This is consistently the root cause of the worst Terraform incidents — corrupted or conflicting state with no clean recovery path.

Applying changes without reviewing the plan output carefully. A plan showing an unexpected resource replacement (not just an update) is a critical signal worth stopping and investigating before applying — Terraform replacing a resource often means downtime or data loss for that resource, not a routine change.

Copy-pasting configuration across environments instead of using modules. This guarantees the environments drift apart silently over time as each copy gets modified independently, defeating the entire “consistent infrastructure across environments” value proposition Terraform is meant to provide.

Frequently Asked Questions

Is Terraform the same across every cloud provider, or does GCP need different knowledge? The core Terraform workflow (plan, apply, state, modules) is identical across providers — what differs is the provider-specific resource types and their arguments, which for GCP means learning the google provider’s resource schema specifically.

Can I import existing, manually-created GCP resources into Terraform management? Yes, via terraform import, which brings an existing resource under Terraform’s state tracking without recreating it — a common and necessary step when adopting Terraform for infrastructure that already exists.

How often should Terraform modules be updated or reviewed? Treat them like any shared library code — version them explicitly (using Git tags or a module registry), and review changes with the understanding that a module update potentially affects every environment consuming it, not just the one being actively worked on.

Does using Terraform mean giving up the GCP Console for making changes? Not entirely, but manual console changes to Terraform-managed resources cause state drift — the console remains useful for inspection and read-only investigation, but changes to Terraform-managed infrastructure should go through Terraform to keep state accurate.

What’s the risk of running terraform apply locally instead of through CI/CD? Local applies bypass the review step a PR-based pipeline enforces, and depending on local authentication setup, may use a long-lived credential rather than short-lived federated tokens — reserving local applies for genuine emergencies, with routine changes going through the reviewed pipeline, is the safer default.

Summary

The Terraform features that get the most tutorial attention — resource syntax, basic provider configuration — are genuinely the easy part. What actually determines whether a GCP Terraform setup remains trustworthy as it grows is remote state with versioning from day one, a module structure that avoids environment drift, a deliberate (and safety-conscious) choice between workspaces and separate directories, and modern, keyless CI/CD authentication. Getting these foundational decisions right early is meaningfully cheaper than retrofitting them onto a year of accumulated infrastructure and institutional habits built around a riskier initial setup.