Schema Evolution: Changing Data Structures Without Breaking Everything Downstream

No schema survives contact with a growing business. The column you swore was always populated becomes optional. The type field needs a sixth value. The identifier that was an integer needs to become a string because you acquired a company whose IDs have letters in them. Schema change is not a failure of modeling — it’s the normal condition of modeling. What separates mature data organizations from fragile ones is not fewer changes; it’s that their changes don’t take down everything downstream.

This post is the practical playbook: which changes are safe and which are breaking, the expand-contract pattern that makes even breaking changes deployable, how schema registries enforce the rules for events, and how the problem shows up differently in databases, warehouses, and lakes.

The Fundamental Asymmetry: Writers and Readers Move Separately

Every schema-evolution problem reduces to one fact: the code that writes data and the code that reads it never deploy at the same instant. A message produced today may be consumed by a service deployed last month; a Parquet file written last year will be read by a query written next year. So every change must be evaluated against two questions:

• Backward compatibility: can new readers handle old data? (Needed whenever historical data persists — which is always, in analytics.)
• Forward compatibility: can old readers handle new data? (Needed whenever consumers deploy on their own schedule — which is always, in event systems.)

From these, the standard classification of changes:

Safe (usually):

• Adding an optional field with a default — old data lacks it (readers use the default); old readers ignore it.
• Widening a type (int → long) — everything old still fits.
• Relaxing a constraint you no longer need.

Breaking (always treat as such):

• Removing or renaming a field — every reader referencing it breaks. A rename is a remove plus an add; there is no safe rename.
• Changing a field’s type or meaning — the second is nastier because nothing crashes: a price that silently switches from dollars to cents corrupts every downstream calculation while all systems report healthy.
• Tightening constraints or making an optional field required — old data violates the new rule.

The “meaning change” case deserves emphasis because tooling can’t catch it. Compatibility checkers validate structure; only contracts, documentation, and review validate semantics. Renaming revenue to net_revenue while changing its formula is two breaking changes, and the schema diff only shows one.

Expand–Contract: The Pattern That Makes Breaking Changes Boring

Since breaking changes are sometimes necessary, the discipline is to never make them atomically. The expand-contract pattern (also called parallel change) splits one breaking change into three boring ones:

Concrete example — splitting full_name into first_name / last_name in a production database:

1. Expand: add the two nullable columns; deploy code that writes all three; backfill old rows with a batch job.
2. Migrate: update each consumer (services, reports, exports) to read the new columns — on their own schedules, verified one at a time.
3. Contract: once query logs confirm nothing reads full_name, stop writing it; drop it a release later.

Each step is individually reversible and non-breaking. The cost is living with duplication for weeks — which is precisely the price of not coordinating a simultaneous deploy of fifteen consumers, a thing that has never once gone well. Note the load-bearing detail in step 3: query logs confirm — usage metadata (see the metadata post in this series) is how you know contraction is safe rather than hoping it is.

Events and the Schema Registry: Evolution with Teeth

Event streams (Kafka and friends) make evolution both more dangerous — messages persist, consumers are many and autonomous — and better-tooled, because this is where the schema registry pattern matured. The mechanics:

• Every topic’s schema (Avro, Protobuf, or JSON Schema) is registered centrally with a declared compatibility mode — BACKWARD (new readers handle old messages), FORWARD, or FULL (both).
• Producers cannot publish a schema version that violates the topic’s mode — the registry rejects it at deploy/CI time, not at 3 a.m. in a consumer’s pager.

// v1
{"type":"record","name":"OrderPlaced","fields":[
  {"name":"order_id","type":"string"},
  {"name":"amount","type":"double"}
]}
// v2 — legal under BACKWARD: new field carries a default
{"type":"record","name":"OrderPlaced","fields":[
  {"name":"order_id","type":"string"},
  {"name":"amount","type":"double"},
  {"name":"currency","type":"string","default":"USD"}
]}

Two practices turn the registry from tooling into culture: run compatibility checks in CI (a producer PR that breaks the contract fails the build — the cheapest possible place to learn), and when a genuinely incompatible change is unavoidable, create a new topic (orders.v2), run both during migration, and retire the old — expand-contract at the topic level.

The same idea has spread beyond streaming as data contracts: explicit, versioned, CI-enforced schemas on the datasets teams publish to each other — the enforcement mechanism that makes data mesh’s “data as a product” (covered earlier in this series) more than a slogan.

The Same Problem in Every Costume

Relational databases have the best tooling — versioned, reviewed migration scripts (Flyway, Liquibase, dbt for the warehouse). The classic operational traps are locks, not logic: adding a NOT NULL column with a default, or building an index non-concurrently, can freeze a large hot table. Know your engine’s fast paths, and remember application deploys and schema deploys are never atomic — expand-contract applies to your own database too.

Warehouses and lakes add the historical dimension: the schema changed in March, but analysts query across January–December. Landing raw data with schema versions attached, then normalizing in transformation layers (the ELT pattern), gives you a place to reconcile epochs. Modern table formats — Delta Lake, Apache Iceberg — track schema per snapshot and handle add/rename/widen cleanly; naive Parquet-on-S3 does not, which is a major reason the formats won.

Document databases put evolution entirely in application hands — the schema_version field and lazy-migration recipe covered in the document design post of this series.

An Operating Checklist

1. Every shared schema has a declared compatibility policy and an owner.
2. Compatibility is checked in CI, not discovered in production.
3. Breaking changes only ever ship as expand → migrate → contract, with usage data gating the contract step.
4. Semantic changes (meaning, units, formulas) are treated as breaking even when structurally invisible — communicated, versioned, documented.
5. Consumers get deprecation windows with dates, and those dates are enforced — an old field that lingers a year past its deadline teaches everyone that deadlines are decorative.
6. Old data remains readable: keep old schema versions registered forever; they describe data you still hold.

The mindset shift underneath all of it: a schema is not private property of the team that writes the data — it is a published interface, and changing it carries the same obligations as changing a public API. Teams that internalize this ship schema changes weekly without drama. Teams that don’t, ship them quarterly, at midnight, with a rollback script and a prayer.