Validating Schema Compatibility in CI

Run the candidate schema against every version still reachable in the log, then add the assertions a structural checker cannot make — coordinate order, CRS explicitness and precision are all type-preserving, so a generic compatibility gate passes the three changes most likely to relocate every geometry in the stream.

This guide sits under Schema Evolution & Versioning for Spatial Events, within Core Event Fundamentals & Architecture. It is the gate that makes Migrating a Spatial Stream Between Schema Versions a deliberate act rather than an accident discovered in production.

When to use this pattern

  • More than one service produces to the stream, so no single review catches every schema edit.
  • The schema lives in a repository and changes through pull requests, which is where a gate can sit.
  • Consumers exist that you do not deploy, so a breaking change cannot be walked back quickly.

What a structural checker sees, and what it misses

Compatibility checkers reason about types and field presence. That catches a removed required field and a narrowed type, and those are worth catching. The changes that hurt a spatial stream are not those.

The dangerous spatial changes are all type-preserving Five schema changes are listed with what a structural compatibility checker reports and what actually happens to the data. Removing a required field is correctly rejected, and widening an integer to a float is correctly accepted; those are the cases generic tooling exists for. The remaining three all pass structurally and all corrupt the stream. Flipping coordinate order from latitude-longitude to longitude-latitude leaves an array of two numbers exactly as it was, and moves every feature to a different hemisphere. Removing an implicit CRS default leaves a string field a string field, and turns every payload that omitted it from well-defined into ambiguous. Reducing stated coordinate precision from six decimal places to four is not a schema change at all — no field, type or constraint moves — and it changes every content hash in the pipeline, so deduplication stops matching across the boundary. A checker that reasons about structure cannot see any of the three, because none of them is structural. what the structural checker reports vs what happens to the data required field removed correctly REJECTED — this is what generic tooling is for int widened to float correctly ACCEPTED — genuinely compatible coordinate order flipped — [lat, lon] to [lon, lat] PASSES — an array of two numbers is still an array of two numbers · every feature moves hemisphere implicit CRS default removed PASSES — a string is still a string · every payload that omitted it becomes ambiguous coordinate precision reduced from 6 dp to 4 dp PASSES — not a schema change at all · every content hash moves, so deduplication stops matching
Figure 1. Three of the five are invisible to structural checking because none of them is structural. The spatial assertions below exist to cover exactly this row group.

Complete runnable implementation

The gate has two halves: a structural check against every reachable version, and a differential replay of real payloads through the old and new readers.

python 
"""ci/check_schema.py — run in CI; exits non-zero on an unsafe change."""
import json
import sys
from pathlib import Path

from shapely.geometry import shape

SCHEMA_DIR = Path("schemas")
CORPUS = Path("tests/corpus/payloads.jsonl")

# Every version still reachable: topic retention plus any replayable archive.
# NOT just the previous one — pairwise compatibility is not transitive.
REACHABLE = (1, 2)


class Incompatible(Exception):
    pass


def structural_check(old: dict, new: dict, path: str = "") -> list[str]:
    """Required fields may not vanish, and types may not narrow."""
    problems = []
    for name in old.get("required", []):
        if name not in new.get("required", []) and name not in new.get("properties", {}):
            problems.append(f"{path}{name}: required field removed")

    for name, spec in old.get("properties", {}).items():
        target = new.get("properties", {}).get(name)
        if target is None:
            continue
        if spec.get("type") != target.get("type"):
            problems.append(
                f"{path}{name}: type {spec.get('type')} -> {target.get('type')}"
            )
        if spec.get("type") == "object":
            problems += structural_check(spec, target, f"{path}{name}.")
    return problems


def spatial_check(new: dict) -> list[str]:
    """The assertions a structural checker cannot make.

    Each of these is type-preserving, so nothing generic will ever flag it.
    """
    problems = []
    props = new.get("properties", {})

    # 1. CRS must be explicit and required. An implicit default is a decision
    #    made by whichever service happened to write the payload.
    if "crs" not in new.get("required", []):
        problems.append("crs: must be required — an implicit CRS is not a contract")

    # 2. Coordinate order must be stated, and must be lon/lat per RFC 7946.
    order = props.get("geometry", {}).get("x-coordinate-order")
    if order != "lon,lat":
        problems.append(f"geometry: coordinate order must be 'lon,lat', got {order!r}")

    # 3. Precision is part of the contract because content hashes depend on it.
    if "x-coordinate-precision" not in props.get("geometry", {}):
        problems.append("geometry: x-coordinate-precision must be declared")

    return problems


def differential_replay(read_old, read_new) -> list[str]:
    """Feed real payloads to both readers and compare the GEOMETRY.

    A change can be structurally fine and still move every feature; only
    comparing the parsed geometry catches that.
    """
    problems = []
    for line in CORPUS.read_text().splitlines():
        payload = json.loads(line)
        try:
            a, b = read_old(payload), read_new(payload)
        except Exception as exc:                     # noqa: BLE001 - reported, not raised
            problems.append(f"{payload.get('feature_id')}: reader raised {exc!r}")
            continue

        moved = shape(a.geometry).distance(shape(b.geometry))
        if moved > 1e-9:
            problems.append(
                f"{payload.get('feature_id')}: geometry moved {moved:.6f} degrees"
            )
    return problems


def main() -> int:
    new = json.loads((SCHEMA_DIR / "event.v3.json").read_text())
    problems = spatial_check(new)

    for version in REACHABLE:
        old = json.loads((SCHEMA_DIR / f"event.v{version}.json").read_text())
        problems += [f"v{version}: {p}" for p in structural_check(old, new)]

    problems += differential_replay(read_v2, read_v3)

    for problem in problems:
        print(f"INCOMPATIBLE: {problem}", file=sys.stderr)
    return 1 if problems else 0


if __name__ == "__main__":
    raise SystemExit(main())
Pairwise compatible, transitively broken Three schema versions are checked pairwise. Version two renamed a field and provided a fallback that reads the old name, so it is compatible with version one. Version three removed that fallback, on the reasonable grounds that version two events all carry the new name, so it is compatible with version two. Each pull request passed its check and each check was correct. But a version one payload read by the version three reader finds neither the new name nor the fallback, so it fails — and version one payloads are still in the log for as long as retention lasts, and in the dead-letter archive for longer. The gap is invisible to a checker that compares each candidate only against its immediate predecessor, which is the default configuration of most compatibility tooling. Checking against every reachable version costs one loop and catches the composition. v1 field named "geom" still in the log and the archive compatible v2 renamed to "geometry", reads "geom" as a fallback compatible v3 fallback removed — v2 events all carry the new name v1 payload read by v3 — neither name present, and it fails Every pull request passed, and every check was correct Pairwise compatibility does not compose. Checking each candidate against every reachable version costs one loop.
Figure 2. Both individual checks were right. The gap only exists between them, which is why the loop over REACHABLE is not a nicety.

Parameter reference

Name Type Spatial constraint Default
REACHABLE tuple[int, ...] Every version in retention plus any replayable archive (1, 2)
x-coordinate-order str Must be "lon,lat" per RFC 7946; the flip is type-preserving "lon,lat"
x-coordinate-precision int Declared, because content hashes depend on it 6
Move tolerance float Degrees; 1e-9 is far below any real change and above float noise 1e-9
CORPUS path Real captured payloads, biased to awkward geometries —
Exit code int Non-zero fails the build; overriding requires an explicit version bump —

Gotchas and spatial edge cases

  1. A synthetic corpus proves nothing. A square in the first quadrant survives a coordinate flip looking merely translated, survives a precision change with no visible difference, and never exercises hole handling or antimeridian logic. Capture the corpus from production and include the antimeridian polygon, the multipolygon with holes, the empty geometry, the null geometry and the largest payload the stream has carried.

  2. distance() on geographic coordinates is in degrees, not metres. The 1e-9 tolerance above is roughly a tenth of a millimetre at the equator and about half that at 60° north. That asymmetry is fine for a “did it move at all” check and useless as a distance measurement — do not reuse the number as a spatial tolerance elsewhere.

  3. Compare geometries, not serialised payloads. Two readers can produce byte-different JSON for the same shape — key order, float formatting, an explicit bbox member — and a string comparison fails on all of it while missing the case where the bytes match and the interpretation differs.

  4. The corpus has to be regenerated. A corpus captured two years ago does not contain the payload shapes a producer added last quarter, so the gate is checking the schema against a stream that no longer exists. Refresh it on a schedule and fail the build if it is stale.

  5. Precision is part of the contract even though it is not in the type system. Reducing declared precision changes every content hash, so deduplication stops matching across the boundary exactly as described in Event Key Generation for Spatial Data. Declaring it as an extension keyword is what makes it reviewable.

A corpus of squares proves the gate runs, not that it works Two corpora are run through the same differential replay. The synthetic corpus is a few hundred axis-aligned squares in the first quadrant: under a coordinate-order flip every square merely appears translated and the geometry comparison may fall inside tolerance, precision reduction produces no visible difference at these magnitudes, and no square has an interior ring, crosses the antimeridian, or is empty — so three whole classes of reader bug are never executed. Every check passes, and the gate reports success while covering almost nothing. The real corpus is captured from production and biased towards awkward shapes: a polygon crossing the antimeridian, a multipolygon with holes, a geometry still in a projected CRS, an empty geometry, a feature whose geometry is null, and the largest payload the stream has ever carried. Under the same flip it fails immediately and loudly. The difference between the two runs is not the code being tested but whether the test data can express the failure. the same differential replay, two corpora synthetic — squares in the first quadrant a coordinate flip looks like a translation precision loss is invisible at this magnitude no holes · no antimeridian · no empty geometry three classes of reader bug never execute every check passes · nothing is covered captured from production, biased to the awkward antimeridian · holes · projected CRS empty · null geometry · the largest payload the flip fails immediately, and loudly the code under test is identical in both runs — only whether the data can express the failure differs This is why the corpus needs a refresh schedule of its own: a corpus captured two years ago cannot contain the payload shapes a producer added last quarter, so the gate is checking the schema against a stream that no longer exists.
Figure 3. A gate is only as good as the inputs it is asked to reject. A corpus that cannot express the failure turns the whole check into a smoke test.

Verification

The gate itself needs a test, because a compatibility checker that has never rejected anything is indistinguishable from one that always passes.

python 
import pytest


BASE = {
    "required": ["feature_id", "crs", "geometry"],
    "properties": {
        "feature_id": {"type": "string"},
        "crs": {"type": "string"},
        "geometry": {"type": "object", "x-coordinate-order": "lon,lat",
                     "x-coordinate-precision": 6},
    },
}


def test_gate_accepts_a_safe_addition():
    """A new optional field must not trip the gate."""
    new = {**BASE, "properties": {**BASE["properties"], "confidence": {"type": "number"}}}
    assert structural_check(BASE, new) == []
    assert spatial_check(new) == []


def test_gate_rejects_a_coordinate_flip():
    """The change no generic checker sees."""
    geom = {**BASE["properties"]["geometry"], "x-coordinate-order": "lat,lon"}
    new = {**BASE, "properties": {**BASE["properties"], "geometry": geom}}
    assert structural_check(BASE, new) == []          # structurally identical
    assert any("coordinate order" in p for p in spatial_check(new))


def test_gate_rejects_an_implicit_crs():
    """Removing crs from required is how an implicit default gets in."""
    new = {**BASE, "required": ["feature_id", "geometry"]}
    assert any("crs" in p for p in spatial_check(new))

The middle test is the one to keep in front of a reviewer: structural_check returns an empty list on a change that relocates every feature in the stream, and that is not a bug in the structural checker — it is the reason the spatial one exists.