Schema Evolution & Versioning for Spatial Events

A spatial event schema is safe to change only when you can say which of three categories the change falls into — compatible, breaking, or silently breaking — and geometry fields put changes in that third category far more often than scalar fields do, because a payload can stay structurally valid while the coordinates inside it come to mean something different.

This topic sits under Core Event Fundamentals & Architecture, the architectural baseline for spatial event systems. Schema changes touch nearly everything downstream: the deterministic keys in Event Key Generation for Spatial Data are computed from the payload’s fields, the routing decisions in Spatial Payload Routing & Parsing read a subset of them, and the canonical projection they assume is set by CRS Normalization Strategies.

Prerequisites

Confirm your stack meets this baseline before changing a live schema. Check off each item as you verify it:

The three kinds of change

Every schema change is one of three things, and the third is the one that causes incidents.

A compatible change is one an old consumer survives unchanged: adding an optional field, widening a numeric range, adding a new enum member that old consumers route to a default branch. A breaking change is one an old consumer fails on loudly: removing a required field, renaming one, tightening a type. These are unpleasant but manageable, because the failure is immediate and obvious.

A silently breaking change is one where every consumer keeps working, every payload validates, and the meaning has changed. Spatial schemas produce these far more readily than ordinary ones, because so much of a geometry’s meaning lives outside its structure: the projection the numbers are in, the axis order, the precision they have been rounded to, the winding direction of a polygon’s rings. None of that is visible to a type check.

Three kinds of schema change, and how each announces itself Schema changes sorted by how they fail. Compatible changes such as adding an optional confidence field or a new enum member are absorbed by old consumers, which keep working; the change announces itself only in the changelog. Breaking changes such as removing a required field or renaming geometry to geom cause an immediate validation error on the first event, so they are caught in staging within seconds. Silently breaking changes such as switching the canonical projection from EPSG:4326 to EPSG:3857, changing coordinate rounding from six to seven decimal places, or reversing polygon ring winding produce payloads that validate perfectly and mean something different, so no error is raised anywhere and the symptom appears days later as features in the wrong place or deduplication that has stopped matching. The third row is the one a schema registry cannot catch, because structural compatibility is exactly what it checks. Compatible — old consumers absorb it a new optional confidence field · a new enum member routed to a default branch · widening a numeric range announces itself in the changelog and nowhere else — which is fine, because nothing changed Breaking — old consumers fail immediately removing a required field · renaming geometry to geom · tightening a type a validation error on the first event — unpleasant, but you find it in staging within seconds Silently breaking — everything works and the meaning has changed canonical CRS EPSG:4326 → EPSG:3857 · rounding 6 d.p. → 7 d.p. · ring winding reversed · axis order swapped no error anywhere. The symptom arrives days later as features in the wrong place, or deduplication that stopped matching — and a schema registry cannot catch it, because structure is exactly what it checks.
Figure 1. The first two rows are handled by any schema registry. The third is not, because every change in it leaves the structure intact — which is why a spatial schema needs a version field even when the wire format already has one.

That last row is why a spatial event needs an explicit schema_version even under Protobuf or Avro. Those formats solve structural compatibility completely and semantic compatibility not at all.

Architecture: version at the envelope, dispatch at the consumer

The version belongs in the envelope, alongside the routing fields, not inside the geometry object. A consumer reads it before parsing anything else and dispatches to the model for that version; the geometry is only deserialised once the consumer knows which contract it is being offered.

Version dispatch: many wire schemas, one internal model An event arrives carrying schema_version in its envelope. The consumer reads that field before parsing anything else and selects the matching Pydantic model: version one, version two which added a confidence field, or version three which changed the canonical projection. Each model parses only the payloads it was written for, and each knows how to upgrade its own output into one shared internal representation, so the rest of the consumer is written once against that representation rather than branching on version at every call site. An unknown version does not fall through to a best guess; it is routed to the dead-letter queue with the version recorded, because a consumer that guesses at an unrecognised contract produces exactly the silent corruption the version field exists to prevent. The upgrade functions are also where a semantic change is absorbed — the version-three upgrade reprojects to the canonical CRS, so downstream code never learns that the producer changed. event envelope schema_version read before any parsing SpatialEventV1 the original contract SpatialEventV2 adds confidence — structurally compatible SpatialEventV3 canonical CRS changed — semantic unknown version → dead-letter never guess at an unrecognised contract upgrade() where a semantic change is absorbed, once one internal model no version branching
Figure 2. Version branching lives in exactly one place — the dispatch table and its upgrade functions. Everything downstream is written against the internal model, so a new version adds one class and one upgrade rather than a conditional at every call site.

Layer breakdown:

  1. Envelope read — schema_version is a flat field in the envelope, so it is readable without deserialising the geometry. This matters because an unknown version must be rejected before you spend CPU parsing a payload you cannot interpret.
  2. Model dispatch — a mapping from version to Pydantic model. Each model is a complete description of one wire contract; none of them use optional-everything to span versions, because a model that accepts all versions cannot tell you which one it received.
  3. Upgrade to internal — each version knows how to produce the current internal representation. A semantic change, such as a producer switching canonical projection, is absorbed here once rather than being handled at every downstream call site.
  4. Unknown version — routed to the dead-letter queue with the version recorded in the envelope, exactly as described in Dead-Letter Queues for Spatial Payloads. A consumer that guesses at an unrecognised contract produces the silent corruption the version field exists to prevent.

Step-by-step implementation

Step 1 — Put the version where it can be read cheaply

The version must be readable without parsing the geometry, which means it lives in the envelope next to the routing fields rather than inside the feature.

python 
from enum import StrEnum
from typing import Annotated, Literal

from pydantic import BaseModel, Field


class SchemaVersion(StrEnum):
    V1 = "spatial-event/1"
    V2 = "spatial-event/2"
    V3 = "spatial-event/3"


class Envelope(BaseModel):
    """The flat header every version shares. Parsed first, cheaply."""

    schema_version: SchemaVersion
    event_id: str
    occurred_at: str
    partition_key: str
    # The geometry is deliberately left as an opaque blob here: the envelope
    # must parse even when the body belongs to a version we cannot interpret,
    # so that the dead-letter record can carry a real version label.
    body: dict

Parsing in two stages — envelope, then body — is what lets an unknown version be dead-lettered with useful context instead of failing as an unparseable blob.

Step 2 — Model each version separately

The temptation is one model with every field optional. Resist it: such a model accepts all versions and can tell you nothing about which one arrived, so every downstream branch has to re-derive the version from which fields happen to be populated.

python 
from shapely.geometry import shape
from shapely.ops import transform
from pyproj import Transformer


class SpatialEventV1(BaseModel):
    schema_version: Literal[SchemaVersion.V1]
    feature_id: int
    geometry: dict          # GeoJSON, EPSG:4326, 6 d.p.


class SpatialEventV2(BaseModel):
    schema_version: Literal[SchemaVersion.V2]
    feature_id: int
    geometry: dict          # unchanged contract
    confidence: float = Field(ge=0.0, le=1.0)


class SpatialEventV3(BaseModel):
    schema_version: Literal[SchemaVersion.V3]
    feature_id: int
    geometry: dict          # EPSG:3857 metres — the semantic change
    confidence: float = Field(ge=0.0, le=1.0)

SpatialEventV3 is the interesting one. Structurally it is identical to V2; a schema registry comparing field names and types would call the change compatible. The difference is entirely in what the numbers mean, which is why it needs a version of its own.

Step 3 — Give every version an upgrade to one internal model

Downstream code should never branch on version. Each version knows how to become the current internal representation, and that is the only place the difference exists.

python 
_TO_4326 = Transformer.from_crs("EPSG:3857", "EPSG:4326", always_xy=True)


class SpatialEvent(BaseModel):
    """The internal model. Always EPSG:4326, always 6 d.p."""

    feature_id: int
    geometry: dict
    confidence: float


def upgrade_v1(e: SpatialEventV1) -> SpatialEvent:
    # V1 predates confidence; absent is not zero, it is "unknown". Using the
    # midpoint would invent information, so callers get an explicit sentinel.
    return SpatialEvent(feature_id=e.feature_id, geometry=e.geometry,
                        confidence=float("nan"))


def upgrade_v2(e: SpatialEventV2) -> SpatialEvent:
    return SpatialEvent(feature_id=e.feature_id, geometry=e.geometry,
                        confidence=e.confidence)


def upgrade_v3(e: SpatialEventV3) -> SpatialEvent:
    # The whole semantic change, absorbed once.
    from shapely.geometry import mapping

    geom = transform(_TO_4326.transform, shape(e.geometry))
    return SpatialEvent(feature_id=e.feature_id, geometry=mapping(geom),
                        confidence=e.confidence)

The upgrade_v1 case is worth dwelling on. When a field is added, old events do not have it, and the upgrade has to say what its absence means. Defaulting to zero would be wrong here — zero confidence is a real value that says “certainly not”, while the absence says “we did not measure”. Encoding one as the other is a data-quality bug that no test will catch, because both are valid floats.

Step 4 — Dispatch, and refuse to guess

python 
import structlog

log = structlog.get_logger()

_MODELS = {
    SchemaVersion.V1: (SpatialEventV1, upgrade_v1),
    SchemaVersion.V2: (SpatialEventV2, upgrade_v2),
    SchemaVersion.V3: (SpatialEventV3, upgrade_v3),
}


class UnknownSchemaVersion(Exception):
    """Raised for a version this consumer was not written against."""


def parse_event(raw: bytes) -> SpatialEvent:
    env = Envelope.model_validate_json(raw)
    entry = _MODELS.get(env.schema_version)
    if entry is None:
        # Deliberately not a best-effort parse. An unrecognised contract that
        # happens to validate against the newest model is the failure mode this
        # whole design exists to prevent.
        log.error("unknown_schema_version", version=env.schema_version,
                  event_id=env.event_id)
        raise UnknownSchemaVersion(env.schema_version)

    model, upgrade = entry
    return upgrade(model.model_validate(env.body | {"schema_version": env.schema_version}))

Step 5 — Roll readers out before writers

The ordering is not a style preference; it follows from which side would meet data it cannot interpret. Deploy consumers that understand the new version first, then producers that emit it. For a removal, reverse it: stop emitting first, drop the reader afterwards.

The overlap window is the deployment, not a step within it Adding version three runs in three stages. First every consumer is deployed with the version-three model and upgrade function, while producers still emit version two; nothing changes on the wire and the new code is exercised only by tests. Second, producers are moved to version three, usually gradually, so the stream carries a mixture and both models are in active use — this overlap window is the deployment itself rather than a step within it, and it must last at least as long as the dead-letter retention period, since a replayed event arrives carrying the version it was produced under. Third, once a per-version counter shows no version-two events arriving for a full retention window, the version-two model can be removed. Removing a version reverses the order: producers stop emitting it first, and readers drop it only after the same drain period. Skipping the wait is what turns a routine deprecation into a batch of dead-lettered replays. 1 · readers gain V3 producers still emit V2 — nothing changes on the wire new code exercised only by tests 2 · writers move to V3 the stream carries a mixture and both models are in active use this overlap IS the deployment 3 · retire V2 only once the per-version counter has read zero for a full window measured, not scheduled How long the overlap must last: at least your dead-letter retention A replayed event arrives carrying the version it was produced under, so retiring V2 early fails exactly the events that already failed once — and they fail a second time for a new reason, which makes the original cause much harder to find. Removals run the same three stages backwards Producers stop emitting first; readers drop support after the same drain period. The rule is unchanged — whichever side would meet data it cannot interpret goes last.
Figure 3. The overlap window is the deployment rather than an awkward interval inside it, and its length is set by your longest replay path — measure the old version draining rather than scheduling its removal.

Spatial validation and error handling

Three failure modes are specific to evolving a spatial schema.

A changed canonical projection passes every structural check. This is the case SpatialEventV3 models above. The defence is the version field plus a bounds assertion in the upgrade function: if a payload claiming EPSG:4326 carries coordinates outside ±180 and ±90, the producer is not sending what it says it is. That magnitude test is described in more detail under Webhook Security Boundaries.

A changed precision policy silently invalidates every derived key. Rounding coordinates to seven decimal places instead of six leaves a valid payload and changes every idempotency key computed from it, so deduplication stops matching across the boundary and the same feature is written twice. Precision is part of the schema; treat a change to it as a version bump.

A new geometry type breaks consumers that pattern-match. Adding MultiPolygon to a stream that previously carried only Polygon is structurally compatible — the field is still a GeoJSON object — but a consumer with an if geom["type"] == "Polygon" branch now silently skips those events. Enumerate accepted geometry types explicitly in the model so the addition fails loudly at validation rather than quietly at the branch.

python 
from typing import Literal

GeometryType = Literal["Point", "LineString", "Polygon", "MultiPolygon"]


def assert_canonical(geometry: dict, version: SchemaVersion) -> None:
    """Catch a producer whose declared version no longer matches its output."""
    if geometry["type"] not in GeometryType.__args__:
        raise ValueError(f"unsupported geometry type {geometry['type']!r}")

    if version in (SchemaVersion.V1, SchemaVersion.V2):
        # These versions promise EPSG:4326. Values past the degree range mean
        # the producer changed projection without changing its version.
        for lon, lat in _iter_coords(geometry):
            if abs(lon) > 180 or abs(lat) > 90:
                raise ValueError(
                    f"{version} promises EPSG:4326 but carries projected metres"
                )

Retry, backoff and delivery guarantees

An unknown schema version is a deterministic failure: it will fail identically on every redelivery, so it must not enter the retry ladder. Classify it as terminal and route it straight to the poison store described in Dead-Letter Queues for Spatial Payloads, which alerts on the first record rather than on a rate.

The subtlety is that an unknown version is often a temporary deterministic failure — it means a producer got ahead of its consumers, and the fix is a deploy rather than a payload correction. That makes those dead-lettered events genuinely replayable once the consumer catches up, unlike an invalid geometry, which will never succeed. Tag the dead-letter record with reason="unknown_schema_version" so the replay job can select exactly that class and re-inject it after the rollout completes, reusing the original idempotency key as Replaying Dead-Letter Spatial Events Safely describes.

Verification

The check that actually catches a bad schema change is a replay of recorded events against the new code, run in CI. A unit test written alongside the change tests what the author expected; a corpus of real payloads tests what producers actually send.

python 
import json
import pathlib

import pytest

CORPUS = pathlib.Path("_verify/events")


@pytest.mark.parametrize("path", sorted(CORPUS.glob("*.jsonl")))
def test_recorded_events_still_parse(path):
    """Every recorded production event must still parse and upgrade.

    The corpus is grouped by the version each file was captured under, so a
    version that has been retired fails here loudly rather than in production.
    """
    for line in path.read_text().splitlines():
        if not line.strip():
            continue
        event = parse_event(line.encode())
        assert event.geometry["type"] in GeometryType.__args__
        for lon, lat in _iter_coords(event.geometry):
            assert abs(lon) <= 180 and abs(lat) <= 90


def test_unknown_version_is_rejected_not_guessed():
    """The gate's own negative case: an unrecognised version must not parse.

    Without this, a dispatch table that silently falls back to the newest model
    would pass every other test in this file.
    """
    raw = json.dumps({
        "schema_version": "spatial-event/99",
        "event_id": "evt-1", "occurred_at": "2026-08-08T00:00:00Z",
        "partition_key": "8828308281fffff",
        "body": {"feature_id": 1, "geometry": {"type": "Point", "coordinates": [13.4, 52.5]}},
    }).encode()
    with pytest.raises((UnknownSchemaVersion, ValueError)):
        parse_event(raw)

The second test is the one worth insisting on. A dispatch table that quietly falls back to the newest model passes every positive test in the file and reintroduces exactly the silent misinterpretation the version field was added to prevent.

Troubleshooting

Symptom Likely spatial cause Fix
Deduplication hit rate drops to near zero after a deploy Coordinate precision or field set changed, so derived keys no longer match Hash over an explicit field list, and bump the schema version when precision changes
Features render in the wrong place, no errors anywhere Producer changed canonical CRS without a version bump Add the magnitude assertion to the upgrade path; treat projection as part of the schema
A consumer silently processes fewer events than it receives A new geometry type was added and a type == branch skips it Enumerate accepted geometry types in the model so the addition fails at validation
Dead-letter queue fills with unknown_schema_version right after a release Producers deployed before consumers Roll readers first; re-inject the dead-lettered batch once consumers are current
Replayed events from last week now fail An old schema version was retired before the dead-letter retention window elapsed Restore the retired model, drain, then retire on a measured zero rather than a date
confidence reads 0.0 for old events that never carried it An added field was defaulted rather than marked unknown Distinguish absent from zero in the upgrade function

FAQ

Do I need a schema version field if I use Protobuf or Avro?

Yes, for a different reason than you need it in JSON. Protobuf and Avro handle structural compatibility for you — an unknown field is preserved, a missing optional field gets its default — but they cannot express semantic changes, which is where spatial schemas actually break. Changing the canonical CRS from EPSG:4326 to EPSG:3857, or changing coordinate rounding from six decimal places to seven, leaves the wire format identical and changes what the numbers mean. An explicit schema version is the only thing that lets a consumer detect that.

Is adding an optional field always a safe change?

Structurally yes, operationally no. Old consumers ignore the field and keep working, so nothing breaks. But if any consumer derives an idempotency key or a content hash from the whole payload, the added field changes that hash, so the same logical event now produces a different key and deduplication stops matching across the rollout boundary. Hash over an explicit field list rather than the whole payload, and adding a field becomes genuinely safe.

How long should I support an old schema version?

At least as long as your longest replay path, which is normally dead-letter retention. A dead-lettered event replayed a week later arrives carrying the schema version it was produced under, so a consumer that dropped support for it will fail on exactly the events that already failed once. Tie the support window to dead-letter retention rather than picking a duration, and instrument a counter per version so you can see when the old one genuinely stops arriving.

Can I change the coordinate precision policy without a version bump?

No — this is the classic silently breaking change. Rounding coordinates to a different number of decimal places produces a valid payload that passes every schema check while changing every derived key: the idempotency hash, the deduplication key, and any content hash used for tile invalidation. Consumers see a stream in which nothing matches anything from before the change, and the symptom is duplicate writes rather than an error. Treat precision as part of the schema and bump the version with it.

Explore this section

Adding a Field to a Live Spatial Event Schema

Read more

Migrating a Spatial Stream Between Schema Versions

Read more

Validating Schema Compatibility in CI

Read more