Deterministic Idempotency Keys for GeoJSON Events

To generate a deterministic idempotency key for a GeoJSON event: strip delivery-envelope fields, recursively normalize coordinate float precision, sort all dictionary keys alphabetically, compact-serialize the result, then hash the canonical UTF-8 string with SHA-256 or BLAKE2b. This produces an identical digest for every structurally equivalent payload regardless of serializer, retry count, or minor formatting drift.

This page is part of Event Key Generation for Spatial Data, which sits within the Idempotency & Spatial Deduplication architecture — the reference section for building duplicate-safe spatial webhook pipelines.

When to use this pattern

Apply this canonicalization-before-hashing approach when:

  • Your webhook provider may deliver the same event multiple times (at-least-once guarantees) and you need to filter duplicates without a unique ID from the sender.
  • You consume GeoJSON from multiple upstream systems — different languages, ORMs, or HTTP clients — that serialize coordinate arrays with varying decimal precision or key insertion order.
  • You need to compare spatial events across delivery attempts where ephemeral envelope fields (delivery_id, received_at, x-signature) must not influence the key.

It is not the right tool when the webhook provider already guarantees a stable, opaque event ID per logical event — in that case, store that ID directly rather than computing a content digest.

Why naive hashing breaks on spatial payloads

The JSON specification (RFC 8259) defines object key ordering as insignificant, yet serializers preserve insertion order. A webhook provider retrying a failed delivery might send {"type":"Feature","geometry":{...}} the first time and {"geometry":{...},"type":"Feature"} on the second attempt. Hashing both raw strings yields different digests, triggering double processing, state overwrites, or corrupted spatial indexes.

GeoJSON compounds this further. Coordinates are deeply nested float arrays, and different languages round IEEE-754 values differently during JSON serialization. The coordinate -122.4194155 may arrive as -122.41941550000001 from a Java client even though the values represent the same geographic point. Without rounding to a shared precision, your event bus treats two deliveries of the same sensor ping as distinct messages.

Robust idempotency and spatial deduplication requires treating the payload as a mathematical object rather than a raw byte stream. The four-step canonicalization pipeline below eliminates all sources of variance.

Canonicalization pipeline — data flow

The diagram below shows how a raw webhook body moves through each transformation stage before reaching the hash function.

GeoJSON idempotency key generation pipeline A five-stage data-flow diagram: raw GeoJSON payload passes through an envelope stripper, then a float normaliser, then an alphabetical key sorter with compact serialisation, and finally a SHA-256 or BLAKE2b hash function that outputs the hex idempotency key. Raw GeoJSON webhook body (dict or string) INPUT 1. Strip Envelope delivery_id received_at … STAGE 1 2. Float Normalise round(coord, 8) recursive STAGE 2 3. Sort Keys + Compact sort_keys=True separators=(",",":") STAGE 3 4. Hash SHA-256 / BLAKE2b hex digest → idempotency key OUTPUT
Figure 1. GeoJSON idempotency key generation pipeline

Complete runnable implementation

The function below is self-contained and uses only the Python standard library. It accepts a raw dict or a JSON string, making it drop-in safe for FastAPI request handlers, Celery tasks, or asyncio message consumers. Before canonicalizing, strip any envelope fields that vary per delivery (webhook gateway metadata, timestamps, signatures) — only the semantic payload should influence the key.

python 
import json
import hashlib
from typing import Any, Union


def _normalize_floats(obj: Any, precision: int) -> Any:
    """
    Recursively round floats to a fixed decimal precision.

    GeoJSON coordinates (EPSG:4326 / WGS84) are deeply nested float arrays.
    Different serializers (Python, Java, Go) produce slightly different
    IEEE-754 string representations for the same geographic value, e.g.
    -122.4194155 vs -122.41941550000001. Rounding to a shared precision
    before hashing neutralises this drift without losing spatial fidelity.
    RFC 7946 recommends 6 decimal places (~0.11 m); use 7–8 for sub-metre work.
    """
    if isinstance(obj, float):
        return round(obj, precision)
    if isinstance(obj, list):
        return [_normalize_floats(v, precision) for v in obj]
    if isinstance(obj, dict):
        return {k: _normalize_floats(v, precision) for k, v in obj.items()}
    return obj  # int, str, bool, None — pass through unchanged


# Fields that vary per delivery attempt and must be excluded before hashing.
# Extend this set for your gateway's envelope schema.
_EPHEMERAL_FIELDS = frozenset({
    "webhook_id", "delivery_id", "received_at", "timestamp",
    "x-signature", "x-hub-signature-256", "attempt",
})


def strip_ephemeral(payload: dict) -> dict:
    """Remove top-level delivery-envelope keys that change on every retry."""
    return {k: v for k, v in payload.items() if k not in _EPHEMERAL_FIELDS}


def generate_geojson_idempotency_key(
    payload: Union[dict, str],
    precision: int = 8,
    algorithm: str = "sha256",
    truncate_bytes: int = 0,
) -> str:
    """
    Return a deterministic hex digest for a GeoJSON payload.

    Args:
        payload:        Raw GeoJSON dict or a JSON string. Mixed CRS payloads
                        should be normalised to EPSG:4326 upstream before
                        calling this function.
        precision:      Decimal places for coordinate rounding (6–8 typical).
                        Values below 6 collapse distinct geographic points.
        algorithm:      Any hashlib name: 'sha256' (safe default) or 'blake2b'
                        (2–4× faster, equally collision-resistant for this use).
        truncate_bytes: Trim digest to this many bytes before hex-encoding.
                        16 bytes (32 hex chars) is safe for practical event
                        volumes while halving index storage.

    Returns:
        Lowercase hex string of the digest (or truncated digest).
    """
    if isinstance(payload, str):
        payload = json.loads(payload)

    # 1. Strip delivery-envelope metadata that changes per attempt.
    payload = strip_ephemeral(payload)

    # 2. Normalize floating-point coordinate precision to remove IEEE-754 drift.
    normalized = _normalize_floats(payload, precision)

    # 3. Produce a canonical JSON string: alphabetically sorted keys, no whitespace.
    #    ensure_ascii=False preserves multi-byte property values (e.g. place names)
    #    without percent-encoding, which could introduce encoding variance.
    canonical = json.dumps(
        normalized,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=False,
    )

    # 4. Hash the canonical UTF-8 bytes.
    h = hashlib.new(algorithm, canonical.encode("utf-8"))
    digest = h.digest()

    if truncate_bytes > 0:
        digest = digest[:truncate_bytes]

    return digest.hex()

Usage example

python 
# Two representations of the same feature — different key order, float drift.
delivery_1 = {
    "webhook_id": "wh-001",          # ephemeral — stripped before hashing
    "type": "Feature",
    "properties": {"sensor_id": "SN-42", "reading": 17.3},
    "geometry": {
        "type": "Point",
        "coordinates": [-73.96535500000001, 40.78286500000002],  # float drift
    },
}

delivery_2 = {
    "webhook_id": "wh-002",          # different delivery envelope
    "geometry": {                     # keys in a different order
        "coordinates": [-73.965355, 40.782865],
        "type": "Point",
    },
    "properties": {"sensor_id": "SN-42", "reading": 17.3},
    "type": "Feature",
}

key1 = generate_geojson_idempotency_key(delivery_1, precision=7, truncate_bytes=16)
key2 = generate_geojson_idempotency_key(delivery_2, precision=7, truncate_bytes=16)

assert key1 == key2  # identical digest — safe to deduplicate
print(key1)          # e.g. '3f8a92b1c4e7d05a' (32 hex chars)

Parameter reference

Parameter Type Spatial constraint Default
payload dict | str Must be valid GeoJSON or a superset; geometry must be pre-projected to EPSG:4326 if you need cross-source deduplication —
precision int 6 = ~0.11 m (RFC 7946 minimum); 8 = ~1.1 mm; do not exceed 10 (amplifies float noise) 8
algorithm str Any hashlib name; sha256 or blake2b recommended; avoid MD5/SHA-1 "sha256"
truncate_bytes int 0 = full digest; 16 = 128-bit, safe for ≤ 10⁹ events; 32 = full SHA-256 0

Gotchas and spatial edge cases

  1. Precision below 6 collapses distinct points. Rounding to 5 decimal places creates a grid cell ~1.1 m wide. Two sensor readings from opposite sides of a road merge to the same canonical form and produce the same key even though they represent different physical events. Use at least 6 decimal places for EPSG:4326 (WGS84) data.

  2. Mixed CRS payloads break cross-source deduplication. A feature in EPSG:3857 (Web Mercator) and the same feature in EPSG:4326 will produce different coordinate arrays and therefore different keys, even after float normalization. Normalize all payloads to a single CRS — ideally EPSG:4326 as required by RFC 7946 — before canonicalization. The approach in handling mixed CRS payloads in Python event handlers covers reprojection before this step.

  3. Coordinate ring orientation differences (Polygon winding order). RFC 7946 mandates counter-clockwise exterior rings, but not all producers comply. A clockwise and a counter-clockwise representation of the same polygon produce identical geometry but different coordinate arrays, yielding different digests. Either validate and normalize ring orientation before hashing, or document that your key covers the serialized form, not the geometric shape.

Does the key identify the bytes or the shape? The same square is described two ways: vertices listed counter-clockwise starting at the south-west corner, and the identical square listed clockwise starting at the north-east. The two are geometrically equal and serialise differently, so the choice is which property the key is supposed to capture. A serialization key hashes the canonical string as-is: it is cheap, it is exactly reproducible, and it treats the two as different events, which is correct if the producer is a single system that never varies its winding. A geometry key normalises ring orientation and vertex start before hashing, so both forms collapse to one key; it costs an orientation pass and it is the right choice when several upstream systems describe the same feature. What must not happen is choosing implicitly. Whichever is picked belongs in the API contract, because a consumer that assumes the other one will either process duplicates or discard legitimate updates, and neither failure announces itself. CCW from SW CW from NE geometrically equal · different bytes Serialization key hash the canonical string as-is → two keys, two events cheap · exactly reproducible right when one producer owns the feed wrong the moment a second system joins Geometry key orient + normalise start, then hash → one key, one event costs an orientation pass per payload right when several systems describe the same feature Either is defensible. Choosing by accident is not. Put the choice in the API contract. A consumer expecting a geometry key while you ship a serialization key processes the same parcel twice; one expecting the reverse discards a legitimate re-survey as a duplicate. Neither failure raises anything — both just change what is on the map.
Figure 2. Winding order forces a decision about what the key is for. Document which one you shipped: the two behave identically until a second upstream system starts describing the same features.
  1. null geometry must be preserved, not filtered. GeoJSON features can have "geometry": null (RFC 7946 §3.2). The recursive normalizer preserves None → null correctly; do not replace it with an empty dict or the key will shift.
null geometry is a value, not an absence RFC 7946 permits a Feature to carry a geometry of null, which means the feature is known but unlocated — a parcel record entered before survey, or a sensor whose fix has been lost. Three plausible normaliser behaviours each change the canonical bytes and therefore the key. Replacing null with an empty object produces a different digest, so a redelivery of the same unlocated feature is treated as novel. Dropping the geometry member entirely produces a third digest and additionally makes the payload invalid GeoJSON. Coercing it to an empty Point invents a location at zero degrees north and zero degrees east, in the Gulf of Guinea, which is worse than losing the key because the feature now claims a position it never had and will match spatial queries over that area. Preserving null as null is the only behaviour that keeps the key stable across redeliveries and keeps the record honest about what is unknown. Input: {"type": "Feature", "geometry": null, "properties": {…}} — valid RFC 7946 null → {} "geometry": {} key shifts — 41ba6d… every redelivery of the same feature looks novel member dropped (no geometry key) key shifts — c70e83… and the payload is no longer valid GeoJSON null → Point(0, 0) [0.0, 0.0] worse than a key shift invents a location in the Gulf of Guinea null → null "geometry": null key stable — 9f2c1a… the record stays honest about what is unknown Why the Point(0, 0) coercion is the dangerous one The other two lose deduplication and you notice: the same feature keeps arriving as new. This one succeeds quietly and gives the feature a position it never had, so it enters spatial joins and bounding-box queries over the Gulf of Guinea as though it belonged there.
Figure 3. "Unlocated" and "at the origin" are different facts, and only one of them is true. A recursive normaliser that treats None as missing rather than as a value converts the first into the second.

  1. Unicode in property strings. ensure_ascii=False lets property values like "name": "São Paulo" or "区域": "北京" serialize as UTF-8 rather than \uXXXX escape sequences. Mixing ensure_ascii=True and ensure_ascii=False across services produces different byte strings for the same semantic content — pick one and enforce it consistently.

  2. Numeric property values are also normalized. _normalize_floats rounds floats inside properties as well as geometry.coordinates. If a property value is a measurement like 17.30000000000001 vs 17.3, they will match after rounding. If you do not want property floats normalized, adapt the function to restrict rounding to coordinate paths only.

  3. Algorithm version changes invalidate all stored keys. If you migrate from SHA-256 to BLAKE2b, existing keys in Redis or PostgreSQL will not match newly computed ones. Version the algorithm in your key store (e.g. prefix with sha256: or store an algorithm column), so you can run both in parallel during rollover.

Verification snippet

Paste this into a test file and run with pytest:

python 
import pytest
from your_module import generate_geojson_idempotency_key  # adjust import path


FEATURE_A = {
    "type": "Feature",
    "properties": {"id": 1},
    "geometry": {"type": "Point", "coordinates": [-73.965355, 40.782865]},
}

# Same feature — key order shuffled, float drift on coordinates, delivery envelope added.
FEATURE_B = {
    "delivery_id": "retry-99",
    "geometry": {"coordinates": [-73.96535500000001, 40.78286500000002], "type": "Point"},
    "properties": {"id": 1},
    "type": "Feature",
}

# Genuinely different feature — same geometry, different property.
FEATURE_C = {
    "type": "Feature",
    "properties": {"id": 2},  # id differs
    "geometry": {"type": "Point", "coordinates": [-73.965355, 40.782865]},
}


def test_retry_produces_same_key():
    """Key must be identical for semantically equivalent retried deliveries."""
    assert (
        generate_geojson_idempotency_key(FEATURE_A)
        == generate_geojson_idempotency_key(FEATURE_B)
    )


def test_different_feature_produces_different_key():
    """Changing a property value must change the digest."""
    assert (
        generate_geojson_idempotency_key(FEATURE_A)
        != generate_geojson_idempotency_key(FEATURE_C)
    )


def test_truncation_produces_shorter_key():
    full = generate_geojson_idempotency_key(FEATURE_A)
    short = generate_geojson_idempotency_key(FEATURE_A, truncate_bytes=16)
    assert len(short) == 32          # 16 bytes → 32 hex chars
    assert full.startswith(short)    # truncation takes the leading bytes


def test_accepts_json_string():
    import json
    as_str = json.dumps(FEATURE_A)
    assert (
        generate_geojson_idempotency_key(as_str)
        == generate_geojson_idempotency_key(FEATURE_A)
    )


def test_precision_boundary():
    """Points that differ only beyond the precision threshold collapse to the same key."""
    near_a = {
        "type": "Feature", "properties": {},
        "geometry": {"type": "Point", "coordinates": [-73.9653550001, 40.7828650001]},
    }
    near_b = {
        "type": "Feature", "properties": {},
        "geometry": {"type": "Point", "coordinates": [-73.9653550002, 40.7828650002]},
    }
    # At precision=8 these differ; at precision=6 they collapse.
    assert (
        generate_geojson_idempotency_key(near_a, precision=6)
        == generate_geojson_idempotency_key(near_b, precision=6)
    )
    assert (
        generate_geojson_idempotency_key(near_a, precision=8)
        != generate_geojson_idempotency_key(near_b, precision=8)
    )

Storing and checking keys in production

Once you have a key, store it in a low-latency idempotency store before processing the event. Using Redis to cache spatial webhook signatures walks through the SET NX PX atomic check-and-set pattern that prevents race conditions under concurrent delivery. Set the TTL to match your webhook provider’s retry window — typically 24–72 hours — and log the key alongside the canonical string length and algorithm version to simplify debugging during payload format migrations.

For payloads where the same geographic boundary may legitimately trigger multiple distinct events (zone entry, sensor threshold breach, status change), add an event_type field to the canonical form before hashing so semantically different events at the same geometry produce different keys.

Frequently asked questions

Why can’t I just hash the raw JSON string from the webhook body?

Raw JSON strings carry insertion-order variance, float representation drift, and whitespace differences across serializers. Two payloads that are semantically identical will produce different digests if key order or decimal precision differs, causing false duplicate misses and double-processing.

How many decimal places should I use for WGS84 coordinates?

RFC 7946 recommends 6 decimal places (~0.11 m resolution). Use 7–8 if you need centimetre-level fidelity. Go beyond 8 only when your source data genuinely carries that precision — extra digits amplify float serialization noise without adding geographic accuracy.

Is SHA-256 or BLAKE2b better for webhook idempotency keys?

Both are collision-resistant for this use case. BLAKE2b is 2–4× faster on modern CPUs and is available in Python’s standard library via hashlib.blake2b. SHA-256 is more universally recognised and works in all compliance contexts. Pick SHA-256 when auditability matters; BLAKE2b for high-throughput sub-millisecond paths.

What if the same geographic boundary triggers multiple distinct events?

Include an event_type field (e.g. "zone_entry", "threshold_breach") in the canonical form before hashing. This ensures semantically different events at the same geometry produce different keys and are not collapsed as duplicates by your idempotency store.