Shedding Spatial Load by Geographic Priority

Classify streams by whether a later event supersedes an earlier one, shed only within that set, and rank by a configured area priority rather than by event density — prioritising by traffic volume encodes “busy” as “important” and degrades sparse regions invisibly, because they generate too few events to move a fleet-wide metric.

This guide sits under Backpressure & Flow Control for Spatial Consumers, within Queue Management, Retries & Delivery Guarantees. It is the escalation after pausing, and it should be rare enough that its rate is itself an alert.

When to use this pattern

  • Pausing has been sustained long enough that the backlog will not clear on its own, which is the precondition — shedding before pausing is throwing away work you had capacity for.
  • At least one stream in the mix is genuinely self-superseding, or there is nothing safe to shed.
  • The alternative is worse: a backlog that grows until the consumer is evicted, taking every stream with it.

Supersession first, geography second

Two filters, and the order matters Streams flowing into a saturated consumer are sorted by a first test: does a later event replace this one? Vehicle position pings pass — another arrives in seconds and is strictly more useful — as do fixed-cadence sensor readings and tile-render requests, which can be regenerated on demand. Cadastral boundary edits fail the test, as do geofence entry and exit transitions and dead-letter replays: nothing resends them, so shedding one is permanent loss with no recovery path. Only the streams that pass the first test reach the second, where a configured area priority decides which to drop first within them. Applying the filters in the other order — ranking everything by area and shedding the lowest-priority events regardless of stream — drops boundary edits from rural areas, which is data loss dressed up as a capacity decision. The area priority itself comes from configuration rather than from measured density, because ranking by event volume means the regions that generate the fewest events are always shed first, and their degradation never shows up in an aggregate. filter 1: does a later event replace this one? yes — sheddable vehicle position pings fixed-cadence sensor readings tile render requests the information arrives again on its own no — never shed cadastral boundary edits geofence entry and exit dead-letter replays nothing resends these — loss is permanent filter 2, applied only to the left column: configured area priority priority areas from configuration a business decision, written down why the order matters Rank everything by area first, and the lowest- priority events shed include rural boundary edits — data loss dressed as a capacity decision why priority is configured, not measured Ranking by event density means sparse regions are always shed first, and because they generate few events their degradation never moves an aggregate metric. "Busy" is not "important" — and encoding it that way makes the bias invisible to the dashboards.
Figure 1. Reversing the two filters produces a system that sheds rural boundary edits under load and reports it as successful capacity management.

Complete runnable implementation

python 
import time
from dataclasses import dataclass
from enum import IntEnum

import h3
from prometheus_client import Counter

SHED = Counter("events_shed_total", "Events deliberately dropped",
               ("stream", "area_class", "reason"))


class Supersession(IntEnum):
    """Whether a later event makes this one redundant."""
    SELF_SUPERSEDING = 0     # another arrives shortly and is strictly better
    NEVER = 1                # nothing will resend this


# Explicit per stream. A stream absent from this table is NEVER, because the
# safe default for "somebody added a topic and forgot" is to keep the data.
STREAM_POLICY: dict[str, Supersession] = {
    "vehicle.position": Supersession.SELF_SUPERSEDING,
    "sensor.reading": Supersession.SELF_SUPERSEDING,
    "tile.render": Supersession.SELF_SUPERSEDING,
    "feature.boundary_edit": Supersession.NEVER,
    "geofence.transition": Supersession.NEVER,
    "dlq.replay": Supersession.NEVER,
}

# Area priority from configuration — a business decision, not event density.
# Lower rank sheds first.
AREA_PRIORITY: dict[str, int] = {}     # H3 res-4 cell -> rank
DEFAULT_AREA_RANK = 50

# A ping already older than this is worthless: its successor has arrived.
STALE_AFTER_SECONDS = 30.0


@dataclass(frozen=True, slots=True)
class ShedDecision:
    shed: bool
    reason: str


def area_class(lat: float, lon: float) -> tuple[str, int]:
    cell = h3.latlng_to_cell(lat, lon, 4)
    return cell, AREA_PRIORITY.get(cell, DEFAULT_AREA_RANK)


def decide(stream: str, lat: float, lon: float, occurred_at: float,
           pressure: float, now: float | None = None) -> ShedDecision:
    """Shed or keep one event. `pressure` is 0.0-1.0 from the work budget."""
    now = now or time.time()
    policy = STREAM_POLICY.get(stream, Supersession.NEVER)

    # Filter 1. A stream whose events do not supersede is never shed, at any
    # pressure — the alternative to keeping it is permanent loss.
    if policy is Supersession.NEVER:
        return ShedDecision(False, "not-superseding")

    cell, rank = area_class(lat, lon)

    # A superseded event is free to drop regardless of pressure: its
    # replacement has already arrived, so processing it is pure waste.
    if now - occurred_at > STALE_AFTER_SECONDS:
        SHED.labels(stream=stream, area_class=cell, reason="stale").inc()
        return ShedDecision(True, "stale")

    # Filter 2. Under pressure, shed from the lowest-priority areas first.
    # Pressure 0.8 sheds rank > 80; pressure 1.0 sheds everything sheddable.
    if pressure > 0.5 and rank > (1.0 - pressure) * 100:
        SHED.labels(stream=stream, area_class=cell, reason="pressure").inc()
        return ShedDecision(True, "pressure")

    return ShedDecision(False, "kept")

The default of NEVER for an unknown stream is the decision that makes this safe to operate. A topic added by another team inherits the conservative policy rather than the convenient one, and the cost of that mistake is a slower consumer instead of missing cadastral edits.

Shedding graduates rather than switching on The proportion of sheddable events dropped is plotted against consumer pressure, split by area priority rank. Below a pressure of zero point five nothing is shed at all except events already stale — the consumer is under load but coping, and discarding work it has capacity for would be pure loss. As pressure rises past zero point five, the lowest-priority areas begin to shed: at pressure zero point seven, areas ranked above thirty are dropped, which in a typical configuration is the long tail of low-priority regions. At pressure zero point nine only areas ranked ten or better survive. At pressure one point zero every sheddable stream is dropped in every area, and only the never-shed streams are still processed — which is the intended floor, because those are the events that cannot be recovered. The graduation matters because a binary switch means the consumer is either wasting capacity or discarding a third of the fleet's telemetry, with nothing in between, and because a gradual curve makes the shed counter a usable pressure gauge rather than an alarm bell. proportion of sheddable events dropped, against consumer pressure 100% 0 pressure 0.0 0.5 1.0 rank > 30 — the long tail of low-priority areas rank 10–30 rank ≤ 10 — priority areas, shed last nothing shed but stale events — the consumer is loaded and coping A binary switch leaves the consumer either wasting capacity or discarding a third of the fleet's telemetry. The graduated curve also makes the shed counter a pressure gauge rather than an alarm bell.
Figure 2. Never-superseding streams are absent from this chart entirely — they sit at zero across the whole range, which is the property the first filter guarantees.

Parameter reference

Name Type Spatial constraint Default
STREAM_POLICY dict Explicit per stream; unknown streams default to NEVER —
AREA_PRIORITY dict H3 res-4 cell to rank, from configuration not density {}
DEFAULT_AREA_RANK int Applied to unconfigured areas; mid-range, not lowest 50
STALE_AFTER_SECONDS float Above the stream’s report interval, so only superseded events qualify 30.0
pressure float From the work budget, 0.0–1.0 —
Shed counter labels — Stream, area and reason — shedding must be countable —

Gotchas and spatial edge cases

  1. An unknown stream must default to never-shed. The convenient default is the dangerous one: a topic added by another team would silently become sheddable, and the first anyone knows is a gap in a dataset nobody was watching. The cost of the safe default is a consumer that sheds less than it could.

  2. Area rank at resolution 4 is coarse, deliberately. A configuration file listing millions of cells is not maintainable, and priority is a regional decision rather than a neighbourhood one. Cells at resolution 4 are roughly the size of a metropolitan area, which matches how the decision is actually made.

  3. Staleness is not the same as pressure shedding, and both need separate counters. A stale event is free to drop at any pressure because its successor has already arrived; a pressure shed is a deliberate sacrifice. Merging them into one counter makes it impossible to tell a healthy pipeline discarding superseded pings from one in distress.

  4. Shedding interacts with deduplication. An event shed at the consumer may already have claimed its deduplication key if the claim happens before dispatch, so the retry finds the key present and suppresses an event that was never processed. Shed before claiming, or release the key on shed — the same ordering problem described in Time-Windowed Deduplication for Moving Assets.

  5. The shed rate belongs in the error budget, not beside it. Shedding is deliberate incompleteness, so it spends the completeness objective defined in SLOs & Alerting for Spatial Webhook Pipelines. A pipeline that sheds routinely without that showing up in an objective has hidden a capacity problem in a feature.

  6. Sustained shedding is a capacity decision, not an operational one. If pressure sits above the shed threshold for hours every day, the correct response is more consumers or a finer partitioning, not a lower threshold. Alert on the duration of shedding rather than only on its occurrence.

Shed before the claim, or the retry is suppressed too An event arrives, claims its deduplication key, and is then shed under pressure. The claim is now held by an event that was never processed, and because the key's lifetime is the retry horizon, every redelivery of that event for the next several hours finds the key present and is suppressed as a duplicate. The shedding decision, which was meant to drop one instance of a self-superseding event, has instead dropped the event permanently — including the retry that the pipeline would otherwise have handled once pressure eased. Ordering the shed decision before the claim avoids it entirely: a shed event never touches the deduplication store, so a redelivery is treated as a first sighting and processed normally. If the ordering cannot be changed, releasing the key on shed achieves the same thing at the cost of one extra round trip and a failure mode when the release itself fails. claim, then shed claim dedup key shed under pressure every retry for hours finds the key and is suppressed a decision meant to drop one instance has dropped the event permanently shed, then claim shed decision a shed event never touches the dedup store a redelivery is a first sighting, and is processed If the ordering cannot be changed, release the key on shed — one extra round trip, and a new failure mode when the release itself fails.
Figure 3. Shedding and deduplication are independently correct and wrong together, which is why their order is part of the shedding design rather than an implementation detail.

Verification

python 
import time
import pytest

NOW = 1_780_000_000.0
BERLIN = (52.5200, 13.4049)


def test_boundary_edits_are_never_shed_at_any_pressure():
    """The property the first filter exists to guarantee."""
    for pressure in (0.0, 0.5, 0.9, 1.0):
        decision = decide("feature.boundary_edit", *BERLIN,
                          occurred_at=NOW, pressure=pressure, now=NOW)
        assert decision.shed is False


def test_unknown_stream_defaults_to_never_shed():
    """A topic somebody forgot to classify must keep its data."""
    decision = decide("some.new.topic", *BERLIN,
                      occurred_at=NOW, pressure=1.0, now=NOW)
    assert decision.shed is False


def test_stale_pings_are_shed_even_at_zero_pressure():
    """Its successor already arrived; processing it is pure waste."""
    decision = decide("vehicle.position", *BERLIN,
                      occurred_at=NOW - 120, pressure=0.0, now=NOW)
    assert decision.shed and decision.reason == "stale"


def test_priority_areas_survive_longer_than_default_ones():
    AREA_PRIORITY[h3.latlng_to_cell(*BERLIN, 4)] = 5
    priority = decide("vehicle.position", *BERLIN,
                      occurred_at=NOW, pressure=0.75, now=NOW)
    default = decide("vehicle.position", 48.1372, 11.5756,
                     occurred_at=NOW, pressure=0.75, now=NOW)
    assert not priority.shed and default.shed


def test_nothing_is_shed_below_the_pressure_floor():
    """Shedding work you have capacity for is loss, not management."""
    decision = decide("vehicle.position", *BERLIN,
                      occurred_at=NOW, pressure=0.4, now=NOW)
    assert decision.shed is False

The second test is the one worth keeping visible in review. It asserts a default rather than a behaviour, and defaults are what get changed by someone tidying a configuration table who does not know that the conservative value was the point.