Migrating a Topic to a New H3 Resolution

Carry the resolution in the envelope so every key is self-describing, keep consumer state under both schemes during the overlap, and cut over one parent cell at a time — a resolution change replaces every partition key simultaneously, which orphans all per-cell state and is a data migration rather than a configuration change.

This guide sits under Spatial Partitioning Strategies, within Core Event Fundamentals & Architecture. It is what to do once Choosing an H3 Resolution from Measured Traffic says the current resolution has aged out.

When to use this pattern

  • Measured traffic says the hottest cell no longer fits a consumer, or that cardinality has grown past what the monitoring stack can carry.
  • Consumers hold per-cell state — counters, last-seen positions, deduplication sets, tile version maps.
  • The stream cannot be paused, which is the case that makes this hard; a stream that can be drained needs none of this.

What a resolution change actually breaks

Seven new keys, and none of them is the old one A consumer holds state under a resolution 7 cell: a running event count of two point one million, a last-seen timestamp, and a set of deduplication keys. The resolution changes to 8, and that parent cell is replaced by seven child cells whose identifiers share no prefix a lookup can use — H3 identifiers are not hierarchical strings, so finding the parent of a cell requires a library call rather than a substring. The next event arrives keyed to one of the children. The consumer looks up that child, finds nothing, and initialises fresh state: the count restarts at one, the last-seen timestamp is now, and the deduplication set is empty so every event in the retry window is admitted a second time. Nothing raises. The count graph shows a cliff to zero that looks exactly like an outage, the deduplication failure appears as a burst of duplicate writes with no error attached, and both are attributed to whatever else happened to deploy that day. The fix is to fold the parent's state into its children before the first child event arrives, which requires knowing the mapping — which the H3 library provides and a substring comparison does not. state under res-7 cell 871f1d4ffff event count: 2 100 000 last seen: 12:04:31Z dedup key set: 48 000 entries tile version map: 900 tiles res 7 → 8 seven res-8 children 881f1d4d61fffff …and six more no shared prefix a lookup can use — H3 ids are not hierarchical strings the parent is a library call away what the consumer does next looks up the child · finds nothing count restarts at 1 · last seen = now dedup set empty — the retry window is admitted a second time Nothing raises, which is why this is worth the ceremony The count graph shows a cliff to zero that is indistinguishable from an outage. The deduplication failure appears as a burst of duplicate writes with no error attached to it. Both get attributed to whatever else deployed that day. Fold the parent's state into its children BEFORE the first child event arrives — the mapping is h3.cell_to_children(), not a substring.
Figure 1. The state is not corrupted; it is unreachable, filed under a key nothing will ever ask for again. That distinction is what makes it survivable if handled before the cutover.

Complete runnable implementation

python 
import h3
from prometheus_client import Counter

MIGRATED = Counter("h3_state_migrated_total", "Parent cells folded into children")

OLD_RESOLUTION = 7
NEW_RESOLUTION = 8


def partition_key(lat: float, lon: float, resolution: int) -> bytes:
    """The key, and the resolution that produced it, together.

    Encoding the resolution means a consumer reading a key can tell which
    scheme it belongs to without consulting configuration that may have
    changed since the event was written.
    """
    return f"{resolution}:{h3.latlng_to_cell(lat, lon, resolution)}".encode()


def parse_key(key: bytes) -> tuple[int, str]:
    resolution, cell = key.decode().split(":", 1)
    return int(resolution), cell


class DualResolutionState:
    """Per-cell state that can answer under either scheme during the overlap."""

    def __init__(self, store, cutover: set[str]) -> None:
        self._store = store
        # Parent cells (at OLD_RESOLUTION) already migrated. Cutting over by
        # parent cell means a parent and its children are never both live.
        self._cutover = cutover

    def resolution_for(self, lat: float, lon: float) -> int:
        parent = h3.latlng_to_cell(lat, lon, OLD_RESOLUTION)
        return NEW_RESOLUTION if parent in self._cutover else OLD_RESOLUTION

    async def migrate_parent(self, parent: str) -> None:
        """Fold one parent's state into its children, then mark it cut over.

        Runs BEFORE any event is keyed to a child, so no consumer ever sees
        an empty child. Order matters more than speed here.
        """
        state = await self._store.get(f"{OLD_RESOLUTION}:{parent}")
        if state is None:
            self._cutover.add(parent)
            return

        children = h3.cell_to_children(parent, NEW_RESOLUTION)

        # Counts must be SPLIT, not copied: copying a parent's 2.1M count into
        # seven children invents 12.6M events. Splitting is also approximate —
        # which is why a count that must be exact should be recomputed rather
        # than folded, and the metric below exists to make the choice visible.
        share = state.get("event_count", 0) // len(children)

        for child in children:
            await self._store.set(f"{NEW_RESOLUTION}:{child}", {
                "event_count": share,
                "last_seen": state.get("last_seen"),        # copied: per-area fact
                "dedup_keys": state.get("dedup_keys", []),  # copied: a superset is safe
                "migrated_from": parent,
            })

        self._cutover.add(parent)
        MIGRATED.inc()
        # Delete the parent only after the children are durable. The reverse
        # order loses everything if the process dies between the two.
        await self._store.delete(f"{OLD_RESOLUTION}:{parent}")

Copying the deduplication set into every child is deliberately over-inclusive: a child inherits keys for events that happened in its siblings, which can only cause extra suppression, never extra admission. Erring in the other direction admits duplicates, so the asymmetry decides the design.

Blast radius is the only real difference Two rollout shapes are compared for the same resolution change. A fleet-wide cutover flips every producer at once: every consumer rebalances simultaneously, every piece of per-cell state migrates simultaneously, and the whole stream is exposed to any mistake in the fold logic at the same moment. Recovery means reverting producers and hoping the old state was not deleted. A parent-cell rollout migrates one region at a time — fold that parent's state, add it to the cutover set, watch its consumers for a reporting period, then take the next. Any mistake affects one region, the rollback is removing one entry from a set, and the fold logic is exercised against real data dozens of times before the busiest region reaches it. The total work is the same and the elapsed time is longer; what changes is that a bug is a rollback instead of an incident, which for a change that silently resets counters is the difference that matters. fleet-wide cutover every producer flips at one instant every consumer rebalances at once every cell's state migrates at once any error in the fold hits the whole stream rollback = revert producers and hope the old state was not already deleted fastest, and untestable at scale one parent cell at a time fold that parent · add it to the cutover set watch its consumers for a reporting period take the next region an error affects one region rollback = remove one entry from a set the fold runs dozens of times before the busiest region reaches it Same total work, longer elapsed time. What changes is that a bug is a rollback rather than an incident — which for a change whose failure mode is a counter silently resetting to zero is the only difference worth paying for.
Figure 2. Ordering the regions from quietest to busiest means the fold logic has been exercised against real traffic many times before it meets the region that matters.

Parameter reference

Name Type Spatial constraint Default
Key format bytes "<resolution>:<cell>" — self-describing, so a key needs no config to read —
Cutover unit H3 cell A parent at OLD_RESOLUTION; never a bounding box or a country name —
cell_to_children call Exactly 7 children per step, except around the 12 pentagons —
Count fold strategy Split, never copy; copying multiplies totals by the child count split
Dedup fold strategy Copy to every child — over-suppression is safe, over-admission is not copy
Delete order — Children durable before the parent is removed —

Gotchas and spatial edge cases

  1. H3 identifiers are not hierarchical strings. A child does not begin with its parent’s identifier, so no prefix scan, no LIKE query and no key-range operation will find related cells. Every relationship needs cell_to_parent or cell_to_children, which means the migration cannot be done in the database with a wildcard.

  2. Pentagons do not have seven children. Twelve cells per resolution are pentagonal and produce six children instead of seven. Code that hard-codes seven — to split a count, to size a batch, to assert a test — is wrong exactly twelve times per resolution, in places that are usually ocean and occasionally not.

  3. Splitting a count is an estimate and should be labelled as one. Seven equal shares of a parent’s total is almost certainly not how the events were distributed. If a count feeds billing or reporting, recompute it from the source rather than folding it, and use the fold only for state that is merely an optimisation.

  4. A moving asset can cross the cutover boundary mid-journey. With region-by-region rollout, a vehicle driving from a migrated region into an unmigrated one produces events keyed at resolution 8 then resolution 7. Consumers must handle both, which is what the resolution prefix is for — and it is why the prefix cannot be dropped the moment the last region cuts over.

  5. Partition count and cell count are different numbers. Changing the resolution does not change how many Kafka partitions exist; it changes how keys hash into them. A finer resolution spreads load better only if the partition count is high enough to receive it, so check both together against Detecting Partition Skew in H3-Sharded Streams.

  6. Events already in the log keep their old keys forever. A replay after the migration reads resolution 7 keys, so the old reader path must survive as long as the retention window and any replayable archive — the same rule as Migrating a Spatial Stream Between Schema Versions.

An asset crossing the cutover boundary produces both key shapes With a region-by-region rollout, some parent cells have been cut over to resolution 8 and others have not. A vehicle drives from a migrated metropolitan region into an unmigrated rural one, so its events are keyed at resolution 8 for the first part of the journey and at resolution 7 for the second — from the same producer, within the same minute, for the same asset. Every consumer must therefore be able to read both shapes for as long as the rollout lasts, which is what the resolution prefix on the key is for: a consumer parsing a key can tell which scheme produced it without consulting configuration that may have changed since. The prefix also cannot be dropped when the last region cuts over, because events already written to the log keep their old keys until retention expires them, and a replay after the migration reads resolution 7 keys from a topic that now produces only resolution 8. migrated region — resolution 8 8:881f1d4d61fffff not yet migrated — resolution 7 7:871f1d4ffffffff one vehicle, one journey, one minute — both key shapes The prefix cannot be dropped when the last region cuts over: events already in the log keep their old keys until retention expires them, and a replay after the migration reads resolution 7 keys from a topic that now produces only resolution 8.
Figure 3. The self-describing key is what makes a partial rollout survivable, and what makes the cleanup deploy a later, separate decision.

Verification

python 
import h3
import pytest

PARENT = h3.latlng_to_cell(53.5400, 9.9300, 7)


@pytest.mark.asyncio
async def test_children_inherit_before_any_event_arrives(store):
    """No consumer may ever see an empty child cell."""
    await store.set(f"7:{PARENT}", {"event_count": 2_100_000,
                                    "last_seen": "2026-08-08T12:04:31Z",
                                    "dedup_keys": ["a", "b", "c"]})
    state = DualResolutionState(store, cutover=set())
    await state.migrate_parent(PARENT)

    children = h3.cell_to_children(PARENT, 8)
    for child in children:
        loaded = await store.get(f"8:{child}")
        assert loaded is not None
        assert loaded["dedup_keys"] == ["a", "b", "c"]


@pytest.mark.asyncio
async def test_counts_are_split_not_copied(store):
    """Copying would turn 2.1M events into 14.7M."""
    await store.set(f"7:{PARENT}", {"event_count": 2_100_000})
    state = DualResolutionState(store, cutover=set())
    await state.migrate_parent(PARENT)

    children = h3.cell_to_children(PARENT, 8)
    total = sum((await store.get(f"8:{c}"))["event_count"] for c in children)
    assert total <= 2_100_000


def test_pentagon_children_are_not_assumed_to_be_seven():
    """Twelve cells per resolution have six children, not seven."""
    pentagon = next(c for c in h3.get_pentagons(7))
    assert len(h3.cell_to_children(pentagon, 8)) == 6


@pytest.mark.asyncio
async def test_uncut_region_still_keys_at_the_old_resolution(store):
    """A vehicle crossing the boundary must produce both key shapes."""
    state = DualResolutionState(store, cutover={PARENT})
    assert state.resolution_for(53.5400, 9.9300) == 8      # migrated region
    assert state.resolution_for(48.1372, 11.5756) == 7     # not yet

The count test uses <= rather than == because integer division loses a remainder — which is itself worth noticing, since the missing events are exactly the kind of small permanent discrepancy that shows up in a reconciliation report months later.