Choosing an H3 Resolution from Measured Traffic

Replay a real sample including its busiest hour, count events per cell at every candidate resolution, and take the lowest resolution whose hottest cell still fits inside one consumer’s throughput — the cell-size table describes geometry, and the thing that makes a partition too hot is the event distribution, which only your own traffic knows.

This guide sits under Spatial Partitioning Strategies, within Core Event Fundamentals & Architecture. The comparison of the indexing schemes themselves is in H3 vs S2 vs Quadkey for Spatial Partitioning; this one assumes H3 has been chosen and asks only how fine to make it.

When to use this pattern

  • You are about to set a partition key for a new stream, or a live one is showing the consumer lag concentrated in a few partitions.
  • You have at least a week of real events to replay, including whatever the stream’s busiest period looks like.
  • The consumer’s throughput is known — events per second per instance, measured rather than assumed.

The table answers the wrong question

Two curves moving in opposite directions The same twenty-four hours of fleet telemetry is bucketed at three H3 resolutions. At resolution 6 there are about nine hundred active cells and the busiest one carries fourteen thousand events per second, which is roughly nine times what one consumer instance can handle, so that partition is permanently behind no matter how many instances are added. At resolution 7 there are about six thousand active cells and the busiest carries three thousand two hundred events per second, still about double a single consumer. At resolution 8 there are about forty thousand active cells and the busiest carries nine hundred, comfortably inside one consumer with headroom for growth. Going further to resolution 9 would bring the busiest cell down to a few hundred but push active cells past a quarter of a million, at which point per-cell metrics and per-cell consumer state become the dominant cost. The choice is the resolution where the descending curve first crosses the consumer's capacity line, and no further, because everything after that point is paid for in cardinality and buys nothing. same 24 hours of telemetry, bucketed at three resolutions resolution 6 ~900 active cells busiest cell: 14 000 ev/s ≈ 9× one consumer — permanently behind, and adding instances cannot help cheap metrics, useless balance resolution 7 ~6 000 active cells busiest cell: 3 200 ev/s ≈ 2× one consumer — still hot closer, not there resolution 8 ~40 000 active cells busiest cell: 900 ev/s inside one consumer, with headroom the first resolution that works Why not go finer still? Resolution 9 would bring the busiest cell to a few hundred events per second and push active cells past 250 000. Each step multiplies cell count by about seven, and every cell is a metric series and a state key. Stop at the first resolution that fits.
Figure 1. The busiest cell falls and the cell count rises, both roughly sevenfold per step. The answer is the first crossing, not the smallest number.

Complete runnable implementation

python 
import collections
from dataclasses import dataclass

import h3

# Measured throughput of ONE consumer instance on this workload, in events
# per second. Measure it; do not take it from a benchmark of a different mix.
CONSUMER_CAPACITY = 1_200

# Headroom for growth and for bursts. A cell at exactly capacity is a cell
# that falls behind the first time anything is slower than usual.
TARGET_UTILISATION = 0.75

# Above this, per-cell metrics and per-cell consumer state dominate.
MAX_ACTIVE_CELLS = 100_000


@dataclass(frozen=True, slots=True)
class ResolutionProfile:
    resolution: int
    active_cells: int
    hottest_cell: str
    hottest_rate: float
    p99_rate: float
    fits: bool


def profile(sample: list[tuple[float, float]], window_seconds: float,
            resolutions: range = range(4, 11)) -> list[ResolutionProfile]:
    """Bucket one sample of positions at each candidate resolution.

    `sample` must include the stream's busiest period. A quiet-Sunday sample
    produces a resolution that is correct on Sundays.
    """
    profiles = []
    for resolution in resolutions:
        counts = collections.Counter(
            h3.latlng_to_cell(lat, lon, resolution) for lat, lon in sample
        )
        rates = sorted((n / window_seconds for n in counts.values()), reverse=True)
        hottest_cell, hottest_n = counts.most_common(1)[0]
        hottest = hottest_n / window_seconds
        p99 = rates[max(0, int(len(rates) * 0.01))]

        profiles.append(ResolutionProfile(
            resolution=resolution,
            active_cells=len(counts),
            hottest_cell=hottest_cell,
            hottest_rate=hottest,
            p99_rate=p99,
            fits=(hottest <= CONSUMER_CAPACITY * TARGET_UTILISATION
                  and len(counts) <= MAX_ACTIVE_CELLS),
        ))
    return profiles


def choose(profiles: list[ResolutionProfile]) -> ResolutionProfile:
    """The COARSEST resolution that fits — finer buys nothing but cardinality."""
    for candidate in profiles:                    # ascending resolution
        if candidate.fits:
            return candidate
    raise ValueError(
        "no resolution satisfies both constraints: the hottest cell is hot at "
        "every resolution coarse enough to keep cardinality bounded — the key "
        "needs a second component, not a finer grid"
    )

The exception message is the important part. When no resolution works, the answer is not resolution 11; it is that geography alone cannot balance this stream, and the partition key needs something else in it.

Parameter reference

Name Type Spatial constraint Default
CONSUMER_CAPACITY int Events/second for this payload mix, measured 1200
TARGET_UTILISATION float Headroom for bursts; 1.0 means permanently at the edge 0.75
MAX_ACTIVE_CELLS int Bound from metric cardinality and per-cell state, not from H3 100000
Sample window seconds Must include the busiest hour, not an average day ≥ 7 days
resolutions range 4–10 covers city to building scale; below 4 is continental range(4, 11)
Chosen resolution int Recorded in the event envelope, so keys are interpretable —

Gotchas and spatial edge cases

  1. A hot cell that stays hot at every resolution is telling you something. A port terminal, a stadium or a depot is a genuine point concentration: subdividing it produces smaller cells that are all in the same building, and one of them still holds the loading bay. The fix is a composite key — cell plus asset identifier hash, or cell plus event type — which trades strict geographic co-location for balance.

  2. Cells are only approximately equal in area, and the pentagons are not. H3 has twelve pentagonal cells per resolution, which are smaller than their hexagonal neighbours and behave differently under grid_disk. They fall in the ocean at low resolutions, but a global stream will eventually key into one; make sure nothing assumes exactly six neighbours.

  3. Resolution changes ordering guarantees. Events for one asset are ordered only within a partition, so a finer grid means a moving asset changes partition more often and its events lose relative order sooner. If ordering per asset matters more than balance, partition by asset and use the cell as a routing attribute instead.

  4. Measure with the events, not with the assets. Ten thousand parked vehicles in a depot produce far more events than a hundred moving ones spread across a region, and an asset-count-based profile inverts the picture entirely.

  5. The distribution moves. A resolution chosen against last year’s traffic is a resolution chosen against last year’s cities. Re-run the profile quarterly and alert when the hottest cell passes the utilisation target, which is the signal that Migrating a Topic to a New H3 Resolution is due.

  6. MAX_ACTIVE_CELLS is about your monitoring stack, not about H3. If cells never appear as metric labels and consumer state is not per cell, the bound can be far higher. State it explicitly, because the next person will assume it came from the library.

When a finer grid cannot help A container terminal generates forty percent of a fleet's events from an area two hundred metres across. At resolution 8 the whole terminal is one cell. At resolution 9 it is seven cells, but six of them are water and access road while the seventh contains the loading bay, so the hottest cell is barely cooler. At resolution 11 the terminal is spread across hundreds of cells, and the loading bay is still one of them carrying most of the traffic, while the active cell count for the whole fleet has passed a million. Subdividing space cannot separate events that genuinely happen in the same place. The fix is to stop using geography alone: hashing the asset identifier into the key alongside the cell splits the terminal's traffic across as many partitions as wanted, at the cost that events from one small area are no longer co-located, which matters only if a consumer needs to reason about neighbours. a container terminal: 40% of fleet events from 200 metres res 8 — one cell 40% of the stream res 9 — seven cells six are water and road; one is the bay res 11 — hundreds the bay is still one cell, and the fleet has 1M+ cells composite key — cell + asset hash splits the terminal across as many partitions as you want cost: events from one area are no longer co-located — only matters for neighbour logic Subdividing space cannot separate events that happen in the same place This is what the `choose()` exception means. Reading it as "try resolution 12" is the mistake; the key needs another component.
Figure 2. The concentration is physical. No grid separates a loading bay from itself, which is why the profiler raises rather than returning the finest resolution it has.
The sample decides the resolution, so it has to be the busy one The same profiler is run over two samples of the same stream. The first is a quiet Sunday: the busiest cell at resolution 6 carries only nine hundred events per second, which already fits inside one consumer, so the profiler correctly returns resolution 6 as the coarsest resolution that works. Deployed against Monday morning, that same cell carries fourteen thousand events per second and the partition is nine times over a consumer's capacity — the recommendation was right about the data it saw and wrong about the stream. The second sample is taken across the busiest hour and returns resolution 8, which holds on Monday and is merely finer than necessary on Sunday. The asymmetry is the whole argument for sampling the peak: a resolution that is too fine costs cardinality, which is a bounded and visible cost, while one that is too coarse costs a permanently saturated partition, which is an outage that recurs every weekday morning. same profiler, same stream, two samples sampled on a quiet Sunday busiest cell at res 6: 900 ev/s — already fits recommendation: resolution 6 on Monday that cell carries 14 000 ev/s nine times a consumer's capacity, every weekday right about the data it saw, wrong about the stream sampled across the busiest hour busiest cell at res 6: 14 000 ev/s — rejected recommendation: resolution 8 holds on Monday merely finer than necessary on Sunday The errors are not symmetric: too fine costs cardinality, which is bounded and visible. Too coarse costs a permanently saturated partition, which is an outage that recurs every weekday morning.
Figure 3. Because the two errors cost differently, a sample taken at a convenient moment biases towards the expensive one.

Verification

python 
import random
import pytest


def clustered_sample(n: int = 200_000) -> list[tuple[float, float]]:
    """40% of events from a 200 m terminal, the rest spread over a city."""
    hot = [(53.5400 + random.gauss(0, 0.0008),
            9.9300 + random.gauss(0, 0.0012)) for _ in range(int(n * 0.4))]
    rest = [(53.50 + random.uniform(0, 0.12),
             9.85 + random.uniform(0, 0.25)) for _ in range(n - len(hot))]
    return hot + rest


def test_coarse_resolutions_are_rejected():
    """A hot terminal must disqualify the coarse end."""
    profiles = profile(clustered_sample(), window_seconds=3600)
    by_res = {p.resolution: p for p in profiles}
    assert not by_res[5].fits
    assert by_res[5].hottest_rate > by_res[9].hottest_rate


def test_choose_returns_the_coarsest_that_fits():
    """Not the finest — cardinality is a cost, not a bonus."""
    profiles = profile(clustered_sample(), window_seconds=3600)
    chosen = choose(profiles)
    finer = [p for p in profiles if p.resolution < chosen.resolution]
    assert all(not p.fits for p in finer)


def test_unbalanceable_stream_raises_rather_than_going_finer():
    """Every event at one point: no grid can help, and it must say so."""
    sample = [(53.5400, 9.9300)] * 100_000
    with pytest.raises(ValueError, match="second component"):
        choose(profile(sample, window_seconds=60))

The third test encodes the judgement the whole exercise exists to produce. A profiler that silently returns resolution 15 for a point source is worse than one that fails, because the resolution it returns will be deployed.