Generator

The floor of each shard, computed from its histogram and then measured

Rendered here at the parameters it defaults to, with every essay that calls it — which is the same list as the blast radius of changing it.
The floor of each shard, computed from its histogram and then measured8 shards of 40,000 arrivals of stationary Zipf — the control, where a window is a smaller sample, partitioned round-robin: every key on every shard, each summarised by 32 counters. The pale bar is the naive estimate: the mass outside the shard's top 32 keys divided by 32. The mid bar is the fixed point, which divides the same mass by the number of slots that are actually churning. The dark bar is the smallest counter the structure settled at. The fixed point is within 0.6% of it on average; the naive estimate is low by about a factor of 1.8.shard 1 · 120 → 119shard 2 · 121 → 120shard 3 · 120 → 120shard 4 · 121 → 120shard 5 · 119 → 118shard 6 · 124 → 123shard 7 · 121 → 120shard 8 · 122 → 121floor, in arrivalsnaive: tail ÷ kfixed pointmeasuredstationary Zipf · round · k = 321.006× the measured floor

The floor of each shard, computed from its histogram and then measured

8 shards of 40,000 arrivals of stationary Zipf — the control, where a window is a smaller sample, partitioned round-robin: every key on every shard, each summarised by 32 counters. The pale bar is the naive estimate: the mass outside the shard's top 32 keys divided by 32. The mid bar is the fixed point, which divides the same mass by the number of slots that are actually churning. The dark bar is the smallest counter the structure settled at. The fixed point is within 0.6% of it on average; the naive estimate is low by about a factor of 1.8.

Drawn at 700 × 400, wide on the page. Everything above is what predicted-floor returns with no arguments; the caption is the generator's own, computed from the numbers in the drawing rather than written beside it.

7 essays call predicted-floor. The drawing above is what it returns with no arguments at all; every call below passes it something, because a placement that passes nothing draws whichever member of the family the generator happens to default to rather than the one its essay argues about — which is what optcheck and figfill exist to catch.

Where it is called

Changing this generator changes every one of these figures.

key 05,416short by 4,163key 1462short by 4,154key 31short by 2,099key 71short by 943key 19401short by 3counter held (bar) against true count (tick)12 counters · 768 bits · no randomnessbound N/(k+1) = 4,615 Structures

The items that survive k counters

shard 1 · 120 → 119shard 2 · 121 → 120shard 3 · 120 → 120shard 4 · 121 → 120shard 5 · 119 → 118shard 6 · 124 → 123shard 7 · 121 → 120shard 8 · 122 → 121floor, in arrivalsnaive: tail ÷ kfixed pointmeasuredstationary Zipf · round · k = 321.006× the measured floor One pass, and no room

The floor a histogram already knows

010020030040021φ 382φ 7203φ 17434φ 42885φ 1012486φ 242measured 409counts chargedlevel of the foldcharged at this levelrunning total64 shards · k = 32 · hashedcharged 409 · measured 409 The floors

The floor charged at every level

0.111010010³10³10⁴floor, in countsarrivals in the shard, nround-robin — n^1.02hashed — fit refusedresidual 2.7%slope 5.2 → 1.19k = 32 · 40,000 arrivalsthe table holds 1.33 of a hashed shard's keys and 0.01 of the stream's The floors

A floor with two variables in it

0.000.250.500.751.00stationarydepartingburstydriftingprediction ÷ measurement, as a factorshare of the top k that moves between halves8 shards · round · k = 327.1× out where the statistic reads 1.00 What is taught wrongly

The histogram that cannot see the order

0.60.81.01.21.41.61.84φ share 0.908φ share 0.8516φ share 0.8032φ share 0.7564φ share 0.70predicted ÷ measuredshards, mlevel floors, uncorrectedleaf floorslevel floors, correctedk = 32 · hashed · 40,000 arrivalsworst 22% against 73% and 46% What is taught wrongly

The floor a merge does not settle at

how far the top k movedthe order warninghow far the floors are from doublingthe regime warningstationary Zipf0.160.53 (54%)one key floods a stretch0.170.59 (54%)a heavy hitter that stops0.170.65 (52%)the popular keys drift0.9134.00 (0%)k = 32 · 40,000 arrivalsin brackets: the leaf model at sixty-four shards What is taught wrongly

The warning that is silent for the right reason

The library, page 3 of 5 — where predicted-floor sits