Algorithms & Data Structures
the bound is a claim, and it is checkable
Concept

Fixed point — where it appears

A value defined in terms of itself, found by iterating an equation until it stops moving. The smallest counter a bounded-space table settles at is one of these: the mass shared among the churning slots depends on how many slots churn, and how many churn depends on that smallest counter. Solving it takes a few dozen steps of arithmetic and no structure at all.

Named by 7 essays across 5 fields — each of them below, with the objects they name alongside it.

0231436845165232shards mergedkeys whose count movedthree orders:folded in one at a timecombined pairwise, in a treefolded in, last shard firstMisra-Gries, k = 32 · hashedworst gap 298 arrivals

What a fold charges per level

Thirty-two counter tables folded in a chain come out wrong by 323 where the same thirty-two combined pairwise are wrong by 148, and the quantile summaries prefer the chain by exactly as much in the other direction. What is being charged in each case is the depth of the fold, and the two families are charged on opposite ones.

bounds · Merge shard 1 · 120 → 119shard 2 · 121 → 120shard 3 · 120 → 120shard 4 · 121 → 120shard 5 · 119 → 118shard 6 · 124 → 123shard 7 · 121 → 120shard 8 · 122 → 121floor, in arrivalsnaive: tail ÷ kfixed pointmeasuredstationary Zipf · round · k = 321.006× the measured floor

The floor a histogram already knows

A summary of thirty-two counters settles at a smallest counter of 119, and the number can be computed from the shard's key frequencies before a single counter is allocated. The obvious way to compute it is wrong by a factor of two, and the reason is that the heavy counters carry no error at all.

streaming · Merge 010020030040021φ 382φ 7203φ 17434φ 42885φ 1012486φ 242measured 409counts chargedlevel of the foldcharged at this levelrunning total64 shards · k = 32 · hashedcharged 409 · measured 409

The floor charged at every level

A key surviving a fold of sixty-four shards is charged 2, then 8, then 20, then 43, then 88, then 248 — the floor of whatever summary it was merged against, level by level. They sum to 409, and the damage read off the merged table is 409. The model that charged sixty-three copies of the leaf floor said 222.

floors · Floor 0.111010010³10³10⁴floor, in countsarrivals in the shard, nround-robin — n^1.02hashed — fit refusedresidual 2.7%slope 5.2 → 1.19k = 32 · 40,000 arrivalsthe table holds 1.33 of a hashed shard's keys and 0.01 of the stream's

A floor with two variables in it

Under round-robin a Space-Saving summary's floor is 0.0203·n^1.018 over a hundred-and-twenty-eight-fold range of shard size, worst residual 2.7%. Under hashing the same measurement has no exponent at all — the local slope runs from n^5.17 to n^1.19 — and a least-squares line through it reports n^1.73 at a 441% residual.

floors · Floor f = 6,3628/8 holdingf = 3,0748/8 holdingf = 1,9638/8 holdingf = 1,3738/8 holdingf = 1,0988/8 holdingf = 9353/8 holdingf = 7690/8 holdingf = 6560/8 holdingpredicted damage, in arrivals — every row totals 967Space-Saving's shareMisra-Gries's share8 shards · round · k = 32bill 967 arrivals

The bill a partition only divides

The two predicted damages for any key sum to the same number under every partition — 967 arrivals here, whatever the arrangement. Round-robin hands nearly all of it to Misra-Gries and hashing hands most of it to Space-Saving, and neither of them is paying more than the other in total.

structures · Merge 0.000.250.500.751.00stationarydepartingburstydriftingprediction ÷ measurement, as a factorshare of the top k that moves between halves8 shards · round · k = 327.1× out where the statistic reads 1.00

The histogram that cannot see the order

A prediction accurate to one per cent on three streams is seven times out on the fourth, and the input that fails is the one every capacity plan is built from. A statistic computed from the same input says in advance which case is in hand — and misses one of the two ways it can go wrong.

wrong · Merge 0.60.81.01.21.41.61.84φ share 0.908φ share 0.8516φ share 0.8032φ share 0.7564φ share 0.70predicted ÷ measuredshards, mlevel floors, uncorrectedleaf floorslevel floors, correctedk = 32 · hashed · 40,000 arrivalsworst 22% against 73% and 46%

The floor a merge does not settle at

Compute a fold's level floors from the shard histograms and the prediction over-shoots by 1.73. A merged summary's floor is not the floor a summary settles at on the same arrivals — it is 0.90 of it at four shards and 0.70 at sixty-four, straight in log₂ m at a 3% residual, because merging preserves the heavy counters and never runs their eviction cascade.

wrong · Merge

Named alongside it

The objects these essays reach for when they reach for this one.

HistogramPartitionShardSpace-savingHeavy hitterMeasurementGuaranteeMerge treeMergeable summaryMisra–GriesEstimatorHonest limit

All concepts