Generator

False-positive rate against bits per element, k = 4, n = 4,000

Rendered here at the parameters it defaults to, with every essay that calls it — which is the same list as the blast radius of changing it.
False-positive rate against bits per element, k = 4, n = 4,000Filled circles are measured: 60,000 queries for keys that were never inserted, counted. The solid line is (1 − e^(−kn/m))^k, the formula every sizing guide prints. The dashed line is the same expression computed from the bits the filter actually has set. At 4 bits per element the measured rate is 16.2% and at 12 it is 0.620%. Not one inserted key tested absent at any size: the errors a filter makes are all in one direction. The keys are drawn at random, which is what the analysis assumes; the two lines then agree with the points to within 2.8%.46810120.010.1bits per element (m / n)false-positive ratemeasured(1 − e^(−kn/m))^kfrom the bits set60,000 absent-key queries per point, seed 80800 false negatives at every size

False-positive rate against bits per element, k = 4, n = 4,000

Filled circles are measured: 60,000 queries for keys that were never inserted, counted. The solid line is (1 − e^(−kn/m))^k, the formula every sizing guide prints. The dashed line is the same expression computed from the bits the filter actually has set. At 4 bits per element the measured rate is 16.2% and at 12 it is 0.620%. Not one inserted key tested absent at any size: the errors a filter makes are all in one direction. The keys are drawn at random, which is what the analysis assumes; the two lines then agree with the points to within 2.8%.

Drawn at 690 × 400, wide on the page. Everything above is what bloom-rate returns with no arguments; the caption is the generator's own, computed from the numbers in the drawing rather than written beside it.

12 essays call bloom-rate. The drawing above is what it returns with no arguments at all; every call below passes it something, because a placement that passes nothing draws whichever member of the family the generator happens to default to rather than the one its essay argues about — which is what optcheck and figfill exist to catch.

Where it is called

Changing this generator changes every one of these figures.

hash rowh1h2h3h4h1 → cell 30: 10,433h2 → cell 29: 10,624h3 → cell 19: 10,575h4 → cell 4: 10,895key 0 occurred 9,579 times · the minimum of the four is 10,433 · over by 854the additive bound at this width is e/w × N = 5,0974×32 counters · 4,096 bits · Zipf s = 1.1exact would take 162,288 bits One pass, and no room

A count that is never under

46810120.010.1bits per element (m / n)false-positive ratemeasured(1 − e^(−kn/m))^kfrom the bits set60,000 absent-key queries per point, seed 80800 false negatives at every size When the algorithm flips a coin

A filter that is allowed to be wrong

46810120.010.1bits per element (m / n)false-positive ratemeasured(1 − e^(−kn/m))^kfrom the bits set60,000 absent-key queries per point, seed 80800 false negatives at every size What is taught wrongly

The formula everybody sizes filters with

bits used · answers that were wrongBloom, bits cleared16,384 bits638 said no wrongly · 14 said yes wronglycounting, 4 bits a cell65,536 bits0 said no wrongly · 42 said yes wronglyfingerprints in two slots32,768 bits0 said no wrongly · 153 said yes wronglyand the condition on the caller1,000 deletions of keys never inserted lost 15 that were2,000 keys, 1,000 deleted, 20,000 absent keys querieda false negative is a different kind of wrong from a false positive When the algorithm flips a coin

The evidence a filter cannot remove

queries answered yesabsent key, the AND0.190%absent key, built on the intersection0.025%in one set only, the AND1.800%in one set only, built on it0.000%bits set: A 6,351, B 6,294, AND 3,232, direct 1,859no common key is ever denied When the algorithm flips a coin

The intersection two filters cannot report

HyperLogLog1024/1024 identicalCount-Min256/256 identicalbottom-k128/128 identicalMisra-Gries20/36 identicalfraction of the state that merged to the identical value→ 1,974→ 10,291→ 2,322→ 8,766two streams of 30,000 · 2,007 distinct keys in the union3 of 4 merge exactly Structures

The summaries that add

0.01%0.1%1%10%100%1,0002,0003,0004,0005,0006,0007,0008,000keys insertedabsent keys answered yesone filterm = 19,171, k = 7; dotted: the design sizedashed: the design rate When the algorithm flips a coin

A filter past its design size

0.01%0.1%1%10%100%1,0003,0005,0007,0009,00011,00013,00015,000keys insertedabsent keys answered yesone Bloom filtera stack of Bloom filtersfingerprints, none reservedfingerprints, 3 reservedforecast 2,000, target 1.0%; dotted: the forecastdashed: the target rate When the algorithm flips a coin

A filter that grows by moving a bit

3579111311.523510positions per key, krate ÷ the independent rateh₁ + i·h₂h₁ + i·h₂, step oddh₁ + i·h₂ + (i³ − i)/6optimal load m·ln 2 / k; one set of hash functions per schemedashed: the account When the algorithm flips a coin

Two hash values and the keys they copy

0.01%0.1%1%10%100%4,00012,00020,00028,00036,00044,00052,00060,000keys insertedabsent keys answered yesfingerprints, none reserved1 bit longer a doubling2 bits longer a doublingfingerprints, 5 reservedforecast 2,000, target 1.0%; dotted: the forecastdashed: the target rate When the algorithm flips a coin

The bits given to the wrong keys

0%1.1%2.1%3.2%4.2%the whole filterblocks of 64blocks of 5120123456distinct 512-bit lines a lookup readsabsent keys answered yes16,384 bits, 2,048 keys, k = 6one line is what a block buys When the algorithm flips a coin

Positions confined to one line

8121620240.000010.00010.0010.01bits a keyfalse-positive ratethe whole filterone block of 512 bitsthe emptier of two 512-bit blocksone block of 1,024 bits2,048 keys · 24 filters a pointdashed: no blocks When the algorithm flips a coin

Two blocks and the chances they add

The library, page 1 of 5 — where bloom-rate sits