Field

When the algorithm flips a coin

A skip list's shape is a sequence of coin flips rather than a property of its keys. Where the randomness is the structure rather than a rule applied to one, the distribution is the result and the average is the least of it.
level 1level 2level 3level 4level 5key 21search: 8 comparisons, 3 hops26 keys, p = 0.5, seed 2026081057 coin flips decided the shape

A structure made of coin flips

What a skip list is, why its shape is decided before a single key is compared, and what changes when the structure of a data structure stops being a function of the data.

share of nodespredictedlevel 149.68% · 50.00%level 225.29% · 25.00%level 312.52% · 12.50%level 46.24% · 6.25%level 53.08% · 3.13%level 61.64% · 1.56%level 70.79% · 0.78%level 80.34% · 0.39%20,000 nodes, p = 0.5, seed 5150worst departure 0.32 points

The height is a distribution, and the coin is a parameter

Why a skip list's height is a random variable rather than a bound, what changes when the promotion probability moves from a half to a quarter, and the three-way trade between comparisons, pointers and random bits that the choice of coin decides.

k/n = 0.1250.0810.1250.169position in the streamshare of runs in which it was sampledAlgorithm R, n = 32, k = 4, 40,000 runsworst departure 3.3% · noise 1.4%

One pass, k slots, and two randomness budgets

How a uniform sample is drawn from a stream nobody has counted, what an off-by-one does to it that no correctness test would catch, and why two algorithms with identical output can differ by a factor of 145 in a resource nothing else measures.

46810120.010.1bits per element (m / n)false-positive ratemeasured(1 − e^(−kn/m))^kfrom the bits set60,000 absent-key queries per point, seed 80800 false negatives at every size

A filter that is allowed to be wrong

What a Bloom filter is, why its false-positive rate is a measurable quantity with a closed form, and what changes when a data structure is permitted to be wrong in one direction only.

average 8.001,0242,048bucket, 0 to 255keys in the bucketthe low bits · keys built for this hashworst bucket 2,048 against 8.0

A hash is a family, not a function

Why a hash function chosen once at compile time has an input that destroys it, what universality actually guarantees, and how a family turns a claim about the data into a claim about the algorithm.

024681,0244,09616,38465,536262,144keys, into as many bucketskeys in the busiest bucketone hashtwo hashes, take the emptierthree hasheslog n / log log nthe average load is 1 at every pointa lookup examines every choice, so two hashes is two probes

The second choice

The power of two choices, measured — how the maximum load stops growing while the mean does not move, and what the second probe costs on every lookup thereafter.

0%25%50%75%100%0.20.30.40.450.50.550.60.70.8keys per slotconstructions that failedconstructions that faileddisplacements, scaled to 36840 constructions per point, 256 slots per tablea lookup is two probes, whatever the keys are

An insertion that can fail

Cuckoo hashing measured — the displacement chain, the load at which it stops terminating, and how the threshold sharpens as the table grows.

1,00010,000110100modulus qverifications in one searchq = 257: 693measuredn / qchosen text: 6%one unit = one character comparison · base 2568 colliding blocks at q = 1,009, none at q = 1,000,003

A match decided by a number

The rolling hash, why its cost has a closed form, why the closed form is an expectation rather than a guarantee, and the arithmetic that decides whether an adversary can build a collision at all.

bits used · answers that were wrongBloom, bits cleared16,384 bits638 said no wrongly · 14 said yes wronglycounting, 4 bits a cell65,536 bits0 said no wrongly · 42 said yes wronglyfingerprints in two slots32,768 bits0 said no wrongly · 153 said yes wronglyand the condition on the caller1,000 deletions of keys never inserted lost 15 that were2,000 keys, 1,000 deleted, 20,000 absent keys querieda false negative is a different kind of wrong from a false positive

The evidence a filter cannot remove

What deletion costs three approximate-membership structures, measured — the false negatives a cleared Bloom filter produces, the space the repairs cost, and the precondition one of them adds.

10,0000.1total bits of stateruns outside ±15%1 × 483 × 485 × 487 × 489 × 48240 runs per design · tolerance ±15% · truth 12,00067 → 3 failures

The estimate that is a median of means

Why averaging buys accuracy and only a median buys confidence, what the two purchases cost separately, and how a failure rate is measured rather than quoted.

1 coefficient13 members2 coefficients169 members3 coefficients2,197 members4 coefficients28,561 members1 key2 keys3 keys4 keys5 keysexact92% gone99% gone100% gone100% goneexactexact92% gone99% gone100% goneexactexactexact92% gone99% goneexactexactexactexact92% goneevery member walked · GF(13) · no tolerance and no seeddegree 1, 2, 3, 4

The independence an estimator spends

k-wise independence as a measured resource — enumerated exactly over GF(13), sampled where it cannot be, and the estimator whose error stops falling when the family is too weak.

queries answered yesabsent key, the AND0.190%absent key, built on the intersection0.025%in one set only, the AND1.800%in one set only, built on it0.000%bits set: A 6,351, B 6,294, AND 3,232, direct 1,859no common key is ever denied

The intersection two filters cannot report

The bitwise AND of two Bloom filters against a filter built directly on the intersection. The AND keeps every shared key, but its false positives for keys in one set only are set by the other filter's load rather than the intersection's size, and a size read off its bits overstates the intersection badly while inclusion–exclusion over the union does not.

0%25%50%75%100%0.30.40.50.60.70.80.850.90.95keys per slotconstructions that failed2 hashes, 1 slot3 hashes, 1 slot2 hashes, 2 slots2 hashes, 4 slots20 constructions a point, 256 bucketsa lookup reads hashes × slots, whatever the keys

More hashes or wider buckets

The cuckoo construction's failure rate against load for several shapes — two, three and four hash functions, and buckets of one, two and four slots. Each shape has a threshold rather than a slope, the measured thresholds sit on the published ones, and the shapes differ in what a lookup reads rather than in how many keys they hold.

2 halves, ties go left2 choices, ties at randomone choiceload 2 or more14,61015,03717,363load 3 or more2675785,250load 4 or morenone11,236load 5 or morenonenone223load 6 or morenonenone36load 7 or morenonenone2load 8 or morenonenone165,536 keys and buckets, seededbar length is log(1 + count)

The tie that breaks left

The power of two choices with random tie-breaking against the asymmetric version — the table split into groups, one choice in each, ties sent to the leftmost group. Deterministic, unfair tie-breaking thins the tail by half and lowers the maximum, and the effect grows with the number of choices.

0.01%0.1%1%10%100%1,0002,0003,0004,0005,0006,0007,0008,000keys insertedabsent keys answered yesone filterm = 19,171, k = 7; dotted: the design sizedashed: the design rate

A filter past its design size

One Bloom filter filled past the number of keys it was sized for, and a scalable stack of tightening filters that is not. The single filter degrades smoothly and without limit, exactly as its formula says; the stack keeps a stated bound on its rate however far it grows, and pays for the guarantee in bits.

independent hashestwo values, h₁ + i·h₂2 choices, load 2+60,41859,9942 choices, load 3+2,2832,3672 choices, load 4+123 choices, load 2+46,45646,2433 choices, load 3+138145262,144 keys and buckets, seededbar length is log(1 + count)

Choices that are not independent

Balanced allocation with d choices generated by double hashing from two hash values, against d independent hashes, on the same keys. The tails agree to within what two independent runs disagree by, for two, three and four choices — the dependence that matters is choices coinciding, and an odd step removes it.

0.01%0.1%1%10%100%1,0003,0005,0007,0009,00011,00013,00015,000keys insertedabsent keys answered yesone Bloom filtera stack of Bloom filtersfingerprints, none reservedfingerprints, 3 reservedforecast 2,000, target 1.0%; dotted: the forecastdashed: the target rate

A filter that grows by moving a bit

A quotient-style fingerprint table that doubles in place, with 0 to 5 bits of its fingerprints reserved for growth, against a single Bloom filter and a stack of tightening Bloom filters, given the same keys up to 32 times a forecast. Reserving log₂ of the growth holds the target rate and costs at the end exactly what a correctly sized table costs; the stack holds its bound at any growth and pays for it in bits that grow faster than the keys.

3579111311.523510positions per key, krate ÷ the independent rateh₁ + i·h₂h₁ + i·h₂, step oddh₁ + i·h₂ + (i³ − i)/6optimal load m·ln 2 / k; one set of hash functions per schemedashed: the account

Two hash values and the keys they copy

Bloom filters of 64 to 4,096 bits with k from 3 to 13, their positions made by independent hashes, by h₁ + i·h₂, with the step forced odd, and with a cubic term added. The excess false positives come from queries that copy a stored key's progression, whole or shifted, and a count of those copies predicts them; a cubic term removes most of the copies and nearly all of the penalty.

0.01%0.1%1%10%100%4,00012,00020,00028,00036,00044,00052,00060,000keys insertedabsent keys answered yesfingerprints, none reserved1 bit longer a doubling2 bits longer a doublingfingerprints, 5 reservedforecast 2,000, target 1.0%; dotted: the forecastdashed: the target rate

The bits given to the wrong keys

Lengthening fingerprints as a resizable filter grows, measured against reserving bits in advance and against a stack of tightening Bloom filters, in rate, in bits and in how far each design can be grown before it cannot store a key at all.

0%1.1%2.1%3.2%4.2%the whole filterblocks of 64blocks of 5120123456distinct 512-bit lines a lookup readsabsent keys answered yes16,384 bits, 2,048 keys, k = 6one line is what a block buys

Positions confined to one line

Blocked Bloom filters measured against whole-filter ones — the rate a block costs on its own at each block size and each density, the cache lines a lookup reads either way, and whether the two-hash-value construction and its cubic repair behave inside a block as they do across a filter.

8121620240.000010.00010.0010.01bits a keyfalse-positive ratethe whole filterone block of 512 bitsthe emptier of two 512-bit blocksone block of 1,024 bits2,048 keys · 24 filters a pointdashed: no blocks

Two blocks and the chances they add

A blocked Bloom filter whose keys choose the emptier of two blocks, measured against one block, one block twice as wide and the whole filter at 8 to 24 bits a key: the spread of keys a block, the false-positive rate, the cache lines an absent and a stored key's lookup reads, and every design at 20 bits a key set against the lines it costs.

8121620240.00010.0010.01bits a keyfalse-positive rateone blockthe emptier of two blocksone block, chosen by athreshold2,048 keys · 24 filters a pointblocks of 512 bits · same total budgetbest from 12 bits a key

A block the lookup can work out

A blocked Bloom filter whose first block carries a threshold on an ordering hash, so a lookup reads one threshold and one block rather than two blocks: the false-positive rate against one-choice and two-choice designs from 8 to 24 bits a key, the load spread each achieves, how wide the threshold has to be, and where the capacity that fills it should sit.

11010010³0.00010.0010.010.1keys held, as a multiple of the forecast of 2,000share of absent keys answered yesthe 1% targetten bits reservedrebuilt from the storerebuilt only when it cannotgrowa stack of Bloom filtersforecast 2,000 at 1%, grown to 2,000,00020,000 absent keys a point

The day a filter cannot grow

A fingerprint table forecast for 2,000 keys at 1% and grown to 2,000,000, measured four ways: rebuilt from the store on its own rate, rebuilt only when it can no longer double, a single table with ten bits reserved for the whole growth, and a stack of tightening Bloom filters. The rate each holds, the bits each spends as the growth proceeds, the keys the rebuilds read back, what the reserve's extra bits are worth per key read if the growth stops early, and the ledger at two million keys.

10,000100,000keys in the filtershare of absent keys answered yes0.04%0.05%0.06%0.08%0.10%the emptier of twoone blockone block, Poisson averageone block, by threshold16 bits a key, 512-bit blocks2,000,000 absent queries a point

A penalty set by the block

Blocked Bloom filters at sixteen bits a key — one block, the emptier of two, and one chosen by a threshold — measured over a 64-fold range of filter sizes and a 16-fold range of block widths, with the one-block rate computed exactly as Bloom's formula averaged over a Poisson load: why the size does not matter, how the busiest block grows without moving the rate, what the width does, and the line count at which a threshold and a wider block give the same rate.

0.00%0.05%0.10%0.15%256 bits · one block0.148%256 bits · a threshold on the first0.089%256 bits · thresholds on both0.095%512 bits · one block0.089%512 bits · a threshold on the first0.062%512 bits · thresholds on both0.066%1024 bits · one block0.066%1024 bits · a threshold on the first0.058%1024 bits · thresholds on both0.052%with no spreadwhat the spread adds16 bits a key, 8,192 keysno spread: every block at its mean load

The bits a second threshold costs

Blocked Bloom filters of 8,192 keys at sixteen bits a key, at block widths from 256 to 2,048 bits: one block a key, a threshold on the first block, thresholds on both of a key's blocks with a third block for keys both refuse, and one block four times as wide — false-positive rates on two million absent keys a point, the spread of block loads, the rate split into what the tables cost in load and what the spread costs, the rate against the bits a threshold is stored in, and the loads themselves.

124816adjacent blocks sharing one thresholdfalse-positive rate0.06%0.08%0.10%0.12%0.14%256-bit blocks512-bit blocks256: one block, no threshold256: one block twice as wide8,192 keys, 16 bits a keyfour-bit thresholds

A threshold knows only its own block

Blocked Bloom filters of 8,192 keys at 16 bits a key with one four-bit threshold shared by each group of 1 to 16 adjacent blocks, the table paid for in blocks, at 256- and 512-bit widths: the false-positive rate against group size beside one block and one block twice as wide, the rate split into its no-spread floor and the spread, where each block's own threshold closes, the share of keys placed in their first block, and two ways to close a shared threshold.

0.0%0.1%0.2%0.3%0.4%0.5%0.6%false-positive rate64-bit blocks1927 blocks128-bit blocks992 blocks256-bit blocks504 blocks512-bit blocks254 blocksreduced by a moduloreduced by its top bitseach dot: one seedupper row of a pair: modulo; lower: top bits

Two hashes that shared their low bits

The one-threshold blocked Bloom filter at 16 bits a key built twice from the same keys, seeds and queries — its block and ordering hashes reduced by a modulo, as the published filters were, and by their top bits: which combinations of ordering value and block the keys reach, the rate over eight seeds at four block widths, the rate against the block count, the threshold's width re-measured over four seeds, and the emptier-of-two filter, which the same dependence leaves untouched.

22.5033.504block width, bitstable bits a block64128256512four bits eachcoded, a line each, offsetscoded, a line eachcoded, with an indexthe entropyabout two thousand blocks at each widthdashed: the closing distribution's entropy

A code that has to fit its worst line

The one-threshold blocked Bloom filter at 16 bits a key, hashes read from their top bits, about two thousand blocks at widths of 64 to 512 bits, with its threshold table stored four bits each, three bits each, as a Huffman code from the filter's own closing distribution packed end to end with an index, and packed a fixed count to a cache line with and without in-line offsets, every table paid for in blocks: where thresholds close and each value's code word, the bits a block each layout takes, what a lookup reads and decodes, the false-positive rate over four seeds, and the table's share of the filter.

2.5033.504block width, bitstable bits a block64128256512four bits eacha bit and four, a line eacha bit and four, end to endcoded, a line eachthree bits eacha bit and three, a line eachmean of four seeds, about two thousand blockslabels at 64-bit blocks

A bit for whether it closed

The one-threshold blocked Bloom filter at 16 bits a key, hashes read from their top bits, about two thousand blocks at widths of 64 to 512 bits, with its threshold table stored as a bit vector of which blocks closed plus a value for each closed block, found by rank — packed a line at a time and end to end — beside plain four- and three-bit tables and a line-packed Huffman code, four seeds a point: bits a block, the line-packed split against its mean, lines read and work done a lookup, false-positive rates by seed, and the saving against block width.

All essays