Generator

Elements written per key inserted, 16,384 random keys

Rendered here at the parameters it defaults to, with every essay that calls it — which is the same list as the blast radius of changing it.
Elements written per key inserted, 16,384 random keysThe same 16,384 insertions into an in-place B-tree and into a log-structured store at four size ratios. The B-tree dirties one leaf per key and that leaf is evicted before it is touched again, so it writes a whole block — 49 elements — for each key. The log-structured store writes each key once per level it passes through and writes in whole blocks, so it moves 2.0 elements per key at T = 4. Neither number appears in any operation count, and the second structure exists entirely because of the first.elements of block written per key insertedB-tree, in place49.3Log-structured, T = 23.0 · 16× less than the treeLog-structured, T = 42.0 · 25× less than the treeLog-structured, T = 81.0 · 49× less than the treeLog-structured, T = 161.0 · 49× less than the treeB = 64, M = 4,096 (M/B = 64)25× between the two structures at T = 4

Elements written per key inserted, 16,384 random keys

The same 16,384 insertions into an in-place B-tree and into a log-structured store at four size ratios. The B-tree dirties one leaf per key and that leaf is evicted before it is touched again, so it writes a whole block — 49 elements — for each key. The log-structured store writes each key once per level it passes through and writes in whole blocks, so it moves 2.0 elements per key at T = 4. Neither number appears in any operation count, and the second structure exists entirely because of the first.

Drawn at 700 × 310, wide on the page. Everything above is what write-amp returns with no arguments; the caption is the generator's own, computed from the numbers in the drawing rather than written beside it.

15 essays call write-amp. The drawing above is what it returns with no arguments at all; every call below passes it something, because a placement that passes nothing draws whichever member of the family the generator happens to default to rather than the one its essay argues about — which is what optcheck and figfill exist to catch.

Where it is called

Changing this generator changes every one of these figures.

elements of block written per key insertedB-tree, in place49.3Log-structured, T = 23.0 · 16× less than the treeLog-structured, T = 42.0 · 25× less than the treeLog-structured, T = 81.0 · 49× less than the treeLog-structured, T = 161.0 · 49× less than the treeB = 64, M = 4,096 (M/B = 64)25× between the two structures at T = 4 When it does not fit

The writes nobody counted

1100ε — the exponent the fanout is B toelements written per key33445679ε = 1 — the B-tree ·fanout 256 · 3transfers a queryε = 0.5 · fanout 16 · 5a querythe number above eachpoint is what a querycostsB = 256, M = 16,384 (M/B = 64)131,072 random keys When it does not fit

One dial between two structures

0.020.050.10.20.5121235810filter bits a keywasted reads per absent lookupno filters: all 3 levelssame rate every levelrates sized to levelsn = 65,536, size ratio 4, first run 1,024 keysthe same memory at every point When it does not fit

The filter each run carries

234681016110size ratioblock transfers · levelsa range of 100 keyslevelsan absent point lookup1,048,576 keys, 5 bits a keyfilters answer one of these two When it does not fit

The read a filter has no key for

0%25%50%75%100%8101214161820222426283032bits of key prefix the filter is built onshare of the empty levels the filter skips83%72%64%46%ranges of 0.1 keysranges of 1 keyranges of 10 keysranges of 100 keys5 bits a key, 4 levels, 2,000 ranges eachdigits: the best length's share When it does not fit

The runs a range query can skip

235810filter bits a key, averaged over the storetransfers a query3.804.204.605.00one length, even bitsa length each, even bitsa length and bits eachthe oraclemixed workload, 2^20 keys, four levelsdotted: the oracle When it does not fit

The small levels' share of a range filter

456filter bits a key, over the whole storetransfers a range query258101216203216 to 24 bits, asked where it fits16 to 24 bits, asked at everylengthone length, chosen for five bitsone length, chosen for each memorythe oraclefour levels, 2^20 keysdotted: the oracle When it does not fit

The resolution a range filter can afford

-0.20000.2000.4000.6000.8001filter bits a keymulti-resolution minus one length25810162032keys in 16 clustersuniform keyshalf the ranges in the gaps, half at the keysbelow zero: multi-resolution cheaper When it does not fit

One length for the gaps and the clusters

-0.300-0.200-0.10000.1000.2000.300filter bits a keytransfers a query saved against one length a level41024the best pair a levelmulti-resolutionhalf the ranges in the gaps, half at the keysabove zero: cheaper than one length When it does not fit

Two lengths that do the work of eleven

shuffled arrivals036925%50%75%100%one cluster at a time036925%50%75%100%share of the 2^20 keys inserteddark: chosen each write · olive: chosen once · orange: none · green: perfect10 bits a key; transfers a range querychosen once: 12 and 22 bits When it does not fit

The filter rebuilt with every write

50%60%70%80%90%100%the store's fill, share of its 1,048,576 keysshare of a perfect filter's saving captured13%25%38%50%75%100%a cluster at a time, dividedshuffled, divideda cluster at a time, evenshuffled, even2 bits a key on averagedashed: the same bits on every level When it does not fit

One bit a key, divided

40%60%80%100%shuffled, 0.5 bits a keyshuffled, 1 bits a keyshuffled, 2 bits a keyshuffled, 4 bits a keya cluster at a time, 0.5 bits a keya cluster at a time, 1 bits a keya cluster at a time, 2 bits a keya cluster at a time, 4 bits a keybar: even split · large dot: fitted · dark dot: rule, δ chosen · small dot: rule, δ = 4charged on unseen queriesshare of the saving When it does not fit

A division chosen before the queries arrive

0.4000.6000.8001share of the keys arrivedshare of a perfect filter's saving1/81/43/81/23/41fitted to the queriesthe size rule, δ = 4the even splitthe shuffled store, 1 bit a keycharged on unseen queries When it does not fit

The sizes a store passes through

0.6000.8001share of the keys arrivedshare of a perfect filter's saving1/81/43/81/23/41fitted to the queriesthe rule, floor of half a bitthe rule, no floorthe shuffled store, 1 bit a keycharged on unseen queries When it does not fit

A floor under every level

shuffled, 0.5 bits a key34.8 span34.8 flooredshuffled, 1 bits a key2.3 span4.1 flooredshuffled, 2 bits a key4.1 span4.2 flooredshuffled, 4 bits a key4.1 span4.1 flooredclustered, 0.5 bits a key9.1 span9.1 flooredclustered, 1 bits a key29.6 span0.9 flooredclustered, 2 bits a key30.7 span0.7 flooredclustered, 4 bits a key30.4 span0.6 flooredworst points behind the fitted division over the filltwo orders, four budgetssix checkpoints When it does not fit

A span the queries do not see

The library, page 5 of 5 — where write-amp sits