Field

What the machine does

The operation count is not the running time. Locality, cache lines and branch behaviour decide the constant, and they rank algorithms differently from the textbook count.
10⁵10⁶10³10⁴10⁵comparisonscache misses (modelled)Insertion sortSelection sortBubble sortMerge sortHeapsortQuicksort, firstQuicksort, median-3Quicksort, randomShellsortMerge + cutofffully associative · 64 lines × 8 elements · LRUa modelled count, not a time

The count is not the time

Why the textbook operation count and the running time diverge, what a stated cache model can and cannot establish, and how carrying two independent counts changes the ranking of the same algorithms.

0%25%50%75%100%645124,09665,536cache holds 512array size n (elements)miss ratefully associative · 64 lines × 8 elements · LRU20,000 random accesses per point

The cliff where the data stops fitting

The working-set curve measured, why the transition from hitting to missing is a cliff rather than a slope, what it means for an algorithm whose cost per element does not change with n, and the two kinds of miss that behave differently.

Merge sort49% sequential · 7,540 accesses2560Heapsort15% sequential · 14,044 accesses2560time (accesses, left to right) · index (bottom to top)one run each, n = 256every access plotted

Where an algorithm looks

The access-trace picture for each sorting algorithm, what its shape says about locality, why the recursive algorithms narrow and the heap-based one does not, and what a measure of sequentiality can and cannot capture.

481632641281010010³10⁴noperations (mean of 60 runs)Insertion trafficMerge trafficInsertion cmpMerge cmptraffic crossessolid: reads + writes · dashed: comparisonstraffic crosses between n = 12 and 16; comparisons never do

Where insertion sort actually wins

The insertion-sort cutoff every standard library implements, measured in two counts at once — the comparison count never crosses and the total read-and-write count does, between twelve and sixteen elements.

10⁵10⁶10⁴10⁵comparisonsmispredictions (modelled)Insertion 0%Selection 1%Bubble 29%Merge sort 52%Heapsort 27%Quicksort, first 24%Quicksort, median-3 39%Quicksort, random 27%Shellsort 51%Merge + cutoff 40%Timsort 41%Introsort 32%pdqsort 43%Dual-pivot 37%2-bit counters, no historysquares are the sorts that ship

The branch the machine guesses

A modelled two-bit branch predictor as the fifth counter, why it re-ranks the algorithms four other counters had already ranked, and what a predictable comparison actually is.

10³10⁴10⁴array length ntotal over 2,000 searchesbranchy, comparisonsbranchless, comparisonsbranchy, mispredictsbranchless, mispredicts2-bit counters, no historysolid: comparisons · dashed: mispredicts

A search with no branch to miss

Branchless binary search measured against the ordinary one, why its mispredictions do not grow with n, and the overflow bug that lived in every published binary search for twenty years.

Sorted array (binary search)2 blocksLevel order4 blocksvan Emde Boas2 blocksmemory address, left to right · alternating outlines are blocksB = 8, M = 64 (M/B = 8)4 blocks against 2, for the same 6 comparisons

Two searches, one comparison count

The same tree in three memory layouts, with the comparison count held identical, so that the entire difference in cost is the arrangement — the clearest case yet of two counters ranking the same work differently.

—sitting—kitten111111101111110-1111110-1-111110-1-1-11110-1-1-10110-1-1-10-11one unit = one subproblem given a value-1, 0, 1 — 3 values, 2 bits each

A column computed in machine words

Myers's bit-parallel edit distance, why the unit has to change before the result can be stated, and the crossover where doing more elementary work stops paying.

balanced, bits49,2616.01 b/chHuffman-shaped, bits39,0614.77 b/chranks per access5.003.95entropy = 3.90 bitsEnglish-like, sigma = 22 · upper bar balanced, lower Huffman-shapedH0 = 3.90 bits/symbol

Rank is the only thing it does

What a rank query actually costs — the two-level directory and its block-length trade, the wavelet tree that turns an alphabet rank into a handful of bit ranks, and the text's entropy appearing as an operation count.

1,00010,00010³bits of select supportpositions inspected, worst casebinary search, no extra bits: 510L=8L=256L=8L=256one position per L onesdense and sparse6,554 ones in 65,536 positions · sub-blocks of 8worst cases, every k

Select is not rank backwards

Constant-time select over a bit vector — why its directory has two cases, what the bound is measured in, and the density at which the structure that costs nothing wins.

text position iphi(i)anchor: a run start25 of thema text that repeats itself · 4 copies of 32r = 26 · pieces 24

A function with r pieces

Why the function that turns one occurrence into all of them is piecewise linear with as many pieces as the transform has runs, and what checking that costs.

0000200121200the parenthesis sequencethe minimum excess of each block of 213 blocks · lookup table 72 bits, sharedblock 2 · 24 parentheses13 blocks

The table that fits inside a block

Where constant time comes from in a succinct range minimum — the ±1 property, the block length that makes an exponential table small, and what a query actually touches.

48163264128110entry width, bytescache lines a lookup readsentries inlinea line of tags in front8,192 slots, load 0.9, buckets of 8keys the table holds

A lookup that stops caring how wide an entry is

A cuckoo table's entries held inline in the bucket against the same table with a line of one-byte tags in front, counted in distinct cache lines a lookup reads, swept over entry width, bucket size, tag width and load — with the accidental tag matches counted rather than assumed.

level 0 · left585 keptlevel 1 · right+351 smallerlevel 2 · right+113 smallerlevel 3 · right+39 smallerlevel 4 · right+40 smallerpositions still in play, and the half the code leaves behindsmaller symbols before position 700: 543rank of "s": 42 — from the same 5 operationsσ 21 · code 011115 ranks, not 105

The count that was already there

Why a compound wavelet-tree walk costs exactly what a plain rank costs, measured on an operation counter, and the other places the same identity was available and unused.

acdefghijklmnopqrstuvwyzroot24 of 26 symbols present54 ranks · 51 nodes

A node costs two ranks

The arithmetic identity that halves an interval-enumeration descent, why no check on its output can catch its absence, and what an operation count has to claim instead.

01e+42e+43e+44e+450100one sampled position in every …bitsplain, one bit a rowcompressed blocksElias-Fano16,384 rowscrosses at one in 4

Twenty bits apart

Where Elias–Fano and a block-classed bit vector cross, why they behave oppositely across the density dial, and what decides between them at the crossing.

Thompson's NFA, a set of states43.724 to buildthe subset DFA, built in full1.011,297 to buildthe subset DFA, built on demand41.824 to buildthe state set in machine words37.224 to buildoperations per character; the note is what was paid once, before the first character4,096 characters · k = 643.7 down to 1

What a character costs on four machines

Four regular-expression machines measured per character on one text, and the two currencies a matcher's cost is paid in.

rounds, if every ready cell ran at oncerow by row65,793column by column65,793anti-diagonal by anti-diagonal513reads that miss the cacherow by row6.3%column by column28.2%anti-diagonal by anti-diagonal31.1%fully associative · 32 lines × 8 elements · LRU263,169 reads and writes per order

The order with the best depth

An edit-distance table filled in three orders on a row-major array, with every read replayed through a cache model. The anti-diagonal order has the smallest depth and the worst locality, row order the reverse, and a cache large enough to hold an anti-diagonal's rows erases the difference.

linear probingchainedcuckoo, two tables0240.10.20.30.40.50.60.70.80.9load factorentries read per lookup0120.10.20.30.40.50.60.70.80.9load factorcache misses per lookup8,192 slots, 64 cache lines of 8a probe is an entry read; a miss is a line fetched

Two probes are two misses

Linear probing, chaining and cuckoo hashing measured in entries read and in cache misses per lookup, across loads. Cuckoo reads the fewest entries at every load it can be built and misses more often than linear probing at every one of them, while a cache that holds the whole table reverses the picture for chaining alone.

linear probingcuckoo, two tablescuckoo, buckets of 8024680.30.450.60.750.850.95load factorentries read per lookup00.511.50.30.450.60.750.850.95load factorcache misses per lookup8,192 slots, 64 cache lines of 8a probe is an entry read; a miss is a line fetched

The bucket that fits a line

Cuckoo hashing with buckets of two, four and eight slots, measured against linear probing and plain cuckoo hashing in entries read, cache misses per lookup, the most lines any one lookup touches, and misses for keys the table does not hold. Aligned buckets bound every lookup to two lines and miss less than linear probing at every load from 0.3 to 0.95; keys the table does not hold cost them a flat 1.87 misses, which linear probing beats until a load between 0.75 and 0.85.

rounds, if every ready cell ran at oncerow order, stored by rows65,793anti-diagonal order, stored by rows513anti-diagonal order, stored by diagonals513reads that miss the cacherow order, stored by rows6.3%anti-diagonal order, stored by rows31.1%anti-diagonal order, stored by diagonals8.7%fully associative · 32 lines × 8 elements · LRU263,169 reads and writes per order

The table stored the way it is filled

An edit-distance table stored by rows and by anti-diagonals, filled in row, column and anti-diagonal order, with every read replayed through caches of 2 to 1,024 lines. The diagonal layout gives the anti-diagonal order a streaming miss rate at its own depth, three streams to row order's two, and needs one more cache line than row order to reach it.

124816326410step width, cellscache lines a stepstored by diagonalsby diagonals, each line-alignedstored by rows512 × 512, lines of 8 cellsone step of an anti-diagonal

Eight cells at once

An edit-distance table's anti-diagonals computed several cells at a time, counted in distinct cache lines per step and per cell on three layouts, swept over the step's width and the table's size — with the step's own resident set measured as the limit on how wide a step is worth making.

a second hash, unordered0.054 entries read on an absent key14.435.79 storedthe bits below the bucket, unordered0.059 entries read on an absent key14.435.79 storedthe bits below the bucket, sorted0.057 entries read on an absent key8.975.21 storedthe bucket's own bits, sorted5.743 entries read on an absent key11.275.80 storedtag comparisons an absent key's lookup makes8,192 slots · buckets of 8 · 8-bit tags38% fewer

A tag that answers more than yes

A cuckoo table's tag array, cut three ways out of a 32-bit hash and kept sorted: the tag comparisons an absent and a stored key's lookup makes, the entries each cut causes to be read, how many of a bucket's keys a tag can tell apart at each width, and what a sorted bucket costs to maintain as the bucket grows.

501001500.2000.4000.6000.800load: keys held ÷ slotsbytes held per keythe entry, 32 bytesslotted entriespacked, a pointer a bucketpacked, groups of 4 bucketspacked, groups of 1616,384 slots, buckets of 8, 32-byte entrieslabels at the lowest load

Entries found by the rank of their tag

A bucketed cuckoo table with sorted one-byte tags, its entries kept either in fixed slots or packed by tag rank behind a pointer per group of buckets: the bytes held per key across load, the bytes an insertion moves including cuckoo kicks, the entry width below which a pointer a bucket costs more than the empty slots save, and what a larger group buys against what it costs to insert.

1248163210010³spare entries a group of 16 buckets keepsbytes moved an insertionslack at the endslack spread, region rewrittenslack spread, nearest gapborrowedslotsload 0.5, groups of 16dashed: the slotted layout

The slack an insertion can reach

Packed layouts of a bucketed cuckoo table of 16,384 slots with 32-byte entries, reallocation charged: slack at the end of each group's region, slack divided among the buckets with the region rewritten when a bucket's run fills, and slack divided among the buckets with the nearest free entry borrowed — bytes moved an insertion against the spare kept, the share that is copying, bytes moved against load, bytes held against bytes moved for every group size and amount of slack, and bytes moved against group size.

05001e+31.5e+3load when the insertion is madebytes moved an insertion50%70%90%95%spare divided evenlyspare by recent insertionsspare by room leftfixed slots16 spare a region of sixteen bucketsdashed: fixed slots

Spare where a bucket can use it

The packed cuckoo table of 2^16 slots in buckets of eight, 32-byte entries packed by tag rank in regions of sixteen buckets with the nearest free slot borrowed, its spare divided at each redistribution evenly, by each bucket's insertions since the last one, or by the entries each bucket can still take: bytes moved an insertion against the load, how full the buckets are, where each rule's spare sits, the bytes of the kick chains, and bytes moved at the highest load against the amount of spare.

05001e+31.5e+3load when the insertion is madebytes moved an insertion50%70%90%95%no capcap 2, overflow in the sparecap 2, overflow in a blocksixteen spare a region, divided by room leftcharged since the previous tenth

The keys a capped chain leaves behind

A packed cuckoo table of 2¹⁴ slots in buckets of eight, 32-byte entries packed by tag rank in regions of sixteen buckets with sixteen spare entries divided by room left, filled to 95% load with each key's kick chain capped at 0 to 8 kicks and the key still displaced put in an overflow, either in the free slot nearest its region's end or in a separate block a region: the bytes an insertion moves against load, the kicks the cap removes, where the bytes go, every cap for both places, and the overflow entries a failed lookup must compare.

filter bits for each overflow keyfailed lookups that reach a block0246810120%25%50%75%100%at 95% loadat 90% loadone filter16,384 absent keys, cap of twodashed: one Bloom filter at those bits

The lookups a filter turns away

A packed cuckoo table of 2¹⁴ slots in regions of sixteen buckets, filled to 95% load with each kick chain capped at two and the stopped keys kept in a separate block a region, with a Bloom filter over each region's overflow keys asked by 16,384 absent keys: the share of failed lookups that reach a block against the filter's bits, the blocks a lookup reads because it asks two regions, double hashing against a mix for every probe, filters sized to the overflow against a fixed filter a region, the stopped keys filed under their first bucket, and the failed and successful lookups an insertion at which the cap breaks even.

00.0200.0400.0600.0800.10002468operations, in multiples of the table's 16,384 entriesshare of keys in overflowkeys stay, 9.2%home at every deletion, 4.0%home when open, 4.9%16,384 entries, buckets of eight, cap of two95% load

The block already open

A cuckoo table of 16,384 entries in buckets of eight, regions of sixteen buckets, a kick chain capped at two and stopped keys filed under their first bucket's region, filled to 85%, 90% or 95% and churned through 131,072 deletions each followed by an insertion: the share of keys in overflow through the churn with keys left in place, moved home at every deletion, and moved home only when their block is read or written anyway; blocks read and written and keys homed for each pair; the settled share at three loads; blocks touched against successful lookups a deletion; and kicks and stops an insertion.

0%2%4%6%8%10%12%share of keys outside their buckets9.2%r = 0room 95.0%9.2%r = 1room 95.7%9.2%r = 2room 96.5%9.0%r = 4room 98.1%9.3%r = 8room 101.3%11.0%r = 16room 108.6%lower: in an overflow block · upper: in the region's reservekeys stay, 95% load

Room the chains could not reach

A cuckoo table of 16,384 entries in buckets of eight, regions of sixteen buckets, a kick chain capped at two and stopped keys filed under their first bucket's region, with 0, 1, 2, 4, 8 or 16 of each region's 128 entries kept back for stopped keys at the same memory, filled to 85%, 90% or 95% and churned through 131,072 deletions each followed by an insertion: the share of keys in overflow blocks through the churn; the split of keys outside their buckets between reserve and block; kicks and stops an insertion under three homing policies; the blocks' share under each policy; the shares at three loads; and overflow blocks touched against successful lookups.

All essays