Field

What the machine does

The operation count is not the running time. Locality, cache lines and branch behaviour decide the constant, and they rank algorithms differently from the textbook count.
10⁵10⁶10³10⁴10⁵comparisonscache misses (modelled)Insertion sortSelection sortBubble sortMerge sortHeapsortQuicksort, firstQuicksort, median-3Quicksort, randomShellsortMerge + cutofffully associative · 64 lines × 8 elements · LRUa modelled count, not a time

The count is not the time

Why the textbook operation count and the running time diverge, what a stated cache model can and cannot establish, and how carrying two independent counts changes the ranking of the same algorithms.

0%25%50%75%100%645124,09665,536cache holds 512array size n (elements)miss ratefully associative · 64 lines × 8 elements · LRU20,000 random accesses per point

The cliff where the data stops fitting

The working-set curve measured, why the transition from hitting to missing is a cliff rather than a slope, what it means for an algorithm whose cost per element does not change with n, and the two kinds of miss that behave differently.

Merge sort49% sequential · 7,540 accesses2560Heapsort15% sequential · 14,044 accesses2560time (accesses, left to right) · index (bottom to top)one run each, n = 256every access plotted

Where an algorithm looks

The access-trace picture for each sorting algorithm, what its shape says about locality, why the recursive algorithms narrow and the heap-based one does not, and what a measure of sequentiality can and cannot capture.

481632641281010010³10⁴noperations (mean of 60 runs)Insertion trafficMerge trafficInsertion cmpMerge cmptraffic crossessolid: reads + writes · dashed: comparisonstraffic crosses between n = 12 and 16; comparisons never do

Where insertion sort actually wins

The insertion-sort cutoff every standard library implements, measured in two counts at once — the comparison count never crosses and the total read-and-write count does, between twelve and sixteen elements.

10⁵10⁶10⁴10⁵comparisonsmispredictions (modelled)Insertion 0%Selection 1%Bubble 29%Merge sort 52%Heapsort 27%Quicksort, first 24%Quicksort, median-3 39%Quicksort, random 27%Shellsort 51%Merge + cutoff 40%Timsort 41%Introsort 32%pdqsort 43%Dual-pivot 37%2-bit counters, no historysquares are the sorts that ship

The branch the machine guesses

A modelled two-bit branch predictor as the fifth counter, why it re-ranks the algorithms four other counters had already ranked, and what a predictable comparison actually is.

10³10⁴10⁴array length ntotal over 2,000 searchesbranchy, comparisonsbranchless, comparisonsbranchy, mispredictsbranchless, mispredicts2-bit counters, no historysolid: comparisons · dashed: mispredicts

A search with no branch to miss

Branchless binary search measured against the ordinary one, why its mispredictions do not grow with n, and the overflow bug that lived in every published binary search for twenty years.

Sorted array (binary search)2 blocksLevel order4 blocksvan Emde Boas2 blocksmemory address, left to right · alternating outlines are blocksB = 8, M = 64 (M/B = 8)4 blocks against 2, for the same 6 comparisons

Two searches, one comparison count

The same tree in three memory layouts, with the comparison count held identical, so that the entire difference in cost is the arrangement — the clearest case yet of two counters ranking the same work differently.

sittingkitten111111101111110-1111110-1-111110-1-1-11110-1-1-10110-1-1-10-11one unit = one subproblem given a value-1, 0, 1 — 3 values, 2 bits each

A column computed in machine words

Myers's bit-parallel edit distance, why the unit has to change before the result can be stated, and the crossover where doing more elementary work stops paying.

balanced, bits49,2616.01 b/chHuffman-shaped, bits39,0614.77 b/chranks per access5.003.95entropy = 3.90 bitsEnglish-like, sigma = 22 · upper bar balanced, lower Huffman-shapedH0 = 3.90 bits/symbol

Rank is the only thing it does

What a rank query actually costs — the two-level directory and its block-length trade, the wavelet tree that turns an alphabet rank into a handful of bit ranks, and the text's entropy appearing as an operation count.

1,00010,00010³bits of select supportpositions inspected, worst casebinary search, no extra bits: 510L=8L=256L=8L=256one position per L onesdense and sparse6,554 ones in 65,536 positions · sub-blocks of 8worst cases, every k

Select is not rank backwards

Constant-time select over a bit vector — why its directory has two cases, what the bound is measured in, and the density at which the structure that costs nothing wins.

text position iphi(i)anchor: a run start25 of thema text that repeats itself · 4 copies of 32r = 26 · pieces 24

A function with r pieces

Why the function that turns one occurrence into all of them is piecewise linear with as many pieces as the transform has runs, and what checking that costs.

0000200121200the parenthesis sequencethe minimum excess of each block of 213 blocks · lookup table 72 bits, sharedblock 2 · 24 parentheses13 blocks

The table that fits inside a block

Where constant time comes from in a succinct range minimum — the ±1 property, the block length that makes an exponential table small, and what a query actually touches.

48163264128110entry width, bytescache lines a lookup readsentries inlinea line of tags in front8,192 slots, load 0.9, buckets of 8keys the table holds

A lookup that stops caring how wide an entry is

A cuckoo table's entries held inline in the bucket against the same table with a line of one-byte tags in front, counted in distinct cache lines a lookup reads, swept over entry width, bucket size, tag width and load — with the accidental tag matches counted rather than assumed.

level 0 · left585 keptlevel 1 · right+351 smallerlevel 2 · right+113 smallerlevel 3 · right+39 smallerlevel 4 · right+40 smallerpositions still in play, and the half the code leaves behindsmaller symbols before position 700: 543rank of "s": 42 — from the same 5 operationsσ 21 · code 011115 ranks, not 105

The count that was already there

Why a compound wavelet-tree walk costs exactly what a plain rank costs, measured on an operation counter, and the other places the same identity was available and unused.

acdefghijklmnopqrstuvwyzroot24 of 26 symbols present54 ranks · 51 nodes

A node costs two ranks

The arithmetic identity that halves an interval-enumeration descent, why no check on its output can catch its absence, and what an operation count has to claim instead.

01e+42e+43e+44e+450100one sampled position in every …bitsplain, one bit a rowcompressed blocksElias-Fano16,384 rowscrosses at one in 4

Twenty bits apart

Where Elias–Fano and a block-classed bit vector cross, why they behave oppositely across the density dial, and what decides between them at the crossing.

Thompson's NFA, a set of states43.724 to buildthe subset DFA, built in full1.011,297 to buildthe subset DFA, built on demand41.824 to buildthe state set in machine words37.224 to buildoperations per character; the note is what was paid once, before the first character4,096 characters · k = 643.7 down to 1

What a character costs on four machines

Four regular-expression machines measured per character on one text, and the two currencies a matcher's cost is paid in.

rounds, if every ready cell ran at oncerow by row65,793column by column65,793anti-diagonal by anti-diagonal513reads that miss the cacherow by row6.3%column by column28.2%anti-diagonal by anti-diagonal31.1%fully associative · 32 lines × 8 elements · LRU263,169 reads and writes per order

The order with the best depth

An edit-distance table filled in three orders on a row-major array, with every read replayed through a cache model. The anti-diagonal order has the smallest depth and the worst locality, row order the reverse, and a cache large enough to hold an anti-diagonal's rows erases the difference.

linear probingchainedcuckoo, two tables0240.10.20.30.40.50.60.70.80.9load factorentries read per lookup0120.10.20.30.40.50.60.70.80.9load factorcache misses per lookup8,192 slots, 64 cache lines of 8a probe is an entry read; a miss is a line fetched

Two probes are two misses

Linear probing, chaining and cuckoo hashing measured in entries read and in cache misses per lookup, across loads. Cuckoo reads the fewest entries at every load it can be built and misses more often than linear probing at every one of them, while a cache that holds the whole table reverses the picture for chaining alone.

linear probingcuckoo, two tablescuckoo, buckets of 8024680.30.450.60.750.850.95load factorentries read per lookup00.511.50.30.450.60.750.850.95load factorcache misses per lookup8,192 slots, 64 cache lines of 8a probe is an entry read; a miss is a line fetched

The bucket that fits a line

Cuckoo hashing with buckets of two, four and eight slots, measured against linear probing and plain cuckoo hashing in entries read, cache misses per lookup, the most lines any one lookup touches, and misses for keys the table does not hold. Aligned buckets bound every lookup to two lines and miss less than linear probing at every load from 0.3 to 0.95; keys the table does not hold cost them a flat 1.87 misses, which linear probing beats until a load between 0.75 and 0.85.

rounds, if every ready cell ran at oncerow order, stored by rows65,793anti-diagonal order, stored by rows513anti-diagonal order, stored by diagonals513reads that miss the cacherow order, stored by rows6.3%anti-diagonal order, stored by rows31.1%anti-diagonal order, stored by diagonals8.7%fully associative · 32 lines × 8 elements · LRU263,169 reads and writes per order

The table stored the way it is filled

An edit-distance table stored by rows and by anti-diagonals, filled in row, column and anti-diagonal order, with every read replayed through caches of 2 to 1,024 lines. The diagonal layout gives the anti-diagonal order a streaming miss rate at its own depth, three streams to row order's two, and needs one more cache line than row order to reach it.

124816326410step width, cellscache lines a stepstored by diagonalsby diagonals, each line-alignedstored by rows512 × 512, lines of 8 cellsone step of an anti-diagonal

Eight cells at once

An edit-distance table's anti-diagonals computed several cells at a time, counted in distinct cache lines per step and per cell on three layouts, swept over the step's width and the table's size — with the step's own resident set measured as the limit on how wide a step is worth making.

All essays