Benchmarking — where it appears
Named by 7 essays across 4 fields — each of them below, with the objects they name alongside it.
Counting instead of timing
A stopwatch measures the laptop it runs on. A counter measures the algorithm. Every number on this site comes from an array that increments a tally each time it is read, written, compared or swapped — which makes the counts exact, reproducible to the last digit, and identical on every machine that has ever built this page.
How close anything gets to the floor
The interesting question about a sorting algorithm is not its complexity class but its distance from the bound nothing can cross. Merge sort comes within 2.2% of the information-theoretic floor. Heapsort uses 96% more than it needs to. Selection sort uses nineteen times. Those three numbers say more than the classification does.
A count over every input
Run five sorts on every one of the 40,320 orderings of eight elements and read off each one's best, mean and worst comparison count. Then mark where the inputs a benchmark generator names — sorted, reversed, nearly sorted, random, few unique — land. For merge sort, heapsort and quicksort with a median-of-three pivot, the worst case is an ordering none of them produces, and for the last of the three every named input lands on its best case.
The rule that pays on a long enough text
With two patterns, the cheap tables cost 106 steps and the scan reads 13,084 characters; the exact tables cost 594 and the scan reads 12,306. Below thirty-two thousand characters the cheap tables win the total, above it the extra skipping pays for them, and with thirty-two patterns there is no crossing at all.
Where the exact rules pay now
With a construction as cheap as the published one, the exact shift rules pay for themselves past eight thousand characters of text at two patterns, four thousand at four, and never at thirty-two — because by thirty-two patterns the two rules make identical decisions.
A parameter that waits for another
Four merge fold shapes over thirty-two evenly loaded shards leave errors of 665, 667, 665 and 667 — a fifth of a per cent apart. Give the same four shapes shards whose loads span seventeen-fold and they leave 148, 183, 323 and 403. The parameter did nothing until a second parameter moved, and every measurement that fixed the second one saw nothing.
The spread a merge sort does not have
On 65,536 random keys quicksort's comparison count has a standard deviation of 42,470 and merge sort's has one of 141.9 — three hundred times smaller, on means only a quarter apart. Merge sort's count is a fixed bill minus 65,535 independent leftovers, one per merge, each worth about a coin toss, so its spread grows as the square root of n where quicksort's grows as n. From 256 keys up a single run of merge sort pins its mean to one per cent, and a third of quicksort's whole spread is decided by its first pivot.
Named alongside it
The objects these essays reach for when they reach for this one.
Comparison countDistributionMeasurementParameter choiceQuicksortTrade offCommentz-walterCounterexampleCrossoverMerge sortMulti patternOperation count