Concept

Memory hierarchy — where it appears

The layered arrangement of registers, caches and main memory whose transfer costs differ by orders of magnitude and appear in no operation count. Its transfer costs differ by orders of magnitude and appear in no operation count, which is why a second counter is needed to explain a clock.

Named by 7 essays across 4 fields — each of them below, with the objects they name alongside it.

block transfersIn order, 0 to n−11,0241.0× a scan · 64.0 elements per transferEvery B-th element (B = 64)65,53664.0× a scan · 1.0 elements per transferUniformly at random61,40760.0× a scan · 1.1 elements per transfera scan of this array is 1,024 transfersB = 64, M = 4,096 (M/B = 64)64× between the cheapest order and the dearest

One access, eight kilobytes

Every count on this site charges one for an array access. A machine charges for a block. The same 65,536 accesses cost 1,024 transfers in one order and 65,536 in another, with nothing about the algorithm's work changed — a factor of 64, which is exactly the number of elements in a block, and which no counter here could see until now.

applied · Transfer
10⁵10⁶10³10⁴10⁵comparisonscache misses (modelled)Insertion sortSelection sortBubble sortMerge sortHeapsortQuicksort, firstQuicksort, median-3Quicksort, randomShellsortMerge + cutofffully associative · 64 lines × 8 elements · LRUa modelled count, not a time

The count is not the time

An operation count is exact, machine-independent, and not a running time. The gap between them is mostly memory, and it is large enough to reorder the rankings. This site carries a second count — modelled cache misses from the same runs — and asserts that the two disagree, because if they agreed the second one would carry no information.

machine · Machine
1010010B (elements per block)block transfers per searchtuned for B = 64Sorted arrayB-tree tuned for B = 64van Emde Boas — told nothingM = 16,384, B as drawnone layout, 7 block sizes, no parameter

The layout that is told nothing

A B-tree is built around a block size somebody looked up. The van Emde Boas layout is given neither the block size nor the memory size, and across seven block sizes spanning a factor of 64 it tracks the best structure that was told them. An algorithm with no parameters making a claim at every level of the hierarchy at once is a strange thing to be able to measure, and this is what it costs.

applied · Transfer
elements of block written per key insertedB-tree, in place49.3Log-structured, T = 23.0 · 16× less than the treeLog-structured, T = 42.0 · 25× less than the treeLog-structured, T = 81.0 · 49× less than the treeLog-structured, T = 161.0 · 49× less than the treeB = 64, M = 4,096 (M/B = 64)25× between the two structures at T = 4

The writes nobody counted

Sixteen thousand keys inserted into a B-tree write 49.3 elements' worth of blocks for every key stored. The same keys into a log-structured store write 2.0. Every operation counter reports the two as the same work — the same insertions, the same comparisons, the same number of updates — and the factor of 24 decides which structure a storage engine is built from.

applied · Transfer
least recently usedthe offline optimumA loop over 7 blocks35 accesses35113.18×Random over 7 blocks35 accesses971.29×A straight sweep35 accesses35351.00×B = 8, M = 48 (M/B = 6)3.2× on the loop

The block that is not a block

This field's model has one memory, one block size, one processor, no prefetcher, no queue and no clock, and every number in it is a count of transfers rather than a duration. A real machine has five levels, reads ahead, issues a hundred requests at once, and charges four orders of magnitude more for one kind of transfer than another. What survives the difference is worth stating exactly, and so is what does not.

wrong · Transfer
levels on the path (outline) · transfers actually paid (filled)B = 411 levels · 6.99 transfers · 16,384 blocks residentB = 166 levels · 3.23 transfers · 4,096 blocks residentB = 644 levels · 1.99 transfers · 1,024 blocks residentB = 2563 levels · 1.30 transfers · 256 blocks residentB = 10243 levels · 1.01 transfers · 64 blocks residentbinary search over the same 4,194,304 keys: 20 transfersB = 1024, M = 65,536 (M/B = 64)20× between binary search and the widest tree

A tree with nodes the size of a block

A B-tree is a binary search tree that has read the hardware manual. Its node holds as many keys as fit in one transfer, so the height falls from log₂ n to log_B n — and the measured cost falls further still, to 1.01 transfers over four million keys, because the top of the tree is small enough to stay in memory. The comparison count goes up.

structures · Transfer
0.020.050.10.20.5121235810filter bits a keywasted reads per absent lookupno filters: all 3 levelssame rate every levelrates sized to levelsn = 65,536, size ratio 4, first run 1,024 keysthe same memory at every point

The filter each run carries

A log-structured store turns every lookup for a missing key into a read of every level, and a Bloom filter on each run buys those reads back with memory. Spread five bits a key evenly across three levels and a missing key still wastes 0.279 reads. Give the small levels more bits and the large one fewer — the same memory — and it wastes 0.201. At four levels the gap is 0.382 against 0.209, because sized filters stop the waste growing with the number of levels.

applied · Transfer

Named alongside it

The objects these essays reach for when they reach for this one.

Block transferCost modelExternal-memory modelB-treeData movementLocalityLsm treeSimulationTrade offWrite amplificationAccess patternBinary search

All concepts