Recently added

What's new

Essays arrive in groups rather than one at a time. The most recent group is below in full, and every earlier one after it, newest first.

Essays arrive in groups rather than one at a time, and a group usually opens up a subject not covered before. Between one group and the next nothing changes, so a reader who has seen the most recent group has seen everything.

15 October 2026

11 essays on when the algorithm is a table, one pass, and no room, the floors, when it does not fit, counting, the data that is not a number, when the algorithm flips a coin, the index that replaces the text, what a bound is, what the libraries do and what the machine does

160170180190200aligned sequence pairs in the corpus (log scale)test alignments optimal, of 200502008003,200the sum, told the propertypositions, fittedsix free classesthe chosen partition200 test pairs, eight seedsoptimal under the generator's costs When the algorithm is a table

A line the corpus finds

Substitution costs built as a transversion penalty plus the distance between the two letters' values on a graded property beat every partition of the pairs, when the model is told the property, and fall below the plainest matrix when told the wrong one. Fitting each letter's position on a line instead was predicted to find the property's order from about four times the told model's data and then behave like it. It finds the order on every corpus of 100 sequence pairs or more but one, and every miss starts with the two letters whose values differ by a fifth of a bit. But knowing the order is not knowing the costs. Its alignments are no better than six free classes' until 800 pairs, sixteen times the data the told model needed for the same score, and on a generator with no graded property at all it finds a line anyway, a quarter as long as a real one at 200 pairs, and pays for it against the plain partition.

6 figures
501002005001,0002,0005,000Zipf exponent of the stream (0 is flat)words for 10% on F3 (log scale)00.250.50.7511.52sample every positionshare 3%, list of 64share 1%, list of 8the best fixed list60,000 arrivals, 2,048 keysa word: a key or a count One pass, and no room

A share of a list of equals

Estimating the third frequency moment by finding the heavy keys first and sampling the rest works best with one counted candidate on steep streams and sixteen at Zipf 0.5, and a system that does not know its stream's skew has to pick. Keeping a long list and counting a candidate only if it holds a stated share of the list's own moment was predicted to come within a few per cent of the best fixed list on every exponent. The share does choose: at 3% it keeps one candidate at Zipf 2, three at Zipf 1, five at Zipf 0.5. But a list the stream shortens is not a short list. Its sixty-four candidates cost 192 words on every stream, four times the whole of the best estimator on the steep ones. On a flat stream every candidate is a near-equal share of a list of equals, and no share can tell that none of them is heavy. No list length and share comes within 10% everywhere; at Zipf 0.5 alone, the chosen list beats every fixed one.

6 figures
0.528321285122048-0.50000.500mean run of one value in a list, m ÷ k keys (log scale)comparisons a key the galloping saves64 keys a list256 keys a list1,024 keys a list4,096 keys a listabove nought: galloping saves64 pairs a point The floors

The runs a short list cannot have

A merge whose comparator can say 'equal' meets the counting floor on lists of repeated values, but it takes the gain only when one list's head happens to meet an equal head in the other. Comparing keys inside a list would find runs of equal keys on purpose, and was predicted to leave the worst case alone and help a great deal on average when the values are few. Solved exactly at four keys a list, it leaves the worst case alone and helps by under a hundredth of a comparison: a list of four has no run worth finding, and an equal answer already places two keys at once. On long lists the same idea, galloping over each run before comparing across again, does pay — a twenty-sixth of the textbook merge's comparisons with one value at 1,024 keys a list — and at every length it stops paying at the same place: when a value's run averages fewer than about thirty-two keys a list.

6 figures
50%60%70%80%90%100%the store's fill, share of its 1,048,576 keysshare of a perfect filter's saving captured13%25%38%50%75%100%a cluster at a time, dividedshuffled, divideda cluster at a time, evenshuffled, even2 bits a key on averagedashed: the same bits on every level When it does not fit

One bit a key, divided

A levelled store of a million clustered keys, each level's range filter a pair of prefix lengths chosen when the level is written, captures 68% of a perfect filter's saving at two bits a key spread evenly when its keys arrive shuffled, and 94% when they arrive a cluster at a time. The proposal was to divide the memory by what each level's filter saves per bit, and it predicted the memory would leave the small levels, whose keys the arrival order had already sorted out. It goes the other way, as it did for single prefix lengths, and much further: the small levels take twelve to sixteen bits a key, the deepest level one, and the store captures 99% of a perfect filter's saving on both orders while leaving part of the budget unspent. Divided, one bit a key on average does what four bits spread evenly cannot.

6 figures
00.500123458shortest run the record keeps, in keyscomparisons a key the pass addsthe pass's own cost, one comparison a key1,024 keys4,096 keys256 keysrandom keys, chains and runs, every answer checkedpass included Counting

The runs too short to drop

A pass between neighbours leaves the two-ended selection sort a record of 674 runs on 1,024 random keys, and the scans after it use them to repay three quarters of the pass. Keeping only runs of three keys or more was predicted to cut the record by two thirds and give back under a tenth of what it repaid. It cuts the record by 63%, and gives back 69%: runs of two keys answer half of everything the record answers, because three quarters of its answers are about two neighbouring positions — the question a scan asks about the key beside the extreme it has just found. The record's cost was never in its short runs. It was in writing each run as three words: every neighbouring pair at two bits a pair is 64 words, a thirty-second of the run list, and keeps all of them.

6 figures
01234block length, bits (log scale)levels' bits a character1531631272555111,023English words, as first builtrepetitive collection, as first builtEnglish words, runs and the flagrepetitive collection, runs and the flagdashed: a class a block, arrangements only16,384 characters, both trees The data that is not a number

The optimum the flag took away

A bidirectional index's wavelet-tree levels were coded in blocks of 63 bits because a sweep found 63 the best length: shorter blocks pay for more classes, longer ones for offsets that approach the block's own length. Then the classes were coded as runs and each offset given a flag to choose a code of its value changes instead. Lengthening the blocks to 127 bits was predicted to save on English words and the repetitive collection and to change nothing on random letters. It saves on all four texts, and there is no best length short of 1,023 bits on three of them: English words fall from 1.431 bits a character at 63 to 1.304 at 1,023. Under the codings the optimum came from, the same levels cost 3.728 at 1,023. The flag turned a long block's offset into the cost of its change points, which no block length moves, and left the length to be set by the one thing still growing with it: the work of a rank.

7 figures
33.203.403.60blocks in the filter (64-bit blocks, 16 bits a key)table bits a block1,93615,497123,885the mean needsized for the worst linemean + 0 sd, overflowmean + 1 sd, overflowmean + 2 sd, overflowthe line-packed split, four-bit valueswhole lines; two seeds When the algorithm flips a coin

The worst of more lines

A blocked Bloom filter's threshold table, stored as a bit for whether each block closed and a value for each block that did, packed a fixed number of blocks to a cache line, costs 3.64 bits a block at 64-bit blocks because every line must hold the most closed blocks any line draws. Sizing the line for the mean plus one standard deviation, and moving a line's excess values to a shared overflow, was predicted to cost about 3.25 bits and send under one closed lookup in a hundred to a third line. The lookups behave as predicted: 0.6% read the overflow, 2.003 lines a lookup. The bits land at 3.38, because every line must carry the offset where its moved values begin. What the measurement found instead is that the worst line is a property of how many lines there are: 3.38 bits at two thousand blocks and 3.62 at a hundred and twenty thousand, while the line sized for the usual line costs the same at every size.

6 figures
0%25%50%75%100%the escape's share of the coding spaceversion 020 filesversion 1040 filesversion 2281 filesversion 44175 filesin each group: growth seen, growth to come, half, one documentat each refit The index that replaces the text

One doubling late

A document array coded by its documents' lengths and refitted each time a growing directory's files double needs an escape for the files added in between, and the escape's right weight is the share of the characters still to come. Weighting it by the share that arrived since the previous refit was predicted to cut the mean loss below a quarter of a bit a symbol. It cuts nothing: 0.518 bits a symbol, against 0.389 for an escape that always takes half the coding space. The growth seen is the growth to come one doubling late — 71%, 50% and 27% where the doublings that followed brought 50%, 27% and 8% — because this directory's growth slowed at every doubling, and before the first refit there is no growth to read at all. A Huffman code also rounds every weight to a whole number of bits, so being right within a factor of two buys nothing. Started from half, the growth seen averages 0.356, the best a store can know, and the weight nobody can know reaches 0.183.

6 figures
the bandthe law, 7 partsthe band's part: 80169412601694169416907 by keys + band in 3 (9 parts)the band's part: 116117011706301172117011701170the law, 9 partsthe band's part: 681402140210241402140212526 by keys + band in 4 (9 parts)the band's part: 72136613649821366136413660%25%50%75%100%rank in the file; numbers are transfers a query in that part2^22 keys, blocks of 1,024a still band 2% wide What a bound is

The two parts the law was not given

The square-root law moves almost every cut of a seven-part file whenever its band of hot queries moves. A background cut once by keys, with only the part holding the band cut again, was proposed instead: a recut would rewrite a seventh of the file. It does, and it beats the law at every drift speed — 365 transfers a query against 417 at a moderate drift — but it wins with nine parts to the law's seven. Given nine, the law costs 322. Held to seven, the fixed background loses to the law at every speed. What survives the fair comparison is narrower and still useful: six parts by keys, with the band's part cut in four, ties the nine-part law on drift for a quarter of its rewriting. It loses on jumps, because a fixed background waits about three hundred queries to believe the band has left before it moves, and it pays double wherever a background cut runs through the band.

7 figures
125102050100buffer, as a share of the filemoves a key of the file (log scale)1/41/161/641/2561/10241/4096pieces by insertionpieces by mergingmerge sort, whole tail65,536 keys, a late tenth from the top fifthmoves counted, sorted checked What the libraries do

Pieces that fit the buffer

A sorted file with a late tenth appended costs the buffer-limited stable merge sort 2.93 moves a key at a buffer of a 64th, fifteen times the least any sort could move, and two thirds of it is the tail sorting itself. Taking the tail a buffer at a time, inserting each key into a sorted piece in the buffer and merging the piece into the head, was predicted to bring that under two moves a key at small buffers. It never comes within twice the merge sort: insertion costs a quarter of a piece a key, and the smaller the pieces the more often each one's merge crosses the head's top fifth. The same pieces sorted in place by merging do pay, at 2.41 moves a key, and for a reason the proposal did not name: a piece that fits the buffer merges into the head without a single rotation. The gain lasts only while the tail belongs near the top of the file.

6 figures
0%2%4%6%8%10%12%share of keys outside their buckets9.2%r = 0room 95.0%9.2%r = 1room 95.7%9.2%r = 2room 96.5%9.0%r = 4room 98.1%9.3%r = 8room 101.3%11.0%r = 16room 108.6%lower: in an overflow block · upper: in the region's reservekeys stay, 95% load What the machine does

Room the chains could not reach

A cuckoo table that caps its kick chain at two, churned at 95% load, keeps 9.2% of its keys outside their buckets while 13.8% of its entries stand empty. Keeping a few entries in each region aside for the keys the cap stops, at the same memory, was predicted to remove most of the overflow for a tenth of a kick an insertion. It removes none of it: the share outside the buckets stays at 9.2% with up to eight entries kept back, because a key is stopped by the pair of buckets its chain visits, never by a region short of room. What the reserve changes is where the stopped keys sit, and that is worth having. Read with the region's header, four entries a region take the overflow blocks from 9.2% of keys to 6.1% when keys stay and from 4.9% to 2.3% when keys go home with an open block, for no kick anyone can measure — until the reserve takes room the chains were using, at sixteen entries when keys stay and at eight when they go home.

6 figures

Before that

Everything published earlier, newest first. Titles only — the cards are on the full listing.

6 October 2026

6 essays on when the algorithm is a table, the floors, when it does not fit, two parameters, the index that replaces the text and what the machine does

3 October 2026

7 essays on one pass, and no room, counting, when the algorithm flips a coin, the data that is not a number, the index that replaces the text, what a bound is and what the libraries do

1 October 2026

9 essays on the floors, the data that is not a number, when it does not fit, two parameters, the index that replaces the text, what a bound is, what the machine does and what the libraries do

29 September 2026

10 essays on when the algorithm is a table, one pass, and no room, when it does not fit, counting, two parameters, when the algorithm flips a coin, what the libraries do and what the machine does

27 September 2026

14 essays on when the algorithm is a table, one pass, and no room, the floors, counting, when it does not fit, when the algorithm flips a coin, two parameters, what a bound is, what the libraries do and what the machine does

25 September 2026

11 essays on when the algorithm is a table, one pass, and no room, the floors, counting, when it does not fit, when the algorithm flips a coin, two parameters, what a bound is, what the libraries do and what the machine does

24 September 2026

11 essays on one pass, and no room, what a bound is, when the algorithm is a table, counting, when it does not fit, when the algorithm flips a coin, two parameters and what the machine does

23 September 2026

6 essays on what the libraries do, when the algorithm is a table, one pass, and no room, when it does not fit, counting and two parameters

20 September 2026

6 essays on one pass, and no room, the floors, counting, when the algorithm flips a coin, what a bound is and what the machine does

17 September 2026

9 essays on one pass, and no room, the floors, counting, when it does not fit, when the algorithm flips a coin and two parameters

16 September 2026

6 essays on one pass, and no room, the floors, when the algorithm is a table and counting

15 September 2026

14 essays on when the algorithm is a table, counting, when it does not fit, when the algorithm flips a coin, two parameters and what the machine does

12 September 2026

19 essays on when the algorithm is a table, counting, when it does not fit, when the algorithm flips a coin, two parameters and what the machine does

11 September 2026

20 essays on two parameters, when the algorithm is a table, counting, when it does not fit, when the algorithm flips a coin and what the machine does

10 September 2026

20 essays on when the algorithm is a table, two parameters, when the algorithm flips a coin, when it does not fit and counting

5 September 2026

50 essays on the floors, the data that is not a number, what the machine does, what a bound is, the index that replaces the text, what is taught wrongly, structures, the other axis and what the libraries do

1 September 2026

20 essays on the data that is not a number, what is taught wrongly, the index that replaces the text, what the libraries do, structures, the other axis, what the machine does and what a bound is

31 August 2026

20 essays on what the machine does, what the libraries do, what a bound is, the index that replaces the text, the data that is not a number, the other axis, structures and what is taught wrongly

30 August 2026

20 essays on what a bound is, what the libraries do, the data that is not a number, the index that replaces the text, the other axis, structures and what is taught wrongly

28 August 2026

20 essays on the index that replaces the text, what the machine does, the data that is not a number, what a bound is, what the libraries do, the other axis and what is taught wrongly

27 August 2026

10 essays on the floors, the other axis, one pass, and no room, structures and what is taught wrongly

26 August 2026

10 essays on what a bound is, what the libraries do, the other axis, one pass, and no room, structures and what is taught wrongly

24 August 2026

10 essays on one pass, and no room, structures, what is taught wrongly, the other axis, what the libraries do and the floors

23 August 2026

10 essays on the data that is not a number, the floors, the index that replaces the text, what the machine does, the other axis and what is taught wrongly

21 August 2026

10 essays on what a bound is, one pass, and no room, the other axis, the floors, structures and what is taught wrongly

20 August 2026

10 essays on the index that replaces the text, when the algorithm is a table, what the machine does, the data that is not a number, the other axis, the floors and what is taught wrongly

19 August 2026

10 essays on one pass, and no room, what a bound is, the other axis, when the algorithm flips a coin, the floors, what is taught wrongly and structures

18 August 2026

10 essays on when the algorithm is a table, the other axis, the data that is not a number, what the machine does, what is taught wrongly and structures

16 August 2026

10 essays on one pass, and no room, when the algorithm flips a coin, the floors, what is taught wrongly and structures

15 August 2026

10 essays on the data that is not a number, when the algorithm flips a coin, what the libraries do, the floors, what is taught wrongly and structures

14 August 2026

8 essays on when it does not fit, what is taught wrongly, the floors, what the machine does and structures

11 August 2026

12 essays on what the libraries do, what the machine does, what a bound is, structures, counting and what is taught wrongly

10 August 2026

12 essays on when the algorithm flips a coin, what a bound is, counting, what is taught wrongly, structures and the floors

5–8 August 2026

34 essays on counting, two parameters, the other axis, what is taught wrongly, what the machine does, the floors, structures and what a bound is

Every essay, by subject · by subject · by thread · search