The thread: Distributions, not averages — page 3
The folklore is about a matcher
Four million steps against three hundred and twenty-nine, on a twenty-character expression matched against twenty characters. One of the two machines doubles with every character of the input and the other does not, and only one of them is what a regular expression is.
What a bound isEvery cut the law moves
A file cut into seven parts by the square-root law, and recut when the law says the saving pays, was charged a full rewrite for every recut. Charging only the parts whose boundaries change was predicted to cut a recut's cost to a tenth on slow drift. It cuts it by 2%. When the band of hot queries moves, the law moves 98% of its cuts, a median of nineteen thousand ranks each, because the parts on either side of the band have to share out what the band left. Holding small moves back keeps a quarter of the file and costs more a query; recutting only the parts around the band costs up to three times as much.
The index that replaces the textOne doubling late
A document array coded by its documents' lengths and refitted each time a growing directory's files double needs an escape for the files added in between, and the escape's right weight is the share of the characters still to come. Weighting it by the share that arrived since the previous refit was predicted to cut the mean loss below a quarter of a bit a symbol. It cuts nothing: 0.518 bits a symbol, against 0.389 for an escape that always takes half the coding space. The growth seen is the growth to come one doubling late — 71%, 50% and 27% where the doublings that followed brought 50%, 27% and 8% — because this directory's growth slowed at every doubling, and before the first refit there is no growth to read at all. A Huffman code also rounds every weight to a whole number of bits, so being right within a factor of two buys nothing. Started from half, the growth seen averages 0.356, the best a store can know, and the weight nobody can know reaches 0.183.
The index that replaces the textRefitting when the escape is wrong
A growing directory's document array, coded by a Huffman code over its files' lengths and refitted each time the files double, loses at best 0.356 bits a symbol for 5.65 final arrays of rewriting, and even an escape weighted by the growth still to come loses 0.183. A clock read off the structure — refit when the share of characters named through the escape passes twice the share its code word stands for — was predicted to refit about six times and halve the loss. It refits 25 times, eleven in the first twelve versions, and rewrites 10.6 arrays for a loss of 0.042. Held back until the characters have grown by a quarter since the last refit, it refits 11 times and loses 0.098 bits a symbol for 4.52 arrays: under every doubling schedule on both counts, including the one that reads the future. It sees anything only if the escape starts small: weighted as half the coding space, the escape's code word is one bit, nothing behind it can ever pass twice a half, and the clock never fires.
Two parametersA switch that hears the traffic stop
A landmark watch switched off by a count of the traffic's stretch ends cannot see traffic slow down, because a slowing is the events stopping, and a count of events keeps its last reading. Letting the open spacing since the last stretch end into the reading, once it is longer than the recorded ones, was predicted to turn the watch back on within four thousand queries of a slowing and cost nothing on steady traffic. It costs nothing on steady traffic, and on ten stretch ends it hears the slowing late, after three to eleven thousand queries, because the silence is averaged with nine fast spacings. On five or three it hears it in time, and the switch on five comes within two cells of the best known policy on every stream.
Two parametersA silence that overrules the count
A landmark watch switched to the corners by a count of the traffic's stretch ends heard a slowing only once the silence outweighed the fast spacings it was averaged with, about 5,900 queries on ten stretch ends. Letting an open spacing of 1,600 queries overrule the count outright was predicted to make the count's length stop mattering on the stream that slows. It hears the slowing within a thousand queries at every count from three to ten, and keeps the corners 8% of the slow half where the count kept them 65 to 75%; the counts still differ, by 3.4 cells a query, because the fast half is where a short count keeps the corners that pay. On the stream that speeds up the override is the cheapest switch at every count, and on steady traffic it is within a cell and a half of the silence.