The thread: The threshold is the algorithm — page 2
A refit that changes one landmark
Choosing four landmarks again after traffic moves costs hundreds of thousands of cells of search, and on drifting streams it never paid. Keeping three and re-choosing the fourth costs a fifth as much — 0.19 of a full selection, not the twelfth the earlier page guessed. On drifting streams, refitting on the counter goes from a loss of up to 13% to within 2% of fitting once, and it wins where the destination moves six cells. When the destination jumps, it is the cheapest policy measured, 12% under the corners at one jump in 10,000 queries — unless the jumps come as often as one in 2,500.
One pass, and no roomA share of a list of equals
Estimating the third frequency moment by finding the heavy keys first and sampling the rest works best with one counted candidate on steep streams and sixteen at Zipf 0.5, and a system that does not know its stream's skew has to pick. Keeping a long list and counting a candidate only if it holds a stated share of the list's own moment was predicted to come within a few per cent of the best fixed list on every exponent. The share does choose: at 3% it keeps one candidate at Zipf 2, three at Zipf 1, five at Zipf 0.5. But a list the stream shortens is not a short list. Its sixty-four candidates cost 192 words on every stream, four times the whole of the best estimator on the steep ones. On a flat stream every candidate is a near-equal share of a list of equals, and no share can tell that none of them is heavy. No list length and share comes within 10% everywhere; at Zipf 0.5 alone, the chosen list beats every fixed one.
When the algorithm is a tableA second word judged by its own share
A spelling checker that offered its commonest nearest word alone when that word's count was many times the next, and two or three words otherwise, beat a coin's mix of one-word and three-word lists by at most two and a half points, and a plain list of two did better. The ratio measures how dominant the first word is, not how likely the second is. Judged instead by its own share of everything within two edits of the misspelling, the second word is shown only when that share is at least 0.15: on short words drawn by use the checker then sits 4.5 points over the coin's line at 1.32 words a query, nearly twice the ratio's lift and above the list of two, and the weight given a two-edit word moves the result by a tenth of a point. A second word with a share under 0.01 is never the source. On misspellings drawn evenly the share is no guide at all, and the list of two wins.
Two parametersThe candidates near where the queries end
Re-choosing one landmark after the traffic moves tried every candidate as the replacement, and most of them could not win. Tried only on the four candidates nearest the new destination, the refit costs 0.085 of a full selection — the twelfth an earlier page guessed — against 0.236, and its placements serve fresh queries within 3% as well. Where the full search's winner lies further away, the refit was barely worth making. A refit that cheap still cannot follow a destination that jumps every 2,500 queries; what holds the counter back was never the refit's price.
When the algorithm flips a coinThe bits a second threshold costs
A blocked Bloom filter whose first block carries a threshold was proposed a second threshold on each key's second block, with a third block for keys both refuse. It does what it was meant to: at 512-bit blocks it removes nearly a third of the load spread the first threshold left. And the rate does not fall — 0.066% against 0.062% — because the second table, paid for in blocks, raises the rate the filter would have with no spread at all by more than the narrower spread saves. The threshold was stored in more bits than it needs: at four to six bits, one threshold beats the eight-bit design by about a tenth on one run, and a second threshold of the best width ties it at 512-bit blocks.
The floorsThe runs a short list cannot have
A merge whose comparator can say 'equal' meets the counting floor on lists of repeated values, but it takes the gain only when one list's head happens to meet an equal head in the other. Comparing keys inside a list would find runs of equal keys on purpose, and was predicted to leave the worst case alone and help a great deal on average when the values are few. Solved exactly at four keys a list, it leaves the worst case alone and helps by under a hundredth of a comparison: a list of four has no run worth finding, and an equal answer already places two keys at once. On long lists the same idea, galloping over each run before comparing across again, does pay — a twenty-sixth of the textbook merge's comparisons with one value at 1,024 keys a list — and at every length it stops paying at the same place: when a value's run averages fewer than about thirty-two keys a list.
One pass, and no roomA list that gives back its places
An estimator of the third frequency moment keeps a list of candidate heavy keys beside a Count-Sketch and samples the rest, and cutting a long list at the end of the pass to the candidates that hold a share of its moment picked the right keys and paid for the whole list. Dropping a candidate during the pass, as soon as it falls below that share, was predicted to make the list's memory follow the stream: under four candidates on steep streams, within a fifth of the best fixed list, and about ten on Zipf 0.5 with most of the earlier win kept. The list ends the pass the right length — three candidates on Zipf 1 — but its memory is its peak, and the peak is set by the twenty-fifth arrival, before any key has stood out: nineteen candidates on Zipf 1, and on flatter streams the most a share θ allows, one over θ. Charged its peak, it beats the end-of-pass cut on every stream from Zipf 0.75 up, 208 words against 364 at Zipf 1, and never reaches the best fixed list, 159.
Two parametersStarting from the corners
A landmark search that refits one landmark when its traffic moves lost to the fixed corners when the destination jumped often, and the accounting blamed the one full selection every refitting policy makes at the start. Starting from the corners instead, and never selecting in full, closes 82% of that gap at one jump in 2,500 queries and is the cheapest policy of all at one in 5,000. It gives the full selection back its advantage when jumps are rare, and on traffic that drifts a step at a time it gives up fitting altogether. Its counter fires when the cost rises, and from the corners concentrated traffic raises nothing: the corners are no worse for it, only no better.
When the algorithm flips a coinA threshold knows only its own block
A blocked Bloom filter that gives each block a threshold on a hash of the key lets a lookup work out which of two blocks a key went to, and it spends a small table to do it. Sharing one threshold among a group of blocks would shrink the table, and was proposed as the next saving. It is a loss at every group size. With one four-bit threshold a block, 512-bit blocks reach a false-positive rate of 0.061%; shared by a pair, 0.071%; shared by sixteen, 0.100%, worse than no threshold at all. Neighbouring blocks fill at unrelated moments, so a shared threshold closes when the first of its group does, and every key it turns away lands in a second block chosen at random.
The floorsA merge that decides per run whether to gallop
Merging two sorted lists of repeated values, a merge that gallops along every equal run beats the textbook merge while the runs average more than about thirty keys a list and loses below, a rule about the whole input. A merge that learns the runs as it goes was predicted to match the better of the two on uniform values and beat both on skewed ones. The proposal's rule — gallop when the last few runs averaged long — cannot learn a run that has not ended: at one, two or four values it walks every key. Timsort's own rule, applied to equal answers — walk until a bar of consecutive equal answers, then gallop, the bar moving by one as galloping pays or fails — does what was predicted. On uniform values it stays within a hundredth of a comparison a key of the better merge; under Zipf 1, at 4,096 keys a list and 1,024 values, it makes 0.447 comparisons a key against 0.595 walking and 0.594 galloping, a quarter under both.
One pass, and no roomA check as slow as what it checks
A burst detector that subtracts a rising background's offset goes on subtracting it after the rise ends, and loses small bursts in the trough that opens. Capping the subtraction by what the difference itself shows, averaged over one slow half-life, was predicted to close the trough to under half a spread. It lifts it from 1.46 spreads to 1.24, because that average is as late as the estimate it checks. Averaged over a quarter of the time it lifts it to 0.70 and catches small bursts in the trough more often than any other design but the one that alarms on a fifth of its readings. Neither cap helps after a large burst, which every slope correction reads as a rise.
Two parametersA refit priced before it is made
A landmark placement that starts at a maze's corners and refits one landmark when a counter sees recent queries getting dearer closed most of its gap on traffic that jumps — and on traffic that drifts it never fired, because from the corners concentrated traffic raises no cost. The chance it missed is cost that could fall. Pricing that chance directly — the current placement on the last twenty queries against four nearby candidates, every two thousand queries, refitting when the estimated saving pays for the refit — costs 120.5 cells a query at one jump in 2,500 against the counter's 128.9 and the corners' 126.8, and 107.4 on a slow drift against 103.3 for a full selection fitted once. It fails where jumps come every thousand queries, and on one stream it stops firing after its placement has aged.
When the algorithm flips a coinTwo hashes that shared their low bits
Two pages on blocked Bloom filters with thresholds each recorded an instability they could not explain: a false-positive rate that collapsed on one seed at 256-bit blocks, and a pair of four-bit thresholds measuring 0.169% where its neighbours measured 0.08%. Both came from how a hash was brought into range. Every filter took its block and its ordering value as a modulo of a multiply-shift hash, and the low bits of such a hash depend only on the low bits of the key, so the two values were dependent whenever the block count had a factor of two to spare. At 992 blocks the threshold filter measures anything from 0.185% to 0.645%; read from the hash's top bits, 0.126% to 0.133%.
When the algorithm flips a coinA code that has to fit its worst line
A blocked Bloom filter's four-bit thresholds are far from uniform: two blocks in five never close, and the rest crowd into the upper values, an entropy of 2.4 to 2.8 bits. A table coded from that distribution was predicted to match the four-bit rate on under three bits a block. Packed end to end it does, at 2.65 to 3.03 bits, and it costs every lookup a second read for the index. Packed a fixed number to a cache line it needs no index, but every line must hold its worst run of long code words, which costs 3.19 bits at 64-bit blocks and eighty-three code words decoded a lookup. Where the table is large enough for the saving to show, a plain three-bit table gets part of it with none of the costs.
Two parametersA horizon the stream can supply
A watch that refits a landmark placement when an estimated saving pays lost where traffic jumped every thousand queries, because its estimate assumed each saving would last a whole window. The horizon proposed to correct it, learned from the intervals between the watch's own refits, is the window every time: the watch looks once a window, so its refits are never closer together than one. A horizon learned from the traffic instead — how long it stays within ten cells of one destination — is safe on every stream and saves 3.4% at one jump in 2,500. At one in 1,000 it reaches 150.7 cells a query against the corners' 142.7, and a horizon read from the future only ties them. At that pace the looking costs what the refits save.
When the algorithm flips a coinA bit for whether it closed
Almost half of a blocked filter's threshold table says one thing: this block never closed. Store that answer as a bit a block, and a closing value only for the blocks that did, found by counting the ones before it. At 64-bit blocks the prediction was 3.1 bits a block and two cache lines. Packed end to end it lands at 3.15 bits and 2.5 lines; packed a line at a time it reads two lines and counts one word, and costs 3.64 bits, because every line must hold the most closings any line happens to draw. At 512-bit blocks, where three blocks in five close, it saves under a tenth of a bit.
Two parametersA watch that looks less
A landmark watch that refits when an estimated saving pays loses to the plain corners when traffic jumps every thousand queries, because its looking costs more than its refits save. Using its learned horizon to look less often there — doubling the interval when the horizon falls below half of it, halving it when the horizon comes back — was predicted to tie the corners at that pace and cost nothing elsewhere. It ties nothing: 150.5 cells a query against the corners' 142.7, since the looks it skips are the looks that found the chances. And at one jump in 2,500, where the watch had won, it loses by twenty cells, because the rule cannot come back down. Halving needs a horizon as long as the stretched interval, and at 8,000 queries no stream here supplies one. A two-state rule that can come back does better and still loses at both paces. The looking was never the waste; at one jump in 1,000 there is nothing to watch for.
The floorsA bar that wins rarely meet
A merge of two sorted lists of repeated values saves most of a textbook merge's comparisons by galloping along a run of equal answers once a bar of them have come in a row, with the bar lowered after a gallop that pays and raised after one that does not. Timsort keeps the same kind of bar for the other kind of stretch, one list winning repeatedly, and a merge of repeated values meets both. One count and one bar for either kind was predicted to save a further tenth under Zipf 1 and change nothing on uniform values. It saves at most 1.0% under Zipf 1, costs up to 1.3% on uniform values, and two separate bars do no better. In two lists drawn from the same distribution a stretch of wins long enough to gallop over is rare: under Zipf 1 at most 18% of the shared bar's gallops are over wins, and where wins dominate the gallops, at a thousand uniform values, the runs of either kind are too short for any gallop to pay.
When the algorithm flips a coinThe worst of more lines
A blocked Bloom filter's threshold table, stored as a bit for whether each block closed and a value for each block that did, packed a fixed number of blocks to a cache line, costs 3.64 bits a block at 64-bit blocks because every line must hold the most closed blocks any line draws. Sizing the line for the mean plus one standard deviation, and moving a line's excess values to a shared overflow, was predicted to cost about 3.25 bits and send under one closed lookup in a hundred to a third line. The lookups behave as predicted: 0.6% read the overflow, 2.003 lines a lookup. The bits land at 3.38, because every line must carry the offset where its moved values begin. What the measurement found instead is that the worst line is a property of how many lines there are: 3.38 bits at two thousand blocks and 3.62 at a hundred and twenty thousand, while the line sized for the usual line costs the same at every size.
Two parametersA watch that switches itself off
A landmark watch that cannot beat the plain corners by looking less was given a switch instead: at every look it prices the corners on the same twenty queries, and when the corners would have been cheaper over its last four looks it goes back to them and stops looking for 8,000 queries. At one jump in 1,000 it works — 145.5 cells a query comparing search costs, 144.4 charging the watch for its own looking and refits, against the corners' 142.7 and the watch's 150.7. At one jump in 2,500, where watching pays, both switches give the watch's advantage away, 125.3 and 126.1 against 116.4, and the charging switch is off for 60% of the stream, as much as at the fast pace. What a look reads is the same at both paces: 12,249 cells a window at one, 11,257 at the other, with a spread of sixty thousand. The watch's value arrives in a few large looks after jumps, and four looks at any pace are mostly the others.
When the algorithm flips a coinAn offset that only has to reach its neighbours
A blocked Bloom filter's threshold table, its cache lines sized for the mean number of closed blocks plus one standard deviation and their excess values moved to a shared overflow, cost 3.38 bits a block where 3.25 was predicted, because every line carries a nineteen-bit offset into the overflow. Grouping lines, each line's offset reaching only its group's part and each group's position held once in an array of its own, was predicted to cost 3.31 bits a block for 2.0064 lines a lookup. It costs 3.301 for 2.0060, as predicted. But the nineteen bits were mostly an offset sized to address a table of moved values, when the overflow this table has needs eleven: sized for the overflow it has, the ungrouped line costs 3.337 for no extra line at all. Grouping's further three hundredths cost a line a moved lookup, and spent on room instead, that line buys more. At equal lines a lookup the grouped table is the dearer, by a hundredth to nine hundredths of a bit.
Two parametersA switch that counts the jumps
A landmark watch whose switch read its own costs could not tell fast traffic from slow, because what a look reads has the same mean at both paces. A switch that reads the traffic instead — the mean spacing of the last ten times the destination moved more than ten cells — was predicted to reach the corners at one jump in 1,000 and keep the watch at every slower pace. It separates the paces where costs could not: 146.6 cells a query at one jump in 1,000 against the watch's 150.7, and on drifting traffic it never switches off at all. But a count of k events is a reading with a delay of k events and a noise of one part in √k, and no k has both: three stretch ends reach the corners' 142.7 at the fast pace and are off 38% of the slower stream; twenty never misfire and save nothing. And when traffic slows, the events that would show it stop arriving, so the count keeps its fast reading and the switch stays off: 135.8 cells a query on a stream that slows halfway, worse than the corners or the watch.
When the algorithm flips a coinA reserve sized for the spill
A split threshold table gives every line the same room, the mean number of closed blocks plus some standard deviations. Giving each line room for the mean and each group of 64 lines a shared reserve at one standard deviation of the group's total was predicted to cost about 3.24 bits a block for 2.006 lines a lookup, under the best ungrouped line. The reserve it predicts holds 52 values where the group's lines spill 145: it is full in every group, the spills go on to a third line, and the table lands on the ungrouped curve. Sized for the spill instead, the reserve puts the table under that curve at every room, by two to three hundredths of a bit a block.
When the algorithm flips a coinThe room the lines lend
A blocked filter's threshold table gives each 512-bit line room for its closed blocks' values, and a group reserve sized for the spill had just put it under the ungrouped curve. A line under its room leaves slots empty that are already paid for, so its neighbours' spills could be carried forward into them, with no reserve at all: predicted at about 3.12 bits a block for 2.012 lines a lookup. The bits are right and the lines are not. At room for the mean a carried value passes 12.8 lines before it finds a slot, because a run of lines that is room-for-the-mean on average is a queue with nothing to spare. Given three quarters of a deviation of room, the walk falls under two lines and the carried table lands a hundredth of a bit under the reserve.
When the algorithm flips a coinA walk that stops at two lines
A blocked filter's threshold table, its lines sized for the mean number of closed blocks and each line's surplus carried forward into the next lines' empty slots, was cheapest in bits — 3.113 a block — and dear in lines, 2.183 a lookup, because a carried value joins a queue with no slack and walks 12.8 lines. Capping the walk was predicted to keep most of the bits, about 3.13, for under 2.02 lines, two hundredths under a reserve sized for the spill. Capped at two lines, the walk sends 53% of its carried values to overflow, and the table costs 3.180 bits a block for 2.027 lines. That is a hundredth under the spill-sized reserve at the same lines and a few thousandths under the uncarried line's curve: at room for the mean most of what joins the queue would wait more than two lines, so the overflow becomes the reserve by another name. With room to spare, every cap loses to the reserve.
When the algorithm flips a coinA queue that drains at intervals
A blocked filter's threshold table, its lines sized for the mean number of closed blocks and each line's surplus carried into the next lines' free slots, is cheapest in bits and dear in lines, because at room for the mean the queue of carried values never drains. Capping the walk turned the cap into an overflow. Giving every eighth line a deviation of extra room was predicted to empty the queue there, bound every walk by eight lines, and cost about 3.15 bits a block for under 2.02 lines a lookup. It costs 3.123 bits for 2.103 lines. After a slack line the queue is empty a third of the time, walks reach 33 lines, and the reason is arithmetic: one line's deviation of room is not room for eight lines' wander, which grows as the square root of eight. The slack lines do buy something — a hundredth or two of a bit a block under spreading the same room evenly, at five of the six spacings — but they buy it at lines no other design here would accept.
What the libraries doA looser budget wastes a larger share
More errors permitted means more work, and the fraction of that work which was never going to help rises with it — from thirty-one per cent at no errors to seventy-five at two. The saving is worth most where the search is most expensive.
What the libraries doThe saving that is a loss
An operation that is seventy-eight times cheaper on a branching search costs twice as much on an exact one. It reports every symbol present in order to hand back the one that was asked for, and a search that knows its character needs none of the rest.
What the libraries doWhere the table starts paying
Five thousand and sixty-five operations before the first character, then one per character. Against nothing before the first character and thirty-nine per character. They cross at two hundred and fifty-six characters, and that crossing is what an engine's compile decision actually is.
What is taught wronglyA cache below the reachable set
A lazy machine with a cache of two hundred and fifty-six states costs fifty-one operations a character and a plain non-deterministic simulation costs fifty-three. At five hundred and twelve it costs eleven. The line is flat across two orders of magnitude and then falls off a cliff.