Series

Symbol — the series

27 essays on one idea, from the one that introduces it to the one that assumes the rest.
  1. 048162432characters every key sharesoperations072,581145,162Character comparisonsRadix sort, characters readElement comparisonsone unit = one character comparisonelement comparisons constant at 3,955

    The comparison that is not one comparison

    Sorting 512 keys costs 3,955 comparisons whatever the keys are, and between 7,849 and 134,409 character examinations depending only on how much those keys have in common. The first number is the one every bound so far is stated in. The second is the one the machine pays, it grows without limit, and nothing here has ever counted it.

    part 1 · text
  2. characters examinedNaive scan520,25663.51 per text characterKnuth–Morris–Pratt16,3211.99 per text characterBoyer–Moore–Horspool8,1290.99 per text characterRabin–Karp00.00 per text characterone unit = one character comparisonone matcher read no characters at all

    The shift the pattern already knows

    On a text of 8,192 characters the naive scan examines 520,256 of them and Knuth–Morris–Pratt examines 16,321 — a factor of 32, and 16,321 is 99.6% of the 2n that no input can push it past. On ordinary random text the same two algorithms examine 20,862 and 20,833. Both measurements are of the same pair of algorithms and only one of them is the reason anybody uses the second.

    part 2 · text
  3. one read per text character24816326495alphabet size, m = 8characters read ÷ text length0.01.12.2Naive scanKnuth–Morris–PrattBoyer–Moore–Horspoolone unit = one character comparison · n = 20,000, m = 8Horspool's best here: 0.132 per character

    The text that answers without reading it

    Boyer–Moore–Horspool finds every occurrence of an eight-character pattern in a twenty-thousand-character text while examining 2,985 characters. Not 2,985 comparisons of eight characters each — 2,985 characters, 0.149 per character of text. It is a correct algorithm returning a complete answer about a text it has mostly not looked at, and the reason it can is a property of the alphabet rather than of the algorithm.

    part 3 · text
  4. 1,00010,000110100modulus qverifications in one searchq = 257: 693measuredn / qchosen text: 6%one unit = one character comparison · base 2568 colliding blocks at q = 1,009, none at q = 1,000,003

    A match decided by a number

    Rabin–Karp replaces a comparison of two strings with a comparison of two integers, and pays for it only when the integers agree by accident. On 65,521 windows at a modulus of 1,009 it verified 79 times against a predicted 64.9. On a text of 1,024 characters chosen with the modulus in hand it verified at 64 windows out of 64 possible, each one costing fifteen of the pattern's sixteen characters — and the closed form is silent about that, because it assumes nobody chose the text.

    part 4 · randomness
  5. 11010010⁴10⁵queries answeredcharacter comparisons, cumulative5 queriesScan each timeIndex, then queryone unit = one character comparison · 4 doubling roundsbreak-even at 5 queries

    The index that is the text

    A suffix array sorts all 4,097 suffixes of a text — 8.4 million characters of string, in total — and examines exactly zero characters doing it. It then answers a search in 91 characters where a scan costs 1,472, and the whole thing pays for itself at six queries. Both halves of that are worth the same amount of attention, and the first is the one that is usually skipped.

    part 5 · structures
  6. 8 patterns of 10 characters over four symbols02,0004,0006,0008,00010,00012,00014,00016,00018,000one cell = 10 positions · shade = fraction read61.6% of the text

    The shift a set of patterns allows

    Aho-Corasick reads every character of the text exactly once, whatever the number of patterns. Commentz-Walter reads backwards inside a window and steps over what it can, and on eight patterns of ten characters it looks at sixty-two per cent of a twenty-thousand-character text. The rule that does the skipping is not the one everybody implements.

    part 6 · text
  7. the same 16 patterns, one of them shortenedshortest 1658.4%mean shift 3.15shortest 1262.8%mean shift 2.99shortest 869.7%mean shift 2.64shortest 680.5%mean shift 2.35shortest 493.4%mean shift 1.98shortest 399.2%mean shift 1.71shortest 2100.0%mean shift 1.3716 patterns, 15 of them 16 characterstext 20,000

    The ceiling the shortest pattern sets

    A matcher that skips is described as faster than one that reads every character, and the description leaves out what decides it. No shift can exceed the shortest pattern in the set, so adding one two-character pattern to fifteen of sixteen characters takes a run from reading fifty-eight per cent of the text to reading all of it twice.

    part 7 · wrong
  8. ·1, 4b1, 3b1, 2b4, 2a4, 2a2, 2a4, 2b4, 2b4, 2b4, 2patterns abbb, baab, bbab · shortest 4d₁, d₂ under each node · a filled node ends a patternthe reversed-pattern trie10 nodes

    The shift somebody published

    The exact rules for shifting a multi-pattern window are a definition that quantifies over every pattern at every offset. The 1979 rules are two tables read off the trie's own failure links, they are computed in one pass, and on this pattern set they agree with the definition at every node.

    part 8 · text
  9. short by 133.6% of pairsshort by 211.2% of pairsshort by 411.2% of pairsthe same shift7994.0% of pairspatterns of 10 over four symbolsthe published rule is never the larger of the twoshift decisions84 pairs · 21 nodes

    What the approximation gives up

    Compared at every one of the 3,736 decisions a search could ask about, the published shift rules and the exact ones agree at all of them on a set of 128 patterns. At two patterns they differ at five of 84, by up to four positions — and the run reads 6.3% more characters.

    part 9 · wrong
  10. 101001,00010010³10⁴10⁵characters of pattern in the setcomparisons before the scanboth rules, exactlybad character onlythe 1979 shift functionsfour symbols · patterns of 10137.35x at 128 patterns

    The table that walks every pair

    The exact shift rules cost 769,724 character comparisons to build for 128 patterns and the published ones cost 5,604. The scan they are both built for reads 41,580 characters, so one of the two constructions is eighteen times the work it is there to save.

    part 10 · bounds
  11. 1,00010,00010³10⁴characters of textcomparisons, built and scannedcrossing at n = 32,000both rules, exactlythe 1979 tables2 patterns of 10 · four symbolscrossing n = 32,000

    The rule that pays on a long enough text

    With two patterns, the cheap tables cost 106 steps and the scan reads 13,084 characters; the exact tables cost 594 and the scan reads 12,306. Below thirty-two thousand characters the cheap tables win the total, above it the extra skipping pays for them, and with thirty-two patterns there is no crossing at all.

    part 11 · practice
  12. 101001,00010010³10⁴10⁵total length of the pattern setprimitive stepsthe definitionfrom the linksthe published rulefour symbols · m = 1085.60x apart at 128 patterns

    The table the links already knew

    The exact good-suffix rule costs 757,058 character comparisons to build from its definition at 128 patterns, and 3,824 from the trie's failure links. Same table, checked at every node — a factor of 198, and the definition was never the algorithm.

    part 12 · text
  13. good suffix, defined757,058good suffix, from links3,824bad character, defined12,666bad character, swept5,168128 patterns · 1,280 characters of patternfour symbols · m = 10198x and 2.5x

    The case a failure link does not cover

    Compute the exact good-suffix rule from the failure links alone and eighteen of twenty-two entries come out too large. The matcher then steps past sixty-six of two thousand and twenty-five occurrences, and every match it does report is a match.

    part 13 · bounds
  14. 101001,00010100total length of the pattern settimes the published rule's precomputationwritten as a definitionbuilt from the linksthe published rulefour symbols · m = 10137x becomes 1.60x

    The ratio that was an implementation

    This collection published a factor of forty-two between two shift rules' preprocessing. Sixty-nine per cent of the denominator was a table the published rule never reads, and the numerator was a definition rather than a construction. The corrected ratio is 1.6.

    part 14 · wrong
  15. 1,00010,00010³10⁴characters of textsteps, precomputation plus scancrossing at 8,000published rulesexact rules2 patterns · four symbolscrossing 8,000 · was 32,000

    Where the exact rules pay now

    With a construction as cheap as the published one, the exact shift rules pay for themselves past eight thousand characters of text at two patterns, four thousand at four, and never at thirty-two — because by thirty-two patterns the two rules make identical decisions.

    part 15 · practice
  16. level 0 · left335 keptlevel 1 · right+207 smallerlevel 2 · left61 keptlevel 3 · right+33 smallerlevel 4 · left8 keptpositions still in play, and the half the code leaves behindsmaller symbols before position 400: 240rank of "m": 8 — from the same 5 operationsσ 21 · code 010105 ranks, not 105

    Every child at once

    A bidirectional extension counts the occurrences of every symbol smaller than the one being added, and this collection did it with one rank per symbol — 139.7 operations an extension on a twenty-six-letter alphabet. One walk down the tree gives the same number.

    part 16 · indexes
  17. level 0 · left585 keptlevel 1 · right+351 smallerlevel 2 · right+113 smallerlevel 3 · right+39 smallerlevel 4 · right+40 smallerpositions still in play, and the half the code leaves behindsmaller symbols before position 700: 543rank of "s": 42 — from the same 5 operationsσ 21 · code 011115 ranks, not 105

    The count that was already there

    A bit vector's rank of one is the position minus its rank of zero. Every walk down a wavelet tree computes one of them at every level, and this collection asked for the other separately for years.

    part 17 · machine
  18. a fixed-length code97,718 bits5.00 ranks · in orderthe best ordered tree77,890 bits3.98 ranks · in orderthe best tree of any shape76,789 bits3.92 ranks · unorderedσ 21 · the ordered tree is 1.43% above the unordered optimum16,384 characters of englishorder costs 1.43%

    The tree the operation insists on

    The compound walk means "everything that went left is smaller", which is true only if the leaves are in the alphabet's order. Huffman's tree is the smallest and its leaves are in frequency order, so the operation that makes a bidirectional search affordable costs the shape that makes an index small.

    part 18 · structures
  19. 051015510152025symbols in the alphabetfactor saved, and per cent paidranks saved, as a factorbits paid, per cent8,192 characters · 2 errorsx14.0 for 0.04%

    A factor of fourteen, for four per cent

    The compound walk removes a factor of σ from every interval extension. It requires a tree whose leaves are in order, which costs between nothing and five per cent of the wavelet tree. Both numbers on one plate, and the trade is not close.

    part 19 · bounds
  20. acdefghijklmnopqrstuvwyzroot24 of 26 symbols present54 ranks · 51 nodes

    Asking about symbols that are not there

    A search extends an interval by every character of the alphabet, and on a deep branch almost all of them produce an empty interval. That is a full rank walk whose entire result is the discovery that nothing was there.

    part 20 · indexes
  21. acdefghijklmnopqrstuvwyzroot24 of 26 symbols present54 ranks · 51 nodes

    A node costs two ranks

    The left child's interval is the position minus the right child's. A descent that calls rank on both children returns exactly the same symbols at twice the cost, and nothing about the answer can see it.

    part 21 · machine
  22. 0100200300102030distinct symbols in the intervalbit-vector ranksthe loop: 320the descentsigma = 32 throughout32x down to 5.16x

    Proportional to the answer, not the alphabet

    At a fixed alphabet of thirty-two, a loop costs three hundred and twenty ranks whether one symbol is present or all of them. The descent costs ten and sixty-two. The experiment has to move the answer without moving the alphabet, and the obvious sweep moves both.

    part 22 · floors
  23. binary · sigma 22.00x2 against 4dna · sigma 42.67x6 against 16protein · sigma 204.76x42 against 200latin · sigma 264.81x54 against 26064-position interval2.00x to 4.81x

    Two at binary, five at twenty-six

    The saving is a factor in the alphabet, so a two-symbol alphabet gets two. Approximate matching in this field is mostly done on DNA, which sits near the bottom of the list at 2.7.

    part 23 · bounds
  24. extensions attempted9,206 dead7,594 livebit-vector ranksthe loop: 168,000the descent: 24,0868 patterns · 1 error · sigma 2054.8% dead · 6.98x

    The branches that find nothing

    An approximate search over a twenty-symbol alphabet attempts sixteen thousand eight hundred extensions and nine thousand two hundred of them produce an empty interval. That is a full rank walk whose entire result is the discovery that nothing was there.

    part 24 · structures
  25. 02040608000.50011.502errors permittedbranches that find nothing, per cent5.75x6.98x9.22xthe label is thesaving at that budget8 patterns · 8 characters31.3% to 74.9%

    A looser budget wastes a larger share

    More errors permitted means more work, and the fraction of that work which was never going to help rises with it — from thirty-one per cent at no errors to seventy-five at two. The saving is worth most where the search is most expensive.

    part 25 · practice
  26. a fixed-length codeleaves in orderacdefghilmnoprstuvwythe best ordered treeleaves in orderacdefghilmnoprstuvwythe best tree of any shapeleaves in frequency ordereahnrstdiloucfgmpvwythe symbols, in the order the descent reports them21 symbols in 2,048 positionsone set, 2 of 3 sorted

    One set, three orders

    The symbols an interval holds do not depend on the tree's shape. The order they come out in does, and a search that accumulates a running count as it reads them computes a plausible number that is wrong on eighty per cent of queries.

    part 26 · wrong
  27. acdefghijklmnopqrstuvwyzroot24 of 26 symbols present54 ranks · 51 nodes

    A walk that does not prune

    Remove the emptiness test and the descent visits every node of the tree, returns exactly the same symbols with exactly the same intervals, and costs sixty per cent more. No test of the answer can see it.

    part 27 · wrong

All series