Concept

Alphabet — where it appears

The set of distinct symbols a text is drawn from, whose size decides how far a matcher may jump and how many states an index needs. Its size decides how far a matcher may jump and how many states an index needs, so it belongs beside any quoted cost in this field.

Named by 23 essays across 7 fields — each of them below, with the objects they name alongside it.

8 patterns of 10 characters over four symbols02,0004,0006,0008,00010,00012,00014,00016,00018,000one cell = 10 positions · shade = fraction read61.6% of the text

The shift a set of patterns allows

Aho-Corasick reads every character of the text exactly once, whatever the number of patterns. Commentz-Walter reads backwards inside a window and steps over what it can, and on eight patterns of ten characters it looks at sixty-two per cent of a twenty-thousand-character text. The rule that does the skipping is not the one everybody implements.

text · Symbol
roothseirhesssolid: a transition on a character · dashed: a suffix or failure link10 states for 12 pattern characters

One pass for every pattern at once

Sixty-four patterns, one text, twenty thousand characters read. Running this site's KMP once per pattern reads 1,595,445 — and an implementation missing one kind of link finds fewer occurrences, reads exactly as much, and looks better on every counter.

text · Automaton
1,00010,00010³10⁴bits · runs12481632characters in the collection · copies aboven·H₃, bitsr, runs in thetransformEnglish-like · base 512 · divergence 0n·H₃ x15.8 · r x1.00

The entropy that cannot see a copy

Two copies of a text have exactly the same symbol statistics as one, so every entropy on this site doubles when the second copy arrives and the second copy carries no information at all. The number of runs in the Burrows-Wheeler transform is 224 at two copies and 224 at thirty-two.

text · Repeat
is·holds·no·the·the·of·of·rather·count·class·that·is·algorithm·o1111111211111151141131111111211122213111111212phrase lengths below each block · a shaded block is a literalEnglish-like · 64 charactersz = 46 · 17 literals

The phrases a text copies from itself

Four thousand characters of English-like text hold 604 phrases in the greedy parse and 1,219 runs in the transform. Thirty-two copies of one text hold 156 phrases and 233 runs. Two measures of repetition, neither of them an entropy, and they do not agree about which text is the more repetitive.

text · Parse
1234152abcbcbccbsolid: a transition on a character · dashed: a suffix or failure link8 states ≤ 9, 9 transitions ≤ 11

Every substring, in fewer states than substrings

A text of 512 characters has 129,416 distinct substrings. A machine that recognises every one of them, and nothing else, needs 831 states — and the bound it is under, 2n − 1, is reached exactly by a string one line long.

structures · Automaton
the same 16 patterns, one of them shortenedshortest 1658.4%mean shift 3.15shortest 1262.8%mean shift 2.99shortest 869.7%mean shift 2.64shortest 680.5%mean shift 2.35shortest 493.4%mean shift 1.98shortest 399.2%mean shift 1.71shortest 2100.0%mean shift 1.3716 patterns, 15 of them 16 characterstext 20,000

The ceiling the shortest pattern sets

A matcher that skips is described as faster than one that reads every character, and the description leaves out what decides it. No shift can exceed the shortest pattern in the set, so adding one two-character pattern to fifteen of sixteen characters takes a run from reading fifty-eight per cent of the text to reading all of it twice.

wrong · Symbol
·1, 4b1, 3b1, 2b4, 2a4, 2a2, 2a4, 2b4, 2b4, 2b4, 2patterns abbb, baab, bbab · shortest 4d₁, d₂ under each node · a filled node ends a patternthe reversed-pattern trie10 nodes

The shift somebody published

The exact rules for shifting a multi-pattern window are a definition that quantifies over every pattern at every offset. The 1979 rules are two tables read off the trie's own failure links, they are computed in one pass, and on this pattern set they agree with the definition at every node.

text · Symbol
100.0010.010.1fraction of the text proposedlog_4 n = 7.101234,56,7seed length, characters · errors allowed abovepattern 24 · text 20,000 · four symbolsrings: the closed form

The filter that proposes everything

Seed-and-extend saves two thousand times the work at zero errors and costs more than doing nothing at four. Between them the selectivity falls through the floor, and where it falls is set by two numbers that can be computed before the filter is run — one of which does not contain the length of the text at all.

wrong · Distance
124816nonephrasescap on the copy chainphrasesworst chainEnglish-like · 8,192 charactersz 156 to 7,351

The term that came back

A phrase index is worth building because 8,192 characters parse into 156 phrases. Cap the copy depth at one and the same text parses into 7,351 — ninety per cent of the characters — and the structure is proportional to the text again.

space · Parse
1234567characters addedoccurrencesgrown leftwardsgrown rightwards"tgatatt"4,000 characters · four symbols1 occurrences

An interval that grows at both ends

A backward search step is two ranks. A bidirectional step is two ranks and the width of the interval for every symbol that sorts before the one being added — 16.5 ranks on four symbols and 138.6 on twenty-six, which is a cost no account of the structure mentions.

indexes · Distance
right to left8302.74xleft to right8772.89xone per piece4471.48xthe scheme3031.00xinterval extensionsall four found the same 6 positions15 characters · k = 22.74x apart

The search that starts in the middle

The same pattern, the same six occurrences, the same index — and 830 interval extensions, or 303, according to which end the search begins at. A pattern cut into three pieces has an error-free one, and only a search with two ends can start there.

text · Distance
run together38,400 bitsσ 21 · 5 bitsone separator38,470 bitsσ 22 · 5 bitsa separator each46,164 bitsσ 35 · 6 bits15 documents of 512 charactersthe separators are the only differenceEnglish-like · 15 documents1.20x for the distinct marks

One separator, or one for each

A shared separator costs one alphabet symbol and is free. Fifteen distinct ones take the alphabet from twenty-two to thirty-five, which crosses a power of two, so every character of every document costs a sixth bit — 1.2 times the packed collection, to tell the boundaries apart.

space · Document
twelve essays — runs0.491generated, matched0.109twelve essays — phrases0.213generated, matched0.046eight source modules — runs0.461generated, matched0.122eight source modules — phrases0.210generated, matched0.050ten revisions of one file — runs0.126generated, matched0.113ten revisions of one file — phrases0.067generated, matched0.047per character24,576 characters a partdivergence 0.02

A corpus that was not generated

Every collection in five strands has been copies of a generated text with a fraction of its characters replaced — three numbers, one dial. Here is one that was not — twelve essays, eight source modules, ten revisions of one file — measured beside the model of it.

text · Document
run together867 bits a characterone separator877 bits a charactera separator each897 bits a charactergenerated, run together215 bits a characterdistinct symbolstwelve essays0.0% premium

Documents that are not the same length

A separator per document costs a whole bit per character — on a generated collection whose alphabet is 21 symbols, where adding a few crosses 32. Real prose has 87 symbols and sits 41 short of the next power of two, so the same separators cost 0.04%.

indexes · Document
① prose144,617 chars12 docs · median 12,335① code106,283 chars8 docs · median 13,150① revisions75,658 chars10 docs · median 7,774② essays1,004,822 chars82 docs · median 12,335② captions549,210 chars2,214 docs · median 230② history174,282 chars14 docs · median 12,963① the first freeze · ② the second6 parts

A million characters of the same thing

Every measurement this collection has published about real text was taken on twenty-four thousand characters, because the phrase count was quadratic. It is linear now, so here is the same corpus at forty times the size — and what forty times does to its own numbers.

text · Document
10⁴10⁵10⁶characters in the collectionshuffled ÷ real, phraseswhat the order is wortha million characters of prose · phrases per characterx1.37 over the sweep

The half of a fall that is the logarithm

Phrases per character on a real collection of essays fall by a factor of 2.34 as it grows. A shuffle of the same characters falls by 1.70. Nearly three quarters of the movement is arithmetic, and no definition of the measure says so.

wrong · Document
① prose12,33512① code13,1508① revisions7,77410② essays12,33582② captions2302,214② history12,96314documentscharacters in a document① the first freeze · ② the secondmedian marked

Two thousand documents of two hundred characters

Every collection this field has measured has been a dozen documents of ten thousand characters. A real collection is usually the other shape, and the other shape moves every term in a document index — one of them by a factor of seventy.

indexes · Document
1001,00010characters in the wordruns · phrasesr, worse orderz, either orderr, better orderFibonacci words · two symbolsspread 1.50x to 3.17x

The measure that cannot see the alphabet

Take a Fibonacci word of 4,181 characters and transform it with a before b — six runs. Transform the same word with b before a — nineteen. The parse gives eighteen phrases either way, and the gap between the two run counts grows with every word in the family.

wrong · Parse
level 0 · left335 keptlevel 1 · right+207 smallerlevel 2 · left61 keptlevel 3 · right+33 smallerlevel 4 · left8 keptpositions still in play, and the half the code leaves behindsmaller symbols before position 400: 240rank of "m": 8 — from the same 5 operationsσ 21 · code 010105 ranks, not 105

Every child at once

A bidirectional extension counts the occurrences of every symbol smaller than the one being added, and this collection did it with one rank per symbol — 139.7 operations an extension on a twenty-six-letter alphabet. One walk down the tree gives the same number.

indexes · Symbol
46812windowswindow lengthspanning a joinin no documentwith a separatorEnglish-like · 8 documents44 invented at m = 12

The occurrences a join invents

Eight documents run together hold forty-nine eight-character windows that span a join, twenty-three of which occur in no document at all. Every index built over the concatenation reports them, and five essays of this collection paid that cost silently.

wrong · Document
level 0 · left585 keptlevel 1 · right+351 smallerlevel 2 · right+113 smallerlevel 3 · right+39 smallerlevel 4 · right+40 smallerpositions still in play, and the half the code leaves behindsmaller symbols before position 700: 543rank of "s": 42 — from the same 5 operationsσ 21 · code 011115 ranks, not 105

The count that was already there

A bit vector's rank of one is the position minus its rank of zero. Every walk down a wavelet tree computes one of them at every level, and this collection asked for the other separately for years.

machine · Symbol
a fixed-length code97,718 bits5.00 ranks · in orderthe best ordered tree77,890 bits3.98 ranks · in orderthe best tree of any shape76,789 bits3.92 ranks · unorderedσ 21 · the ordered tree is 1.43% above the unordered optimum16,384 characters of englishorder costs 1.43%

The tree the operation insists on

The compound walk means "everything that went left is smaller", which is true only if the leaves are in the alphabet's order. Huffman's tree is the smallest and its leaves are in frequency order, so the operation that makes a bidirectional search affordable costs the shape that makes an index small.

structures · Symbol
051015510152025symbols in the alphabetfactor saved, and per cent paidranks saved, as a factorbits paid, per cent8,192 characters · 2 errorsx14.0 for 0.04%

A factor of fourteen, for four per cent

The compound walk removes a factor of σ from every interval extension. It requires a tree whose leaves are in order, which costs between nothing and five per cent of the wavelet tree. Both numbers on one plate, and the trade is not close.

bounds · Symbol

Named alongside it

The objects these essays reach for when they reach for this one.

Index sizeMeasurementRepetitionEntropyWavelet treeCorpusRun-lengthSelf-indexAho-corasickBidirectional indexCompound operationCompressibility

All concepts