Concept

Construction — where it appears

The algorithm that fills a table, as distinct from the definition that says what the table contains. The two can differ by orders of magnitude in cost, and a measurement of the second reported as a property of the first is how a rule comes to look unbuildable.

Named by 10 essays across 6 fields — each of them below, with the objects they name alongside it.

0%25%50%75%100%0.30.40.50.60.70.80.850.90.95keys per slotconstructions that failed2 hashes, 1 slot3 hashes, 1 slot2 hashes, 2 slots2 hashes, 4 slots20 constructions a point, 256 bucketsa lookup reads hashes × slots, whatever the keys

More hashes or wider buckets

A cuckoo table with two hash functions and one slot per bucket cannot be built past about half full. Give it a third hash function and it builds to 0.92. Keep two hashes and give each bucket two slots and it builds to 0.89; four slots, past 0.95. Every shape keeps the worst-case lookup the plain table was built for, and every shape pays for its threshold in a different place.

randomness · Randomness
1,00010,00010³10⁴characters of textsteps, precomputation plus scancrossing at 8,000published rulesexact rules2 patterns · four symbolscrossing 8,000 · was 32,000

Where the exact rules pay now

With a construction as cheap as the published one, the exact shift rules pay for themselves past eight thousand characters of text at two patterns, four thousand at four, and never at thirty-two — because by thirty-two patterns the two rules make identical decisions.

practice · Symbol
124816nonephrasescap on the copy chainphrasesworst chainEnglish-like · 8,192 charactersz 156 to 7,351

A parse that will not follow a long chain

A greedy self-referential parse bounds the copy depth by nothing at all — thirty-two copies of a text give a position costing twenty-two phrase follows. Restricting every phrase to sources no deeper than D holds it at D, and the whole question is what that costs.

text · Parse
good suffix, defined757,058good suffix, from links3,824bad character, defined12,666bad character, swept5,168128 patterns · 1,280 characters of patternfour symbols · m = 10198x and 2.5x

The case a failure link does not cover

Compute the exact good-suffix rule from the failure links alone and eighteen of twenty-two entries come out too large. The matcher then steps past sixty-six of two thousand and twenty-five occurrences, and every match it does report is a match.

bounds · Symbol
101001,00010010³10⁴10⁵total length of the pattern setprimitive stepsthe definitionfrom the linksthe published rulefour symbols · m = 1085.60x apart at 128 patterns

The table the links already knew

The exact good-suffix rule costs 757,058 character comparisons to build from its definition at 128 patterns, and 3,824 from the trie's failure links. Same table, checked at every node — a factor of 198, and the definition was never the algorithm.

text · Symbol
11010³cap on the depthphrases9.8% more phrasesevery occurrence consideredthe earliest only4,096 charactersworst at cap 4

The cap an automaton cannot see

A state of a suffix automaton stands for a set of occurrences and hands back one of them. So a capped parse driven by it can ask whether the earliest occurrence is shallow enough and cannot ask whether any occurrence is — which costs up to 9.8% of the phrases, and only at the caps that bind.

bounds · Parse
1,00010⁴10⁵10⁶charactersoperationscharacter comparisonstransitions and linksrepetitive · 8 copies346 phrases at 8,192

The parse in one pass of the text

The same parse, phrase for phrase, from 1,324,336 character comparisons or from 25,420 transitions and suffix-link steps. One of those numbers grows with the text and the other grows with its square, and the difference is why every measurement about a depth cap here was taken on a few thousand characters.

text · Parse
rank among the boundaries, sorted by the text before themsorted by the text after0 in the rectanglepattern "ss is un"split after 40 end with the left half0 begin with the rightEnglish-like · 2 copies of 512z = 156 · 0 crossing

A rectangle over a permutation

Two orderings of one set of boundaries are two permutations, so a phrase index's intersection is a rectangle over a permutation grid — the one point set a wavelet tree stores exactly, at one bit per point per level and no coordinates at all.

structures · Grid
101001,00010100total length of the pattern settimes the published rule's precomputationwritten as a definitionbuilt from the linksthe published rulefour symbols · m = 10137x becomes 1.60x

The ratio that was an implementation

This collection published a factor of forty-two between two shift rules' preprocessing. Sixty-nine per cent of the denominator was a table the published rule never reads, and the numerator was a definition rather than a construction. The corrected ratio is 1.6.

wrong · Symbol
11010⁴cap on the depthphrasesno capcap 165,536 characters · 16 copies26x at cap 1

What a quadratic construction was setting

A depth cap of one costs 90% of an 8,192-character text in phrases, and 86% of a 65,536-character one. The ladder's conclusions hold at thirty-two times the size — and the number the ladder could not reach, the deepest chain a real collection produces, turns out to be fourteen.

wrong · Parse

Named alongside it

The objects these essays reach for when they reach for this one.

MeasurementTrade offCommentz-walterDefinitionPhraseShift ruleGood-suffix ruleLempel zivMulti patternParsePreprocessingRepetition

All concepts