Field

When the algorithm is a table

A dynamic program's cost is settled before its input is touched, by how many distinct subproblems its recurrence has. The unit is the subproblem; the second number is how many must be held at once; and the methods worth knowing are the ones that compute more in order to keep less.
—bcab—abca6388328188632571322513518753111111one unit = one invocation of the recurrence481 calls, 25 distinct subproblems

The cost is the number of subproblems

Why a dynamic program's cost is settled before its input is touched, how a recursion graph turns an exponential into a rectangle, and what the ninth counter on this site measures.

—sitting—kitten012345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455one unit = one subproblem given a value56 cells, filled in row order

The same table, filled two ways

What separates a memoised recursion from a bottom-up fill when the table is the same table, why the evaluation order is a decision the recurrence does not make, and where each strategy wins.

—abrocadabro—abracadabra012101221012210122112222123321233212332123321233212322one unit = one subproblem given a value54 of 144 cells, 90 skipped

A band as wide as the answer

Ukkonen's banded dynamic program, why the guarantee is one-sided, and the measured gap between a run that has the answer and a run that can certify it.

acgtacgtseen ->0313303113033130a gap of k characters costs 2krows: expected · columns: seen · unit: bitslinear gaps

A cost that is not one

What unit edit cost assumes, how a substitution matrix is derived from a stated mutation process rather than quoted, and the measured consequence — a finer model choosing among twenty-one alignments a coarser one cannot tell apart.

01234567891011120123456789101112123456789101112123456789101112345678910123456789123456781234567123456123451234123121one unit = one subproblem given a value91 cells, 364 transitions, 4.0 per cell

The cells are not the cost

Interval dynamic programs measured in two numbers rather than one — how many subproblems a recurrence has and what each of them costs — and why a table's shape does not reveal its class.

0%25%50%75%100%11.522.533.545what a matching character is worthreported as the shared region, of the longer sequencea random pair scores zero hereunit cost, alphabet acgtthe region found: 13 characters at the left, 34 at the right

The zero that moves the answer out of the corner

Local alignment as a change of four lines, why its answer leaves the corner, and the statistical precondition that decides whether the region it reports is a region at all.

—agcacacggatcag—ccagggagta09101112131415161718192021229110101212141416171819192122101021011121314151718191920221110113101212141516171920192112121012411131314151718202119131312111351214131416181921211414131312146131413151719202115141514131314714151316181921161614161514141571416141719191717171517161515168151615182018171818151816161617816171519one unit = one subproblem given a value495 cells across three tables, 980 transitions

A cell that has to know where it is

Gotoh's construction measured — why an affine gap needs a cell to know which route it arrived by, what the one-table version silently computes instead, and the alignment it fragments.

01234567891011120123456789101112122332223333122222332231222222223122222233122222221222222122222122221222122121one unit = one subproblem given a value91 cells, 156 transitions, 1.7 per cell

The argmin that cannot go backwards

Knuth's monotonicity, measured — why the split point that wins for an interval cannot precede the one that wins for its left neighbour, how the ranges telescope, and what the saving depends on.

—execution—intention0136101521283645247111622293746555812172330384756649131824313948576572141925324049586673792026334150596774808527344251606875818690354352616976828791944453627077838892959754637178848993969899one unit = one subproblem given a value100 cells, filled in diagonal order

The order that has a depth

Work and span on a dynamic-programming table — how many cells a fill computes, how many rounds an order takes, and the longest chain the recurrence itself contains.

cost per column of the alignmentab / ba1.00 over 2 · 0.67 over 3kitten / sitting0.43 over 7 · 0.43 over 7intention / execution0.56 over 9 · 0.50 over 10gattaca / gactata0.29 over 7 · 0.29 over 7abracadabra / abrocadabro0.18 over 11 · 0.18 over 11unit cost, in edits per columnupper bar: the optimum, divided · lower bar: the best rate

A distance divided by a length is not a rate

Why the length-normalised edit distance is a different optimisation from the edit distance, how a parametric search finds it, and what the two disagree about.

pairs where the two disagreerestricted · unrestrictedab → bca3 against 2ac → cba3 against 2ba → acb3 against 2bc → cab3 against 2ca → abc3 against 2triples the restricted rule breaksab → bca costs 3, but by way of ba it costs 2ac → cba costs 3, but by way of ca it costs 2ba → acb costs 3, but by way of ab it costs 240 strings, 1,600 pairs, 64,000 triplesunrestricted: 0 triples broken

The edit that reaches back two rows

The restricted and unrestricted transposition distances, enumerated — where they disagree, why one of them is not a metric, and what a fourth transition does to a recurrence dependency structure.

—gactacgat—gattacagt112223334244352461647one unit = one subproblem given a value21 matching pairs, 21% of the rectangle

The cells that were never worth having

Sparse dynamic programming on the matches alone, why the alphabet decides whether it is an improvement, and what it means that the subproblem set is a choice.

stay 0.9, 400 pairsacgtacgt0525505225045240stay 0.5, 400 pairsacgtacgt0212202112022120cost of aligning the row letter with the column letterpseudocount 1

The matrix a corpus wrote

A log-odds substitution matrix estimated from corpora of aligned pairs, and what happens to optimal alignments when the corpus changes. Divergence rewrites every entry, a small corpus rewrites several, and more than half of a set of test alignments change between two plausible matrices.

123456123456cost to open a gapcost to extend a gapintentionexecution571 settingsinte---ntion---execution3 settingsinte-ntion-execution1 setting-intentionexec-ution1 settingintention against execution, affine costseach colour is one optimal alignment

The parameter plane has few answers

The optimal alignment of a fixed pair of strings over a grid of cost settings, coloured by which alignment wins. Hundreds of settings produce a handful of alignments, most of the plane is a single region, and under a linear model the boundaries are lines through the origin whose slopes can be computed by hand.

01234567801234567891724303539424410182531364043111926323741122027333813212834142229152316one unit = one subproblem given a valueeach number is a storage offset, of 45 slots

A triangle stored in a square

One interval dynamic program's reads replayed at the addresses three storage layouts and two fill orders give them. Packing saves space and changes nothing, storing by diagonal doubles the misses, and every layout converges on half its reads missing, because a cell's split scan reads a row and a column and no row-major layout can keep both close.

cache misses per split point consideredsquare array, by length1.1063,128,465 missestwo copies, by rows0.212598,455 missessquare array, split scans0.095268,386 missesfully associative · 32 lines × 8 elements · LRU256 keys, 32,896 cells

The split scan cut into blocks

An interval dynamic program over 16 to 256 keys, filled cell by cell in four arrangements, stored twice, and filled by a recursive fill that splits every cell's scan across blocks, each replayed through caches of 8 to 1,024 lines. Reordering whole cells stays on the half-miss floor; cutting the scans goes under it by a factor of ten on every cache from 32 lines up.

10³10⁴10⁵words in the vocabularysubproblems given a valueevery table in full · 1.16best so far · 0.99answer known · 0.96one unit = one subproblem given a valuesubproblems given a value, n from 250 to 2424

The bound the search finds for itself

The nearest word to 300 misspelt queries over vocabularies of 250 to 2,424 words from a fixed corpus of essays on algorithms, found with full tables, with a best-so-far bound in vocabulary order and in length order, and with the true distance supplied in advance. The discovered bound computes a quarter of the full tables' cells, all four strategies grow linearly with the vocabulary, and the gap to knowing the answer narrows from 2.6 times to 1.2 times as the queries carry more errors.

rounded to whole bitsunroundeda resample, 400 near pairs0.860.9340 pairs at stay 0.90.820.928 pairs at stay 0.90.710.88400 pairs at stay 0.70.680.86400 pairs at stay 0.50.620.790.5: no prediction200 test pairs, 16 directionsdashed: a coin flip

The ties a rounded matrix makes

The distance from each of 200 optimal alignments to its nearest tie under a fitted substitution matrix, measured along random directions in cost space, set against which alignments five refitted matrices move — with the matrices rounded to whole bits and unrounded. The distance ranks alignments by how likely a resample is to move them, and rounding creates most of the ties it finds.

10³10⁴10⁵words in the vocabularysubproblems given a valueevery table in full · 1.16a trie, no bound · 0.99best so far · 0.99a trie, best so far · 0.82one unit = one subproblem given a valuesubproblems given a value, n from 250 to 2424

The columns the candidates share

The nearest word to 300 misspelt queries, searched over a vocabulary stored as a list and as a trie, with no bound, with the best distance found so far, and with the true answer supplied. Sharing prefixes changes how the cost grows with the vocabulary rather than dividing it by a constant, and it makes the order the candidates are met in stop mattering.

1632649612810³10⁴10⁵table sizesplit points appliedevery splitbounded per blockbounded per cell, by lengthweights satisfying the quadrangle inequalityall three compute the same table

The bound a block can and cannot have

The monotone bound on an interval table's best split, applied to a cache-oblivious blocked fill rather than to a cell at a time — how often each side of the bound is available, how much work it removes, and why removing work removes almost no transfers.

alignments on a tiedistinct costs · predictionwhole bits633 of 6 · 0.86half bits793 of 6 · 0.91quarter bits414 of 6 · 0.93a grain of 0.2235 of 6 · 0.89eighth bits156 of 6 · 0.93a grain of 0.05205 of 6 · 0.95unrounded156 of 6 · 0.93200 test pairs, 16 directionsbar: alignments within 0.01 of a tie

The lattice that decides the ties

The same substitution matrix fitted at whole, half, quarter, fifth, eighth and twentieth bits and with randomised rounding, set against how many of 200 test alignments sit on a tie and how well the distance to a tie predicts which alignments a resample moves. The grain does not order the ties; the number of distinct costs surviving it does.

1101001,000queries seenshare of the trie's prefixes ever visited10%20%30%50%70%100%misspelt vocabulary wordsmisspelt words it lacks7,709 prefixes, 2,424 wordsthe bounded walk, bound discovered

A trie fitted to its queries

A bounded nearest-word walk over a trie of 7,709 prefixes, measured on a stream of 2,000 misspelt vocabulary words and 600 misspelt words the vocabulary lacks: how much of the trie a stream visits, how concentrated the visits are, the distances the answers lie at, what a fixed starting bound costs against discovering it and knowing it, and a bound learned from the stream on streams that mix the two kinds of query.

1.522.533.544.55fitted cost, bitsgenerator: 1.644generator: 4.644ag1.667 ± 0.103ct1.711 ± 0.084gt4.323 ± 0.248cg4.687 ± 0.239ac4.706 ± 0.234at4.879 ± 0.291one corpus, 400 pairs at stay 0.9bars: two bootstrap standard errors

The grain a corpus chooses for itself

A substitution matrix choosing its own precision from the data: the lattice that separates all six fitted costs against the grouping of costs the corpus can distinguish at two standard errors, scored against the generator's own two costs — agreement, churn between resamples, the effect of corpus size, and the threshold.

01020share of queries for words the vocabulary lacksthousands of cells a query0%10%20%30%discover the boundone try at the medianone try at twoone, then two, then discovertold the distanceevery attempt chargeddotted: told the answer's distance

A second try before any feature

A bounded nearest-word walk over a trie of 7,709 prefixes, on streams mixing 2,000 misspelt vocabulary words with 600 misspelt words the vocabulary lacks: whether a query's first letters, length or unseen letter-triples predict its answer's distance, what a starting bound chosen per class costs against the best a feature allows in hindsight, what a schedule of two or three tries costs, and why a schedule chosen from a price list loses to a fixed one.

1001,000aligned sequence pairs in the corpusresamples choosing the generator's classes0481216held-out pairsa fixed penalty a classa penalty that growstwo classes, 16 seeds a size203 partitions each

The classes held-out pairs cannot rule out

Choosing a substitution matrix's classes inside the fit: all 203 partitions of six substitution pairs scored by five-fold held-out likelihood, by a fixed penalty a class (AIC) and by a penalty that grows with the corpus (BIC), on corpora from 25 to 3,200 pairs drawn from a generator with two cost classes and one with a third class planted half a bit away — how often each finds the generator's classes, what a spurious split gains, and what the choices do to alignments.

010203040the cap on an answer's distance, in editsthousands of cells a query123456nonewords it lacksa fifth unfamiliarmisspelt vocabulary wordsevery attempt charged2,000 + 600 queries

A checker that stops at two edits

A spelling checker over a trie of 2,424 words, walked at one edit, then two, then at a cap C, and stopping there, on 2,000 misspellings of vocabulary words and 600 of words the vocabulary lacks, with each query's source word replayed from the stream's seed: cells a query against the cap, what each cap returns for words the vocabulary lacks, the distance to their nearest vocabulary word, what a cap of two returns by the length of the writer's word, and cells against the share of unfamiliar queries.

25501002004008001,6003,200sequence pairs in the corpusclasses chosen123456six different costsmedian chosentwo terms: three parameterssixteen corpora a sizedot area: corpora choosing it

A cost built from two properties

Corpora from a generator whose six substitution costs are a transversion term plus a term in the difference of a number carried by each letter, fitted two ways at 25 to 3,200 sequence pairs: every partition of the pairs with the best chosen by a growing penalty, and a log-linear model with the two terms: the classes chosen, the six costs of each fit on one line, excess bits on a fresh corpus, alignments optimal under the generator's costs, and both fits on the plain two-class generator.

00.1000.2000.30000.2500.5000.7501misspelt vocabulary words correctedunfamiliar words given a false suggestionat 1at 2at 3at 4cap aloneeach point: one threshold, with the cap of twoup and right: the cap alone

The triples a typo writes

A spelling checker over a trie of 2,424 vocabulary words, on 2,000 misspelt vocabulary words and 600 misspelt words the vocabulary lacks, each query declined before any walk when it carries at least one to four letter-triples no vocabulary word contains, and otherwise answered under a cap of two edits: how many unseen triples each kind of query carries, where they come from, the corrections kept against the false suggestions removed, the same by the query's length, and the work each threshold saves.

00.1000.2000.3000.40000.2500.5000.7501misspelt vocabulary words correctedunfamiliar words given a false suggestionuniformdrawn by use0.50.5each point: one threshold; the right-hand end is the cap alonea tenth per edit

The counts only short words carry

A spelling checker over a trie of 2,424 vocabulary words, its answers capped at two edits and declined when the nearest words' combined count in a 19,279-word corpus, times a tenth for each edit, falls under a threshold: how skewed the counts are by word length, where the scores of corrections and false suggestions fall on streams drawn evenly and drawn by use, the trade at each threshold, the same by length, three weights for an edit, and every test priced in corrections given up a false suggestion removed.

160170180190200aligned sequence pairs in the corpus (log scale)test alignments optimal, of 200502008003,200the sum, told the propertypositions, fittedsix free classesthe chosen partition200 test pairs, eight seedsoptimal under the generator's costs

A line the corpus finds

Substitution costs for a four-letter alphabet from a generator whose mismatch costs are a transversion term plus the difference of the letters' values on a graded property, fitted from corpora of 25 to 3,200 aligned pairs, sixteen seeds a size, by the two-term sum told the property, letters' positions on a line fitted freely over every order, six free classes, and the partition a growing penalty chooses: how often the fitted order is the generator's; the fitted positions against the true ones; excess bits on a fresh corpus; alignments optimal under the generator's costs; excess bits on a generator with no property; and the fitted line's length on both generators.

All essays