Field

When the algorithm is a table

A dynamic program's cost is settled before its input is touched, by how many distinct subproblems its recurrence has. The unit is the subproblem; the second number is how many must be held at once; and the methods worth knowing are the ones that compute more in order to keep less.
bcababca6388328188632571322513518753111111one unit = one invocation of the recurrence481 calls, 25 distinct subproblems

The cost is the number of subproblems

Why a dynamic program's cost is settled before its input is touched, how a recursion graph turns an exponential into a rectangle, and what the ninth counter on this site measures.

sittingkitten012345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455one unit = one subproblem given a value56 cells, filled in row order

The same table, filled two ways

What separates a memoised recursion from a bottom-up fill when the table is the same table, why the evaluation order is a decision the recurrence does not make, and where each strategy wins.

abrocadabroabracadabra012101221012210122112222123321233212332123321233212322one unit = one subproblem given a value54 of 144 cells, 90 skipped

A band as wide as the answer

Ukkonen's banded dynamic program, why the guarantee is one-sided, and the measured gap between a run that has the answer and a run that can certify it.

acgtacgtseen ->0313303113033130a gap of k characters costs 2krows: expected · columns: seen · unit: bitslinear gaps

A cost that is not one

What unit edit cost assumes, how a substitution matrix is derived from a stated mutation process rather than quoted, and the measured consequence — a finer model choosing among twenty-one alignments a coarser one cannot tell apart.

01234567891011120123456789101112123456789101112123456789101112345678910123456789123456781234567123456123451234123121one unit = one subproblem given a value91 cells, 364 transitions, 4.0 per cell

The cells are not the cost

Interval dynamic programs measured in two numbers rather than one — how many subproblems a recurrence has and what each of them costs — and why a table's shape does not reveal its class.

0%25%50%75%100%11.522.533.545what a matching character is worthreported as the shared region, of the longer sequencea random pair scores zero hereunit cost, alphabet acgtthe region found: 13 characters at the left, 34 at the right

The zero that moves the answer out of the corner

Local alignment as a change of four lines, why its answer leaves the corner, and the statistical precondition that decides whether the region it reports is a region at all.

agcacacggatcagccagggagta09101112131415161718192021229110101212141416171819192122101021011121314151718191920221110113101212141516171920192112121012411131314151718202119131312111351214131416181921211414131312146131413151719202115141514131314714151316181921161614161514141571416141719191717171517161515168151615182018171818151816161617816171519one unit = one subproblem given a value495 cells across three tables, 980 transitions

A cell that has to know where it is

Gotoh's construction measured — why an affine gap needs a cell to know which route it arrived by, what the one-table version silently computes instead, and the alignment it fragments.

01234567891011120123456789101112122332223333122222332231222222223122222233122222221222222122222122221222122121one unit = one subproblem given a value91 cells, 156 transitions, 1.7 per cell

The argmin that cannot go backwards

Knuth's monotonicity, measured — why the split point that wins for an interval cannot precede the one that wins for its left neighbour, how the ranges telescope, and what the saving depends on.

executionintention0136101521283645247111622293746555812172330384756649131824313948576572141925324049586673792026334150596774808527344251606875818690354352616976828791944453627077838892959754637178848993969899one unit = one subproblem given a value100 cells, filled in diagonal order

The order that has a depth

Work and span on a dynamic-programming table — how many cells a fill computes, how many rounds an order takes, and the longest chain the recurrence itself contains.

cost per column of the alignmentab / ba1.00 over 2 · 0.67 over 3kitten / sitting0.43 over 7 · 0.43 over 7intention / execution0.56 over 9 · 0.50 over 10gattaca / gactata0.29 over 7 · 0.29 over 7abracadabra / abrocadabro0.18 over 11 · 0.18 over 11unit cost, in edits per columnupper bar: the optimum, divided · lower bar: the best rate

A distance divided by a length is not a rate

Why the length-normalised edit distance is a different optimisation from the edit distance, how a parametric search finds it, and what the two disagree about.

pairs where the two disagreerestricted · unrestrictedab → bca3 against 2ac → cba3 against 2ba → acb3 against 2bc → cab3 against 2ca → abc3 against 2triples the restricted rule breaksab → bca costs 3, but by way of ba it costs 2ac → cba costs 3, but by way of ca it costs 2ba → acb costs 3, but by way of ab it costs 240 strings, 1,600 pairs, 64,000 triplesunrestricted: 0 triples broken

The edit that reaches back two rows

The restricted and unrestricted transposition distances, enumerated — where they disagree, why one of them is not a metric, and what a fourth transition does to a recurrence dependency structure.

gactacgatgattacagt112223334244352461647one unit = one subproblem given a value21 matching pairs, 21% of the rectangle

The cells that were never worth having

Sparse dynamic programming on the matches alone, why the alphabet decides whether it is an improvement, and what it means that the subproblem set is a choice.

stay 0.9, 400 pairsacgtacgt0525505225045240stay 0.5, 400 pairsacgtacgt0212202112022120cost of aligning the row letter with the column letterpseudocount 1

The matrix a corpus wrote

A log-odds substitution matrix estimated from corpora of aligned pairs, and what happens to optimal alignments when the corpus changes. Divergence rewrites every entry, a small corpus rewrites several, and more than half of a set of test alignments change between two plausible matrices.

123456123456cost to open a gapcost to extend a gapintentionexecution571 settingsinte---ntion---execution3 settingsinte-ntion-execution1 setting-intentionexec-ution1 settingintention against execution, affine costseach colour is one optimal alignment

The parameter plane has few answers

The optimal alignment of a fixed pair of strings over a grid of cost settings, coloured by which alignment wins. Hundreds of settings produce a handful of alignments, most of the plane is a single region, and under a linear model the boundaries are lines through the origin whose slopes can be computed by hand.

01234567801234567891724303539424410182531364043111926323741122027333813212834142229152316one unit = one subproblem given a valueeach number is a storage offset, of 45 slots

A triangle stored in a square

One interval dynamic program's reads replayed at the addresses three storage layouts and two fill orders give them. Packing saves space and changes nothing, storing by diagonal doubles the misses, and every layout converges on half its reads missing, because a cell's split scan reads a row and a column and no row-major layout can keep both close.

cache misses per split point consideredsquare array, by length1.1063,128,465 missestwo copies, by rows0.212598,455 missessquare array, split scans0.095268,386 missesfully associative · 32 lines × 8 elements · LRU256 keys, 32,896 cells

The split scan cut into blocks

An interval dynamic program over 16 to 256 keys, filled cell by cell in four arrangements, stored twice, and filled by a recursive fill that splits every cell's scan across blocks, each replayed through caches of 8 to 1,024 lines. Reordering whole cells stays on the half-miss floor; cutting the scans goes under it by a factor of ten on every cache from 32 lines up.

10³10⁴10⁵words in the vocabularysubproblems given a valueevery table in full · 1.16best so far · 0.99answer known · 0.96one unit = one subproblem given a valuesubproblems given a value, n from 250 to 2424

The bound the search finds for itself

The nearest word to 300 misspelt queries over vocabularies of 250 to 2,424 words from a fixed corpus of essays on algorithms, found with full tables, with a best-so-far bound in vocabulary order and in length order, and with the true distance supplied in advance. The discovered bound computes a quarter of the full tables' cells, all four strategies grow linearly with the vocabulary, and the gap to knowing the answer narrows from 2.6 times to 1.2 times as the queries carry more errors.

rounded to whole bitsunroundeda resample, 400 near pairs0.860.9340 pairs at stay 0.90.820.928 pairs at stay 0.90.710.88400 pairs at stay 0.70.680.86400 pairs at stay 0.50.620.790.5: no prediction200 test pairs, 16 directionsdashed: a coin flip

The ties a rounded matrix makes

The distance from each of 200 optimal alignments to its nearest tie under a fitted substitution matrix, measured along random directions in cost space, set against which alignments five refitted matrices move — with the matrices rounded to whole bits and unrounded. The distance ranks alignments by how likely a resample is to move them, and rounding creates most of the ties it finds.

10³10⁴10⁵words in the vocabularysubproblems given a valueevery table in full · 1.16a trie, no bound · 0.99best so far · 0.99a trie, best so far · 0.82one unit = one subproblem given a valuesubproblems given a value, n from 250 to 2424

The columns the candidates share

The nearest word to 300 misspelt queries, searched over a vocabulary stored as a list and as a trie, with no bound, with the best distance found so far, and with the true answer supplied. Sharing prefixes changes how the cost grows with the vocabulary rather than dividing it by a constant, and it makes the order the candidates are met in stop mattering.

1632649612810³10⁴10⁵table sizesplit points appliedevery splitbounded per blockbounded per cell, by lengthweights satisfying the quadrangle inequalityall three compute the same table

The bound a block can and cannot have

The monotone bound on an interval table's best split, applied to a cache-oblivious blocked fill rather than to a cell at a time — how often each side of the bound is available, how much work it removes, and why removing work removes almost no transfers.

alignments on a tiedistinct costs · predictionwhole bits633 of 6 · 0.86half bits793 of 6 · 0.91quarter bits414 of 6 · 0.93a grain of 0.2235 of 6 · 0.89eighth bits156 of 6 · 0.93a grain of 0.05205 of 6 · 0.95unrounded156 of 6 · 0.93200 test pairs, 16 directionsbar: alignments within 0.01 of a tie

The lattice that decides the ties

The same substitution matrix fitted at whole, half, quarter, fifth, eighth and twentieth bits and with randomised rounding, set against how many of 200 test alignments sit on a tie and how well the distance to a tie predicts which alignments a resample moves. The grain does not order the ties; the number of distinct costs surviving it does.

All essays