A list as long as the doubt

The commonest word at the distance priced an assumption every spelling essay before it had made. A checker searching a trie of 2,424 vocabulary words for the words nearest a misspelt query had been credited with a correction whenever the intended word was anywhere among its nearest words, and on a query of three or four letters that list averages three or four words. Offering only the commonest of them, ranked by how often each is used in the prose corpus the vocabulary came from, kept 77% of the whole list’s corrections on short words drawn as writers make them. Offering the three commonest kept 96%.

Its closing section proposed the checker between the two. Offer the commonest word alone when its count is far above the next word’s, as the is above them, twenty-six to one, and offer two or three words when the counts are close, as was and way are at thirty-four to thirty-three. The list would grow only where the counts leave doubt. It predicted that on misspellings drawn by use, such a list, averaging about 1.3 words on short queries, would recover nine tenths of what the three commonest recover, and that on misspellings drawn evenly from the word list the ratio would mislead, as the ranking had.

The prediction is met on its own terms, and its terms turn out to be the wrong ones. The counts carry real evidence about which word was meant. A list sized by that evidence is barely better than a list sized at random.

One word when the counts agree, more when they do not #

The checker, the cap of two edits and the two pairs of streams are the earlier essays’. The stream drawn by use takes 2,000 misspelt vocabulary words with their sources in proportion to their prose counts; the even stream draws its sources uniformly from the word list; both give each source one or two random edits, the edits a typo writes. Every misspelt vocabulary word is answered under the cap, and the only question is how many of its nearest words to show.

For each answered query, the nearest words are ranked by prose count. If the commonest word’s count is at least $r$ times the second’s, the checker offers the commonest alone; otherwise it offers the $K$ commonest, with $K$ two or three. A query with a single nearest word always gets that word. At $r = 1$ the rule is the one-word checker, since the commonest is always at least as common as the second, and once $r$ exceeds every ratio that occurs it is the $K$-word checker. Ties are scored as the earlier essay scored them, by expectation: a source tied at the top with two others earns a third of a correction from a one-word list. Corrections are counted as a share of every misspelt vocabulary word in the length band, answered or not, and the list’s length as the mean number of words it shows on the answered ones.

The rule has a baseline that the closing section did not name, and it decides how to read everything after it. A checker that flips a biased coin for each query, offering one word with some probability and three otherwise, reaches every point on the straight line between the one-word and three-word checkers, at whatever mean length it likes, without reading a single count. A list sized by the counts earns its place only by sitting above that line.

Above the line by two and a half points #

On misspellings of three or four letters drawn by use, a list of 1.32 words a query, the one the prediction named, corrects 70.4%, 88% of the three commonest’s 79.7%. A blind mix of one word and three at the same length corrects 69.0%. The prediction’s nine tenths is nearly met. But the one-word checker alone already reaches 80% of what three words reach, so a measure taken against the three-word list was always going to look generous. Against the line a coin reaches, the ratio rule at that setting is 1.4 points better.

That is the most the rule ever does on this stream. Its greatest lift over the line is 2.5 points, at $r = 10$, where it corrects 76.8% showing 1.65 words. The plain list of two words, which reads the counts only to rank, corrects 76.5% showing 1.59. The rule never reaches a point a fixed list of two cannot nearly match. With two words in doubt rather than three it does a little better, 76.2% at 1.47 words at $r = 20$, which saves the fixed list a tenth of a word a query for three tenths of a point.

So the obvious checker exists and it is not much better than its two obvious neighbours. The ratio buys an eighth of a word a query on short misspellings, at the corrections a list of two already makes.

The ratio is evidence #

Drawn by use, the commonest nearest word is the intended word 36% of the time when it is under twice as common as the next, 64% at five to twenty times, and 91% at over a hundred. The rise is steady across six bands and large, a factor of two and a half from the bottom band to the top. The closing section’s premise holds. The ratio of the top two counts says how far the counts can be trusted, and a checker that reads it knows, query by query, how likely its single word is to be right.

The reason the ratio carries evidence is the reason the counts carried evidence at all. When misspellings follow use, the writer’s intended word is drawn in proportion to its count, so among the words a typo landed near, each is the source with a chance roughly proportional to how often it is written. A word ten times as common as its nearest rival is about ten times as likely to be the source, and the counts only short words carry built its score on exactly that. The ratio is that likelihood ratio, read off the two largest terms.

The trouble is in the bars beneath the line. Of the 415 short queries with a choice, 77% have a ratio between one and twenty and 3% more are tied, and in that range the commonest word is right a third to two thirds of the time. The ratio sorts the queries well. It does not sort many of them into the confident band.

A second word earns wherever the ratio is under twenty #

A second word adds 0.27 of a correction a query where the commonest is under twice as common as the next, 0.32 where it is two to five times as common, and 0.20 at five to twenty times; above twenty it adds 0.04. The value of a second word is nearly flat across the bands where most queries lie, and then it falls off a cliff. A list sized by the ratio can save words only where the second word earns nothing, and that is the 20% of queries above a ratio of twenty, plus the 41% with one nearest word, which every rule already answers with one word.

That is why the rule’s curve hugs the line. Moving $r$ from 1 to 20 adds the second and third words to queries in bands where the second word adds 0.2 to 0.3 a query, roughly the average it adds everywhere, which is what the coin adds too. The rule beats the coin only by withholding words from the top band, and the top band is small.

The third word earns little anywhere: under a tenth of a correction a query in every band. That accounts for the rule doing slightly better with two words in doubt than with three, at the same corrections and fewer words. A source that is not among the two commonest of its nearest words is usually not among the three commonest either; it is a rare word whose typo landed among common ones, and no ranking by use will reach it.

Drawn evenly, the ratio points the wrong way #

On misspellings drawn evenly from the word list, the ratio rule falls below the line a coin reaches at every ratio from 1.5 to 100, by as much as 2.6 points at $r = 5$. Reading the counts is worse than ignoring them, which the prediction expected in kind. The calibration plate shows why. On this stream the commonest nearest word is right 13% to 31% of the time in every band that holds more than a handful of queries, and it is right least often, 13%, at ratios of five to twenty, where on the stream drawn by use it was right 64% of the time.

The earlier essay found the reason. Drawn evenly, the short sources are mostly the vocabulary’s rarely used short words, hood, cuts, tens, and a typo of a rare word lands among common ones as easily as a typo of the does. So the dominant word in the list is usually a common word the writer did not mean, and the more dominant it is the more confidently the rule offers it alone. The ratio is still a likelihood ratio. It is the likelihood under a model of the writer, misspellings in proportion to use, that this stream does not follow.

A checker cannot tell from its own counts which stream it is serving. What insurance against an estimate costs found the same shape in a query planner: an estimate’s confidence is a statement about the model that produced it, and when the workload leaves the model, the confident cases are the ones that go wrong.

Over every length, the same small lift #

Over misspellings of every length drawn by use, the ratio rule’s greatest lift over the coin’s line is 2.3 points, at $r = 5$: 86.1% corrected at 1.33 words, against 83.8% for a blind mix and 86.7% at 1.40 words for a plain list of two. Long queries make the picture milder rather than different. Most of them have one nearest word, so every rule shows them one word and corrects them, and the lists shorten for all policies alike.

Two words unless the commonest is twenty times the next corrects 76.2% of three- and four-letter misspellings at 1.47 words and 97.7% of seven- and eight-letter ones at 1.12, within 3.5 points of the three commonest on short words and within 1.6 from five letters up, while showing a word and a half or fewer. By length the rule does what its designer would hope: it shows more words where the lists are long and the counts are close, which is on short words, and almost always one word from seven letters up. But a fixed list of two would show nearly the same lengths, because a long query’s nearest-word list is almost always a single word, and a list of two cannot show a second word that does not exist.

That is the plain reason the doubt-sized list and the fixed list of two coincide so closely. On every query with one nearest word, both show one. On short queries with several, both usually show two, because the ratio is under twenty three times in four. They differ only on the one short query in five whose commonest word dominates, and there the second word was worth 0.04 of a correction.

What the counts can and cannot say #

The counts answer two questions, and the measurement separates them. Which word is likeliest? The commonest, and on the stream drawn by use the ratio of its count to the next says how much likelier, faithfully, from bands under twice as common to bands over a hundred times. How many words should a list show? That depends on how much the second word would add, and the second word’s value is nearly the same, 0.2 to 0.3 of a correction, across every ratio where most queries sit. The ratio is evidence about the first question and nearly silent on the second.

Put as a decision, a second word is worth showing whenever the chance that it is the source is worth the reader’s glance. Under use, that chance is about one in four for ratios up to twenty, and it does not shrink much as the ratio grows within that range, because a larger ratio means a commoner first word, not a rarer second word. The second word is rare in absolute terms, not relative to the first, and its chance of being the source depends on its own count against the whole neighbourhood’s. A rule that read the second word’s share of the neighbourhood’s total count, rather than its ratio to the first, would be asking that question directly.

A trie fitted to its queries found that the queries a checker serves decide its structure; this checker’s best list length is decided by the same thing, and on the even stream no setting of the ratio helps at all.

What the prediction got right #

The closing section predicted that on short misspellings drawn by use, a list averaging about 1.3 words would recover nine tenths of what the three commonest recover. At 1.32 words it recovers 88%. It predicted that on the even stream the ratio would mislead, as the ranking had, and it does, falling below a blind mix at every ratio from 1.5 to 100.

What the prediction measured against was the three-word list, and on short words the one-word list already reaches 80% of it, so any rule that mixes the two clears most of the prediction’s bar without reading the counts. The question it asked, whether a checker can tell from its counts when they are not enough, has a two-part answer. It can tell how likely its first word is: on the stream drawn by use, between 36% and 91% across the bands of the ratio. It cannot tell from that how much a second word would add, and showing two words on every short query is within a quarter of a point of the best list the ratio can size.

Three things the measurement assumes #

One corpus of counts. The ranking is by counts in the 19,279-word prose corpus the vocabulary was drawn from, and the stream drawn by use draws its sources from the same counts, so on that stream the counts are as right as counts can be. A checker whose counts come from different text than its writers write would sit somewhere between the two streams, and nothing here says where.

Small bands. The 415 short queries with a choice drawn by use, and 178 drawn evenly, are divided into six bands of the ratio, some holding a few dozen queries and one holding three. A result the size of its own noise is the standing caution: the trend across bands on the stream drawn by use is large and monotone, and a difference of a few points between neighbouring bands is not evidence of anything.

A word shown is a word read. A list’s cost is counted as the words it shows, each counted alike. A reader skimming a list of two may read the second word at less than full cost, or ignore it, and a checker’s real trade is between corrections and the reader’s attention, which nothing here measures.

Still open: a second word judged by its own share #

The ratio of the top two counts measures the first word’s dominance, and the second word’s value does not follow it. What the second word is worth is the chance that it is the source, and under misspellings drawn by use that chance is its count over the counts of every word the typo could have come from, which the trie’s walk has already found: every word within two edits, weighted for distance.

The measurement that follows offers the second word whenever its share of that weighted neighbourhood exceeds a threshold, swept, and the third likewise, and charts corrections against words shown on both streams beside the ratio rule and the fixed lists. The prediction is that it beats the coin’s line by about twice what the ratio rule does on short words drawn by use, five points rather than two and a half, because it withholds the second word exactly where its chance is small, whatever the first word’s count, and that on the even stream it fails as the ratio did, since its shares come from the same counts. It could fail on the weighting. The neighbourhood includes words two edits away, and how much less likely a two-edit typo is than a one-edit typo sets the share; the commonest word at the distance found the noisy-channel choice needed that weight near a hundredth before it stopped losing corrections, and a share computed at the wrong weight would be a ratio in disguise. The question is whether the reader’s doubt about a list is a property of the list’s own words or of everything the typo could have been.