Concept

Model selection — where it appears

Choosing among candidate models by how well each would predict data it was not fitted to, rather than by how well it fits the data in hand. A penalty on parameters such as BIC, or a score on held-out pairs, stops the choice drifting to the model with the most free numbers.

Named by 2 essays across one field — each of them below, with the objects they name alongside it.

1001,000aligned sequence pairs in the corpusresamples choosing the generator's classes0481216held-out pairsa fixed penalty a classa penalty that growstwo classes, 16 seeds a size203 partitions each

The classes held-out pairs cannot rule out

A substitution matrix can choose how many distinct costs it has as part of the fit: try all 203 ways of grouping its six pairs into classes and keep the one that best predicts pairs it was not fitted to. On corpora from a generator with two costs, that choice finds the two on about a third of resamples, and no more often at 3,200 pairs than at 50. A spurious split gains the same bit or two of fit at every size, and held-out scoring charges it a fixed price. A penalty that grows with the corpus finds the two every time, and finds a third class half a bit away from 400 pairs.

tables · Cost
25501002004008001,6003,200sequence pairs in the corpusclasses chosen123456six different costsmedian chosentwo terms: three parameterssixteen corpora a sizedot area: corpora choosing it

A cost built from two properties

A substitution matrix fitted as a partition gives each class of pairs one cost, and the best number of classes can be chosen from the data. When the true costs are a sum of two terms — a transversion penalty and a graded difference between the letters — every pair costs something different, and the chosen partition's number of classes becomes a statement about the corpus: two classes at 25 sequence pairs, four at 200, five at 1,600. A model with the two terms and three parameters predicts a fresh corpus better than any partition at every size, and gets 191 of 200 alignments right from 50 pairs where the chosen partition needs 3,200. Told the wrong property it is worse than the plain two-class matrix, and nothing in its own fit says so.

tables · Cost

Named alongside it

The objects these essays reach for when they reach for this one.

AlignmentCorpusCost modelEstimatorFittingHonest limitOverfittingSubstitution matrixCross validationParameter choice

All concepts