Concept

Generated collection — where it appears

A collection made by copying a base text and replacing a fraction of each copy's characters. It is an instrument rather than a sample: one dial moves the repetition monotonically, and every property it has no dial for is fixed by construction — one document length, one edit rate, one alphabet.

Named by 6 essays across 4 fields — each of them below, with the objects they name alongside it.

100100occurrences of the patternreadsmost frequent ", and " · 388 occurrencesthe scanthe chain12 documents · 144,617 characters3 of 6 won

What the generated collection was right about

Five strands of conclusions, drawn on collections made by one line with one dial, checked against a corpus nobody made. Most hold. One headline was a property of the generator's alphabet, and one crossing that was guessed at turns out to be met — but only with the structure the strand on range minima built.

practice · Document
twelve essays — runs0.491generated, matched0.109twelve essays — phrases0.213generated, matched0.046eight source modules — runs0.461generated, matched0.122eight source modules — phrases0.210generated, matched0.050ten revisions of one file — runs0.126generated, matched0.113ten revisions of one file — phrases0.067generated, matched0.047per character24,576 characters a partdivergence 0.02

A corpus that was not generated

Every collection in five strands has been copies of a generated text with a fraction of its characters replaced — three numbers, one dial. Here is one that was not — twelve essays, eight source modules, ten revisions of one file — measured beside the model of it.

text · Document
run together867 bits a characterone separator877 bits a charactera separator each897 bits a charactergenerated, run together215 bits a characterdistinct symbolstwelve essays0.0% premium

Documents that are not the same length

A separator per document costs a whole bit per character — on a generated collection whose alphabet is 21 symbols, where adding a few crosses 32. Real prose has 87 symbols and sits 41 short of the next power of two, so the same separators cost 0.04%.

indexes · Document
02468101214mean 5.71depthshare of the collectionten revisionstwelve essays75,658 characters · 2,966 phrasesworst 14, mean 5.71

The shape of a real history's depth

Four essays here are about capping how far an extraction follows a chain of copies, and every number in them came from a generated collection. Here is the depth histogram of a real version history, which is a bell, and of twelve unrelated essays, which is nearly the same bell.

text · Parse
ten revisions of one filecap 10worst 14 · mean 5.7fourteen revisions of anothercap 10worst 17 · mean 7.2twelve unrelated essayscap 10worst 14 · mean 4.9eight source modulescap 10worst 78 · mean 5.9a generated collectioncap 10worst 13 · mean 4.6within a twentieth of the free parsefour to eight was the model

The cap that would ship

The published sweep put the knee at four to eight. On four real collections it is at nine to twelve, a cap of one costs forty-eight times the free parse rather than twenty-one, and the number a system should actually set is none of those.

practice · Parse
1.0%10.0%0.11fraction of characters changedper characterthe real runsthe real phrasesgenerated runsgenerated phrasesten revisions of one file · 24,580 characters2.32% against 3.24%

The dial that has no setting

The generator has one parameter. The real version history's run count asks it for 2.3% and its phrase count asks for 3.2% — and the reason is not that the dial is badly calibrated. Real edits average 7.3 characters a block and generated ones average 1.04.

wrong · Document

Named alongside it

The objects these essays reach for when they reach for this one.

CorpusControlMeasurementPhrase countVersion historyIndex sizeAlphabetDepthDocument collectionDocument listingExtractionModel

All concepts