Generator

One tug-of-war register: 32 keys, each pulled one way or the other

Rendered here at the parameters it defaults to, with every essay that calls it — which is the same list as the blast radius of changing it.
One tug-of-war register: 32 keys, each pulled one way or the otherEvery bar is one key of the stream, its height its exact count and its direction the ±1 the hash family drew for it. The register holds the sum, which is 9,034; the estimate is that number squared, 81,613,156, against a true F2 of 36,931,352 — 121.0% out on this draw. The bars very nearly cancel, and that is the mechanism rather than an accident: the cross terms cancel in expectation and what survives is the sum of the squares. Model: cash register, 20,000 updates.+1−1keys, most frequent firstΣ s(x)·f(x) = 9,034squared: 81,613,156true F2: 36,931,352121.0% outa polynomial of degree 4 · cash-register model · counts exactone register, 32 bits

One tug-of-war register: 32 keys, each pulled one way or the other

Every bar is one key of the stream, its height its exact count and its direction the ±1 the hash family drew for it. The register holds the sum, which is 9,034; the estimate is that number squared, 81,613,156, against a true F2 of 36,931,352 — 121.0% out on this draw. The bars very nearly cancel, and that is the mechanism rather than an accident: the cross terms cancel in expectation and what survives is the sum of the squares. Model: cash register, 20,000 updates.

Drawn at 700 × 224, wide on the page. Everything above is what tug-of-war-run returns with no arguments; the caption is the generator's own, computed from the numbers in the drawing rather than written beside it.

7 essays call tug-of-war-run. The drawing above is what it returns with no arguments at all; every call below passes it something, because a placement that passes nothing draws whichever member of the family the generator happens to default to rather than the one its essay argues about — which is what optcheck and figfill exist to catch.

Where it is called

Changing this generator changes every one of these figures.

+1−1keys, most frequent firstΣ s(x)·f(x) = 9,034squared: 81,613,156true F2: 36,931,352121.0% outa polynomial of degree 4 · cash-register model · counts exactone register, 32 bits One pass, and no room

The estimate that squares the stream

1 coefficient13 members2 coefficients169 members3 coefficients2,197 members4 coefficients28,561 members1 key2 keys3 keys4 keys5 keysexact92% gone99% gone100% gone100% goneexactexact92% gone99% gone100% goneexactexactexact92% gone99% goneexactexactexactexact92% goneevery member walked · GF(13) · no tolerance and no seeddegree 1, 2, 3, 4 When the algorithm flips a coin

The independence an estimator spends

Count-Min 4×64exact0 under200 overrms 303Count-Sketch 4×64-2,774-1,38701,3872,774120 under80 overrms 256cash register · insertions only, Zipf s = 1.1 · counts exact8,192 bits each Structures

A sketch that is allowed to be under

0.313103000.250.50.7511.251.52Zipf exponent s of the stream — 0 is flatvariance of one copy ÷ moment²sampling, F4sampling, F3sampling, F2signs, F22,048 keys, 60,000 items, exactdashed: tug-of-war One pass, and no room

The moments past the second

00.250.50.7511.5210010³Zipf exponent of the streamwords for 10% on F3sample every positionheavy keys first60,000 arrivals, 2,048 keysa word: a key or a count One pass, and no room

The keys worth counting before sampling

501002005001,0002,0005,000Zipf exponent of the stream (0 is flat)words for 10% on F3 (log scale)00.250.50.7511.52sample every positionshare 3%, list of 64share 1%, list of 8the best fixed list60,000 arrivals, 2,048 keysa word: a key or a count One pass, and no room

A share of a list of equals

010203005001e+31.5e+32e+3arrivals so farcandidates on the listZipf 0.5: ends at 7Zipf 1: ends at 3faint levels: the length at 60,000 arrivalsθ = 0.03 One pass, and no room

A list that gives back its places

The library, page 5 of 5 — where tug-of-war-run sits