Algorithms & Data Structures
the bound is a claim, and it is checkable
Theme

The thread: The check must reject — page 2

Page 2 of 2, continuing the same thread in the same order.
keys in each sorted run of the inputwords read a question (log scale)0.111048163264128256512sortedregisterskept past a breakanswering from insidetwo bits a pair, plain1,024 keys, a pass firstlogarithmic scale Counting

A register dropped by what it already knew

Two registers beside a two-ended selection sort's extremes cut the reading of its record of adjacent order to a forty-fifth on short runs, and made it worse than reading the record plainly on long ones: 53.5 words a question on a sorted array of 1,024 keys against 22.1. The swaps were blamed, and the proposed repair kept the part of a broken run beyond the break as a register of its own. It changes the reading by under 1% anywhere. On the sorted array the survivors are kept 523,775 times and asked 1,022 times, and the sort makes no swap at all: every one of the 263,167 relearning drops there is a pair the record already held, rewritten at the same value by a neighbour's question that the previous scan's chains answered. The swap never broke the register. Learning what it already knew did.

 range consideredminimum atC[at] < lonew document[476, 489)483 · document 4yesyes[476, 483)477 · document 1yesyes[476, 477)476 · document 0yesyes[478, 483)482 · document 2yesyes[478, 482)478 · document 0nono[484, 489)486 · document 3yesyes[484, 486)484 · document 0nono[487, 489)487 · document 2nono8 steps, 5 documents, and the two columns never differrows 476–489 · 8 documents8 steps Structures

A document already in the answer

The same walk, with the chain's test replaced by a lookup in the answer so far. It reports the same documents at the same cost, and two exchanged lines make it lose nine of seventeen without failing.

 equal lengths5.00 bits100.0%lengths as one over rank4.15 bits83.4%a few long, many short4.31 bits86.7%one document holding most of the text1.69 bits39.5%log₂ 3232 documents · 8,192 characters100.0% to 39.5% What is taught wrongly

A saving quoted without its collection

A check asking whether a compressed document array is smaller than the plain one passes on the rounding whenever the document count is not a power of two. It would report a saving of nothing as sixteen per cent, on a collection that has no redundancy at all.

 a fixed-length codeleaves in order␣acdefghilmnoprstuvwythe best ordered treeleaves in order␣acdefghilmnoprstuvwythe best tree of any shapeleaves in frequency order␣eahnrstdiloucfgmpvwythe symbols, in the order the descent reports them21 symbols in 2,048 positionsone set, 2 of 3 sorted What is taught wrongly

One set, three orders

The symbols an interval holds do not depend on the tree's shape. The order they come out in does, and a search that accumulates a running count as it reads them computes a plausible number that is wrong on eighty per cent of queries.

 acdefghijklmnopqrstuvwyzroot24 of 26 symbols present54 ranks · 51 nodes What is taught wrongly

A walk that does not prune

Remove the emptiness test and the descent visits every node of the tree, returns exactly the same symbols with exactly the same intervals, and costs sixty per cent more. No test of the answer can see it.

 01e+42e+43e+44e+450100one sampled position in every …bitsplain, one bit a rowcompressed blocksElias-Fano16,384 rowscrosses at one in 4 What is taught wrongly

Where the sparse representation loses

At every row marked, Elias–Fano costs twice the plain vector. The crossing is at one row in four, which is a sampling rate a real index uses — so the choice between the two is a choice, not an improvement.

 bits heldthe model3,591the structure3,905one rank, in operationsbinary search10 stepsselect and walk1 select + 0.7016,385 rows · one in 32 marked14x on the operation What is taught wrongly

A price with no structure under it

A class in this collection has charged Elias–Fano's price for several strands and stores an array of positions searched by binary search. The accounting is right about space to a bit per thousand and wrong about one operation by a factor of eight.

 factor against the published loopthe compound walk11xthe enumeration78xthe two, multiplied860xnothing measures thisan index with both78xthe shortfall is 11x, and the walk's factor is 11x6 patterns · 1 error · sigma 21the larger of the two, not the product What is taught wrongly

Two factors that do not multiply

Eleven times and seventy-eight times against the same baseline, so an index with both should be eight hundred and sixty. It is seventy-eight, and the shortfall is eleven — the first factor, exactly, because the second operation already contains it.

 a second hash, unordered0.054 entries read on an absent key14.435.79 storedthe bits below the bucket, unordered0.059 entries read on an absent key14.435.79 storedthe bits below the bucket, sorted0.057 entries read on an absent key8.975.21 storedthe bucket's own bits, sorted5.743 entries read on an absent key11.275.80 storedtag comparisons an absent key's lookup makes8,192 slots · buckets of 8 · 8-bit tags38% fewer What the machine does

A tag that answers more than yes

A byte of hash in front of each entry has one bit of output per slot: this tag matches or it does not. Cut it from an ordered field instead and a bucket's tags can be kept sorted, so a lookup stops at the first tag above its own — 8.97 comparisons for an absent key instead of 14.43, for 3.55 tag-byte shifts an insertion. The obvious ordered field is the trap. Take the tag from the bucket index's own bits and every key in a bucket shares them: 5.74 entry reads on an absent key instead of 0.054, a hundred and seven times more.

 10100states the cache holdsoperations a characterthe plain NFA: 52.5the whole set fits: 19.94,096 characters · k = 8the knee is at 512 states What is taught wrongly

A cache below the reachable set

A lazy machine with a cache of two hundred and fifty-six states costs fifty-one operations a character and a plain non-deterministic simulation costs fifty-three. At five hundred and twelve it costs eleven. The line is flat across two orders of magnitude and then falls off a cliff.

 00.2000.4000.600residents drawn, the oldest of them kicked (8: the bucket's oldest)kicks whose key found room12348cap of 2cap of 8cap of 3295% load, after churnshare of kicks What the machine does

An old key is a crowded key

A cuckoo table that makes room by kicking each full bucket's oldest resident keeps its newest keys at home and brings a lookup asked by recency to 1.04 reads, the cheapest design measured — but only at a cap of two kicks. At a cap of 32 it leaves 6.6% of its keys in overflow where random kicks leave none, and the explanation offered was a loop: an old key kicked to its other bucket is old there too, and the next chain moves it back. Sparing every key the chain has already moved changes nothing at caps of two and eight and makes cap 32 slightly worse, 7.1%; sparing the last sixteen insertions' moves gets it to 5.0%. There was no loop. A kicked resident's other bucket has room 22% of the time when the resident is chosen at random and 7% when it is the oldest: in a table that always moves the oldest, a key grows old because both of its buckets stay full. Kicking the older of four drawn at random keeps much of the gain at a long cap, 1.055 reads by recency and 0.5% overflow, and still costs more than the oldest kick at a cap of two.

All threads