Field

The floors

Some work cannot be avoided. The information-theoretic bound on comparison sorting is computable exactly, and the gap between it and each algorithm is a number.
every algorithm that makes at most 4 comparisonsthe 24 orderings of 4 elementsroot16 leaves16 seated · 8 with no leafone comparison per level, two outcomes per comparison⌈log₂(4!)⌉ = 5 comparisons

The floor under every comparison sort

The decision-tree argument for the comparison-sorting lower bound, why it is a statement about every algorithm rather than any one of them, how log₂(n!) is computed exactly without overflowing, and what the word "comparison" is doing in the theorem.

the floorMerge sort1.02×Quicksort, first1.23×Quicksort, random1.24×Merge sort + cutoff1.29×Quicksort, median-31.32×Shellsort1.45×Heapsort1.96×Insertion sort9.69×Bubble sort19.26×Selection sort19.38×floor = log₂(256!) = 1,684 comparisonsmean of 16 runs, against a proved bound

How close anything gets to the floor

Every sorting algorithm on this site measured against log₂(n!), why the ratio is more informative than the complexity class, and what an algorithm within 2% of an unbeatable bound says about how much room is left.

comparisons (bar length is log-scaled)sorting, floor43,250sorting, merge sort43,976searching, floor13searching, binary13searching, linear4,096green outline: a proved floor · blue: a measured run3,327× between the two floors

The floor moves when the question does

How the decision-tree argument gives different floors for sorting and searching, why binary search is optimal rather than merely good, and what happens to the bound when the algorithm is allowed to look at values instead of only comparing them.

with the edge — connected01234567all 8 edges presentwithout it — two components01234567one edge withheld, everything else identicalV = 8, E = 8Ω(E), by adversary rather than by counting

The adversary who hides the edge

The adversary argument for connectivity, why the decision-tree method that settles sorting says nothing here, and what a lower bound proved by an opponent rather than by counting looks like.

the floor that applieslog₂(n!)Merge sort2.3×1,682Shellsort2.3×1,714Merge sort + cutoff2.6×1,938Heapsort4.2×3,101Quicksort, first6.6×4,887Quicksort, random7.2×5,291Quicksort, median-37.8×5,728Insertion sort18.5×13,644Bubble sort43.1×31,820Selection sort44.2×32,6408 distinct values, n = 256, seededthe two floors are 2.28× apart

The floor when the values repeat

What the information-theoretic bound on sorting becomes when keys are not distinct, why quicksort's comparison count triples on such inputs, and where counting sort's linear time comes from.

bits per elementε = 0.13.3 → 4.8 (+1.5)ε = 0.035.1 → 7.3 (+2.2)ε = 0.016.6 → 9.6 (+2.9)ε = 0.0038.4 → 12.1 (+3.7)ε = 0.00110.0 → 14.4 (+4.4)ε = 1e-413.3 → 19.2 (+5.9)floor log₂(1/ε) filled; Bloom's log₂(1/ε)/ln 2 outlined44.3% above the floor at every rate

A floor on the bits

The information-theoretic floor on approximate membership, why it is a bound on space rather than on time, and what it means that a Bloom filter sits a constant factor above it at every accuracy.

10110100runs in the transform, rbitsthe structurelog₂ N(r)12 characters · 2 symbols · all 4,096 texts walkedgap 14.1x–92.0x

A floor under a run count

The counting floor under any run-based representation, computed exhaustively for every text of twelve characters, and the exact slack in the encoding that stores runs.

10⁴10010³n (elements)block transfersmeasured sortthe bound3.20×2.67×2.29×2.94×2.67×2.40×B = 32, M = 512 (M/B = 16)3.20× the floor at worst

The floor under moving data

The external-memory sorting lower bound, where it comes from, why it is a floor rather than an aspiration, and the measured gap between it and a real sort.

thousands of bits · the floor is 31.9kthe text as it is34.0k streamto undo itnothinga shuffle agreed in advance34.4k streamto undo itnothingthe characters sorted0.3k streamto undo it31.8kthe Burrows–Wheeler transform14.8k streamto undo itnothingmodel: order 0 after move-to-front · Words from a fixed vocabularydashed: coding the text as it stands

What a reordering costs to undo

Four permutations of the same 8,192 symbols — none, a shuffle agreed in advance, a full sort, and the Burrows–Wheeler transform — each followed by move-to-front and coded at its zeroth-order entropy, with the side information its inverse needs added. The sort's saving is exactly cancelled, and every string of eight characters over three letters is enumerated to check that it is.

bits per symbolhuffman3.9371.01× the floorarithmetic (adaptive)3.8981.00× the floorLZSS (window 4096)2.1130.54× the floorLZ783.1470.81× the floorH₀ = 3.89216,384 symbols · alphabet 21 · floor shown is H0 = 3.892 bits/symbolmodel: order 0 · Words from a fixed vocabularybest here: 2.113 bits/symbol

The bits a coder emits

Shannon's bound as a measured quantity — what the entropy of a stream is a floor on, how close each coder gets, and why this floor moves when nothing about the data changes.

011228101214161820universe size ubits of statea u-bit bitmaplog₂ C(u, u/2)⌈log₂(u+1)⌉, a counterat u = 12: 924 subsets, 8 bitstwo collide → answers 6 and 7floor computed exactly · collision found by exhaustion at u = 12floor 9.85 bits

The floor under a summary

The site's third kind of lower bound, counting memory states rather than comparison outcomes, performed by exhaustion rather than quoted — and what changes when the answer is allowed to be approximate.

1,00010³window length W, in arrivalsbits of state heldW bits, exactε = 0.2ε = 0.1ε = 0.05sliding-window model · 40,000 arrivals · state from the shape of the structure2,808 bits at W = 8,000

The floor under a window

The memory-state lower bound for exact sliding-window counting, performed exhaustively, and what is measured rather than proved about the approximate case.

SETHno algorithm for k-SAT beats exhaustive search for every korthogonal vectorsno N^(2-e) algorithmedit distanceno n^(2-e) algorithmperformed heresplit and list, checked on 55,754 formulasquoted, not performed herequoted, not performed heresolid: an implication performed here · dashed: one that is quoteda conditional floor

A floor that holds if something else does

The conditional lower bound under edit distance — split-and-list from satisfiability to orthogonal vectors, verified by exhaustion, and the identity that makes a quadratic scan exactly exhaustive search.

read from the start40,000 readsa ring of 4,096 keys and 4,096 stamps184,320 bitsread from the end4,096 reads792 counters, no stamps50,688 bitsitems readsame answer: 1×700, 2×341, 3×224…W = 4,096 · φ = 0.02 · stationary Zipf9.8× fewer reads, same answer

The pass that runs the other way

What making the direction of a pass a parameter reveals about the sliding-window model, why the same trick is worth nothing on a question about the prefix, and where a backward pass is not available at all.

universe 0…11 · prefixes of 6 · candidate keeps 9 bitsthe two prefixes that collideA01234567891011B01234567891011first differencesame state — the candidate stores "7" after boththen both read the same suffix -1, -2, 12, 13, 14true median of A3true median of B4and one answer for both924 prefixes · floor ⌈log₂ C(12,6)⌉ = 10 bits10 bits collide on none

A floor one pass cannot get under

The sixth lower-bound argument here and the third of the memory-state kind — the construction, the exhaustive search that performs it, why it is tight, and what a second pass does to it.

010020030040021φ 382φ 7203φ 17434φ 42885φ 1012486φ 242measured 409counts chargedlevel of the foldcharged at this levelrunning total64 shards · k = 32 · hashedcharged 409 · measured 409

The floor charged at every level

What a Space-Saving fold actually charges a surviving key, why the account is exact, and why a model counting only the shard floors was right at eight shards and half the answer at sixty-four.

0.111010010³10³10⁴floor, in countsarrivals in the shard, nround-robin — n^1.02hashed — fit refusedresidual 2.7%slope 5.2 → 1.19k = 32 · 40,000 arrivalsthe table holds 1.33 of a hashed shard's keys and 0.01 of the stream's

A floor with two variables in it

Why a Space-Saving floor is linear in a shard's mass under one partition and not a power law at all under another, and what the second variable is.

1010010³documents in the answeroperations inside the structureevery occurrencerange minimumone descent17 to 324 rows5.14 to 1.94 per document

Work that falls as the answer grows

What proportional-to-the-answer means when the constant is not constant, measured across a sweep where the range grows twenty-five-fold and the answer fivefold.

0100200300102030distinct symbols in the intervalbit-vector ranksthe loop: 320the descentsigma = 32 throughout32x down to 5.16x

Proportional to the answer, not the alphabet

The cost of an interval enumeration against how many symbols the interval holds, at fixed sigma, and the logarithmic discount that makes the per-symbol cost fall.

occurrences reportedin text order16from the right2 — 14 lostevery one it reports is real, so the answer is short rather than wrong1 of them were found by the boundary search and never propagated16 copies · 6-character pattern87.5% lost

From the right, two of sixteen

What a traversal order is worth, measured by reversing it, and the second silent shortfall of the same shape in two strands.

probe 1: 0–1 absent9 still to ask10 of 10 as good as anyprobe 2: 0–2 absent8 still to ask9 of 9 as good as anyprobe 3: 0–3 absent7 still to ask8 of 8 as good as anyprobe 4: 0–4 present6 still to ask7 of 7 as good as anyprobe 5: 1–2 absent5 still to ask6 of 6 as good as anyprobe 6: 1–3 absent4 still to ask5 of 5 as good as anyprobe 7: 1–4 present3 still to ask4 of 4 as good as anyprobe 8: 2–3 absent2 still to ask3 of 3 as good as anyprobe 9: 2–4 present1 still to ask2 of 2 as good as anyprobe 10: 3–4 absentdecided1 of 1 as good as any5 vertices · 10 pairs · 59,049 states solvedsolid: present · dotted: absent · coloured: this probe

Every pair must be asked

The adjacency-matrix model solved exactly: every partial state of the probing game enumerated at up to six vertices (14,348,907 states), every property of four-vertex graphs checked, the parity argument that settles most of them in a line, the two controls that show what the conjecture's hypotheses are doing, and how far below the worst case the optimum sits on a random graph.

length of the longer list, nshorter, m1234561234561floor 12floor 22floor 23floor 33floor 33floor 33floor 34floor 45floor 45floor 56floor 55floor 56floor 67floor 67floor 77floor 78floor 79floor 89floor 810floor 911floor 10the optimum is one above the floorthe floor is reachedevery cell solved exactly · m ≤ n ≤ 6large: the optimum · small: ⌈log₂ C(m+n, m)⌉

The floor a merge cannot reach

The information floor for merging, log₂ C(m + n, m), checked against the exact optimum found by solving the merge game over every state of knowledge up to seven keys a list: where the worst case misses the floor and by how much it grows, the adversary argument that forces 2m − 1 on equal lengths, how close the best average gets, and four real merges measured against both floors over every interleaving and at ratios up to a thousand to one.

01234characters of context the sort reads, kbits a symbol after move-to-front012481632the full transform, 1.802ties kept in text orderties sorted by symbol: the stream alonethe text's own floormodel: order 0 after move-to-front · Words from a fixed vocabularyexactly the transform from k = 24

The order inside a tie

Rotations sorted by their first k characters for k from zero to the full context, under two rules for the order inside a tie. Every string of eight characters over three letters is enumerated to count what each costs to undo, 8,192 symbols from three sources are coded after move-to-front, and the characters the sort reads are counted, to find where the sort and the transform sit in one family.

All essays