Field

The other axis

Auxiliary space, counted at the point it is taken. "Sorts in place" turns out to cover three behaviours that differ by a factor of n, and one of them runs out of stack.
peak slots held at once, logarithmic18645124096Insertion sort11Selection sort11Bubble sort11Heapsort11Shellsort11Quicksort, median of three22log nQuicksort, random pivot28log nQuicksort, first-element29log nMerge sort with a cutoff4,106nMerge sort4,110nn = 4,096, random inputone slot = one array element or one stack frame

Measuring what an algorithm keeps

The alloc and enter primitives, why a call stack is auxiliary space that no array counter reaches, and why peak and total have to be tracked separately rather than summarised as one number called "space".

10010³1010010³nstack framesa stack of 512 framespivot: first-elementpivot: median of threepivot: random pivotMerge sortalready sorted input, n from 64 to 4,096one frame charged as one slot

The stack nobody counts

Recursion depth as auxiliary space — why total calls and peak depth are different numbers, the three space profiles of one traversal, and the two-line change that bounds quicksort's stack on every input.

10⁵10⁶10⁷11010010³10⁴comparisonspeak auxiliary slotsInsertion sortSelection sortBubble sortMerge sortHeapsortQuicksortQuicksortQuicksortShellsortMerge sort with a cutoffn = 8,192, random input5 on the frontier, 5 dominated

The frontier between time and space

The Pareto set over comparisons and peak auxiliary space, what being dominated means that being slower does not, and how completely the frontier is rearranged by changing the input.

pale: total slots ever allocated · dark: peak held at onceone buffer, allocated once16,38416,400peak 100% of na buffer per merge229,37616,386peak 100% of nn = 16,384, random input208,687 comparisons each — identical in time

The space the model does not see

The limits of counting auxiliary space in slots — bytes, allocator overhead, the excluded input, the interaction with cache — and why the time–space product bound was attempted and cut rather than quoted.

—execution—intention87777776569888888765one unit = one subproblem given a value100 cells computed, 20 held at once

The table nobody has to keep

Why the number of cells a dynamic program computes and the number it holds at once are independent resources, what a rolling frontier costs, and the answer it can no longer give.

—execution—intention0123456789112345667822234567773333455678434345667854444567776555555678766666656787777776569888888765one unit = one subproblem given a value110 cells for one divide step, 30 held

The alignment that fits in one line

Hirschberg's divide and conquer for linear-space alignment, why the extra work is exactly twice and not approximately, and what the general time-for-space exchange looks like.

71 s60 ticks11100 ms600 ticks1410 ms6,000 ticks171 ms60,000 ticks270.001 ms60,000,000 ticksclock resolutionbits per stamp⌈log₂ 2D/r⌉no arrival rateappears in itD = 60 s · key 32 bitscomputed, not measured

The clock that cannot see the burst

Why a clock's resolution is a floor on the burstiness a system can observe, what a timestamp costs as the resolution is refined, and why the resolution is almost never a decision anybody made.

1,00010³window length W, in arrivalsbits of state heldW bits, exactε = 0.5ε = 0.2ε = 0.1ε = 0.05ε = 0.02sliding-window model · 40,000 arrivals · state from the shape of the structure5,670 bits at W = 8,000

What a window costs in bits

State against window length for the exponential histogram, the crossover with keeping the window outright, and where the bits actually go.

100,000110bits held by the whole indexLF steps per located occurrence1 in 11 in 21 in 41 in 81 in 161 in 321 in 641 in 128one point per sampling rate · 66 occurrences located each timeEnglish-like

The text that does not have to be kept

Extraction and location from a self-index — why reproducing the text is what makes the size claim mean anything, and the sampling rate that trades bits against the work of turning a row into a position.

10³10⁴10⁵10⁶passes over the datapeak bits of stateone pass, exact: 1,048,576 bitsradix, uniformsample, uniformradix, paretosample, paretoevery point exact · 32,768 values320 bits at best

What a second pass buys

Peak state against passes for exact selection — the interval that narrows, the exponent fitted to the sweep, and why a split rule that needs no data is a bet on the distribution.

sampled positions3,855grows with nsample marks2,056grows with nrun starts2,016grows with rrun lengths per symbol1,885grows with rrun heads1,563grows with rC table525grows with r11,900 bits in total16,384 characters · r = 224 · sample one in 6450% is the sampling

What is still proportional to n

The term a run-length index does not remove — why the sampling is proportional to the length, what fraction of the structure it becomes, and what changing it costs.

1,000248163264high-biased, α = 0.56low-biased, α = 0.56none-biased, α = 0.74tuples keptshards mergedlog-normal, σ = 1.2 — a latency distribution · 20,000 valuesα 0.56 / 0.56 / 0.74 · worst residual 6.5%

The cheap tail and the expensive merge

How the choice of error function decides both the size of a quantile summary and the exponent its merged size grows by, and why the two effects point the same way.

ε = 0.0515 → 29 (1.93×)ε = 0.0238 → 73 (1.92×)ε = 0.0177 → 136 (1.77×)ε = 0.005152 → 270 (1.78×)ε = 0.002397 → 674 (1.70×)reportedoccupied at the peak20,000 arrivals · lognormalpeak = resident + period, to 14%

The tuples a summary does not report

Why a quantile summary's peak is its resident size plus its compression period, what that ratio is across ε, and why the number the structure reports is the wrong one to size an allocation from.

10,000110sampling, bitssteps per occurrence1 in 21 in 41 in 81 in 161 in 321 in 641 in 128r-indexEnglish-like · 16 copies · r = 233pattern " time "

Every occurrence at the same price

Why the size-against-locate trade a sampling rate buys is a curve, and what it means that a structure sized by the run count is not on it.

1,00010,00010phrases followed12481632characters in the collection · copies aboveworst depthmean depthEnglish-like · z = 156median 18 · 90th 31

The character that costs a chain

Why an index built on a parse produces a character by following copies, how deep those chains get, and what bounds them — which is nothing.

phrase table5,148 bitsboundary orders3,744 bitsintersection grid1,696 bitspropagation grid1,544 bitsz = 156 · ⌈log₂ z⌉ = 8 levels16,384 characters · 12,132 bitsgrids 26.7% · 20.8 bits a point

The structure paid for before the first query

What a range structure costs in bits before any query is asked, where the o(z log z) actually goes, and how much of the grid the index was already paying for.

124816nonephrasescap on the copy chainphrasesworst chainEnglish-like · 8,192 charactersz 156 to 7,351

The term that came back

What happens to a structure sized by a measure of repetition when the constraint that makes it readable is tightened all the way, and what the collapse says about the measure.

ranks to compute D72extensions removed27,906extensions remaining12,051one search · k = 3 · 4,000 charactersand one more index: 17,033 bits4,000 characters · m = 16388 extensions a rank

A bound that has to be paid for

What a lower-bound pruning costs in space, why the second index cannot be avoided, and how a saving in one currency is paid for in another.

run together38,400 bitsσ 21 · 5 bitsone separator38,470 bitsσ 22 · 5 bitsa separator each46,164 bitsσ 35 · 6 bits15 documents of 512 charactersthe separators are the only differenceEnglish-like · 15 documents1.20x for the distinct marks

One separator, or one for each

What each way of separating documents costs an index, why the cost is invisible until the alphabet crosses a power of two, and which structures require the expensive one.

parentheses131,07216.7%select support115,71214.8%superblock minima16,3982.1%superblock table80,97010.3%block minima147,46518.8%block tables291,25537.2%the partsbitstotal 782,872 · the segment tree 2,228,224shared lookup table 82,944, not counted65,536 values · openings in index order, found by select2.85x smaller

Two bits a value, and what undoes them

What a succinct range minimum actually costs, in the parts the published bound is a sum of, and why the term that decides it is the map from an index to its parenthesis.

one index35,3354.31 bits/charthe forward half35,3354.31 bits/charthe reverse half35,3354.31 bits/charboth, which is the structure70,6708.63 bits/charbits8,192 characters · sample 322.00x one index

The structure that was supposed to halve

What a bidirectional index actually costs, why the second transform cannot be avoided, and what a structure named as a space fix turned out to be a fix for instead.

00.2500.5000.750101234567891011level of the wavelet treebits a bitthe parse's grida uniform permutation3,612 points · 12 levels0.997 bits a bit

A bit for every bit

What a wavelet tree over a permutation costs, where its entropy is, and why a structure whose levels are balanced by construction has almost nothing for a compressor to take.

forward · wavelet tree18,377forward · rank directories5,037forward · C table100forward · sample marks8,193forward · sampled positions3,598reverse · wavelet tree18,377reverse · rank directories5,037reverse · C table100reverse · sample marks8,193droppedreverse · sampled positions3,598dropped8,192 characters · sampling every 3216.7% of both halves

An index that cannot locate

A counting-only FM-index built, stripped and measured, with the parts it keeps, the parts it drops, and what a bidirectional index costs when one half is that.

tree · suffix array2,363,886tree · text656,635tree · document array1,050,616tree · previous-occurrence chain2,363,886tree · range minimum4,727,772succinct · suffix array2,363,886succinct · text656,635succinct · document array1,050,616succinct · previous-occurrence chain2,363,886succinct · range minimum1,483,854chainless · suffix array2,363,886chainless · text656,635chainless · document array1,050,616chainless · range minimum1,483,854chainless · reported bitmap256131,327 characters · 256 documents6 parts

What the chain cost

The size of the previous-occurrence chain measured against every other part of a document index, and what the apparatus costs once it is gone.

run together1.00xsigma 21 · 5 bitsone separator1.00xsigma 22 · 5 bitsa separator each1.20xsigma 36 · 6 bitsruns 1.00xruns 1.00xruns 1.02x16 documents of 512 characterspacked bits, against running them together

Two currencies for one separator

The three document joins priced in packed bits and in runs of the transform, and why the two answers diverge as a collection is cut more finely.

4,097 rows · one in 32 marked3.1% of the rows

The array that says where is twice the samples

Where an FM-index's locating apparatus actually is, why the marks do not shrink when the sampling thins, and the floor every curve in the strand flattens onto.

range minimum and chain838,797255.0%range minimum, no chain285,46086.8%one descent over D119,97236.5%the document array, the chain and a range minimumthe document array, a range minimum and one bit per documentthe document array, in a wavelet tree64 documents · 16,447 characters14.3% of the published apparatus

The apparatus, three times smaller again

The document-listing apparatus priced at three stages, what each removal was worth, and where the remaining bits are.

both halves locate250,584100.0%the reverse half counts only226,50490.4%and its marks are Elias-Fano210,93484.2%the saving spent on sampling241,47596.4%16,384 characters · one in 3284.2% smaller, or the same size and faster

The ladder, and the rung that spends

Three size changes to a bidirectional index applied cumulatively, and a fourth step that converts all of them into speed at almost no net size.

phrase lengths2,71222.6%phrase sources2,71222.6%phrase literals1,1309.4%boundary orders5,42445.3%three permutations of 226 elements: 1,808 bits each, and one is the inverse of another226 phrases · 4,096 characters45.3% in the orders

Half an index is three permutations

Where a Lempel–Ziv index's bits actually are, the permutation that is derivable from its neighbour, and a time-for-space choice never priced here.

copy 1copy 8copy 16copy 24copy 32position within the copy32 copies × 2,048 charactersdivergence 0.0005

Sparse on average, dense where it gathers

The run-start values of an r-index over 32 copies of 2,048 characters of English, drifted at divergences from 0 to 0.01, against the marks of an FM-index sampled one row in 32: the run-start values along the collection; bits a value as compressed blocks, as one Elias–Fano and partitioned into chunks that each choose a bitmap, their own Elias–Fano or a run; the chunks' forms along the set; bits a value against chunk size and against divergence; and the bits a select reads at each chunk size.

bitmap chunkElias–Fano chunka chunk of 32 beginscopy 1copy 8copy 16copy 24copy 32position within the copy32 copies × 2,048 charactersdivergence 0.0005

Cuts that follow the gathering

The run-start values of an r-index over 32 copies of 2,048 characters of English, drifted at divergences from 0 to 0.01, and the marks of an FM-index sampled one row in 32, partitioned by the cheapest set of cuts — a shortest path through the sorted values, its upper-level charge re-estimated until it settles — with and without a cap on the values a chunk holds: where the cuts fall along the collection, bits a value against one Elias–Fano, fixed chunks and the floor for an arbitrary set, bits against the bits a select reads, the saving under each cap, and how large the optimum's chunks are.

468567891011log₂ bits a select readsbits a valueone Elias–Fano≤32≤64≤128≤256∞t 256t 64t 16capped, no samplesuncapped, samplest: values a sampleexact copiesa model of reads, not a timing

A select that starts near its answer

The run-start values of an r-index over 32 copies of 2,048 characters of English, drifted at divergences from 0 to 0.01, and the marks of an FM-index sampled one row in 32, partitioned by the cheapest set of cuts with and without a cap on chunk length, and with a select sample every 256, 64 or 16 values inside each chunk charged in the shortest path: bits a value against bits a select reads on exact copies, at 0.0005 and at 0.01, the reads of the uncapped optimum, the capped one and the sampled one on every set, and what the samples cost in bits and in chunks.

5.8066.206.406.60567891011log₂ bits a select readsbits a valuet 256t 64t 16s 1024s 512s 256s 128s 64select samples, t valuesrank samples, s bitsdivergence 0.01a model of reads, not a timing

A sample that has to be found

The run-start values of an r-index over 32 copies of 2,048 characters, drifted at divergences from 0 to 0.01, and the marks of an FM-index, partitioned by the cheapest uncapped set of cuts with select samples every 256, 64 or 16 values or rank samples every 1,024 to 64 bits of each chunk's scanned vector, all charged in the shortest path: bits a value against bits a select reads on exact copies, at 0.005 and at 0.01, what a select reads in the search over rank samples and in the scan after it, and the smallest index of either kind within a budget of reads on every set.

3.203.403.603.8056789log₂ bits a select readsbits a valuet 64, s 64two levelsselect samples alonerank samples aloneexact copiesa model of reads, not a timing

A second level that pays in one band

The run-start values of an r-index over exact copies, over copies at divergences from 0.0002 to 0.01, and over scattered marks, cut into the cheapest uncapped partition of plain, Elias–Fano and run chunks with every directory's bits charged in the path. Select samples alone, rank samples alone, and a two-level directory with a select sample every 64, 128 or 256 values holding a rank block's number and rank samples every 64, 128 or 256 bits: bits a value against bits read a select on exact copies and at divergence 0.01, what a two-level select reads by part on every set, the rank samples it steps, and the saving per set at two budgets of reads.

All essays