Concept

Interval symbols — where it appears

The operation that returns every distinct symbol a range of a sequence holds, together with the sub-interval each occupies, from one descent of a wavelet tree. Its cost follows how many symbols are present rather than how many the alphabet has, so a search branching over an interval never attempts an extension that finds nothing.

Named by 13 essays across 7 fields — each of them below, with the objects they name alongside it.

0100200300102030distinct symbols in the intervalbit-vector ranksthe loop: 320the descentsigma = 32 throughout32x down to 5.16x

Proportional to the answer, not the alphabet

At a fixed alphabet of thirty-two, a loop costs three hundred and twenty ranks whether one symbol is present or all of them. The descent costs ten and sixty-two. The experiment has to move the answer without moving the alphabet, and the obvious sweep moves both.

floors · Symbol
acdefghijklmnopqrstuvwyzroot24 of 26 symbols present54 ranks · 51 nodes

A node costs two ranks

The left child's interval is the position minus the right child's. A descent that calls rank on both children returns exactly the same symbols at twice the cost, and nothing about the answer can see it.

machine · Symbol
binary · sigma 22.00x2 against 4dna · sigma 42.67x6 against 16protein · sigma 204.76x42 against 200latin · sigma 264.81x54 against 26064-position interval2.00x to 4.81x

Two at binary, five at twenty-six

The saving is a factor in the alphabet, so a two-symbol alphabet gets two. Approximate matching in this field is mostly done on DNA, which sits near the bottom of the list at 2.7.

bounds · Symbol
025507510000.50011.502errors permittedfactor against the published loop64.5%55.8%74.7%the walk: 11xthe descentthe label is the shareof dead extensions6 patterns · sigma 2198x to 105x

Flat in the budget, and not

One saving is eleven times at every error budget, because it is a property of the alphabet. The other moves between ninety-eight and a hundred and five, because it follows the share of extensions that find nothing. Two savings, two shapes, and neither line crosses the other.

bounds · Distance
every occurrence5610 documentsrange minimum and chain32510 documentsrange minimum, no chain9610 documentsone descent over D3810 documentsthe document arraythe document array, the chain and a range minimumthe document array, a range minimum and one bit per documentthe document array, in a wavelet tree56 occurrences in 10 documentsoperations inside the structure

The tree answers the question

The distinct documents in a range of rows are the distinct symbols of the document array in that range. A wavelet tree enumerates those in one descent, so the range minimum, the chain, the bitmap and the recursion all go at once.

indexes · Index
acdefghijklmnopqrstuvwyzroot24 of 26 symbols present54 ranks · 51 nodes

Asking about symbols that are not there

A search extends an interval by every character of the alphabet, and on a deep branch almost all of them produce an empty interval. That is a full rank walk whose entire result is the discovery that nothing was there.

indexes · Symbol
extensions attempted9,206 dead7,594 livebit-vector ranksthe loop: 168,000the descent: 24,0868 patterns · 1 error · sigma 2054.8% dead · 6.98x

The branches that find nothing

An approximate search over a twenty-symbol alphabet attempts sixteen thousand eight hundred extensions and nine thousand two hundred of them produce an empty interval. That is a full rank walk whose entire result is the discovery that nothing was there.

structures · Symbol
a loop over the alphabet1,478,400the compound walk, inside the loop134,40011xone descent per node18,91678x6 patterns · 1 error · sigma 2155.8% of the extensions were dead

Three savings on one structure

A factor of eleven on an extension, a factor of seven on a branching search, and a sixth of the bits. Applied to one bidirectional index they do not give a factor of seventy-seven, and the reason is that two of the three are the same saving.

indexes · Distance
02040608000.50011.502errors permittedbranches that find nothing, per cent5.75x6.98x9.22xthe label is thesaving at that budget8 patterns · 8 characters31.3% to 74.9%

A looser budget wastes a larger share

More errors permitted means more work, and the fraction of that work which was never going to help rises with it — from thirty-one per cent at no errors to seventy-five at two. The saving is worth most where the search is most expensive.

practice · Symbol
a fixed-length codeleaves in orderacdefghilmnoprstuvwythe best ordered treeleaves in orderacdefghilmnoprstuvwythe best tree of any shapeleaves in frequency ordereahnrstdiloucfgmpvwythe symbols, in the order the descent reports them21 symbols in 2,048 positionsone set, 2 of 3 sorted

One set, three orders

The symbols an interval holds do not depend on the tree's shape. The order they come out in does, and a search that accumulates a running count as it reads them computes a plausible number that is wrong on eighty per cent of queries.

wrong · Symbol
acdefghijklmnopqrstuvwyzroot24 of 26 symbols present54 ranks · 51 nodes

A walk that does not prune

Remove the emptiness test and the descent visits every node of the tree, returns exactly the same symbols with exactly the same intervals, and costs sixty per cent more. No test of the answer can see it.

wrong · Symbol
factor against the published loopthe compound walk11xthe enumeration78xthe two, multiplied860xnothing measures thisan index with both78xthe shortfall is 11x, and the walk's factor is 11x6 patterns · 1 error · sigma 21the larger of the two, not the product

Two factors that do not multiply

Eleven times and seventy-eight times against the same baseline, so an index with both should be eight hundred and sixty. It is seventy-eight, and the shortfall is eleven — the first factor, exactly, because the second operation already contains it.

wrong · Distance
a loop over the alphabet107.9the compound walk10.0the cheapest hereone descent per step19.016 exact patterns of 12ranks per step · sigma 21

The saving that is a loss

An operation that is seventy-eight times cheaper on a branching search costs twice as much on an exact one. It reports every symbol present in order to hand back the one that was asked for, and a search that knows its character needs none of the rest.

practice · Distance

Named alongside it

The objects these essays reach for when they reach for this one.

Wavelet treeDescentRankBacktracking searchAlphabet sizeBidirectional indexCheckCompound walkDead branchError budgetCompositionIndex size

All concepts