Concept

Quantile — where it appears

The value below which a stated fraction of the data lies, and a question whose error can be measured on the rank or on the value. Its error can be measured on the rank or on the value, and the two are different questions with different answers on the same run.

Named by 3 essays across 2 fields — each of them below, with the objects they name alongside it.

quantilepromisedone summarymerge of 8q = 0.5100.01418q = 0.920.0183q = 0.992.005overq = 0.9990.214overrank error, in items out of 20,000ε = 0.01 · high-biased · 8 shards, round2 of 4 quantiles over the single summary's promise

The promise that does not survive the tree

A high-biased quantile summary at ε = 1% answers the 99th percentile of twenty thousand values to within two ranks. Merge thirty-two of them and the answer is seventeen ranks out — inside the merged bound, eight times outside the one anybody was quoted.

wrong · Rank
0.1%1%10%100%0.50.750.90.990.999quantile asked forrank error ÷ (1 − q)Greenwald–Khanna7,080 bitshigh-biased56,064 bitst-digest5,952 bits8 streams per point · ε = 0.01, δ = 100denominator: the tail

An error measured against the answer

A quantile summary asked for the 99.9th percentile answered 9,694 where the truth was 256, and violated nothing — its promise was a rank error under one per cent of the stream and it delivered a tenth of one per cent. One per cent of the stream is a thousand per cent of the tail, and no amount of extra state changes that.

streaming · Rank
Greenwald–Khanna7,008 bits802high-biased55,008 bits802t-digest6,144 bits675the truth, 802ε = 0.01, δ = 100 · 40,000 valuesq = 0.52

The digest that promises nothing

The t-digest is the quantile structure most widely deployed and the only one with no proven bound on its rank error at any quantile. Measured, it beats the structure that does have one — and on two clusters with a gap between them it returns 431.5, where the data holds nothing at all between 40 and 800.

wrong · Rank

Named alongside it

The objects these essays reach for when they reach for this one.

GuaranteeRank errorTailGreenwald–KhannaHeavy tailMeasurementState bitsSummaryCounterexampleEpsilonEstimatorFalsification

All concepts