Concept

False alarm — where it appears

An alarm raised when nothing happened: the detector's threshold was crossed by the ordinary variation of what it watches. A detector is specified by the rate at which that happens on a quiet stream, and the threshold is whatever achieves that rate, however it is estimated.

Named by 6 essays across one field — each of them below, with the objects they name alongside it.

010200100200seconds since the burstestimated rate, arrivals a secondthe burstexponentialexact polynomialthree exponentialsrestarted forwardfaint: the same streamswith the burst removedmatched at 5.0% spread · burst of 300 in one secondthe same event, four descents

What a heavier tail actually buys

Four decays, each tuned until it is equally quiet on a stream with nothing happening, then given one burst. At a burst of three hundred they see it for 54, 63, 68 and 104 seconds — indistinguishable. At nine thousand the exact polynomial reaches 384 seconds and the exponential 156. The shape of a tail does not decide how far back a scheme can see. It decides how fast that distance grows with the size of the event, and the exchange rate is the reciprocal of the exponent.

streaming · Decay
3009003,0009,00010100arrivals in the bursthalf-detection horizon, sthe oracle's thresholdscale held, R = 64scale held, R = 16scale learns, R = 64scale learns, R = 16ten arrivals a second, sixty streamsdashed: R = 16

A detector that learns its own quiet

A burst detector built from a fast decayed counter needs a threshold, and the one it was measured with came from sixty independent runs of the background — an oracle no deployed system has. Two counters and a running mean square of their difference can supply it from the stream itself, and on a stream that does not move they hit a 5% false-alarm rate at every setting tried. What they lose is the burst: a scale that learns every reading learns the burst as noise and forgets it in 20 to 40 seconds whatever its size. Holding the scale while alarmed brings the oracle's horizon back for small bursts, turns a slow drift into an alarm that will not stop, and still cannot see past the lag at which the slow counter holds more of the burst than the fast one.

streaming · Decay
5%10%20%40%rise in the background rate every 1,000 sreadings that alarm0%5%20%40%60%scale heldalarm clockpositive clockrebase on the positive clockscale learnsR = 16, 24 streams a pointdashed: 5%

The clock a burst cannot outlast

A burst detector that holds its scale while alarmed keeps a burst visible to a ceiling set by two half-lives, and cannot stop alarming on a slow drift. The repair proposed for it was a schedule: learn again once an excursion has outlasted anything a burst could produce. Timed by how long the alarm has lasted, the schedule changes almost nothing, because on a gentle rise the alarm breaks every few seconds and the clock never reaches the ceiling. Timed by how long the difference between the counters has stayed positive, it keeps every burst to within about a second of the ceiling and falls below the target false-alarm rate on steep drifts — and one quantity, the drift's offset measured in spreads of the difference, says where it still fails.

streaming · Decay
0.310.621.242.484.859.7the rise's offset in d, in spreads of dreadings that alarm0%5%25%50%75%100%held while alarmedthe positive clockcorrected for the slopesolid: R = 16; dashed: R = 64dashed rule: a still stream's 5%

The slope the slow counter already shows

A burst detector built from a fast and a slow decayed counter reads a steady rise in its background as an offset in their difference, β(R − 1)H/ln 2, and alarms on it. Every repair so far waited the offset out. This one subtracts it: the slow counter's own change over one slow half-life is a reading of the slope, and the offset follows from it. At R = 16 the corrected detector alarms on 5.5% to 7.8% of readings on rises that make the held scale alarm on 11% to 63%, and it costs a burst at most 3.7 seconds of detection. The estimate has to be quick: four times slower, it has recovered only 71% of the offset when alarms start to count.

streaming · Decay
-2-1012305001e+3seconds from the end of the risemean difference, in spreads of dthe rise endsuncorrected, 20% risecorrected, 20% riseuncorrected, 40% risecorrected, 40% risemean of 24 streams, R = 16zero: the background's own level

The rise the correction kept subtracting

A burst detector that subtracts a rising background's offset, read off its own slow counter, goes on subtracting it after the rise has stopped. The prediction was a deficit for about one slow half-life that would cost bursts of 300 arrivals. Measured, the deficit is deeper and longer: down to 1.46 spreads below zero, below half a spread for seventeen minutes after a 40% rise. Bursts of 300 are never missed. What the deficit costs is small bursts — 30 arrivals caught 27% of the time where the rise saw 77% — and a fifth of a large burst's visible time.

streaming · Decay
-2-101-505101520minutes from the end of the risemean corrected difference, in spreads of dthe rise endscorrected, no capcapped, mean over 213 scapped, mean over 53 smean of 24 streams, 40% risezero: the background's own level

A check as slow as what it checks

A burst detector that subtracts a rising background's offset goes on subtracting it after the rise ends, and loses small bursts in the trough that opens. Capping the subtraction by what the difference itself shows, averaged over one slow half-life, was predicted to close the trough to under half a spread. It lifts it from 1.46 spreads to 1.24, because that average is as late as the estimate it checks. Averaged over a quarter of the time it lifts it to 0.70 and catches small bursts in the trough more often than any other design but the one that alarms on a fifth of its readings. Neither cap helps after a large burst, which every slope correction reads as a rise.

streaming · Decay

Named alongside it

The objects these essays reach for when they reach for this one.

Design parameterEstimatorExponential decayHalf lifeStreaming modelHonest limitMeasurement designVariancePredictionState bitsFittingForward decay

All concepts