The rise the correction kept subtracting

The slope the slow counter already shows repaired the one thing a burst detector built from two decayed counters could not survive on its own. The detector keeps a fast counter with a half-life $H$ of 13.3 seconds and a slow one with sixteen times that, 213 seconds, and alarms when their difference $d$ exceeds 1.85 times a spread it has learned. On a background whose rate rises steadily, the slow counter lags further behind than the fast one, so $d$ sits at an offset of $\beta(R-1)H/\ln 2$, where $\beta$ is the rise’s slope, and a detector that learns its spread while sitting on that offset alarms on a large share of its readings. The repair read $\beta$ off the slow counter itself, as its change over one slow half-life, decayed that reading into an estimate with a half-life of one more slow half-life, and subtracted the predicted offset from $d$. On rising backgrounds at R = 16 it held the alarms to 5.5–7.8% of readings, against 11–63% for the held scale.

Its closing section named the weakness it had not tested. A rise ends. The estimate is a reading of the slow counter’s recent past, and for some time after the background levels off it still shows a slope, so the detector subtracts an offset that is no longer there and $d$ is pushed below zero. A burst arriving in that stretch starts from a deficit. The section predicted that the deficit would last about one slow half-life. In that time the corrected detector would miss bursts of 300 arrivals that the held scale and the positive clock catch, while bursts of 900 and more would be caught regardless. The number it wanted was the smallest burst the correction still catches after a rise ends.

The deficit is real and larger than predicted. The prediction pointed at the wrong bursts.

A background that rises and stops #

Every stream here is the earlier pages’ Poisson background at ten arrivals a second, rising by 20% or 40% of that rate every 1,000 seconds until 2,000 seconds in, and level from then on. Three detectors run on each stream, all at R = 16 with their thresholds at 1.85 spreads. The held scale learns its spread except while alarmed. The positive clock holds its scale while alarmed and learns again once $d$ has stayed positive longer than any burst could keep it. The corrected detector is the held scale with the slope correction, the estimate decayed over one slow half-life, the setting the earlier page recommended.

A burst is a given number of extra arrivals spread over one second, placed at a stated lag after the rise ends. A burst counts as caught when the detector alarms at any reading in the ten seconds from its start. A burst’s visible time is the seconds from its start to the first reading that no longer alarms. Each point is an average over 24 to 60 streams, and the same streams serve every design, so the comparisons between designs are paired.

The deficit, deeper and longer than one half-life #

After a 40% rise ends, the corrected difference falls to 1.46 spreads below zero within four minutes and stays more than half a spread below zero for over seventeen minutes, five slow half-lives. After a 20% rise the trough is 0.84 spreads, and the deficit lasts about as long. The prediction said one half-life. While the background rises, the corrected difference is centred on zero, as the earlier page found, −0.17 spreads at the moment the rise stops on the steeper rise and 0.06 on the shallower.

The length follows from what the estimate reads. When the background levels off, the slow counter does not; it is still 0.6 arrivals a second behind the new level after a 20% rise, and it closes that lag with its own half-life of 213 seconds. Its slope over the last half-life therefore stays positive for several half-lives, and the estimate, decayed over one more half-life, follows that reading with a further lag. Meanwhile the uncorrected difference falls as the slow counter catches up with the fast one, on the same timescale. Two quantities that decay at similar rates, one of them delayed by the estimate’s own window, give a difference that opens, peaks a minute or two after the slower one starts to fall, and closes only when both have reached zero. The trough is 59% to 68% of the rise’s original offset, and the time to close is set by the slow counter, not by the estimate.

Two slow things, and the one the prediction left out #

The arithmetic of the trough has three timescales in it, and the prediction counted one. The estimate’s window is one slow half-life, 213 seconds, and its decay another. The slow counter’s own catching up is a third, and it is the longest. The counter with no window in it showed that a counter fading by half every $S$ behaves like a window of $1.44S$ only on a steady stream; when the rate changes, it closes the gap to the new rate exponentially, a factor of two every $S$. After a 20% rise the gap is 0.61 arrivals a second when the rise stops. Halving every 213 seconds, it is still 0.28 four minutes later and 0.13 after eight. The slope the estimate reads is the rate at which that gap closes, positive for as long as there is a gap.

So the estimate is not stale in the sense the prediction meant, a reading of the old rise that has not yet been forgotten. It is a correct reading of the slow counter’s present slope, and the slow counter’s present slope is its own lag, not the background’s rise. The count that outlives its arrivals met a counter reporting a key long after the key stopped arriving; here an estimate reports a rise long after the rise stopped, for the same kind of reason: the number it reads has its own memory, and nothing in it marks when the thing it measured ended.

The subtraction then does what the error of a difference found any difference of estimates doing. While the rise lasts, both the uncorrected difference and the predicted offset are large and close, and their difference is small with a small error. After the stop both shrink towards zero at similar rates, but one of them lags, and the error of their difference is a fixed share of the quantity each had been estimating. It does not shrink with the quantity the detector now cares about, which is zero.

A quiet detector on the level stretch #

On the level stretch the corrected detector alarms on 2.2% of readings after a 20% rise and 1.2% after a 40% rise, the fewest of the three designs; the held scale alarms on 11.1% and 20.5%. A deficit is a detector biased away from alarming. On a quiet background, where every alarm is a false one, that is an improvement. The held scale is the opposite case. It never removed the rise’s offset from its difference and learned its spread around a value that was already high, so the offset that decays after the stop keeps carrying its difference over the threshold.

The positive clock sits between them for a reason of its own. During the rise its schedule saw $d$ stay positive past the burst ceiling and learned again, widening its spread to hold the offset. After the stop that wide spread puts its threshold far above a level background’s noise, and it alarms on 2.0% to 4.3% of readings until it relearns a narrower one. The clock a burst cannot outlast built that clock to stop alarming on slow drift, and the plate shows that it does. It is the same widened scale that will cost it bursts below.

Small bursts, missed in the trough #

A burst of 20 arrivals, caught 68% of the time by the corrected detector while a 20% rise is under way, is caught 37% to 43% of the time from 200 to 600 seconds after the rise ends. The positive clock catches 58% to 62% of the same bursts in that stretch, and the held scale 70% to 85%. The corrected detector is the weakest of the three for small bursts during the rise too, by a smaller margin. Its threshold is set by a spread learned around zero, and the other two have offsets that lift a small burst over their thresholds, offsets they pay for in false alarms.

Twenty arrivals in one second at R = 16 raise the fast counter by about one arrival a second, two spreads of $d$, against a threshold of 1.85. A burst that size sits on the edge of detection on a level background, and a deficit of half a spread to a spread takes it over the edge on the wrong side. That is what the trough costs: not the bursts the prediction named, but the ones near the threshold.

After the steeper rise, bursts of 30 arrivals are caught 77% of the time by the corrected detector during the rise and as few as 27% of the time 600 seconds after it ends. The deficit here is deeper, so a larger burst is lost in it, and for longer: the corrected detector is below 50% from 200 to 600 seconds. The positive clock does better in the trough, 43% to 55%, and worse during the rise, 63% against 77%, because its widened scale is a cost it carries the whole time.

The corrected detector catches 92% of bursts arriving exactly at the end, more than before it. A burst at that moment arrives before the deficit has opened, and the correction is still nearly exact. The trough is not at the stop. It begins a minute or two after, at the point where the slow counter’s slope and the estimate’s reading of it part company.

So the smallest burst the correction still catches, the number the earlier page wanted, depends on when it arrives. On a level background with no rise at all, each of the three designs catches a burst of fifteen arrivals 54–55% of the time and one of twenty 71–73% of the time. During a rise the correction keeps close to that, 68% for twenty. In the trough after a 20% rise the half-detection size grows to between 20 and 30 arrivals, and after a 40% rise to over 30. A burst of 300 is far above any of these.

Large bursts, never missed, seen for less time #

Every burst of 300 arrivals is caught by every design at every lag, on all sixty streams. What the deficit costs it is visible time: 32.5 seconds at 600 seconds after a 40% rise ends, against 40.2 seconds during the rise and 41.8 on a level background. That is a fifth of the burst’s visible life. A burst of 300 raises the fast counter by about fifteen arrivals a second, over thirty spreads, so it is caught at once whatever the deficit. Its excursion then decays with the fast counter’s half-life of 13.3 seconds, and the detector stops seeing it when it falls back under the threshold. A deficit of one spread moves that crossing earlier by about $H \log_2 \frac{2.85}{1.85}$, eight seconds.

The positive clock is worse here than the correction. At the moment the rise ends a burst of 300 stays visible for 24.9 seconds under the positive clock, 15 seconds less than under the correction. The positive clock’s widened scale lifts its threshold for every burst, and on a background that had risen 40% that costs more than the correction’s trough. It recovers as it relearns, reaching 36.4 seconds 800 seconds after the stop. The held scale sees bursts for longest, 62 seconds during the rise, because its threshold sits on top of an offset it never removed, and the plate on the level stretch above shows what it pays for that.

A detector that learns its own quiet measured how long a burst stays detected as the half-life at which half the streams still alarm. The visible time here is the mean over streams, not the lag at which half still alarm, and it differs from that measure by a few seconds: 41.8 seconds on a level background against the 39.6 the earlier page measured for the same burst. The comparisons between designs use the same measure throughout.

What the prediction got right and wrong #

The prediction got the mechanism right. The estimate goes on reading a slope after the slope has gone, and the detector starts each burst from a deficit. It was wrong about the length and about what the deficit costs, and both errors come from treating the estimate as the only slow thing in the detector. The slow counter is slow too. After the rise it keeps climbing towards the new level for several of its half-lives, and an estimate that reads its slope faithfully reads that climb. The deficit lasts as long as the slow counter takes to settle.

It was wrong about the bursts because it judged them by size, not by margin. A burst of 300 arrivals is caught by a detector that is a spread or two off, since it arrives thirty spreads above zero. What a deficit of a spread can take away is a burst that arrives two spreads above zero, and those are bursts of twenty to thirty arrivals at R = 16. A detector’s weaknesses show first on the bursts nearest its threshold, and a test with a large burst hides them. The clock that cannot see the burst met an instrument that could not represent a burst at all; this one represents every burst and loses only the ones a spread or two above its noise.

So the correction is safe to deploy in the sense the earlier page meant. Every burst of the sizes it was tested on is still caught after a rise ends, and the level stretch after a rise is quieter under the correction than under either clock. It is not safe for small bursts in the minutes after a rise ends. A system whose bursts of interest are near the threshold, say twenty or thirty arrivals over a background of ten a second, would lose half of them for a quarter of an hour after every rise that stops.

What the measurement leaves out #

Rises that stop at once. Every rise here ends abruptly, a ramp turning into a level line. A rise that slows gradually would give the slow counter less lag to make up at the end and the estimate a slope that falls with it. The trough would be shallower, and nothing here measures by how much.

One ratio, one threshold. Everything is at R = 16 and 1.85 spreads. The slope the slow counter already shows found the correction’s error multiplied fourfold at R = 64, and the trough there would be correspondingly deeper.

Bursts of one second. A burst spread over a minute meets the trough differently, since the slow counter takes in part of it and the estimate reads that as slope.

Visible time as a mean. The visible time is a mean over 40 streams, and the plates show it rather than a distribution. The comparisons between designs are paired on the same streams, which is why a difference of a few seconds is readable.

Still open: a correction that checks its estimate against the offset it can see #

The detector holds two readings of the same quantity. The estimate predicts the offset the rise should put into $d$, and $d$’s own recent mean is that offset, measured. During a steady rise they agree, since that agreement is the earlier page’s result. After a rise ends they part. The measured offset falls as the slow counter catches up, and the prediction keeps reading a slope that is only the counter’s own catching up. The correction could subtract the smaller of the two: the estimate’s predicted offset, capped by a decayed mean of the uncorrected $d$ over, say, one slow half-life.

The measurement that follows gives the correction that cap and repeats every plate here, with the rise of the earlier page’s streams as a check that it costs nothing while the background is still rising. The prediction is that the trough falls from 1.46 spreads to under half a spread after a 40% rise, and that bursts of 30 arrivals in the trough are caught at least as often as by the positive clock. It could fail if the decayed mean of $d$ is itself pulled up by bursts. A burst would then raise the cap and the subtraction with it for a slow half-life after the burst, so the cap would give back, burst by burst, the visible time the trough took.