A margin the count sets itself

A landmark search on a maze of 2,500 weighted cells keeps four landmarks at the corners, which cost nothing to maintain, or a watch that refits landmarks near where recent queries end, which costs its looking and pays only when the destination stays put long enough. A switch that counts the jumps chooses between them by the traffic’s pace, read as the mean spacing of the last $k$ stretch ends — the moments the destination has moved ten cells from where its current stretch began — and keeps the corners while that reads under 1,600 queries. A silence that overrules the count let an open spacing of 1,600 queries since the last stretch end turn the watch back on whatever the count said.

A gap between off and on separated the two directions. Switching the watch on was the silence’s job, so switching it off could use a lower threshold, and on three stretch ends an off threshold of 1,200 saved 3.5 cells a query at one jump of the destination in 2,500 queries: 117.8 against the override’s 121.3. It cost 1.7 at one jump in 1,000, where the corners are the right placement and the gap delayed the return to them. On five stretch ends no gap helped anywhere, because a mean of five spacings is seldom fooled by a bunch of them, and a margin fitted to the noise of three was too wide for the noise of five.

That essay’s closing section proposed letting the count set its own margin. The standard error of a mean of $k$ spacings is their spread over $\sqrt{k}$, which the switch can compute from the spacings it already holds; it could switch the watch off only when its reading plus $Z$ standard errors is under 1,600. A reading of 1,100 from three wildly different spacings would then not switch the watch off, and the same reading from five similar ones would. The prediction was that at $Z = 1$ on three stretch ends the switch would match the fixed gap’s 117.8 at the moderate pace and lose less than a cell at the fast one, because fast traffic’s spacings are short and similar, so its standard error is small. It named the failure: jumps every thousand queries may produce stretch ends bunched as well as fast, with a spread as large as their mean.

A margin read from the spacings #

The mazes, streams and charges are the earlier essays’. At each window the switch reads the count: the mean of the last $k$ spacings between stretch ends, or the silence once that alone reaches 1,600. While it keeps the corners, it stays with them until the reading reaches 1,600, as before. While it watches, it returns to the corners only when there is no silence of 1,600 and the mean plus $Z$ times the spacings’ standard deviation over $\sqrt{k}$ is under 1,600. Margins of a half, one and two standard errors were run on three, five and ten stretch ends, beside the override with no margin and the fixed gap of 1,200 on three, on four steady paces and the two streams that change pace.

The same choices as the fixed gap #

Every way of setting the off threshold trades the moderate pace against the fast one: the override on three stretch ends costs 121.3 cells a query at one jump in 2,500 and 142.6 at one in 1,000; one standard error on three ends costs 117.8 and 144.5, exactly the fixed gap of 1,200 at the moderate pace and 0.2 more at the fast; and the override on five ends, 118.2 and 141.7, is the one point no other beats on both. The plate is a trade-off curve with almost no slack in it. Moving down the moderate axis by a few cells costs a cell or two on the fast axis, whatever produces the move — a fixed gap, a margin of standard errors, or a longer count. The points that are beaten on both counts are the long counts with wide margins, ten stretch ends at one or two standard errors, which are slow at the fast pace and gain nothing more at the moderate one.

At the moderate pace the prediction holds: one standard error on three stretch ends costs 117.8, the fixed gap’s figure to the tenth. At the fast pace it fails: 144.5 against the override’s 142.6, a loss of 1.9 cells where under one was predicted, and no better than the fixed gap’s 144.3.

Fast spacings are not similar #

At one jump in 1,000 the margin costs on every count: one standard error on three stretch ends costs 144.5 cells a query against the override’s 142.6, 1.9 more where under a cell was predicted, and on ten ends two standard errors cost 148.1, near the watch’s 150.7 — fast spacings are not similar enough for a margin to clear quickly. The proposal’s reasoning needed fast traffic’s spacings to be short and close together, so that their standard error would be small and the margin would let the switch return to the corners almost as soon as the override did. Fast traffic here is jumps every thousand queries plus the stretch ends that walking queries add between them, and the spacings that result are bunched: some of a few hundred queries, some of nearly two thousand. A mean of 1,100 from such spacings would carry a standard error of several hundred, and on three of them the margin keeps the switch watching about as long as a fixed gap at 1,200 does.

On five stretch ends the margin’s cost grows with $Z$, from 1.7 cells at a half to 4.6 at two, as the corners’ share falls from 47% to 28%. On ten ends the override alone already loses 3.4 cells to the corners at this pace, since ten fast spacings take a long time to accumulate after every slow stretch, and a margin adds to that.

Where the margin does its work #

At one jump in 2,500 the margin does what it was built to: on three stretch ends the corners’ share falls from 15% with none to 7% at two standard errors, on ten from 5% to 2% — and the cost moves by under a cell on either, 121.3 to 117.6 and 117.7 to 117.9, except on three ends between no margin and a half. The whole of the margin’s value at the moderate pace is in its first half standard error on three stretch ends: 3.5 cells, the same as the fixed gap’s. After that, keeping the corners less often saves nothing, because the spells it removes are no longer the costly ones. Two standard errors bring three stretch ends to the 7% at the corners the proposal of the earlier essay hoped for, and to 117.6 cells a query, 1.2 above the watch’s 116.4, which is as close as any switch has come.

A count of ten stretch ends reaches the same place without any margin, 117.7 cells a query with the corners kept 5% of the time, because a mean of ten spacings, if they are about as spread as their mean, already has a standard error near a third of the spacing. That is the earlier essay’s finding seen from the other side: the margin and the count’s length are two ways of buying the same certainty, and on this traffic they buy it at the same price.

Streams that change pace #

On the streams that change pace three stretch ends with any margin cost within half a cell of each other — slowing 124.3 at one standard error, speeding 119.3 — while ten stretch ends cost 126.7 and 123.1: a long count is slow to see the fast half of either stream, and no margin makes it faster. On the stream that slows, the slowing is heard by the silence, which no margin touches, and every switch on three ends costs 124.3 to 124.5. On the stream that speeds up, the fast second half is the fast pace again, and the switches that are slow to switch off there pay for it: one standard error on ten stretch ends costs 123.1, close to the watch’s 123.8, against three ends’ 118.9 to 119.3.

So a long count, which was as good as a margin at the moderate steady pace, is worse on traffic that changes, because its history is longer. A margin on a short count keeps the short count’s quickness and buys certainty only when the spacings disagree.

Six streams, one number #

Averaged over the six streams, every switch on three or five stretch ends lies within 1.4 cells a query of the best, 113.78, which is two standard errors on three stretch ends; on three ends the margin moves the mean by about half a cell at most, and ten ends cost 1.4 more at their best. The first four places are within an eighth of a cell of each other, and they are four different mechanisms: a margin of two standard errors, a margin of a half, a fixed gap, and no gap at all on a longer count. Over a mix of paces like this one, the off threshold’s construction decides almost nothing. What decides more is the count’s length: ten stretch ends lose more than a cell on average, whatever their margin, because of the streams that change pace.

The mean is over a mix chosen for the earlier essays, two streams that change pace and four steady paces from fast to slow, and a different mix would weigh the trade differently. A store that served mostly moderate traffic would want the margin; one that served mostly fast traffic would want the override on five; neither would gain or lose more than a few cells a query by the choice.

What a misreading costs, and why it is the same everywhere #

The trade on the first plate has a reason that does not depend on how the threshold is set. A switch makes two kinds of mistake. It switches the watch off on traffic that is not fast, which at the moderate pace means throwing away landmarks fitted to the current destination and paying the corners’ cost, about ten cells a query more than the watch, until the silence or a slower count brings the watch back. Or it keeps watching on traffic that is fast, which at the fast pace means paying the watch’s looks and refits, about eight cells a query more than the corners, until the count catches up. Every rule here — a gap, a margin, a longer count — reduces the first mistake by demanding stronger evidence of fast traffic, and the same demand makes the second mistake last longer.

So the rules differ only in how much evidence they demand and when, and on this traffic the evidence of fast traffic arrives at the same rate whatever reads it: one stretch end at a time, every thousand queries or so at the fast pace. A rule that waits for one more stretch end before switching off pays one more thousand queries of watching at the fast pace, roughly a cell a query over a stream with a few dozen fast stretches; and it avoids, at the moderate pace, the misreadings that one more stretch end would have corrected. The two costs are of the same size because they are counted in the same unit, stretch ends waited for, and that is why so many different rules sit on one curve.

Starting from the corners established why the corners are a safe default: they cost nothing to keep and are never far from the best placement on traffic that moves. A watch that looks less showed that the watch’s looking is what makes it dear on fast traffic. Between them, the switch’s job is to tell which of two known costs it is paying, and the plates say that a count of stretch ends can do that job to within about a cell a query on average, no better, however its threshold is shaped. The candidates the watch chooses among, the candidates near where the queries end, and the horizon it saves over, a horizon the stream can supply, are where the remaining cells are, not in the switch.

What the count can and cannot tell #

The closing question was whether the evidence a count needs before it acts can be read from the count, or whether the right margin is a fact about the traffic that only the costs reveal. The count’s own standard error is the right instrument in one respect: it tells a three-end count when its reading is weak, and on a moderate pace that is exactly when the count misreads, so a margin of one or two standard errors stops the misreadings. It is the wrong instrument at the fast pace, where the spacings are bunched and a correct reading of fast traffic has a large standard error too. There the margin delays the right switch as much as it delayed the wrong one, because the count’s uncertainty does not distinguish noise from bunching.

A watch that switches itself off priced the corners against the watch on the same sample at every look, and was the switch the earlier essays started from because counting is free and pricing is not. The plates here say that counting has reached what it can do on this traffic: three different readings of the count, at their best, cost the same to an eighth of a cell, and each loses on one pace what it gains on another.

What the plates assume #

Four maps and one mix of streams. Every cost is a mean over four mazes; differences under half a cell between switches are not separable from the maps. The six-stream mean is one mix.

A standard error that assumes independence. The margin treats the $k$ spacings as independent draws. Bunched stretch ends are not independent, and a margin computed with their correlation would be wider at the fast pace, which is the wrong direction for the fast pace’s cost.

One on threshold. The silence and the switching on stayed at 1,600 throughout.

Still open: the count read against the costs it saves #

Every switch measured so far has read the traffic and none has read the money. The count is free because it reads the destinations of queries already served; the watch’s looks are not free, but the watch already prices the corners against its own placement on each look’s sample, which a watch that switches itself off used and abandoned because one look’s price was too noisy. A switch could combine them: use the count, with a margin of half a standard error, to propose switching off, and require the watch’s last look to have priced the corners within a few cells of its own placement before accepting — so the count decides when to ask and the price decides the answer.

The measurement that follows runs that switch on three and five stretch ends, with the price’s tolerance swept from nought to ten cells a query, on every stream here, and charges the looks as before. The prediction is that at a tolerance of about three cells it keeps the moderate pace’s 117.8 and recovers the fast pace’s override figure, 142.6, because at the fast pace the price confirms the count’s proposal and at the moderate pace it refuses the misreadings the margin was refusing. It could fail on the price’s own noise: one look prices one sample of a thousand queries, and at the moderate pace the corners and the watch may price within three cells of each other often enough that the price confirms the count’s mistakes too. The question is whether two noisy readings of the same choice, one of the traffic and one of its cost, together know what neither knows alone.