Attack item R-8 · 2026-07-30 · harness
harness/tau_hat_calibration.py · 1,000 null draws × 4 regimes × 2 lengths
Iteration 11 showed a sub-slice standardisation works, but that the constant is not universal — fixed 2/5 fails by 58% on binary-valued data — and concluded that any repair must estimate τ², with Chatterjee Thm 2.3's plug-in as the route. That was an argument from theory. "A consistent estimator exists" and "this implementation's numbers are calibrated on my data" are different claims, so this tests the second.
Thm 2.3 promises consistency, so the reported sd should match empirical within ~3% in every regime, and the p-value should be Uniform(0,1). Risk I named in advance: at binary Y the normal approximation might be poor, giving the right spread but the wrong tail — and the tail is what a threshold consumes. That risk did not materialise.
regime n empirical sd reported sd ratio verdict continuous 997 0.019733 0.020030 1.015 ok continuous 3990 0.010087 0.010013 0.993 ok tied31 997 0.019415 0.020030 1.032 ok tied31 3990 0.009471 0.010013 1.057 MISMATCH levels5 997 0.022395 0.021849 0.976 ok levels5 3990 0.010920 0.010915 1.000 ok binary 997 0.032165 0.031670 0.985 ok binary 3990 0.015634 0.015831 1.013 ok
7 of 8 within 5% — including both binary rows, which is the case that defeats the fixed constant. Checked against the no-ties value, the estimator is demonstrably responding to discreteness rather than passing it through:
reported sd / sqrt(2/5n): continuous 1.0000 tied31 1.0000
levels5 1.0908 binary 1.5811
1.5811 is exactly √(1/0.4) — the Bernoulli(½) case where τ²=1. The estimator recovers the
theoretical value. So the sign-valued-kernel exposure iteration 11 flagged (a shipped column,
bar_sign_markov_flux) is handled by τ̂²ₙ.
regime n mean p P(p<.10) P(p<.05) P(p<.01) continuous 3990 0.496 0.107 0.057 0.011 tied31 3990 0.482 0.091 0.045 0.013 levels5 3990 0.494 0.105 0.054 0.007 binary 3990 0.492 0.093 0.042 0.008
8 of 8 uniform. The normal approximation holds even at two levels.
tied31@3990 reports a sd 5.7% above empirical. First I checked my own harness:
the construction produces 2,757 distinct of 3,990 — a tie fraction of exactly 0.309, matching the
real LINKUSDT@100 rate it is meant to model. So the harness is faithful and the mismatch is real.
The mechanism is visible in the ratio table: at 30.9% ties the estimator returns exactly the no-ties constant (ratio 1.0000), while the true sd sits slightly below it. So τ̂²ₙ judges that tie structure to have negligible variance effect, and is marginally wrong in the conservative direction.
That direction matters: overstating the sd understates the test statistic, which inflates p-values. The calibration table confirms it — tied31 shows P(p<.05) of 0.040 and 0.045 against a nominal 0.05, i.e. slightly under-rejecting. The inference stays valid; it just loses a little power. With 1,000 draws the empirical sd itself carries ~2.2% relative standard error, so 5.7% is about 2.5 se — real but small, and both tied31 rows lean the same way, which is why I read it as a genuine small bias rather than noise.
The repair route iteration 11 identified is green. τ̂²ₙ correctly handles the regime where the fixed constant fails badly, produces uniform p-values throughout, and its one inaccuracy is small and conservative. A standardisation built on it would be sound — which is the missing piece that made "repairable, not deletable" actionable rather than merely arguable.
Nothing was changed. Landing that standardisation moves every sub-slice comparison and remains the operator's call.
Evidence: harness/tau_hat_calibration_evidence.json. Reproduce:
VIRTUAL_ENV="" uv run --python 3.13 --no-project --with numpy python3
harness/tau_hat_calibration.py