iter 3 — de-randomised ξ lands, and the oracle gate stops flaking FIXED

Attack item R-2 · 2026-07-29 · harness harness/xi_derandomisation.py · library rotation_robust_eval.chatterjee_xi_derandomised

Two deliverables, one root cause

Iter 2 established that ξ is randomized under X-ties. That same randomness had already infected the gate landed in iter 1: re-running it moved L2_quadratic_x100's p-value 0.0060 → 0.0050 → 0.0030 against a 0.00167 failure threshold. A gate that can flake is not a gate. Both the estimator and its own oracle needed the same fix, so they are one item.

Prediction, written before running

Averaging R rearrangements gives MC-SE = sd/√R exactly; with sd = 0.00436 (iter 2), resolving the 0.0008 margin at 1 SE needs R ≈ 30, and 3× headroom needs R ≈ 270. Continuous kernels need R = 1. Seeding XICOR's draws makes the gate reproducible.

All four held. The R figures came out slightly higher (37 and 330) because this iteration's calibration draw landed at sd = 0.00484 rather than 0.00436 — same law, different draw.

1. The 1/√R law holds — verified, not assumed

The draws are not obviously independent: they share the same data and differ only in tie ordering. So the reduction was measured across 200 independent replications of the whole averaged estimator.

     R    sd(est)  predicted   ratio  mean mc_se  vs margin
     1   0.004840   0.004840   1.000         nan       6.05x
     2   0.003148   0.003422   0.920    0.002688       3.93x
     8   0.001555   0.001711   0.909    0.001539       1.94x
    32   0.000781   0.000856   0.913    0.000804       0.98x
   128   0.000407   0.000428   0.951    0.000406       0.51x

Ratios 0.899–0.985 — the law holds, and the observed sd falls very slightly faster than predicted. Note also that the self-reported mc_se tracks the true sd closely (R=32: 0.000804 reported vs 0.000781 actual). That is the property that makes this safe to use without a pre-registered threshold: a caller checks a margin against a measured quantity rather than trusting a point estimate.

2. The compute budget is derived, not chosen

   target SE                           meaning  R required
     0.00080 resolve the 0.0008 margin at 1 SE          37
     0.00040        2x headroom on that margin         147
     0.00027        3x headroom on that margin         330
     0.00060        one tenth of the PASS moat          66

The caller supplies the margin it must adjudicate; the arithmetic returns the budget. No dial is set anywhere — R is compute, and more of it is strictly better.

3. Free for the kernels that do not need it

tie exposure 0.000 · reps_used=1 (asked 128) · mc_se=0.0 · 1.0 ms  -> ok

chatterjee_xi_derandomised short-circuits when x carries no ties, because there the estimator is already exact. Continuous kernels pay nothing; the cost falls only on the lattice-valued kernels iter 2 identified as the exposed ones.

4. Adoption is cheap

R=   1       1.0 ms/cell      0.06 s for 62 cells
R=  32      27.9 ms/cell      1.73 s for 62 cells
R=  64      59.8 ms/cell      3.71 s for 62 cells
R= 128     112.6 ms/cell      6.98 s for 62 cells

Against a 41-minute R2 run, even R=128 across every cell costs 7 seconds. Cost is not an argument against adoption.

5. The oracle gate is now bit-reproducible

XICOR breaks X-ties with rank(ties.method="random") and exposes no seed argument — but R's set.seed() does control it, and XICOR's own manual says so:

"This version does not use a seed as argument, if reproducibility is an issue, set a seed before calling the function."
— CRAN XICOR 0.4.1 reference manual

Seeding per draw (distinct seeds, so the reference remains a genuine sample of the rearrangement distribution) makes the gate deterministic. Two consecutive runs:

md5: 7a6763205a5fd38302bfc9be2dded086  /  7a6763205a5fd38302bfc9be2dded086
BIT-IDENTICAL — gate is now reproducible

worst case  L2_quadratic_x100: p=0.00600  alpha_adj=0.00167  margin=3.6x

What deliberately did NOT happen

The cascade was not rewired. xi_max_pair and xi_cell_vote_stable still call the single-draw chatterjee_xi. Wiring the de-randomised estimator in would move every verdict — which belongs to the operator-gated re-baseline, not to an unattended loop. This iteration lands the capability and its evidence; adoption is a separate, supervised step, and it is queued rather than assumed.

No band, threshold, tolerance, percentile or sub-slice fraction was touched.

Carried forward

Evidence: harness/xi_derandomisation_evidence.json. Reproduce: VIRTUAL_ENV="" uv run --python 3.13 --no-project --with numpy --with pandas python3 harness/xi_derandomisation.py · gate: mise run xi:oracle