Attack item X-2 ยท 2026-07-29 ยท harness
harness/future_perturbation_invariance.py ยท draft remedy
PRE-REGISTRATION-DRAFT-regime-tercile-lookahead.md
The operator's standing rule is perturb test rows, train-side discovery must be unchanged โ phrased for a predictive walk-forward. These probes are redundancy screens with no train/test split, so the rule does not transfer verbatim, and iteration 7 flagged that waving the analogy through would produce a gate that looks like compliance and tests nothing. The invariant that does transfer:
Perturbing FUTURE rows must not change any label or reading assigned to PAST rows.
regime_labels() computes volatility with a trailing window โ correct โ then
buckets it against globally computed terciles:
v = vol[idx] q1, q2 = np.nanquantile(finite, [1/3, 2/3]) # over the WHOLE cell vol_lab = where(v <= q1, "low", where(v <= q2, "mid", "high"))
A bar's volatility regime is assigned relative to volatility that had not happened yet.
I predicted vol_lab would relabel 5โ25% of past rows while trend and session stayed at 0%. The direction and the discrimination were right. The magnitude was wrong in both directions, because it turns out to depend entirely on the scenario โ which is itself the finding.
| Scenario | % of PAST value rows relabelled |
|---|---|
| Operational โ append 5% more bars (i.e. a re-run) | 2.25% |
| Operational โ append 10% | 2.47% |
| Operational โ append 25% | 5.24% |
| Operational โ append 50% | 5.24% |
| Stress โ future half replaced by a 2.5ร volatility regime | 65.74% (max 81.43%) |
Controls, which are what make this a measurement rather than an alarm:
sess_lab 0.00% invariant trend_lab 0.00% invariant xi sub-slice reading on untouched rows: bit-identical
The test discriminates. It is not flagging everything โ it isolates the one axis that consults a global statistic.
The 65.74% is a stress bound and should not be quoted as the operational number. The operational figure is 2โ5%: every re-run relabels a few percent of historical value rows simply because time passed. That is smaller, but it is structural and unavoidable under the current rule.
This is not a prediction leak โ nothing is being forecast. The harm is irreproducibility of the stratification, and it lands on the campaign's own bookkeeping:
No verdict in the existing record is invalidated by this alone. Within a single run the stratification is internally consistent; the defect is about comparability between runs.
Four remedy options are drafted โ expanding-window terciles, fixed absolute thresholds, frozen per-cell terciles from a declared reference window, or accept-and-document. Each carries a decision: the first three change every historical stratum assignment and therefore invalidate all accumulated coverage. That is the operator's call, not an unattended loop's.
The harness is runnable as /xi:future-perturbation but is deliberately not added
to /xi:check-full. Wiring a currently-failing gate into the suite would turn the whole
suite red and force the decision by making CI unusable โ which is coercion, not reporting.
vol_lab was traced to its source. Other global statistics may exist elsewhere in
the pipeline; this iteration establishes one leak, not the absence of others.Evidence: harness/future_perturbation_invariance_evidence.json.
Reproduce: mise run xi:future-perturbation