Navigation

← Audits

Batch-6 Chatterjee ξ filter

SETTLED2026-07-02

Ran the newly declared redundancy test on the eight candidate features that had already cleared an earlier screen — four came out clearly independent and four landed in the grey zone — and along the way showed that the obvious way of computing this test on rolling-window features produces false answers.

Why it carries this status

Lifecycle, not result. This says where the audit sits in its process — never whether what it found was good.

**Result (of record — PR #564 §B worst-cell bands): 4 PASS · 4 WATCH · 0 BAN.**
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/CLAUDE.md

2026-08-17. Derived from folder evidence, then adversarially challenged; the challenge pass was upheld.

The verdict

The audit's own conclusion, reproduced in full from the source below. Not a summary — this is the document, rendered. Links inside it that point at unpublished files are shown as plain text rather than as links that would 404 here.

verdict.md — Chatterjee-ξ filter, batch-6 candidates (2026-07-02, amended same day)

Question: of the 8 batch-6 candidates that PASS the Spearman worst-regime/worst-cell gate, which survive the project-declared Chatterjee threshold?

> ⚠ AMENDED 2026-07-02. The original verdict (Part 2 below) applied the legacy single > cut XI_FLAG = 0.80 — at the time the only ξ threshold in the constants registry. Later > the same day, PR #564 merged the operator-declared ξ bands > (findings/evolution/audits/2026-06-30-chatterjee-xi-threshold-calibration/CHATTERJEE-THRESHOLD-DECLARATION.md), > which (a) bind the promotion gate to the §B worst-cell statistic, (b) BAN the > median rule as a promotion axis (spoke 08), and (c) supersede any single-cut read. > Part 1 is the verdict of record. Part 2 is retained as the historical calibration > read (same measurements, superseded bands).


Part 1 — §B verdict (CURRENT · PR #564 worst-cell bands)

Declared rule applied (PR #564 §B):

1. BAN?    xi_worst > 0.95  AND  breadth ≥ 0.80   → BAN
2. PASS?   xi_worst < 0.50                        → PASS
3. else                                           → WATCH

xi_worst = max ξ over all floor-met slices; breadth = share of slices with ξ ≥ 0.50. Measurement conventions unchanged from Part 2: xi_max both directions (xicorpy, codec_foci_eval), FLOOR_1000, 750-dbps tier excluded (spearman_only), raw h_norm ≥ 0.05 degeneracy gate, per-cell quantile ranks, non-overlapping stride-200 windows (CONTROL_FINDINGS.md), 97,811 pooled windows / 336 cells.

Substrate note (pre-documented in xi_probe_batch6.py header): at the declared stride-200 windows a single grid cell cannot clear FLOOR_1000, so the finest floor-valid slice on this substrate is the regime pool (7 regimes) — xi_worst is therefore read as the worst floor-met regime, and breadth as the share of the 7 regimes with ξ ≥ 0.50. §B's cell-slice axis maps to regime-slices here by the declared floor itself, not by choice.

Result: 4 PASS · 4 WATCH · 0 BAN

CardCandidatexi_worstworst sliceworst competitorbreadth (≥0.50)§B level
78hoeffding_phi_squared_midreturn_duration0.078bear_2022bar_sign_markov_flux0/7PASS
60vg_time_directed_clustering_kld0.171bear_2022bar_cox_stuart_trend_z0/7PASS
59hvg_forward_visibility_horizon_mean_bar0.175bear_2022bar_cox_stuart_trend_z0/7PASS
50rogers_satchell_var_bar0.312bear_2022bar_sign_markov_flux0/7PASS
58hvg_mean_degree_excess0.609bull_2021close1/7WATCH
45heikin_ashi_open_gap_sign0.659current_2026bar_cox_stuart_trend_z1/7WATCH
108sequitur_grammar_compression_ratio_logret_sign0.710bull_2021high4/7WATCH
107etc_effort_to_compress_logret_sign0.771bull_2021high6/7WATCH
  • PASS (4): 78, 60, 59, 50 — all under the 0.50 cluster boundary (certified-orthogonal ceiling 0.494), decided by check 2 alone.
  • WATCH (4): 58, 45, 108, 107 — in the §B residual band. Per the declaration: *document
  • re-probe before shipping; do NOT auto-ban.* Note 107's breadth (6/7 ≈ 0.86) clears the BAN breadth dial, but BAN requires both dials and 0.771 is far below the 0.95 / ≥0.9995 duplicate cluster.
  • BAN (0): nothing approaches xi_worst > 0.95.

Null-cohort context (unchanged data, §B read): the shipped, dedup-proven batch-5 columns score up to 0.809 worst-slice through the identical pipeline — i.e. the four WATCH candidates sit inside the range shipped ground truth itself occupies. That supports "WATCH = operator judgment," not "WATCH = redundant."

Promotion consequence

The batch-6 promotable-now set is the 4 PASS — cards 78, 59, 60, 50 — each still carrying its readiness blocker (78 B-perf · 59 no-FOSS-oracle · 60 _kld_meangap rename · 50 window-mean reduction ratification). The 4 ξ-WATCH — cards 58, 45, 108, 107 — are held: per §B WATCH semantics they may only ship after documentation

  • re-probe and an explicit operator ruling. The PR stack in

IMPLEMENTATION_PLAYBOOK.md §6 (which led with 108/107) must be re-sequenced before the loop fires.

Machine-readable §B scoring: xi_bands_sectionB.json.


Part 2 — Original verdict (HISTORICAL · legacy XI_FLAG = 0.80 single cut, superseded by PR #564)

Declared rule applied: XI_FLAG = 0.80 (constants_registry.toml, "ξ≥0.80 ≈ functional duplication"), xi_max = max(ξ(a→b), ξ(b→a)) via xicorpy (codec_foci_eval convention), FLOOR_1000 convergence floor, 750-dbps tier excluded (spearman_only), h_norm ≥ 0.05 degeneracy gate on raw values, per-cell quantile ranks (dedup_grid pooling), non-overlapping 200-bar windows (see CONTROL_FINDINGS.md for why), pooled 97,811 windows over 336 cells; verdict = REDUNDANT if pooled or any floor-met regime ≥ 0.80.

Result (superseded): ALL 8 CANDIDATES PASS the 0.80 flag — none filtered

Candidatepooled ξ_maxworst-regime ξ_maxworst regimeworst competitorVerdict
hoeffding_phi_squared_midreturn_duration0.0520.078bear_2022bar_sign_markov_fluxORTHOGONAL
vg_time_directed_clustering_kld0.1240.171bear_2022bar_cox_stuart_trend_zORTHOGONAL
hvg_forward_visibility_horizon_mean_bar0.1370.175bear_2022bar_cox_stuart_trend_zORTHOGONAL
rogers_satchell_var_bar0.1760.312bear_2022bar_sign_markov_fluxORTHOGONAL
heikin_ashi_open_gap_sign0.2970.659current_2026bar_cox_stuart_trend_zORTHOGONAL
hvg_mean_degree_excess0.4840.609bull_2021closeORTHOGONAL
sequitur_grammar_compression_ratio_logret_sign0.6050.710bull_2021highORTHOGONAL
etc_effort_to_compress_logret_sign0.6810.771bull_2021highORTHOGONAL

Caveats carried at the time (vindicated by the §B re-score): the two compression features (108, 107) run closest to the flag (worst-regime 0.710 / 0.771 vs price high in bull_2021) — recorded as ξ-WATCH-adjacent. heikin_ashi jumps to 0.659 in current_2026 vs cox_stuart (both trend-sign objects) — family resemblance. Under the §B bands all four of these are now formally WATCH.

Null-cohort validation (shipped batch-5, ground-truth orthogonal)

Scored through the IDENTICAL pipeline: 7/8 ORTHOGONAL at the 0.80 flag. One marginal exceedance: bar_sign_markov_flux worst-regime 0.809 vs low in bull_2021 — 0.009 over the flag, in a shipped, dedup-proven column. Read: the method's empirical null ceiling in worst regimes is ≈0.81, i.e. residual upward noise near the boundary. Every candidate scores below the null cohort's own maximum (0.771 < 0.809 worst-regime; 0.681 < 0.755 pooled).

Method provenance (2 artifacts caught + corrected — CONTROL_FINDINGS.md)

  1. Overlap inflation: per-cell ξ on ~1-bar-stride rolling stats (199/200 shared bars) flags shipped pairs at up to 0.971 — i.i.d. violation, not redundancy. Fix: stride=200 (zero overlap).
  2. Point-mass → cell-identifier: aggregation_density (=1 on 99.999% of bars) per-cell-ranked becomes a cell label; ξ reads regime-location as dependence. Fix: the probe's declared h_norm ≥ 0.05 gate applied to RAW pooled values (drops aggregation_density + 4 constant exchange_session_* columns).

Artifacts: xi_probe_batch6.py (probe, resume-safe), xi_summary.json (telemetry of record), xi_null_control.py (the null-control experiment), CONTROL_FINDINGS.md (method postmortem). Raw per-cell .npz under xi_cells/ are reproducible via the script and are NOT committed (34 MB binary).

source: findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md

Still owed 3

What it claims, and what backs each claim 16

Every row pairs a claim with the file it came from and the verbatim text in that file. The sources sit above the deploy root, so the quote is embedded and the path is printed as text rather than linked — a link would resolve on a laptop and 404 here.

ClaimEvidence
Of the eight batch-6 candidates, four pass the declared redundancy bands, four land in the WATCH band, and none is banned.
MEASURED
n = 8 candidates; PASS 4 (cards 78, 60, 59, 50), WATCH 4 (cards 58, 45, 108, 107), BAN 0
## Result: 4 PASS · 4 WATCH · 0 BAN
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md
The four passing candidates read xi_worst between 0.078 and 0.312 and are elevated in none of the seven regime slices.
MEASURED
card 78 = 0.078, card 60 = 0.171, card 59 = 0.175, card 50 = 0.312; breadth 0/7 regimes each
| 78 | `hoeffding_phi_squared_midreturn_duration` | 0.078 | bear_2022 | bar_sign_markov_flux | 0/7 | **PASS** |
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md
The four WATCH candidates read xi_worst between 0.609 and 0.771, with the highest one elevated in six of seven regimes.
MEASURED
card 58 = 0.609 (1/7), card 45 = 0.659 (1/7), card 108 = 0.710 (4/7), card 107 = 0.771 (6/7)
| 107 | `etc_effort_to_compress_logret_sign` | 0.771 | bull_2021 | high | 6/7 | **WATCH** |
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md
Card 107 clears the ban rule's breadth dial but not the ban rule, because a ban requires both dials.
CONFIRMED
breadth 6/7 ≈ 0.86 ≥ 0.80 threshold; xi_worst 0.771 vs 0.95 ban floor
Note 107's breadth (6/7 ≈ 0.86) clears the BAN breadth dial, but BAN requires **both** dials and 0.771 is far below the 0.95 / ≥0.9995 duplicate cluster.
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md
An earlier same-day read of the identical measurements under the legacy single 0.80 cut called all eight orthogonal; it was superseded and kept as a historical record.
REFUTED
n = 8 candidates; pooled ξ_max 0.052–0.681, worst-regime ξ_max 0.078–0.771
## Result (superseded): ALL 8 CANDIDATES PASS the 0.80 flag — none filtered
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md
Computing this dependence measure naively per cell on rolling-window features is invalid: it flagged already-shipped, dedup-proven columns as duplicates.
REFUTED
4 of 15 shipped batch-5 pairs above the 0.80 flag; worst inflated readings 0.971 / 0.964 / 0.950 / 0.842
4/15 shipped pairs breach the 0.80 flag. **The naive method would retroactively condemn the shipped set itself → the method fails its null, not the candidates.**
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/CONTROL_FINDINGS.md
The root cause is that the estimator assumes independent samples while the features are 200-bar rolling statistics sampled roughly every bar, so adjacent samples share 199 of 200 bars.
CONFIRMED
199 of 200 bars shared between adjacent samples at 1-bar stride
Chatterjee ξ is an **i.i.d.-sample estimator**. All these features are trailing 200-bar rolling statistics sampled at ~1-bar stride — adjacent samples share **199/200 bars**
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/CONTROL_FINDINGS.md
The fix is to evaluate windows at a stride equal to the lookback so windows never overlap.
CONFIRMED
stride = 200 bars, zero overlap
1. **Decorrelate:** evaluate windows at `stride = LOOKBACK_COUNT = 200` → **zero overlap**.
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/CONTROL_FINDINGS.md
A second artifact was caught by the null cohort: a near-constant column turns into a cell identifier when ranked per cell, manufacturing fake dependence.
CONFIRMED
aggregation_density = 1 on 871,907 of ~872,000 rows (99.999%+) on BTCUSDT@250; 5 columns dropped by the raw h_norm ≥ 0.05 gate
per-cell pct-ranking a point-mass column collapses each cell's ties to a single value ≈ 0.5+1/2n — *distinct across cells* because n differs — so the pooled ranked column becomes a **cell identifier**.
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/CONTROL_FINDINGS.md
With the corrected recipe, the shipped ground-truth-orthogonal cohort scores clean except for one marginal exceedance, putting the method's empirical noise ceiling at about 0.81.
MEASURED
7 of 8 null-cohort columns orthogonal; bar_sign_markov_flux worst-regime 0.809 (0.009 over the 0.80 flag); null ceiling ≈ 0.81
Scored through the IDENTICAL pipeline: **7/8 ORTHOGONAL** at the 0.80 flag. One marginal exceedance: `bar_sign_markov_flux` worst-regime 0.809 vs `low` in bull_2021 — 0.009 over the flag, in a *shipped, dedup-proven* column.
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md
Every candidate scores below the shipped null cohort's own maximum, so no candidate exceeds what proven-orthogonal columns already read.
CONFIRMED
candidate max worst-regime 0.771 vs null 0.809; candidate max pooled 0.681 vs null 0.755
**Every candidate scores below the null cohort's own maximum** (0.771 < 0.809 worst-regime; 0.681 < 0.755 pooled).
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md
Because non-overlapping windows cannot clear the declared convergence floor at grid-cell granularity, the finest valid slice on this substrate is the seven-regime pool, not the cell.
CONFIRMED
97,811 pooled windows over 336 cells; 7 regime slices; FLOOR_1000 per pooled pair
at the declared stride-200 windows a single grid cell cannot clear `FLOOR_1000`, so the finest floor-valid slice on this substrate is the **regime pool** (7 regimes)
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md
The promotable-now set is the four PASS cards, each still carrying its own readiness blocker, while the four WATCH cards are held pending an explicit operator ruling.
OPEN
4 promotable, 4 held; blockers: 78 B-perf · 59 no-FOSS-oracle · 60 rename · 50 window-mean ratification
The **4 ξ-WATCH** — cards **58, 45, 108, 107** — are **held**: per §B WATCH semantics they may only ship after documentation + re-probe and an explicit operator ruling.
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md
The already-planned implementation PR stack must be re-ordered because it led with two of the now-held candidates.
OPEN
cards 108 and 107 both WATCH
The PR stack in `IMPLEMENTATION_PLAYBOOK.md` §6 (which led with 108/107) must be re-sequenced before the loop fires.
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/verdict.md
No new thresholds were invented for this filter; it consumes the declared constants and does not use the banned median rule anywhere.
ASSERTED
bands from PR #564 §B; xi_max both directions; FLOOR_1000; 750 dbps excluded; h_norm ≥ 0.05; LOOKBACK_COUNT = 200
## Declared constants used (no new knobs)
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/CLAUDE.md
The recipe is declared binding on future work: any future ξ pass on rolling features must reuse it and must score a shipped null cohort.
ASSERTED
**Any future ξ pass on rolling features must reuse this recipe** — and must score a shipped null cohort to prove it.
findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/CLAUDE.md

The audit folder 3 markdown files

Source of record: findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/ — not published, so these are listed rather than linked.

FileRole
CLAUDE.mdHub: scope, verdict of record, the load-bearing methodology finding, file index, and the declared constants used.
CONTROL_FINDINGS.mdMethod postmortem: why naive per-cell ξ is invalid on rolling features, the two artifacts found by the null cohort, and the corrected v2/v3 recipe.
verdict.mdTwo-part verdict — Part 1 the current worst-cell band scoring with the promotion consequence, Part 2 the superseded legacy single-cut read plus the null-cohort validation.
Generated by findings/dashboard/build_audits.py from findings/evolution/audits/2026-07-02-batch6-chatterjee-xi-filter/AUDIT_LEDGER.json — never hand-edited. Each quote was verified to occur in the file named beside it when the ledger was written.