An attempt to answer whether a feature that looks independent today will still look independent tomorrow: across 384 slices of six years of crypto and currency history it found that about 94โ95% of today's independent features stay independent into the next market regime, that today's redundancy margin ranks the ones about to flip with 0.88โ0.95 accuracy, and that not one of 92 features tested is independent in every regime โ so every such statement has to name the market conditions it applies to.
Lifecycle, not result. This says where the audit sits in its process โ never whether what it found was good.
**Status:** PROPOSED โ for Terry to refine (add/remove/approve) before any build. Nothing here is ratified or deployed.
2026-08-17. Derived from folder evidence, then adversarially challenged; the challenge pass was upheld.
The audit's own conclusion, reproduced in full from the source below. Not a summary โ this is the document, rendered. Links inside it that point at unpublished files are shown as plain text rather than as links that would 404 here.
Date: 2026-05-26 Status: Phase 1 + 2 complete ยท LICENSE-AUDIT complete ยท Phase 3 smoke spike executed (8 of 10 originally-planned slices โ 2020 pre-2021 BTCUSDT@250 not in CH).
> Can we predict whether a feature that is orthogonal today will still be orthogonal tomorrow?
No โ not with full confidence. Forward-looking orthogonality cannot be proved. What the 2010โ2025 literature provides is a set of bounded probabilistic guarantees, each valid under stated assumptions:
| Question | Predictable? | Best primitive |
|---|---|---|
| Is the feature orthogonal today with FDR control? | Yes, finite-sample | Model-X Knockoffs (Candรจs et al. 2018) |
| Will the selection be stable under resampling? | Yes, PFER bound | Stability Selection (Meinshausen & Bรผhlmann 2010) |
| Will it stay orthogonal under a specified class of future regime shifts? | Yes, probabilistic | Invariant Causal Prediction (Peters, Bรผhlmann, Meinshausen 2016) |
| Is apparent orthogonality an artifact of backtest mining? | Yes, PBO quantifies it | Bailey, Borwein, Lรณpez de Prado, Zhu (2017) |
| Will it stay orthogonal tomorrow in an arbitrary future regime? | No, provably impossible | โ (Rosenfeld et al. 2020 on IRM limits) |
Realistic confidence ceiling: ~8/10. This is the SOTA asymptote. It cannot be raised further because of four structural limits documented in the literature:
The catalogue in RESEARCH-DIGEST.md supports a robust pipeline that combines methods, where each layer removes a different kind of uncertainty:
| Layer | What it removes | Primitive |
|---|---|---|
| 1 | Sample noise | Block / stationary bootstrap |
| 2 | Parameter noise | Bayesian VAR + spike-and-slab + MCMC |
| 3 | Echo confound | Wold conditional innovation extraction |
| 4 | Linear-only blindness | Transfer entropy on innovations |
| 5 | Drift noise | Particle filter (sequential Monte Carlo) |
| 6 | Distribution-free coverage | EnbPI conformal prediction |
| 7 | Regime fragility | Invariant Causal Prediction |
Detailed recipe and cost estimates in PIPELINE-7-STEP-STACK.md.
Phase 2 โ FOSS implementation lookup. For each method in the catalogue, identify whether a maintained FOSS implementation exists. Three outcome categories per method:
foss_candidates.jsonlAfter Phase 2 completes, full cost-benefit visibility unlocks Phase 3 (pilot empirical spike on a Phase-2-validated subset of methods).
Tier-A smoke on aggression_ratio against 5-feature active set across 8 regime-distinct slices (BTCUSDT @ 250 dbps, 2021-2025, 78,773 bars total, single-thread Python, ~10s wall-clock):
| Metric | Result | ||||
|---|---|---|---|---|---|
| Spearman max\ | ฯ\ | (Sub-test a) | 8 of 8 PASS โ orthogonal to active set in every regime (max\ | ฯ\ | range: 0.62โ0.79, below FRAMEWORK PASS = 0.85) |
| Granger min p-value (Sub-test c) | 2 of 8 PASS at conventional p < 0.05; S03 (p=1.5e-07) and S04 (p=5.0e-19) strongly significant; the other 6 inconclusive | ||||
| Originally-planned 2020 regimes (S01, S02) | Dropped โ BTCUSDT@250 cache does not extend pre-2021-01-01 | ||||
Original candidate lookback_hurst | Discovered 100% null in production (column defined but never populated); switched to aggression_ratio |
Honest caveat on Granger: threshold (p < 0.05) is Fisher convention, not project-mandated. No permutation null baseline, no Holm-Bonferroni correction for 3-lag min-p selection, no effect-size floor. The 6 FAIL_GRANGER slices have small bar counts (509โ6,258) and may be power-limited rather than truly non-predictive. Strong PASSes (S03, S04) are robust to any reasonable threshold.
What the spike validates: (1) the harness mechanically works end-to-end on real production data, (2) Steps 3 + 4a (Wold AR + Granger F-test) of the 7-step pipeline produce real verdicts, (3) the verdict distribution across regimes is non-uniform โ the structural signal the methodology was designed to detect.
What the spike does NOT validate: Steps 1, 2, 5, 6, 7 (block bootstrap, Bayesian VAR, particle filter, EnbPI conformal, ICP). The full pipeline still requires those layers wired in.
Full artifact: spike_2026-05-26/results.md, with per-slice machine-readable verdicts in spike_2026-05-26/per_slice.jsonl. The 8 Parquet slices the spike consumed are not committed (they exceed reasonable repo-binary thresholds and reproduce deterministically from bigblack ClickHouse); to regenerate them, run spike_2026-05-26/spike.py on bigblack, which re-pulls the same slice scope from fxview_cache.forex_bars / opendeviationbar_cache.open_deviation_bars and writes parquets to spike_2026-05-26/data/ (gitignored).
Full design in PHASE-3-PROBE-DESIGN.md. Key points:
MODEL-SYNTHETIC-INTERNAL (knockoffs, Bayesian-VAR, particle filter, GAM-DVQR) โ flagged, kept optional.papers.jsonl and SOURCES.mdsearch_log.jsonl with timestamps and result URLsfoss_candidates.jsonlLICENSE-AUDIT.mdspike_2026-05-26/per_slice.jsonl (Parquet inputs gitignored; reproducible via spike.py)Every row pairs a claim with the file it came from and the verbatim text in that file. The sources sit above the deploy root, so the quote is embedded and the path is printed as text rather than linked โ a link would resolve on a laptop and 404 here.
| Claim | Evidence |
|---|---|
| The central question cannot be answered with certainty, and the literature says so: what exists are bounded guarantees under stated assumptions. ASSERTED ~30 SOTA methods across 6 layers; stated confidence ceiling ~8/10 | **No โ not with full confidence.** Forward-looking orthogonality cannot be proved. What the 2010โ2025 literature provides is a set of **bounded probabilistic guarantees**, each valid under stated assumptions: findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/verdict.md |
| Four published structural limits cap how confident anyone can be, and the audit names all four rather than working around them. ASSERTED 4 limits: linear-only Wold orthogonality, required stationarity, drifting VAR coefficients, orthogonality being relative to the active set | **Realistic confidence ceiling: ~8/10.** This is the SOTA asymptote. It cannot be raised further because of four structural limits documented in the literature: findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/verdict.md |
| The first small test on real data worked mechanically but only two of eight market periods passed both of its sub-tests. MEASURED 8 regime slices, 78,773 bars, ~10 s; 8/8 pass the redundancy sub-test (max |ฯ| 0.618โ0.787); 2/8 pass the Granger sub-test; 6 fail on slices of 509โ6,258 bars | - **PASS** (max\|ฯ\| โค 0.85 AND Granger p < 0.05): **2 / 8** findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/spike_2026-05-26/results.md |
| The originally chosen feature had to be swapped because the production column it needed was empty. CONFIRMED 1 column 100% null | | Original candidate `lookback_hurst` | Discovered 100% null in production (column defined but never populated); switched to `aggression_ratio` | findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/verdict.md |
| Every one of the 50 catalogued forward methods was checked parameter by parameter against its own paper; none is genuinely knob-free, so the test was reframed as arbitrary-knob versus principled-knob. CONFIRMED 50 methods classified; 27 magic-number-free, 22 offenders, 1 impossibility result; verification sweep of ~976K tokens and 236 tool uses | > **Key finding:** No forward matrix is fully *parameterless* (forecasting tomorrow requires specifying temporal structure). The decision axis is therefore **magic-number vs principled-parameter**, not parameterless vs parameterized. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/MATRIX-PARAMETER-CLASSIFICATION.md |
| The test grid was built from named market events rather than arbitrary calendar windows, and every cell but one had enough data. CONFIRMED 384 cells (crypto 210 + forex 136 + 38 session cells); 384/384 have data; 383/384 clear the 1,000-bar floor; 1 thin cell (BTCUSDT@750 in ftx_2022, 952 bars) | **Verdict: the combined regime grid is fully populated and dense.** Every regime slice has data on every probe-faithful (symbol, threshold) cell; one of 384 cells is sub-floor and is dropped by the existing density gate. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/COMBINED-REGIME-SLICING.md |
| The two substrates are badly lopsided: crypto has only 24 usable feature columns against 140 on the currency side, partly because the recently promoted features were never backfilled into history. CONFIRMED Crypto 24 of 102 columns covered, 78 empty (32 quote/spread, 22 intra_*, 16 lookback_*, 8 recently promoted bar_*); forex 140 of 151 covered | **Implication for the forward matrices:** the *effective* feature set is **24 crypto vs 140 forex** โ treat the asymmetry honestly. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/COMBINED-REGIME-SLICING.md |
| A two-cell dry run caught two design errors before the expensive run โ a mis-wired reference pair and three metrics aimed at the wrong question. CONFIRMED 2 cells, 13 crypto + 140 forex features, Nโ2000/cell; enbpi Rยฒ โ 0.000โ0.004 and icp invariance 0.000 on both substrates; a memory guard tripped at 2,055 MB and self-aborted with nothing written | **Issue 2 โ three metrics test the wrong question.** `granger` / `enbpi` / `icp` use **next-bar return** as the target. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/TIER-A-METRIC-SCALE-SPIKE.md |
| The full grid ran cleanly but the pre-registered success count was only 1 of the required 5, and the audit records that number without inflating it. MEASURED 384/384 cells in 112.2 min, 0 failures, RSS flat 251 MB; 3,344 crypto + 24,122 forex feature-cell evaluations; 1 metric useful on both substrates (adv_auc, 40/40 combos), 1 forex-only (19/19), 1 crypto-only (21/21) | By the **pre-registered rule** (โฅ5 metrics USEFUL on both substrates = GO; 3โ4 = CONDITIONAL; <3 = PIVOT), the strict count is **1 cross-substrate USEFUL โ PIVOT territory**. We do NOT inflate that number. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/FULL-GRID-SCALE-VERDICT.md |
| No feature at all has a regime-independent relationship to the rest of the panel โ a result that was then replicated on complete data. CONFIRMED 0 of 91 then 0 of 92 features invariant at ฮฑ=0.05 across 10 regime environments, ~2K bars per regime, 4 active predictors | - **Crypto: 92 features, 0 regime-invariant โ REPLICATED on complete data.** The "forward declarations must be regime-conditional" finding stands. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/E4-FINAL-VERDICT.md |
| Today's redundancy margin is itself the best predictor of which features will stop being independent tomorrow, and the result survives a control that shuffles the today-to-tomorrow pairing. MEASURED 19,196 transitions (crypto 2,992 / forex 16,204); persistence among orthogonal-at-t 95.1% crypto / 93.6% forex; conditional AUC 0.945 (n=803) crypto and 0.880 (n=8,189) forex; shuffled-pairing null collapses to [0.472,0.522] and [0.491,0.509] | | **redundancy margin (t)** | **0.997 [0.996, 0.998]** | **0.945** (n=803) | **0.983 [0.982, 0.985]** | **0.880** (n=8,189) | findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/E123-WALK-FORWARD-RESULTS.md |
| A blind statistical screen picked out exactly the four crypto features the earlier orthogonality audits had promoted, with a bounded false-discovery rate and anti-mining checks passed. CONFIRMED 4 of 16 crypto features and 72 of 141 forex features declared at BH=BY q=0.10; PBO 0.00 crypto, 0.20 forex; deflated max-z 22.15 vs expected 1.53 crypto and 12.49 vs 2.45 forex | **Crypto declared set (FDR โค 10%):** `aggression_ratio`, `bar_dispersion_entropy`, `bar_katz_fd`, `bar_petrosian_fd` โ **exactly the features the orthogonality audits promoted**, now independently confirmed to persist across regime transitions with controlled false-discovery. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/E123-WALK-FORWARD-RESULTS.md |
| The claim that adaptive error bars "fixed" the currency-side calibration was downgraded by the audit's own pre-registered battery: the information is real but the price exceeded the line agreed in advance. REFUTED 7 pre-registered rules; forex passes R1 (0.38ร climatology), R3 (Winkler 0.441 < 0.464 < 0.903), R7 (shuffled width explodes to 1.79ร), R4, R5; fails R2 (1.328ร vs 1.3ร) and R6; crypto graded COSMETIC | | R2 price | **1.328ร fixed โ 2.8% over the pre-registered 1.3ร line** | โ | findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/E2-BATTERY-VERDICT.md |
| A later re-run on fresh data reproduced the whole headline campaign exactly, derived one trustworthy threshold, and showed the project's 0.85 redundancy line cannot be justified from the data. CONFIRMED 210/210 cells in 4.5 min, peak RSS 216 MB under a 3 GiB cap, 0 writes; persistence 95.1%, conditional AUC 0.945, PBO 0.00, deflation 22.15 vs 1.53, ICP 0/92 all reproduced; adv_auc matched-null threshold โ0.522 (KS 0.529); redundancy median 0.991 over 3,344 feature-cells with 64% above 0.95 | โ **TAU = 0.85 fails the sensitivity/plateau test โ it remains POLICY, not derived.** findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/SPIKE-2026-06-16-RECONFIRMATION.md |
| The strongest of the four intended verdict grades is empty in practice, and the boundary between the two middle grades cannot be drawn because a measurement was never taken. CONFIRMED STABLE-ORTHOGONAL passed by 0 of 91 features; only 2 of 4 statuses numerically assignable today; 5 thresholds in the stack classified as un-nulled policy (0.85 WATCH, h_norm 0.05 with K=5, min_regime 1.0, FLOOR_1000, the 1.3ร line) | - **FRAGILE vs CONDITIONAL cannot yet be split per feature** โ the ICP run reported only the binary "full invariance 0/91," not the per-environment pass count (k of 10) the boundary needs. **That measurement is missing.** findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/STATUS-GROUNDING-AND-THRESHOLDS.md |
| Of nine proposed extra measurements, seven were rejected as adding nothing trustworthy โ including the obvious fix for the un-justified 0.85 threshold. CONFIRMED 9 candidates: 7 EXCLUDE, 2 INCLUDE-IF (anytime-valid live monitor; nonlinear/relaxed invariance test as a one-shot falsifier) | **Net:** 7 of 9 add no new, trustworthy information. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/METRIC-ADDITION-GATE.md |
| A later literature refresh re-checked every citation against the source archive, caught and corrected one misattributed author, and produced no new numbers on the project's own data. CONFIRMED 14 candidates surveyed, 4 survive as INCLUDE-IF; 15/15 arXiv IDs resolve with matching titles; 1 author string corrected (arXiv:2502.02710) | - **IS NOT:** new empirical evidence on ODB data. **The deep research produced zero new numbers on our bars.** findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/SOTA-RESEARCH-2026-06-23.md |
| The part of the design that would actually raise the alarm when a feature stops being independent is the weakest part: it is unbuilt, never power-tested, and built on an assumption a later campaign showed is violated by this kind of data. CONFIRMED Grading side ~9/10, monitor ~4โ5/10, lifecycle automation ~3/10; 9 named gaps (3 decisive); the violated assumption produced 11โ57% false alarms instead of 5% and killed 8 instruments in the sibling campaign | | Live-expiry MONITOR (the actual expiry) | **~4โ5/10** | Sound design, but one law-violating default (raw exchangeability null on dependent bars) + unbuilt + E6 never run. | findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/FEATURE-EXPIRY-ROBUSTNESS-ASSESSMENT-2026-07-15.md |
| On the currency side the regime-invariance test still returns nothing at all, so no currency feature can be graded on that axis. CONFIRMED 0 environments returned for forex; 92 features graded on crypto | 1. Forex ICP wiring defect (0 environments) โ debug pass needed. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/E4-FINAL-VERDICT.md |
| One auxiliary metric behaves backwards on crypto โ high apparent freshness predicts flipping โ and the reversal reproduces, so it is flagged rather than used. CONFIRMED conditional AUC 0.278 (below 0.5) on crypto, replicated across v2 and v3 runs; 0.617/0.623 on forex | 2. wold_R conditional inversion on crypto replicates (0.278) โ investigate the mechanism before using it as a gate. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/E4-FINAL-VERDICT.md |
| Licences were verified per method by reading the actual licence file, and a third of the catalogue needs process isolation or a rewrite before it could be used. CONFIRMED 50 idea-ids checked: 27 permissive, 9 GPL-family, 2 non-commercial-only, 3 with no licence file, 1 unknown, 8 with no code at all | license-compatibility verdict (27 use-as-is ยท 8 R-via-Rscript ยท 1 Python-via-subprocess ยท 6 reimplement ยท 8 no-FOSS) findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/CLAUDE.md |
| The proposed live system that would turn all of this into a running product is a proposal only โ nothing is built, ratified or deployed. ASSERTED 4 decisions locked with the operator; 6 named tasks outstanding before go-live including threshold calibration ("Not done yet") and a crypto feature backfill | **Status:** PROPOSED โ for Terry to refine (add/remove/approve) before any build. Nothing here is ratified or deployed. findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/ADR-2026-07-15-LIVE-ORTHOGONALITY-DECLARATION-ENGINE.md |
Source of record: findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/ โ not published, so these are listed rather than linked.
| File | Role |
|---|---|
ADR-2026-07-15-LIVE-ORTHOGONALITY-DECLARATION-ENGINE.md | Proposed architecture for a live per-regime declaration board with alerts and an append-only expiry history; awaiting review, nothing built. |
CAMPAIGN-GROUNDED-SUMMARY.md | Campaign close-out: the candidate funnel, what each of the 8 evaluated methods grounded, the one-paragraph answer, and five carried-forward items. |
CLAUDE.md | Audit hub โ the question, scope, provenance model and a per-file index covering every phase of the campaign. |
COMBINED-REGIME-SLICING.md | The 384-cell regime grid: named market events instead of calendar windows, density-verified, with the crypto/forex column asymmetry. |
E123-WALK-FORWARD-RESULTS.md | Leakage-gated walk-forward: forecast skill, calibration and panel error control, with the four declared crypto features. |
E2-BATTERY-VERDICT.md | Seven-rule falsification battery on the adaptive error bars; downgrades the forex calibration claim to PARTIAL. |
E4-FINAL-VERDICT.md | Clean re-run on complete data confirming skill, calibration and the four stamped features; lists four open items. |
FEATURE-EXPIRY-ROBUSTNESS-ASSESSMENT-2026-07-15.md | Assessment of the existing expiry design against two later-discovered laws; nine gaps and twelve prioritised additions. |
FULL-GRID-SCALE-VERDICT.md | The 384-cell run: per-metric scorecard, the zero-invariance finding, named defects, and the campaign candidate accounting. |
LICENSE-AUDIT.md | Per-method licence verdict (use-as-is, process isolation, or reimplement) read from the actual licence files. |
MATRIX-PARAMETER-CLASSIFICATION.md | Per-parameter classification of all 50 methods plus the production baseline; the source of the 27-free/22-offender split. |
METRIC-ADDITION-GATE.md | Default-exclude decision record on nine proposed extra metrics, plus the prioritised roadmap of what to fix next. |
PHASE-3-PROBE-DESIGN.md | Robustness-first method prioritisation, the five tiers, the gate ladder and the verdict definition; records two operator exclusions. |
PIPELINE-7-STEP-STACK.md | Design-only recipe combining the methods into one seven-layer pipeline, each layer removing a different uncertainty. |
RESEARCH-DIGEST.md | Canonical F001โF050 method table by SOTA layer with paper ID, repo, commit SHA and licence. |
S0-PRE-REGISTRATION.md | Frozen pre-registration for the calibration campaign โ thresholds, nulls, splits and budgets fixed before the test split was touched. |
SOTA-RESEARCH-2026-06-23.md | Literature refresh with a blocking citation re-verification gate; refines the two include-if buckets, adds no data. |
SOURCES.md | Annotated bibliography organised by SOTA layer with abstracts. |
SPIKE-2026-06-16-RECONFIRMATION.md | Fresh-data replication of the headline numbers plus the first data-derived threshold and the finding that the 0.85 line stays policy. |
STATUS-GROUNDING-AND-THRESHOLDS.md | Numbers-first audit of which forward status each threshold can actually support; marks every un-nulled policy number. |
TIER-A-METRIC-SCALE-SPIKE.md | Two-cell de-risk dry run that caught the anchor-wiring defect and the wrong-target metrics before the full grid. |
spike_2026-05-26/results.md | Per-slice results of the first smoke spike on one feature across eight regime epochs. |
verdict.md | Root verdict: the honest "no, not with full confidence" answer, the four structural limits, the smoke-spike findings and the Phase-3 design decisions. |
findings/dashboard/build_audits.py from findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/AUDIT_LEDGER.json โ never hand-edited. Each quote was verified to occur in the file named beside it when the ledger was written.