Orthogonality, redirected: from correlation to regime-invariance

Design preview · 2026-07-22 · opendeviationbar-py forward-predictiveness campaign · scratch reading artifact — not an ADR, not for review

The direction in one picture. Steps ①→④; the two axes are the load-bearing new idea. The clock (B-05) and the validity spine are the two enablers that make the engine trustworthy rather than a “silent liar”.

1 · Layman’s summary — the core concepts

Think of your model as a hiring manager building a team of specialists. Each feature is a candidate. You want a team that is (a) full of distinct skills and (b) still competent when conditions change. Five ideas do all the work here:

Orthogonal feature — a feature whose information doesn’t overlap the ones you already ship. A distinct specialist, not a duplicate hire.
Regime — a market “climate”: calm, crash, bull-blowoff, range-bound, etc. The market is not one distribution — it’s a sequence of these.
Correlation-orthogonality (the old lens) — “is this feature redundant right now?” Cheap hygiene, but blind to the future.
Regime-invariance orthogonality (the new lens) — “does this feature stay distinct and keep its meaning across many different climates?” This is what survives when the weather turns.
Forward-predictiveness — the machinery that answers: “a feature is orthogonal today — will it still be orthogonal tomorrow, and how confident are we?”

One-line version: we stop hiring specialists who only looked good last quarter, and start hiring the ones who stay useful in every climate we’ve lived through.

2 · The two axes — and how each one helps

These measure two completely different things, which is exactly why they combine into something stronger than either alone. Axis 1 asks “which way will price go, robustly?” Axis 2 asks “how fragile / how liquid is this?”

Axis 1 · OOD-Robust (regime-invariance)

What it measures: whether a feature’s relationship to the target is stable across regimes, using causal-invariance and instrumental-variable tests (seqICP, Causal Dantzig, HSIC-X, Anderson–Rubin) rather than a correlation score.

How it helps: out-of-distribution failure = the future regime differs from the past. A feature admitted here has already been stress-tested against regime change, so it’s your best bet to carry into a climate you haven’t seen.

Honest limit: unconditional “orthogonal in any future regime” is provably impossible — the campaign measured the ceiling (ICP 0/92, ~8/10). Every verdict is regime-scoped, never “forever”.

Axis 2 · Cascade-Free (liquidity & crowding)

What it measures: how liquid and un-crowded a position is — a different information axis from price. Because it doesn’t encode “which way will price go,” it is orthogonal by construction to your price-microstructure panel.

How it helps: it flags fragility — where a liquidation cascade could ignite and how hard it would be to exit — so strategies can size down / gate capacity before the stampede, not after.

Scope note: the richest crowding signals (funding, open interest, liquidation heatmaps) are derivatives data — blocked by the spot-only policy. So we use spot-legal proxies: Amihud illiquidity, Kyle-λ, order-book imbalance, and the effective-spread estimators already probed on 2026-07-22 (edge/roll/corwin/parkinson).

Why two axes beat one: Axis 1 gives you a signal that keeps working when the regime turns; Axis 2 keeps you from being trampled while you hold it. A strategy needs both to be simultaneously OOD-robust and liquid/cascade-free — which is precisely your stated goal.

2·5 · Axis 2 harvest — the live loop that grounds it OPEN SURFACE

Axis 2 is the open frontier of this campaign. Grounding it needs two harvests (per the grounding plan), both collected by an append-only /loop — nothing is evaluated on our data yet:

(A) Candidate features — the spot-legal liquidity / crowding / fragility estimators that will be the Axis-2 columns.
(B) Measurement instruments — metrics that quantify the cascade-free / liquidity property (the 23 generic orthogonality matrices carry over for free; the domain-specific ones are the gap).

What the loop sweeps, exhaustively, family by family: illiquidity & price-impact (Amihud, Kyle-λ, Hasbrouck, MRR) · effective-spread from OHLC/trades (Roll, Corwin–Schultz, Abdi–Ranaldo, EDGE) · range/HL vol (Parkinson, Garman–Klass, Rogers–Satchell, Yang–Zhang) · order-flow toxicity & crowding (VPIN, OFI, CVD, taker-skew) · microstructure invariance (Kyle–Obizhaeva) · capacity/turnover (Days-ADV) · liquidity-cascade / fire-sale risk · jump/tail fragility for ignition (realized jumps, tail-index, Hawkes self-excitation).

Load-bearing constraint: spot-only (principle #8) — funding / open-interest / liquidation-heatmaps are derivatives → catalogued but flagged out-of-scope. Seed catalog: 16 spot-legal / L2 candidates + 3 derivatives-flagged.

Open the Axis-2 Cascade-Free harvest — full grounding scope + live iteration ledger →

3 · How forward-predictiveness surfaces candidate orthogonal features

The engine grades every shipped feature, per live regime, and assigns the highest tier its evidence earns (cheapest test first, early-exit):

BAN — redundant today vs the shipped set WATCH — real information, but regime-sensitive ORTHOGONAL — holds across regimes → a genuine candidate

The point that makes this a discovery tool and not just a scorecard: the campaign proved the tomorrow-verdict is predictable — features orthogonal today persist 94–95% into the next regime, and the flip-vs-hold ranking scores AUC 0.88–0.95. So the ORTHOGONAL tier isn’t a snapshot; it’s a forward filter that surfaces the features worth carrying.

Non-negotiable underneath it all: every test must use a dependence-aware null. On autocorrelated bars, an independence-assuming null false-alarms at 11–57% (it already killed 8 instruments). Without this, the board is a confident liar — so it’s the first thing that has to be certified.

4 · The direction we’re heading — and a correction to sharpen it

You said: “a new way of evaluating orthogonality based on regime invariance, rather than correlation.” You’re right — that’s the core shift. Two refinements so it’s exact:

Correlation isn’t thrown away — it’s demoted to hygiene. Correlation-redundancy is still the cheap BAN screen (“duplicate today → reject”). Regime-invariance becomes the bar for the ORTHOGONAL ship-decision. So: correlation for hygiene, invariance for the decision.
The invariance idea is already grounded — the genuinely new frontier is the second axis + the strategy bridge. Axis 1 (OOD-robust) is largely built and correctly bounded. What’s underbuilt is Axis 2 (cascade-free/liquidity) and the realness/usefulness bridge that turns an admitted feature into a tradeable strategy. That’s where the next real edge is.

So “orthogonal” is redefined as necessary-but-not-sufficient: non-redundant and regime-stable earns a feature into the pool — it does not by itself prove tradeable edge. Step ③ is the honest gate between “admitted feature” and “strategy.”

5 · First step — pilot runs to find candidate features

Before any ADR, we do small, honest pilot runs to surface a shortlist:

Read-only & capped — ClickHouse --readonly=2, spare-only, watchdog-guarded. No production touch.
Spot-legal features only — price-microstructure (Axis 1) + spot liquidity proxies (Axis 2). No derivatives data.
Leakage-gated first — the T1–T4 guard (future-perturbation invariance + shuffled-null collapse) runs as a hard-stop before we trust any number. Too-good-to-be-true is treated as a leak, not a win.
On real regime slices — the 20 named crypto/forex archetypes, so “across regimes” is measured, not assumed.
Output — a candidate table per axis with a provisional BAN / WATCH / ORTHOGONAL and the evidence behind each.

The pilot’s job is not to be right — it’s to de-risk the design and hand the ADR a concrete, grounded candidate set to reason about.

6 · What the ADR will be (once we’re aligned)

A PROPOSED design ADR — same route as #633 (ground → ADR → Terry refines → implement), nothing ratified on landing. Its scope, in one breath:

Adds Axis 2 — a spot-legal liquidity/crowding feature axis the engine grades alongside price-microstructure.
Defines the pilot protocol — exactly the read-only, leakage-gated candidate-discovery run above.
Sequences the enablers — (1) dependence-aware null + E6 replay as the correctness spine; (2) the B-05 regime-turn detector as the clock; (3) grow the crypto ORTHOGONAL witness leg beyond the single e-ICP-fp.
Names the successor — the realness/usefulness ladder (step ③) as the explicit bridge from “admitted feature” to “OOD-robust, cascade-free strategy.”

It stays a docs-only artifact until you and Terry steer it. This preview exists so we agree the picture above is the right picture first.

Pages: this reader · Axis-2 Cascade-Free harvest (live loop + ledger)

Campaign reader (dashboard twin) · findings/dashboard/probes/regime-invariant-orthogonality-campaign-2026-07-22.html · SSoT: findings/evolution/audits/2026-07-22-regime-invariant-orthogonality-campaign/ · docs-only; deploy to the nasimubd site is a separate gated step. Grounded in: ADR-2026-07-15-LIVE-ORTHOGONALITY-DECLARATION-ENGINE.md · FEATURE-EXPIRY-ROBUSTNESS-ASSESSMENT-2026-07-15.md · CAMPAIGN-GROUNDED-SUMMARY.md · 2026-07-22 rotation-probe head-to-head.