โ€บNavigation

Architecture Decision Record ยท for Terry to refine

Live Orthogonality Declaration Engine

The grounded matrices are the graders. This engine is the outcome: for each live market regime it runs those matrices over every shipped feature and declares each one BAN / WATCH / ORTHOGONAL โ€” published live, alerted on Telegram, with an append-only expiry trail.

2026-07-15 PROPOSED โ€” for Terry's steer โ†’ implement Quality bar: 10/10, no defective version

What "live expiry" means here. A feature's ORTHOGONAL declaration expires the moment the current regime's matrices stop declaring it orthogonal โ€” it drops to WATCH, then BAN. Re-promotion is the reverse. The board is the running truth; the history of transitions is the expiry record.

Context โ€” the forward-predictiveness research this rests on

This engine is the operational face of a multi-month research arc with one question: "a feature is orthogonal today โ€” will it still be orthogonal tomorrow, and how do we declare that?" The chain you're reviewing:

StageWhat it establishedLiving page
Forward-predictiveness campaign
2026-05-26
A feature orthogonal today stays orthogonal into the next regime 94โ€“95% of the time; which ones flip is rankable (AUC 0.88โ€“0.95); calibrated error bars (ACI) hold; a "stays-orthogonal" set is declarable at FDR โ‰ค 10%. ICP 0/92 โ‡’ every declaration is regime-scoped (~8/10 structural ceiling; unconditional invariance is provably impossible).๐Ÿ”ฎ Forward โ€” Orthogonal Tomorrow
Matrix-admission campaign
2026-07-02 โ†’ 07-14
Certified 23 measurement matrices (16 admits + 7 grounded) through a 6-gate attack ladder; 45 kills; discovered the two load-bearing laws โ€” dependence-aware nulls and anytime-validity. These matrices are the graders this engine runs.Matrix Admission Paradox Loop
Tri-tier proposal
2026-07-11
Collapsed the four feature statuses into BAN / WATCH / ORTHOGONAL, each grounded matrix mapped to the tier it informs โ€” the grammar this engine publishes.Tri-Tier BAN / WATCH / ORTHOGONAL

In one line: the forward campaign proved the declarations are predictive, the admission campaign built and proved the graders, and this ADR ships them live. The full trail is in the provenance appendix at the bottom.

The three tiers (grounded matrices behind each)

โ— BAN

Redundant today vs the shipped set โ€” reject.
G0 Spearman+h_norm ยท G2 cond-AUC ยท G3 ICP ยท G5 adv_auc ยท G7 conformal ยท G10 shuffled-null ยท G11 FDR ยท PID-red #15

โ— WATCH

Real info but regime-sensitive โ€” document + re-probe.
k_v verdict-flip ร—2 ยท MMD #13 ยท Wasserstein #14 ยท calibration-gap #12 ยท O-info #16

โ— ORTHOGONAL

Holds across regimes โ€” ship (scoped, never "always").
e-ICP-fp ยท seqICP ยท IAS ยท IPP ยท HSIC-X ยท Causal Dantzig ยท Staigerโ€“Stock ยท Andersonโ€“Rubin (crypto = e-ICP-fp only โ€” Q5)

Decisions locked with you (2026-07-15)

#DecisionChoice
D1Refresh cadenceRegime-anchored + tiered โ€” work scales with regime changes, not the clock
D2TelegramRegime change + tier flips โ€” mirrors ODB ops (notify/telegram.py, @obdpy_bot)
D3UniverseAll shipped features ร— live regime, per substrate โ€” 24 crypto / 140 forex
D4ProcessADR โ†’ Terry โ†’ implement โ€” no prod build first

How it runs (five layers)

  โ‘  REGIME LAYER (the clock)
     live_tail rolling window โ†’ classify archetype (crash/bull/bear/range/parity/parabola) + session
     cheap always-on regime-turn watcher  โ”€โ”€ fires โ”€โ”€โ–บ  full re-declaration
          (B-05 detector, de-aligned + certificate-attacked; daily-floor fallback until B-05 unGATED)
                                   โ”‚
                                   โ–ผ
  โ‘ก DECLARATION LAYER (cheapest-first, early-exit, per feature ร— live regime)
     BAN screen (cheap) โ”€โ–บ WATCH check (medium) โ”€โ–บ ORTHOGONAL check (heavy chain)
       โ””โ”€โ”€ assign the HIGHEST tier the evidence earns; thresholds operator-calibrated
                                   โ”‚
                                   โ–ผ
  โ‘ข VALIDITY SPINE  (mandatory โ€” what makes it 10/10)
     every live null is DEPENDENCE-AWARE (block/circular de-alignment OR the certified
     SKIT betting e-process). No declaration publishes unless it passed the certificate attack.
                                   โ”‚
             โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
             โ–ผ                                             โ–ผ
  โ‘ฃ PUBLISH                                       โ‘ค NOTIFY (Telegram, @obdpy_bot)
     living dashboard board  +                     regime change  โ†’  board summary
     append-only CH table                          tier flip       โ†’  ORTHOGONALโ†’WATCH/BAN = EXPIRY
     (expiry history queryable)                     (design only โ€” nothing sent without your go)

Resource efficiency (first-class)

MechanismHow it saves
Tiered cadenceCheap BAN screen hourly; heavy WATCH+ORTHOGONAL suite on regime-turn + daily floor โ€” not every 10 min recomputing identical results.
Event-drivenHeavy suite fires only when the regime-turn watcher trips โ†’ work scales with changes, not time.
Incremental / cachedRe-run a (feature, regime) cell only when the slice changed; cache prior declarations; skip < 1000-bar cells.
Rust kernelsKernel two-sample, ICP folds, block bootstrap in Rust; Python does I/O only.
Read-only + cappedClickHouse --readonly=2; ~2c/2G always-on watcher; heavy suite spare-only + watchdog.
De-alignment reuseDependence-aware null computed once per slice, reused across features โ€” not once per feature.

Before it's authoritative โ€” the 10/10 gate

1 ยท Certificate attack (the IID law): every live matrix must hold โ‰ค 10% false-alarm on de-aligned true nulls, or it doesn't publish. This is the exact test that killed 8 instruments โ€” it's why the board won't cry wolf on autocorrelated data.

2 ยท E6 replay (positive + negative controls): a feature known to flip when a regime turns โ†’ the board must flip it; a known-stable feature โ†’ it must not. Wrapped in the leakage guard (T1โ€“T4).

3 ยท Until both pass, the board ships as EVALUATING / shadow, not authoritative.

Where your steer would help most

You've already set the direction โ€” these are the spots where your opinion and suggestions would sharpen it.

  1. 1 Crypto ORTHOGONAL gap โ€” it rests on one instrument (e-ICP-fp) vs forex's nine. Ship with a single-witness flag, or grow the crypto leg first? Our lean: ship as ORTHOGONALโป (flagged) and keep growing the crypto leg.
  2. 2 Prioritise the live regime-turn detector? Building the B-05 change-detector family is what lets the board auto-refresh on real regime turns (vs a daily-floor fallback) โ€” worth pulling forward? Its gate is ours to lift; your read on priority is what helps.
  3. 3 Anything you'd add โ€” SOTA angles, robustness checks, or scope โ€” to make the live engine stronger.

On our plate before go-live (operator tasks โ€” for visibility, not sign-off)

One paragraph for Terry

We grounded 23 matrices and sorted them into BAN / WATCH / ORTHOGONAL. This ADR turns that into a live engine: for each market regime it runs the matrices over every shipped feature and publishes a BAN/WATCH/ORTHOGONAL board, refreshed when the regime turns (cheap screen hourly, full suite on-turn + daily) โ€” so it's resource-light โ€” with a Telegram alert on regime changes and tier flips, and an append-only history so every "orthogonal โ†’ watch/ban" expiry is auditable. The one thing that makes it a real 10/10 rather than a silent liar: every matrix that runs live uses a dependence-aware null (the law we grounded the hard way) and must pass a certificate attack + E6 replay before the board is authoritative. Your read and any suggestions are welcome โ€” then we build.

Provenance & telemetry โ€” the full trail (everything we have so far)

So you review with the exact view. Dashboard = living pages, deployed at odb.epatner.com (clickable). Audit / telemetry = the append-only forensic record + code + raw JSON on GitHub main (terrylica/opendeviationbar-py).

โ‘  Dashboard โ€” matrix evaluation

PageWhat it is
Matrix Admission Paradox LoopThe campaign hub โ€” 103 iterations, 23 certified matrices, 45 kills, the two laws.
Tri-Tier BAN / WATCH / ORTHOGONAL proposalYour operator-requested tier mapping โ€” the grammar this engine publishes.
Iteration 103 โ€” handoverEvery metric in plain words + the matrix-side live-expiry ADR + flowchart.
Matrix Evaluation for OrthogonalityThe Q1โ€“Q6 evaluation loop โ€” incl. the original live-expiry monitor spec (Q4).
Forward-Orthogonality Matrix-Scale EvaluationThe 384-cell leakage-gated walk-forward behind the Forward probe.

โ‘ก Dashboard โ€” harvesting

PageWhat it is
Matrix Harvest & Status-Grounding LoopThe candidate-sourcing campaign โ€” 300โ€“400 matrices harvested, provenance-verified, M0/M1-screened, fed to the admission loop as batches B-01โ€ฆB-06.

โ‘ข Dashboard โ€” forward-predictiveness + supporting probes

PageWhat it is
๐Ÿ”ฎ Forward โ€” Orthogonal TomorrowThe forward-predictiveness probe (persistence 94โ€“95%, flip-AUC, FDR sets, ICP 0/92).
Probes hub ยท The Gate ยท Orthogonal ยท Parameterless ยท AgnosticThe three-axis probe family + the promotion gate.
Three-Axis Formalization ยท Registry Robustness SOTA ยท Parameterless Discovery 100 ยท Rotation Operational CampaignSupporting probes: axis formalization, registry robustness, parameterless discovery, rotation operationalization.
This packet (pending deploy): Declaration-Engine ADR (this page) ยท Feature-Expiry Robustness AssessmentThe two new pages under review โ€” sibling links resolve once deployed alongside the others.

โ‘ฃ Audit folders โ€” the append-only forensic record (GitHub main)

FolderWhat it holds
2026-05-08 ยท orthogonal-features-btcusd64 candidate orthogonal features for BTCUSD, two-sided provenance.
2026-05-16 ยท orthogonality-measurement-instruments~95 SOTA measurement instruments across 3 tiers ร— 8 categories (FOSS status per metric).
2026-05-26 ยท forward-orthogonality-prediction this ADR lives hereThe forward-predictiveness methodology + grounded campaign + the G12 live monitor. This ADR + the robustness assessment are in this folder.
2026-06-09 ยท three-axis-probe-formalizationThe orthogonal / parameterless / agnostic discipline turned into runnable probes + a promotion gate.
2026-06-17 ยท crypto-forex-parameterless-discoveryParameterless orthogonal candidate discovery, crypto + forex.
2026-06-26 ยท rotation-probe-meta-evaluation-auditMeta-evaluation of the rotation probe.
2026-06-30 ยท chatterjee-xi-threshold-calibrationThe Chatterjee-ฮพ threshold calibration โ€” the banlist method the tri-tier scheme inherits.
2026-07-02 ยท batch6-chatterjee-xi-filterBatch-6 ฮพ filter (4 PASS ยท 4 WATCH ยท 0 BAN); the corrected worst-cell recipe.
2026-07-02 ยท matrix-admission-paradox-loop the campaign23 certified matrices ยท 45 kills ยท the two laws ยท the ADRs ยท LEDGER/BONEYARD/FRONTIER + 45 exam scripts & result JSONs.
2026-07-04 ยท matrix-harvest-status-grounding-loopCandidate harvesting โ†’ the FRONTIER-FEED B-01โ€ฆB-06 batches; the 300+ CANDIDATE-CATALOG.
2026-07-07 ยท feature-realness-usefulnessFeature realness/usefulness census (the REAL ร— DENSE axis, orthogonal to the statuses).
2026-07-08 ยท crypto-cost-realism-labelsCrypto cost-realism labels ADR (your 2026-07-07 handoff).

โ‘ค Telemetry artifacts โ€” the raw evidence

ArtifactWhat it is
LEDGER.md117 append-only iteration verdicts โ€” every ADMIT / EXCLUDE with its evidence.
BONEYARD.md60 do-not-re-propose rows โ€” every kill + its re-entry clause.
FRONTIER.mdBatch closure map B-01โ€ฆB-06 (the graders' provenance).
M0 pre-registration ยท GATE-LADDER-M0-M5The frozen rulebook (hash f9594240) + the 6-gate spec that certified the matrices.
Matrix-expiry ADRThe separate matrix-certificate expiry line โ€” context, distinct from this feature-declaration ADR.
CANDIDATE-CATALOG.md ยท FRONTIER-FEED.mdThe harvest telemetry: 300+ catalogued candidates + the B-01โ€ฆB-06 handoff batches.
PHASE-3-PROBE-DESIGN ยท STATUS-GROUNDING ยท GROUNDED-SUMMARYThe forward-side design: the G0โ€“G12 gate ladder, every threshold's trust basis, the "orthogonal tomorrow" answer.
Feature-Expiry Robustness Assessment ยท this ADR pageThis packet's two pages (the ADR you're reading + the robustness assessment). The markdown SSoT for both ships in the PR.
manifest.jsonPer-iteration structured telemetry (layman summary + verdict per iteration) behind the campaign dashboard.

PROPOSED ยท nothing ratified, nothing deployed, no Telegram sent ยท append-only. SSoT: findings/evolution/audits/2026-05-26-forward-orthogonality-prediction/ADR-2026-07-15-LIVE-ORTHOGONALITY-DECLARATION-ENGINE.md. Sources: tri-tier proposal (LEDGER row 105) ยท the 23 grounded matrices ยท the two laws (rows 99โ€“117) ยท COMBINED-REGIME-SLICING.md ยท notify/telegram.py ยท robustness assessment 2026-07-15.