โ€บNavigation

Iteration 8 โ€” Permutation k-of-N Passes Its Own Null ยท First FRAGILEโ†”CONDITIONAL Split Measured INSTRUMENT INCLUDE-IF FIRST k DISTRIBUTION

What happened in this iteration, in plain words: version 2 of the per-regime counting instrument โ€” the one where every feature's regime-break score is judged against that same feature's own shuffled-regime distribution โ€” passed the exact safety test that killed version 1: with regime structure deliberately destroyed, it now correctly reads "holds almost everywhere" (9.35 of 10, where a calibrated truth-teller should read ~9.5, and version 1 read a broken 1.91). With a trustworthy instrument in hand, the campaign gets its first-ever fine-grained answer to "who survives regime changes, and how often": out of 26 features with real historical data, a clean split appears โ€” a six-member club (volume, buy volume, sell volume, trade counts, bar duration โ€” the market's activity dial) holds its behavior in 9 of 10 regimes, while 15 features hold in 2 or fewer. FRAGILE vs CONDITIONAL is measurable for the first time. Two honesty notes shipped alongside: (1) 75 of the ~105 columns turn out to be genuinely empty in the historical windows (so iteration 7's "100 features" included ~74 phantom gradings โ€” its per-feature numbers are hereby corrected on the record); (2) iteration 7's kill stands, but its mechanism story was part artifact โ€” a NaN-handling defect in the inherited code was a co-cause, and the clean recheck shows the textbook test still fails calibration (7.81, needs โ‰ฅ9) while version 2 passes on identical data.

Preflight (A0)

2026-07-03 11:19 UTC โ€” load1 3.94 / 32 cores (โ‰ค24) ยท no competing nasimubd jobs ยท 21 GiB available, si/soโ‰ˆ0 ยท clickhouse-server active ยท sidecar active + /health healthy โ†’ ALL PASS. Three capped read-only runs totaling <2 min (main 0.35 min), readonly=2, 5c/5G scope, pre-authorized (03b), zero generated values (03c โ€” permutations of real rows only).

The instrument and its acceptance

DESIGN (successor to the killed parametric variant):
  per (feature, regime): Chow statistic (env vs rest) judged against the
  permutation distribution of the SAME statistic under 1,000 random
  re-assignments of REAL rows (sizes preserved) โ€” matched null BY CONSTRUCTION,
  no distributional assumptions. Global per-feature standardization (preserves
  regime scale breaks; difference from the old per-env wiring recorded).
  Vectorized via sufficient statistics: full run 0.35 min inside the cap.

ACCEPTANCE (must pass before the measurement is trusted):
  A1  destroyed-env null   mean k = 9.35 of 10   need โ‰ฅ 9.0   โœ“ PASS
      (version 1 read 1.91 here โ€” the exact failure that killed it)
  A2  real < null          3.92 < 9.35                        โœ“ PASS

THE FIRST k DISTRIBUTION (26 real-data features, crypto BTCUSDT@250):
  k =  1: โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 10 features        โ† FRAGILE-material
  k =  2: โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 5
  k =  4: โ–ˆ 1
  k =  5: โ–ˆโ–ˆ 2
  k =  7: โ–ˆโ–ˆ 2
  k =  9: โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 6                      โ† CONDITIONAL-material:
          volume ยท buy_volume ยท sell_volume ยท individual_trade_count ยท
          agg_record_count ยท duration_us   (the market's ACTIVITY/SIZE family)

Honesty notes (corrections on the record)

#Note
1Feature universe corrected: 75 of ~105 numeric columns are ALL-NULL in the 2018โ€“2024 window subsamples (microstructure/plugin columns unpopulated there โ€” consistent with the committed "24 crypto effective columns" note). Iteration 7's "100 features graded" therefore included ~74 NaN-phantom gradings; its per-feature k values are unreliable (LEDGER row 12).
2Iteration 7 mechanism refined, verdict unchanged: the inherited v3-style fill leaves all-NaN columns as NaN โ†’ NaN test statistics counted as rejections (a third silent-failure mode in the v3 lineage, now on record). Clean-pool recheck: the parametric null improves 1.91 โ†’ 7.81 but STILL fails the โ‰ฅ9 bar while the permutation successor reads 9.35 on identical data/seed โ†’ residual parametric size distortion under regime mixture is real. Kill stands; mechanism = NaN defect (major co-cause) + mixture distortion (residual, confirmed). LEDGER row 12 supersedes row 9's mechanism sentence only.

Technical record

ItemValue
Ledger rows11 (ADMISSION โ†’ INCLUDE-IF, permutation k-of-N; gates M0โ€“M3 walked, M4/M5โ€ฒ legs pending) ยท 12 (NOTE โ€” mechanism correction, supersedes 9's mechanism only)
INCLUDE-IF conditions(i) M4 incremental-value + M5โ€ฒ real-data power legs before Loop-2 consumes k for statuses ยท (ii) block-permutation refinement (within-env autocorrelation) ยท (iii) Pโ‰ฅ10k perms if BH-gated declarations needed (current min p โ‰ˆ 0.001)
Conventionsฮฑ=0.05 (G3) ยท BH q=0.10 (G11) ยท P=1000 (resolution recorded) ยท seeds STRUCTURAL ยท zero new magic numbers
Numberspool 20,604 rows ร— 26 features ยท A1 9.35 (pass โ‰ฅ9) ยท A2 3.92<9.35 ยท mean k_BH 3.88 ยท wall 0.35 min main + recheck
Harnessesk_of_n_permutation_calibrated.py (sufficient-statistics vectorization) ยท k_of_n_mechanism_recheck.py (Q1 universe + Q2 clean-pool parametric)
Compute / prod impactthree sub-2-min capped runs ยท ~30 readonly=2 SELECTs ยท zero writes outside audit folder + dashboard

โ–ถ Next iteration

Iteration 8 ยท 2026-07-03 ยท Frontier #2 successor: instrument INCLUDE-IF'd, first k distribution, two corrections on record ยท capped read-only ยท zero synthetic data ยท append-only