iteration 17 · 2026-07-23 · usefulness axis · laptop-drives-bigblack
Permutation importance: train a model, shuffle one feature out-of-sample, and measure how much accuracy it costs. The model-based usefulness check — and the one that survives the substitution effect via ONC clustering. All five §7 gates pass, robust across 5 seeds.
readonly=2. 5c/5G/no-swap capped, single-thread BLAS. Wall ~44 s.| Gate (§7 row 7) | Result (seed range) | Target | |
|---|---|---|---|
| Admit demo | (mean−std) 0.0006–0.0034 > 0 AND > null-thr | (mean−std)>0 & >null-95 | PASS |
| Power @ IC=.03 | 0.825–0.883 | ≥ .8 | PASS |
| Both nulls FPR ≤ .05 | block-perm 0.000–0.042, permuted-target 0.008–0.033 | ≤ .05 | PASS |
| ONC co-cluster ≥95% & recover ≥.8× | co-cluster 1.00, recovery 0.96–1.07× | ≥.95 & ≥.8× | PASS |
| Harden — pin base + interaction blind | interaction missed every seed (importance ≈ 0) | FNR=1 | PASS |
A driver and its exact clone each look weak individually: permute one and the model leans on the other, so neither gets credit. ONC (silhouette-selected KMeans on the correlation distance, AFML ch.6) co-clusters the driver+clone 100% of the time, and clustered-MDA — permuting the whole cluster together — recovers 0.96–1.07× the lone-driver importance. The redundant pair is scored at its true combined worth, not halved.
At the nominal §7 "null-95pct" threshold, the null FPR sits at ≈.05 ± noise (seed 303's block-perm null hit 0.058). A conservative 97th-percentile empirical-null threshold controls both nulls robustly (≤0.042) while power stays ≥0.825 — the signal-to-null gap is ~10× (importance 0.002 vs null-threshold 0.0002), so the higher bar barely costs power. This is the same conservative-control pattern grounded in #4 / #5 / #10, not construction-shopping.