Before spending compute on roughly 230 research ideas, someone checked which ones the data on hand can actually support โ then measured the check itself and found that 41 of its 69 confident "impossible" verdicts were wrong, leaving the missing order book as the only genuine data gap.
Lifecycle, not result. This says where the audit sits in its process โ never whether what it found was good.
**Status:** **COMPLETE (offline gate shipped) ยท PARTIALLY PROBED (present-but-NULL).**
2026-08-17. Derived from folder evidence; the adversarial challenge pass CORRECTED the first reading. The quote exists verbatim at CLAUDE.md:4 โ but it is the proposal's own worst witness: the half it cites in full reads "**COMPLETE (offline gate shipped) ยท PARTIALLY PROBED (present-but-NULL).**" DONE was inferred from the first clause while the second clause says the measurement is incomplete. The folder's ยง9 is titled "Open items (not self-fixable)" โ self-declaring that the audit cannot advance
The audit's own conclusion, reproduced in full from the source below. Not a summary โ this is the document, rendered. Links inside it that point at unpublished files are shown as plain text rather than as links that would 404 here.
In one line: we now know which research ideas can actually be tested with the data we have, and we know it because we counted rather than because we read notes.
Before spending compute evaluating ~230 research candidates, check the cheap thing first: does the data each one needs actually exist in the slices we pinned?
Only one genuine data absence exists: the order book. We counted every row in the table โ hundreds of millions โ and the bid/ask columns hold zero values. Not sparse. Zero. So ~20 candidates needing order-book depth are properly blocked until someone acquires that feed. This is the single claim the measurement confirmed.
Everything else that looked like a wall was either a stale note or a one-line code choice.
The data we do have is healthy. Twenty-eight slices sampled: no all-NULL columns, no all-NaN columns, no constant columns, no slice too thin to work with.
The recommended procedure โ a cheap read-only pass over the candidate list, marking what is uncomputable โ is exactly what was run first. It produced 69 confident "impossible" verdicts.
Then we measured, and 41 of those 69 were wrong.
Every wrong answer was stated confidently, with a citation, in a machine-readable file that a launcher would have obeyed. Had we stopped at the paperwork stage and shipped it, we would have written off 41 perfectly testable ideas and felt organised about it.
So the honest reading: the paperwork pass is a hypothesis generator, not a finding. It is still worth running โ it is what told us where to point the measurement โ but its output must not be trusted until measured. That is the inverse of how the source checklist frames it.
A smaller version of the same lesson appeared inside the build. The name-matching logic silently attached the wrong data requirement to a candidate because one name is a substring of another. Plausible shortcut, confident wrong answer. It now refuses to guess.
petrosian_fd failure happened: real inputs in, nonsense out, nothing raised.Before, this knowledge lived as prose scattered across three documents that disagreed, and no program read any of it. Now it is one file a program reads; anything expensive refuses to start on a blocked candidate; a negative control deliberately breaks the refusal to prove it refuses; and a drift guard fails loudly if the sources change without a rebuild.
The ledger itself was never touched โ its append-only hash chain is intact. Substrate verdicts are derived data and live beside it, not inside it.
Every row pairs a claim with the file it came from and the verbatim text in that file. The sources sit above the deploy root, so the quote is embedded and the path is printed as text rather than linked โ a link would resolve on a laptop and 404 here.
| Claim | Evidence |
|---|---|
| The cheap offline desk-check produced 69 confident "impossible" verdicts, and measurement showed 41 of them were wrong. MEASURED 69 offline blocks โ 49 after measurement; computable 55 โ 75; 41 blocks overturned | **Then we measured, and 41 of those 69 were wrong.** findings/evolution/audits/2026-08-14-substrate-computability-audit/verdict.md |
| The register's committed counts match those figures: 232 entries across two cohorts, 75 computable and 49 blocked on proven substrate. CONFIRMED total 232 entries; computable 75; blocked 157 of which 49 proven-substrate and 108 unknown-requirement; by class L2-REQUIRED 20, CROSS-ASSET-NOT-IN-SLICE 18, UNDECLARED-REQUIREMENT 105, UNPARSED 3, EXTERNAL-METADATA 4, ABSENT-FROM-SLICE 3, MULTI-VENUE 2, DERIVATIVES 1, METAORDER-LABELS 1; cohorts axis2-catalog 122 and ledger 110 | "blocked_proven_substrate": 49, findings/evolution/shared_data/substrate_register.json |
| The one thing the measurement confirmed is that the quote columns hold no data at all โ not sparse, zero. CONFIRMED 0 non-NULL rows table-wide across spread_open, spread_high, spread_low, spread_close, spread_mean, spread_variance and tick_count_with_quotes | "spread_columns_ever_populated": 0, findings/evolution/shared_data/substrate_register.json |
| Columns marked "dormant" in the code, which reads like empty, in fact hold hundreds of millions of values. REFUTED lookback_hurst 151,671,941; lookback_permutation_entropy 151,671,963; intra_hurst 48,185,745; intra_permutation_entropy 50,014,672; is_orphan 300,041,943 non-NULL rows | "lookback_hurst": 151671941, findings/evolution/shared_data/evidence/substrate-null-probe-2026-08-14.json |
| A frozen pre-registration's premise that only Bitcoin carries the cost labels was true when written and has quietly stopped being true โ twenty-one symbols carry them. REFUTED 21 distinct symbols across 75 symbolรthreshold rows carry label_fwd_3s_roundtrip_cost_bps; filed as D45 against frozen PREREG.md:29 R7 | "symbols_with_cost_labels": 21, findings/evolution/shared_data/substrate_register.json |
| The block asserting that the pinned slice cannot tell you how long a bar took is simply wrong: consecutive timestamps give it, with no malformed values. REFUTED BTCUSDT at 4 thresholds, 80,000 deltas each: 80,000/80,000 positive, 0 non-positive; median 68,986,315.5 ยตs (69.0 s) at 100 dbps rising to 516,478,000 ยตs (516.5 s) at 750 dbps | | Bar duration not derivable | **REFUTED** โ 80,000/80,000 positive `close_time_us` deltas, 0 non-positive, median 69.0 s @100 โ 516.5 s @750 | findings/evolution/audits/2026-08-14-substrate-computability-audit/CLAUDE.md |
| The data that does exist is healthy: no all-NULL, all-NaN, constant or too-thin columns anywhere in the sampled slices. CONFIRMED 28 cells sampled (4 axis-2 cells ร 8 columns + 24 prespec cells ร 8 columns), 80,000 rows per cell, 0 degenerate columns and no cell below the 200-bar lookback floor | "degenerate_columns_found": 0 findings/evolution/shared_data/substrate_register.json |
| Only the order-book blocks are genuine data absences; the cross-asset blocks are a one-line code choice, not missing data. CONFIRMED 20 genuine L2 blocks; 18 cross-asset blocks reclassified as slice scope | Only **20 L2 blocks are genuine data absences.** The 18 cross-asset blocks are a slice-scope choice (`fetch_cell()` hardcodes `BTCUSDT`), not missing data. findings/evolution/audits/2026-08-14-substrate-computability-audit/CLAUDE.md |
| The two candidate lists are two research programmes against two different pinned slices, so the planned merge was abandoned. CONFIRMED ledger 110 rows on prespec-441cell (6 symbols ร 21 windows); axis2-catalog 122 rows on axis2-btc-14col (BTCUSDT, 14 columns); overlap 4 names; the 31-row campaign registry reaches only 8 ledger rows | `candidate_ledger.jsonl` (110) and `CANDIDATE-CATALOG.md` (122) overlap by **4 names** (CorwinโSchultz, AbdiโRanaldo, GarmanโKlass, RogersโSatchell). They are **two research programmes against two different pinned slices**. findings/evolution/audits/2026-08-14-substrate-computability-audit/CLAUDE.md |
| The catalogue's existing "Spot?" column answers legality under the no-derivatives principle, not computability โ a conflation the audit separates. CONFIRMED 19 L2 rows added on legality grounds | The catalogue's `Spot?` column answers **legality** under principle #8 (no derivatives), not **computability**. Iter 6 makes it explicit โ it adds 19 `L2` rows arguing *"L2 order-book depth is spot-legal โฆ so within principle #8."* Legal, yes. Computable, no: no order-book capture exists. findings/evolution/audits/2026-08-14-substrate-computability-audit/CLAUDE.md |
| The append-only candidate ledger was never mutated; substrate verdicts are derived data living beside it, and its tamper-evident hash chain still verifies. CONFIRMED candidate_ledger.jsonl byte-unchanged; D6 chain verified | **The ledger was never mutated.** `candidate_ledger.jsonl` is byte-unchanged and its D6 hash chain verifies. findings/evolution/audits/2026-08-14-substrate-computability-audit/CLAUDE.md |
| The build itself surfaced a silent wrong-substrate defect: name matching by substring attached the wrong data requirement to a candidate, producing a confident incorrect "computable" verdict. CONFIRMED 1 mis-assignment found and pinned as a regression test | Substring matching assigned `close` to *"Bartels rank von-Neumann ratio"* because slug `vonneumann_ratio` is inside that string, while the correct slug `bartels_rank_vn_ratio` carries `logret`. Replaced with **token-subset** matching, which refuses both. Pinned as a regression test. findings/evolution/audits/2026-08-14-substrate-computability-audit/CLAUDE.md |
| Unknown is kept strictly apart from blocked, so the register cannot claim more knowledge than it has. CONFIRMED 105 undeclared and 3 unparsed requirements held separate from the 49 proven substrate blocks | "UNDECLARED-REQUIREMENT": 105, findings/evolution/shared_data/substrate_register.json |
| Both halves of the cross-repo checklist claim that prompted the audit are wrong โ the sibling repo's denominator and the accusation that this repo burned a campaign on uncomputable candidates. REFUTED mql5 counts.total = 41 but examined 74 untested in-scope candidates and blocked 38 plus 3 prior deferrals, its own dashboard saying "41 blocked of 115 in-scope" โ not 41 of 120; odb-py's axis-2 halted at 0/120 after 18 agent firings with zero candidate compute | | odb-py "burned a whole campaign" | **False.** The axis-2 campaign halted at 0/120 on **harness defects** (D10: `REPO = EVAL.parents[4]`, one level short โ `ModuleNotFoundError` at import), not on uncomputable candidates. Cost was **18 agent firings, zero candidate compute**. findings/evolution/audits/2026-08-14-substrate-computability-audit/CLAUDE.md |
| The ruling that was supposed to park uncomputable candidates existed but had nothing behind it โ the disposition it referenced was never authored, so no candidate was ever parked. CONFIRMED 0 candidates parked before this audit; the new preflight caller is the first to author F0 | What was missing is that **F0 was never authored**, so the ruling was a tripwire with nothing behind it and zero candidates were ever parked. findings/evolution/audits/2026-08-14-substrate-computability-audit/CLAUDE.md |
| The silent-failure risk is not fully closed: the probe covered shared column sets, not each candidate's own derived inputs, which is exactly how an earlier kernel failed. OPEN 4 axis-2 + 24 prespec cells probed; all register entries remain null_probe=UNPROBED | Entries still read null_probe=UNPROBED because that sweep covered the SHARED column sets, not every candidate's own derived inputs โ a kernel can still return all-NaN from live inputs, which is exactly how petrosian_fd failed. findings/evolution/shared_data/substrate_register.json |
| The measurement was read-only and its telemetry is hash-verified and drift-guarded, so a source edit cannot land without a rebuild. CONFIRMED readonly=2, all statements through ch_concurrency.ch(), 8-column ceiling; sha256 fb7eb3a9โฆ9c9236e verified byte-identical; registry:substrate-check-fresh byte-diffs a fresh build against the committed one | Probe telemetry `evidence/substrate-null-probe-2026-08-14.json`, sha256 `fb7eb3a9โฆ9c9236e`, verified byte-identical against the bigblack original before duplication. findings/evolution/audits/2026-08-14-substrate-computability-audit/CLAUDE.md |
| Three pre-existing defects were encountered and deliberately left unfixed, including a repo-wide validation gate that is red on main. CONFIRMED 3 pre-existing defects: 6 registry rows pointing at absent paths; spike:check-full unrunnable on a laptop (passes --hermetic, 33 modules, 0 offenders); spike-install does not forward its argument | - `registry:validate` **red on `main`** โ 6 rows point at `findings/dashboard/forex-candidates/โฆ`, absent on disk. Verified against the pristine file at HEAD. findings/evolution/audits/2026-08-14-substrate-computability-audit/CLAUDE.md |
| The audit's own verdict is that the cheap read-only pass is a hypothesis generator rather than a finding โ the inverse of how the source checklist frames it. ASSERTED 41 of 69 offline verdicts wrong, each stated with a citation in a machine-readable file a launcher would have obeyed | So the honest reading: **the paperwork pass is a hypothesis generator, not a finding.** It is still worth running โ it is what told us where to point the measurement โ but its output must not be trusted until measured. That is the inverse of how the source checklist frames it. findings/evolution/audits/2026-08-14-substrate-computability-audit/verdict.md |
Source of record: findings/evolution/audits/2026-08-14-substrate-computability-audit/ โ not published, so these are listed rather than linked.
| File | Role |
|---|---|
CLAUDE.md | Audit hub in the 11-section skeleton: scope, refutation of both halves of the prompting claim, the two-cohort correction, what was built, the two defects the build surfaced, the measurement table, verdict, open items, pre-existing defects, and provenance. |
verdict.md | Plain-English conclusion: the one confirmed data absence, the three refuted walls, the uncomfortable part (41 of 69 offline blocks wrong), what is still unknown, what changed structurally, and the three next steps. |
findings/dashboard/build_audits.py from findings/evolution/audits/2026-08-14-substrate-computability-audit/AUDIT_LEDGER.json โ never hand-edited. Each quote was verified to occur in the file named beside it when the ledger was written.