2026-07-31 ยท โ Axis-2 evaluation ledger ยท PR #686 ยท commits cdc39da2, 44ff0d92 ยท LEDGER rows 0lโ0o
The gate due was SEAL′ at 750 dbps โ the smallest cell, per the frozen slicing order. It was stopped before touching a candidate, under rule 0: a defect in the machinery that evaluates candidates outranks everything, including STOP.
Two defects surfaced. Neither is subtle in hindsight; both had survived multiple reviews. The first was found by running the harness โ which, as it turns out, nobody had ever successfully done.
Measured on bigblack, in the campaign worktree the runner actually deploys to (/home/nasimubd/lab/2026-07-24-axis2-cascade-free-evaluation/repo):
Traceback (most recent call last):
File ".../evidence/seal_prime_controls.py", line 43, in <module>
import axis2_seal_kernel as K
ModuleNotFoundError: No module named 'axis2_seal_kernel'
REPO = EVAL.parents[4] is one level short. EVAL is <repo>/findings/evolution/audits/<campaign>/axis-2-cascade-free/evaluation, whose parents[4] is <repo>/findings โ not the repo root. Two consequences, both fatal:
| consumer | resolved to | exists? |
|---|---|---|
sys.path insert | <repo>/findings/findings/evolution/shared_data | no |
| C4 fixture paths | <repo>/findings/tests/fixtures/โฆ | no |
This answers a standing question rather than raising one. seal_prime_controls.py had never executed against data, and the reason was assumed to be an outstanding precondition. It was not. It was arithmetic โ the module crashed at import in the exact directory it deploys to.
Sibling ch_concurrency.py computes the same root correctly from a different depth, so there was no shared convention to inherit the error from. One file, counted wrong.
parents[4] โ parents[5]A positional index cannot be checked by reading. You have to count directory segments in your head and be right โ which is precisely what failed. The fix walks up for an ancestor carrying both pyproject.toml and findings/evolution/shared_data/ (both required, so a stray pyproject.toml in a parent cannot satisfy it), and raises RUN VOID naming the search root instead of surfacing as a ModuleNotFoundError four frames from the cause.
run_seal_prime.sh's header listed the 2.5 GiB per-query ceiling among the preconditions it had closed, quoting a cell count and a peak-RSS figure. Against that:
| record | says |
|---|---|
AXIS2-ENVELOPE.json _validation_status | PENDING โ sweep MUST be re-run before SEAL′ fires |
| LEDGER row 0k | "Validation PENDING" |
| dashboard attention A2 (severity blocking) | "The 2.5 GiB per-query ceiling is unvalidated" |
And no artifact carrying those numbers exists โ not in the repo, and not on bigblack, where the campaign folder's evidence/ and results/ are both empty.
Whether the sweep was ever run is beside the point. Amendment 24a's own rule is that a result sitting only in /home/nasimubd/lab is an unbacked result; rule 0 says a gate you cannot say why it passed is a defect. A precondition marked closed with nothing to cite is that same failure wearing a different hat โ and it sat in the last thing an operator reads before firing, where a comment reads as clearance.
The header now states each precondition as what its SSoT records, and the ceiling became an actual runtime gate that asks AXIS2-ENVELOPE.json and refuses on PENDING โ with UNREADABLE treated as PENDING, because the one thing it must not do is fire because it could not find out.
$ bash run_seal_prime.sh 750
SEAL' โ 750 dbps โ campaign folder /home/nasimubd/lab/2026-07-24-axis2-cascade-free-evaluation
REFUSING TO FIRE โ the per-query memory ceiling is not validated.
AXIS2-ENVELOPE.json _validation_status: PENDING โ the 07-27 feasibility sweep MUST be re-run โฆ
โฆ
Run the sweep, commit its artifact, then set _validation_status.
EXIT=1
This is an expected fail-closed refusal, so it is not itself a defect โ it is the gate that was missing. Attention entry A2 now blocks by machinery rather than by memory.
Rule 0b is the clause a loop under time pressure skips. Five legs, every one red before its fix:
| leg | asserts | before | after |
|---|---|---|---|
| clean-interpreter start | the harness starts with PYTHONPATH stripped and cwd a tmpdir โ reproducing the bigblack traceback verbatim | RED | GREEN |
| repo-root constant | REPO carries pyproject.toml and SHARED exists | RED | GREEN |
| C4 invariant paths | the per-firing determinism assert is handed fixtures that exist | RED | GREEN |
| runner vs envelope | no validation claim while _validation_status is PENDING | RED | GREEN |
| reverse direction | once the status flips, a ceiling artifact must exist under evidence/ | SKIP | SKIP โ arms when PENDING clears |
The runner-vs-envelope leg is a deliberately blunt substring check. It caught the fix commit's own first-draft comment, which quoted the banned phrase while explaining why the phrase was banned. That is the behaviour wanted from it, so the comment was reworded rather than the test weakened.
| suite | result | hermetic |
|---|---|---|
tests/test_axis2_seal_controls.py | 13 passed, 1 skipped (was 9) | yes |
axis2:self-test | 12/12 | yes |
ch_concurrency_smoke_test.py โ live on bigblack | 9/9 | no |
The smoke test returned n = 88,149 for BTCUSDT@750 FINAL, which independently corroborates the distinct-close_time_us census frozen in amendment 30b. It also confirmed readonly=2 live on the connection, the limiter holding at max 2 in flight across 6 threads, and the guard refusing write / smuggle / heredoc-smuggle / egress.
| clause | what happened |
|---|---|
| a. STOP | gate not finished, no verdict written, no candidate ledger row appended |
| b. failing test first | 5 legs, all red before their fix; one reproduces the bigblack traceback verbatim |
| c. three suites green | above |
| d. test and fix together | cdc39da2, 44ff0d92 โ each carries its own test |
| e. quarantine | none required โ the ledger stood at 0 decided, so the defective versions produced no verdict to supersede |
| f. resume from the beginning | SEAL′@750 restarts next firing, not from where it stopped |
SEAL′ is now blocked on real work rather than on a broken tool: re-run the 2026-07-27 feasibility sweep at max_memory_usage = 2 684 354 560 against the worst cell this campaign touches, commit the artifact under evaluation/evidence/, then close _validation_status.
A Code 241 there is evidence, not a failure: the campaign serialises to one query at a time. The cap is not raised in either branch.
Eleven defects have now been found in this machinery before it produced a single verdict. Nine were found by adversarial agents, a smoke test, or careful reading. D10 is the first found by executing the thing โ and it was the one that made execution impossible in the first place.
That ordering is itself a small piece of evidence about where the remaining defects are likely to be: in the paths that only open once the harness gets far enough to reach them.
Amended later the same firing: the count above is stale. An adversarial audit dispatched during this firing returned 16 further findings, taking the standing count to 27. See the section below. The prediction in the paragraph above โ that the remaining defects live in the paths that only open once the harness gets far enough to reach them โ turned out to be exactly right, which is the least comforting way to be right.
An adversarial read-only audit ran in parallel with the repairs above. It executed the real kernel against the committed fixture rather than reasoning from the source, and returned 16 findings โ 5 of them blockers. They are queued in DEFECT-QUEUE.md rather than fixed here: rule 0 keeps the gate stopped until the machinery is sound, but it does not require one firing to fix everything.
run_c2 compares the ClopperโPearson upper 95% bound against FWER_MAX = 0.05 over 200 draws:
max hits that still passes: 3 of 200 (observed size <= 0.0150) P(hits <= 3 | true size = alpha = 0.05) = 0.00905 P(all 9 cells pass) = 4.07e-19
C2 is a correctly sized ฮฑ = 0.05 test by construction, so E[hits] = 10 and the gate demands โค 3. It also contradicts the sibling gate in the same file: choose_block_length accepts a block length only if its size lands inside binomial_size_band(300) = (0.0267, 0.0767). A size of ~0.05 sits inside that band and outside C2's pass region โ whenever calibration succeeds, C2 fails.
The harness turned ยง5's "FWER โค 0.05" into "size significantly below ฮฑ". The loop is deliberately not choosing the fix โ size-band versus one-sided exact binomial is a methodological choice against a frozen control, which is amendment territory. Raised as attention A7.
| id | defect |
|---|---|
| D13 | N_MIN.json's sha256 binding stores a basename for an artifact written outside the repo, so load_n_min can never resolve it โ F1 and F4 permanently unrunnable. The mechanism that binds the number to its evidence is guaranteed to reject its own output. |
| D14 | derived N_min is floored by nb! permutation granularity, not by power. Measured at |ic| = 1.000 โ a perfect signal โ p = 0.240 at nb=3 and p = 0.430 at nb=2, i.e. undetected. Nothing records that a floor was hit. |
| D15 | the envelope preflight is fail-OPEN: an unset SPIKE_CAMPAIGN_DIR silently skips all of E1โE5, with no field recording the skip, beneath a comment reading "Fail-closed preamble. Order matters." |
| D16 | the preflight that does run verifies the base 5 GiB envelope, not the campaign's 2.5 GiB โ the shim installs the base file and the runner exports AXIS2_ENVELOPE_JSON while preflight reads SPIKE_ENVELOPE_JSON. Repointing does not help: E1 is an equality tripwire, so 2.5 GiB would fail as DRIFT. AXIS2-ENVELOPE.json's claim that "E1 enforces this direction" is false. |
D16 compounds D11 above. The ceiling was not merely unvalidated โ the machinery could not have applied it. Two concurrent queries could have reached 10 GiB, the exact scenario the campaign envelope was written to prevent.
Recorded so the next firing does not re-litigate them: the look-ahead shift direction (correct), the known_positive entry-vs-exit leg claim (accurate against labels_schema.sql), the block_perm padding mask (no leak), the peak_rss_mib divisor (right for both platforms), every K./C./SI. call signature, and four more. An audit finding is a claim, not a defect, until it is reproduced.
axis-2-cascade-free/evaluation/LEDGER.md rows 0l (amendment 30d recorded), 0m (PR re-pinned to #686), 0n (D10), 0o (D11)PREREG.md as amended by 24a ยท 30a ยท 30b ยท 30c ยท 30dtelemetry.json, firing 1manifest.json, attention A5 / A6cdc39da2 (D10), 44ff0d92 (D11), both on PR #686