2026-07-31 · ← Axis-2 evaluation ledger · ← iter 1 · PR #686 · LEDGER rows 0q–0r
run_c2 compared the Clopper–Pearson upper 95% bound against 0.05 over 200 draws, which tests whether size is significantly below α rather than bounded by it. C2 is correctly sized by construction, so E[hits] = 10; the gate admitted at most 3.
P(hits <= 3 | size = 0.05) = 0.00905 per cell P(all nine cells pass) = 4.07e-19 for the control
A failed control hard-stops the loop, so the campaign as built could never have reached its first candidate. It also contradicted the sibling gate in the same file: choose_block_length accepts an L only if its size lands inside a band that contains 0.05. Whenever calibration succeeded, C2 failed.
Operator ruling: the size band, frozen as amendment 31a. On the over-rejection side the band and the one-sided alternative are identical at m=200 (both fail at ≥17 hits), so two-sidedness is free where §5's wording is directive. What it buys is the low end — where a degenerate null, one whose block permutation ran out of distinct orderings, would otherwise pass as “conservative”. That is not hypothetical: it is defect D14, still open.
Both reproduced against the pre-fix code before being touched, because an audit finding is a claim until it is.
D13: *** RUN VOID — N_MIN.json references result artifact
.../seal_prime_controls_result_750.json, which is absent
D24: load_n_min(250) -> 99999 (250's own artifact was never hashed)
D13 pointed at a path nothing ever wrote, so load_n_min failed forever and F1/F4 were permanently unrunnable — the mechanism whose whole purpose is binding a number to its evidence was guaranteed to reject its own output. D24 kept one hash at the top level and overwrote it each firing, so every threshold but the last was served unbound.
Binding is now per-threshold and path-relative to N_MIN.json; emission is gated on SEAL-PASS, not merely on the power stage having run. Given D12, every run to date would have been a SEAL-FAIL that still seeded the SSoT.
The check sat inside a truthiness test on SPIKE_CAMPAIGN_DIR. Unset it, and all of E1–E5 vanished — no shim on PATH, no readonly=2, no memory cap — and ~180,000 permutations were dispatched at a shared production ClickHouse anyway. Nothing printed; no field recorded the skip. The comment above it read “Fail-closed preamble.”
An unset variable is now the loudest failure rather than the quietest, which is the right ordering: not knowing what the caps are is strictly worse than knowing they are wrong. Set-but-wrong fails identically, or the variable is a rubber stamp.
The audit found the verification half. Reading the enforcement path turned up the half that actually mattered, and it is measurable, so it was measured — through the real shim, against the live server:
without SPIKE_ENVELOPE_JSON max_memory_usage = 5368709120 (5 GiB, the BASE) with SPIKE_ENVELOPE_JSON max_memory_usage = 2684354560 (2.5 GiB, the campaign) readonly = 2 and max_threads = 2 in both
clickhouse_client_shim.sh caps the server off SPIKE_ENVELOPE_JSON; the runner exported AXIS2_ENVELOPE_JSON and never that. So the halving was decorative and two concurrent queries could reach 10 GiB — exactly what the campaign envelope was written to prevent.
The verification half compounded it: preflight.py read the same unset variable and passed while confirming the untightened base. Repointing it was not a fix, because E1 is an equality tripwire — 2.5 GiB failed as DRIFT. AXIS2-ENVELOPE.json's own claim that “preflight.py E1 enforces this direction” was false, and is now corrected in-file.
| envelope | question | gate |
|---|---|---|
BASE SPIKE_BASE_ENVELOPE_JSON | has the frozen declaration drifted? | E1 — still equality |
ACTIVE SPIKE_ENVELOPE_JSON | is the server capped the way this run declares? | E3–E5 |
| and does it only tighten the base? | E1b — new |
E1 was deliberately not loosened to ≤. The probe-arbitration campaign states its measured results against those exact frozen numbers; weakening the tripwire would silently re-scope another campaign's evidence — precisely what this envelope's own _why_not_edit_the_shared_file exists to prevent. When no campaign envelope is declared, both paths resolve to the same file and behaviour is unchanged for every existing caller.
ACTIVE keeps the name SPIKE_ENVELOPE_JSON on purpose: that is the variable the shim reads. Verification and enforcement must resolve the same file, or the check is theatre — which is this defect in one sentence.
readonly is excluded from the ≤ comparison and requires exact equality. It is a mode, not a budget: readonly=0 is numerically smaller and strictly more dangerous, so a naive tightening rule over every key would wave the most important weakening straight through.
POSITIVE E0 E1 E1b E2 E3 E4 E5 L1 L2 L3 all PASS
E3: server observes max_memory_usage = 2684354560
NEGATIVE widened ceiling + changed readonly:
E1 PASS (base still matches frozen)
E1b FAIL max_memory_usage (10737418240, 5368709120, 'looser than base')
readonly (1, 2, 'must be identical — it is a mode')
A gate never observed to fail is not a gate. E1b was observed both ways.
| suite | result |
|---|---|
tests/test_axis2_seal_controls.py | 32 passed, 1 skipped (9 at iter 0) |
axis2:self-test | 12/12 |
ch_concurrency_smoke_test.py — live | 9/9 |
| preflight E0–E5 + L1–L3 — live | all PASS |
D14 is the next blocker: derived N_min is floored by nb! permutation granularity rather than by power. At |ic| = 1.000 — a perfect signal — the p-value floor is 0.43 at nb=2 and 0.24 at nb=3, so nothing is detectable. The planned fix is derived rather than chosen: refuse grid points below nb ≥ 4, so at least 1/α = 20 distinct orderings exist and p ≤ 0.05 is attainable at all — and record the refusal rather than reporting a floored number as if it were a measurement.
And a distinction worth keeping straight: the 2.5 GiB ceiling is now enforced. It is still not validated — nobody has shown the worst cell survives at that ceiling. Those are different claims, the second is attention A2, and run_seal_prime.sh still refuses to fire until a sweep artifact is committed.
27 defects found, 7 closed, 20 open. Still no candidate evaluated, and the harness has still never run a gate end to end.
Iter 1 predicted the remaining defects would live in the paths that only open once the harness gets far enough to reach them. D16's enforcement half fits that shape exactly: it was invisible until someone traced what the shim — not the checker — actually loads.