โ€บNavigation

iteration 02 ยท card 78 S2+S3 kernel ยท batch-6 implementation ยท 2026-07-02

Card 78 โ€” Rust kernel proven bit-exact against the oracle S3 DONE ยท โ‰ค1e-9 GREEN

← campaign board ยท previous: iter 01 (the answer key)

In plain language โ€” what happened this iteration

Last iteration built the answer key; this iteration built the engine part and proved it gives the same answers. The production pipeline computes everything in Rust (a fast systems language); the trusted reference from iteration 01 was Python. Today the Rust version of Hoeffding's ฮฆยฒ โ€” the "is move size related to bar duration, in any shape?" measurement โ€” was hand-written and then made to reproduce all 19,602 frozen reference answers (9,801 per market, Bitcoin and Ethereum) to within 0.000000001. It passed on the first full run.

Why so paranoid about "the same answers"? This feature is built on rankings (which bar had the 1st, 2nd, 3rdโ€ฆ biggest move). Rankings are brittle: if two moves are nearly identical and two systems round the 16th decimal differently, the ranking flips and the final number jumps visibly. A past feature (Bartels, incident I2) failed exactly this way. The fix, reused here: both systems rank a quantity computed by a single division โ€” an operation the floating-point standard guarantees is bit-identical everywhere โ€” so a ranking flip is impossible by construction.

The kernel was also cross-examined three independent ways: (1) a deliberately slow, dumb counting version written in Rust re-derives every number from scratch and must agree โ€” a bookkeeping bug in the fast version can't confirm itself; (2) a property battery checks known truths (perfectly linked inputs score near 1, the score never leaves its range, scaling all prices by 4 changes nothing โ€” bit for bit); (3) the Python generator already double-checked itself with exact whole-number arithmetic in iteration 01.

One genuine discovery: the textbook says ฮฆยฒ lives in [0, 1] โ€” true for the idealized continuous formula. Our finite 199-pair grid version can reach 1.0000631 (a hair over 1) but ONLY under mathematically perfect lock-step dependence, which real markets never produce. This was found because a property test honestly failed; it's now precisely documented in three places rather than papered over.

Plumbing that came along: the engine's "memory" (checkpoint) โ€” what lets it resume seamlessly after a restart โ€” now also carries the rolling window of bar durations this feature needs. Old checkpoints from before this change still load fine: the feature simply warms up for 200 bars while every existing feature continues uninterrupted.

The one red light, and why it's good news: a watchdog test ("schema contract") now fails on purpose. It noticed the new measurement exists in the engine but not yet in the database blueprint โ€” the exact class of silent mistake that once shipped 8 features whose values were computed and then thrown away (incident #556). The watchdog stays red until the next iteration wires the database side, which is precisely its job.

Grounded this iteration

S2 (xcheck) + S3 (kernel) for bar_hoeffding_phi_squared_midreturn_duration, plus the processor core of S4: kernel + bar_close_ordinal_ranks (stable-tie, numpy-mergesort-identical) in bar_close_features.rs; 8-test battery green (brute-force independent leg, monotoneโ†’~1, bounds+determinism, bit-exact scale/monotone-transform invariance, NaN policy, no-Inf, rank stability); โ‰ค1e-9 oracle gate GREEN on BTC+ETH via the new two-substrate walk seam driving the exact production windowing helper; durations rolling window + checkpoint persistence (serde-default back-compat + self-healing restore) + PyO3 checkpoint dict round-trip; workspace compiles clean; neighboring suites 78/79 green (the 1 red = the intended schema-contract tripwire).

stage3/16
oracle gateโ‰ค1e-9 GREEN ร—2
kernel tests8/8
neighbor suites78/79
tripwireRED (by design)
compared bars19,602

Decisions made (for operator review at PR time)

#DecisionWhy
1First two-substrate kernel: (closes, durations_us) slices; durations window is a sibling VecDeque in the processorCard 78's formula needs bar duration; the close-only window API can't express it
2Old checkpoints restore with an EMPTY durations window (self-heals in โ‰ค200 bars); 12 existing features unaffectedBackward compatibility without a warm-up band for anything already shipped
3Dedicated walk seam bar_hoeffding_โ€ฆ_walk(closes, durs); the 12-column walk tuple untouchedAvoids touching 12 existing selectors; hoeffding column in the default walk is not consumed by any gate
4Range documented as "effectively [0,1]" with the exact finite-m bound (1.0000631 at m=199)Property test honestly caught the textbook-vs-grid discrepancy; documented, not clamped (probe parity)

Next fire picks up here

S4 write surfaces (the #556 guard): feature_manifest.toml entry (group=bar_close) โ†’ schema.sql column + column_comments.py โ†’ constants.py BAR_CLOSE_FEATURE_COLUMNS โ†’ arrow_export.rs (schema/builder/append/finish) โ†’ helpers.rs dict serializer โ†’ clickhouse_writer/row.rs (CORE_COLUMNS + struct + from_completed_bar) โ†’ dead_letter.rs โ†’ bump tripwires 85โ†’86 (Arrow) and 90โ†’91 (CORE_COLUMNS, + the Python mirror in test_dead_letter_roundtrip.py) โ†’ schema-contract + completeness tests back to green. Then S5 causality / S6 determinism (robustness harness registration). Worktree: ~/eon/odb-worktrees/batch6-78-hoeffding, all work uncommitted on disk, ledger is truth.