iteration 01 ยท card 78 S1 oracle ยท batch-6 implementation ยท 2026-07-02
Card 78 โ Hoeffding ฮฆยฒ oracle grounded S1 DONE
In plain language โ what happened this iteration
The mission of this campaign is to add four new "measurements" (features) to every price bar the pipeline produces. Think of a bar as a snapshot of trading activity; each feature is a number that describes some aspect of it โ this batch adds numbers that are deliberately different from everything already measured (they passed two independence audits), so a machine-learning model gets genuinely new information.
This iteration started feature #1 of 4: card 78, "Hoeffding's ฮฆยฒ (phi-squared)". In plain terms it asks: over the last 200 bars, is there any relationship โ of any shape โ between how much the price moved and how long the bar took to form? Classic correlation only spots straight-line relationships; Hoeffding's ฮฆยฒ spots any pattern (curved, U-shaped, threshold-like). It answers on a 0-to-1 scale: 0 = "move size and bar duration are unrelated", 1 = "one completely determines the other".
What was actually built today is the answer key, not yet the engine part. Before any feature is allowed into the production Rust engine, we first write a slow-but-trusted reference version in Python and freeze its answers on 10,000 real Bitcoin bars and 10,000 real Ethereum bars ("the oracle"). Later stages must reproduce these answers to within 0.000000001 โ that's how we prove the fast production code computes exactly the same mathematics as the reference.
Two safeguards worth explaining: (1) the reference computed every answer twice, via two completely different mathematical routes โ a grid-counting method and an exact whole-number formula โ and required both routes to agree on all 19,602 windows before writing anything (they did, to better than one part in a trillion). (2) The price inputs were passed through the same rounding step the production engine uses, so the reference and the engine literally see identical numbers โ a past incident (I3) taught us that even a rounding difference in the 9th decimal can flip a rank and corrupt this kind of statistic.
One surprise: the shared test dataset used by earlier features stores only closing prices, but this feature also needs each bar's duration. When we went back to the production database to fetch durations for the same bars, we found the database had been legitimately repaired since May (gap-healing), so those exact old bars no longer exist. Rather than corrupt the old dataset, card 78 got its own fresh, self-contained dataset (price + duration), fingerprinted (SHA-256) so any future tampering or drift is detected instantly.
Grounded this iteration
S1 (oracle) for bar_hoeffding_phi_squared_midreturn_duration: authored
tests/fixtures/regenerate_bar_close_dur_samples.sh (read-only ClickHouse SELECT, pinned spec) โ
committed-ready 2-col fixtures (close, duration_us) for BTC+ETH; authored
scripts/gen_hoeffding_oracle.py (clean-room numpy; I2 ratio-rank substrate; I3 FixedPoint
pre-quantization drift guard, drift=0; I4 fail-closed dual-leg: sweep-kernel grid construction vs
exact-integer closed-form, tolerance 1e-12, green on all 19,602 windows) โ oracle CSVs for BTC+ETH,
9,801 populated rows each, all values finite and inside [0,1].
Artifacts + integrity pins
| Artifact | sha256 (prefix) | Note |
|---|---|---|
BTCUSDT-bars-close-dur-sample-10k.csv | 26bfea6f887ea114โฆ | 10,000 real bars, close+duration_us, 250 dbps from 2024-01-01 |
ETHUSDT-bars-close-dur-sample-10k.csv | 1627ec312e502354โฆ | โฅ2-symbol gate (scale-bug detector) |
BTCUSDT-hoeffding-oracle-sample-10k.csv | 2ec3992da85820ccโฆ | 9,801 rows, mean ฮฆยฒ=0.0281 |
ETHUSDT-hoeffding-oracle-sample-10k.csv | 686761231929d20aโฆ | 9,801 rows, mean ฮฆยฒ=0.0213 |
Decisions made (for operator review at PR time)
| # | Decision | Why |
|---|---|---|
| 1 | New self-contained 2-col fixtures; close-only fixtures untouched | Production cache repaired since 2026-05-29 (45 scattered diff hunks vs committed closes) โ old bar set irreproducible; existing oracle SHAs must not churn |
| 2 | Returns computed WITHIN the 200-close window (m=199 pairs), pair = (close[t+1]/close[t], dur[t+1]) | Bartels precedent; probe convention pairs the return landing in bar i with bar i's duration |
| 3 | Rank the IEEE-exact gross-return ratio, not ln | I2 guard โ rank-identical to logret, immune to libm-vs-glibc near-tie rank flips |
| 4 | I4 second leg = exact-integer closed form (no R) | Canonical FOSS ref (copBasic::hoefCOP) is GPL-2 + R-side; registry records PORT / foss_oracle_validated=false; same convention as card 59's closed-form leg |
| 5 | Whole-window NaN policy (not probe's row-dropping) | bar_close family kernel contract; wiring maps NaNโNone/NULL |
Next fire picks up here
S2/S3 (xcheck + kernel): add compute_bar_hoeffding_phi_squared_midreturn_duration(closes, durations_us)
to bar_close_features.rs (first two-substrate kernel โ needs a durations window alongside
recent_bar_closes in the processor), ordinal stable ranks, property battery + in-Rust independent
leg + โค1e-9 oracle parity over BTC+ETH. Then S4 wire: 13th walk-tuple column (all 12 existing destructures
gain a field), push_and_compute_bar_close gains durations, manifest entry, and the #556 write
surfaces. Worktree: ~/eon/odb-worktrees/batch6-78-hoeffding (branch
feat/bar-orth/batch6-78-hoeffding off origin/main @ 4c0ee656). All work uncommitted on disk โ
ledger is the state truth.