โ€บNavigation

Iteration 3 โ€” Q1 RESOLVED-AS-SCOPED โ€” empty result live ClickHouse real data

Read-only evaluation ยท live clickhouse-client --readonly=2 queries this firing ยท capped envelope (5 cores / 3 GiB / no-swap / idle-IO) ยท watchdog load1 9.53 โ‰ค 24 ยท matrix-evaluation-for-orthogonality loop ยท PR #543 ยท 2026-06-24

What changed this iteration (operator decision + a correction)

Operator decision (2026-06-24): scope stays crypto-only. Forex is intentionally out of scope, consistent with the locked pre-spec.md v2 ("Scope narrowed to crypto-only 'for now', operator directive 2026-05-30") and the probe code itself (orthogonality_probe.py: "CRYPTO-ONLY (forex removed from scope 2026-05-30)").

Therefore Q1 is no longer a blocker. It is re-graded BLOCKED โ†’ RESOLVED-AS-SCOPED: forex regime-invariance is not graded by design, not because of an unresolved problem. The forex ICP wiring defect (forex active-set columns are crypto-named, so the forex cell came back empty) is logged for if/when forex is reintroduced โ€” it is not on this campaign's critical path.

Q1 โ€” final answer (crypto-only)

Q1 asked: "Can features be graded on regime-invariance at all?" Read the result before the verdict word: on crypto the instrument runs, but the graded result is EMPTY โ€” 0 of 91 crypto features are regime-invariant across the 10 environments (Type-I-controlled ICP, ฮฑ=0.05; STABLE-empty). The question is therefore answerable on crypto and has been answered โ€” that is all "RESOLVED" means here; it does NOT mean an orthogonal-and-stable set was found (none was). Forex is resolved by descoping, not by measurement. There is no outstanding blocker for Q1.

Live real-data grounding (this firing queried ClickHouse directly)

Per the operator's standing instruction that the evaluation use real ClickHouse data, this iteration ran live read-only queries against the production cache on bigblack โ€” not a re-read of pre-computed files. Database opendeviationbar_cache, table open_deviation_bars, every query --readonly=2, each wrapped in the capped systemd scope.

Query 1 โ€” real scale & recency of the crypto substrate

systemd-run --user --scope -p CPUQuota=500% -p MemoryMax=3G -p MemorySwapMax=0 \
  -p CPUWeight=10 -p IOWeight=10 nice -n 19 ionice -c3 taskset -c 0-4 \
  clickhouse-client --readonly=2 --max_threads=4 --query "
    SELECT count(), min(open_time_us), max(close_time_us)
    FROM opendeviationbar_cache.open_deviation_bars
    WHERE symbol='BTCUSDT' AND threshold_decimal_bps=250
    SETTINGS max_execution_time=30, max_memory_usage=2000000000"
FieldReal value (live)Meaning
n_bars871,376BTCUSDT @ 250 dbps bars in the live cache
first bar (UTC)2018-01-16 00:00:00history depth
last bar (UTC)2026-06-24 10:11:24most recent committed bar is from today โ€” recency confirmed by this direct read (does not by itself prove the sidecar is currently running)

Query 2 โ€” real redundancy / orthogonality on the most recent 50,000 bars

WITH recent AS (
  SELECT open, high, low, vwap, aggression_ratio, bar_dispersion_entropy,
         bar_katz_fd, bar_petrosian_fd
  FROM opendeviationbar_cache.open_deviation_bars
  WHERE symbol='BTCUSDT' AND threshold_decimal_bps=250
  ORDER BY close_time_us DESC LIMIT 50000)
SELECT corr(open,vwap), corr(high,low),
       abs(corr(bar_katz_fd, bar_petrosian_fd)),
       abs(corr(aggression_ratio, bar_dispersion_entropy)),
       abs(corr(aggression_ratio, bar_katz_fd)),
       abs(corr(bar_dispersion_entropy, bar_petrosian_fd))
FROM recent
SETTINGS max_execution_time=30, max_memory_usage=2500000000  -- --readonly=2, capped
Pair (real columns)|Pearson| on 50k live barsReading
open vs vwap0.999976price levels co-move โ†’ โ‰ˆ1 (metric-calibration anchor โ€” shows what near-maximal correlation looks like; not feature redundancy)
high vs low0.999975same levels artifact (co-moving prices) โ€” not feature redundancy
bar_katz_fd vs bar_petrosian_fd0.7923two fractal-dimension estimators: correlated, not duplicate
bar_dispersion_entropy vs bar_petrosian_fd0.7941complexity family: moderately correlated
aggression_ratio vs bar_dispersion_entropy0.2460order-flow vs complexity: largely orthogonal today
aggression_ratio vs bar_katz_fd0.2242order-flow vs complexity: largely orthogonal today
Layman's reading: on the live data, none of the four stamped-feature pairs measured is a near-copy of another โ€” the order-flow feature (aggression_ratio) shares only ~22โ€“25% linear co-movement with the bar-shape/complexity features, and the two complexity pairs measured top out at ~0.79. (Only 4 of the 6 unique pairs were computed โ€” bar_katz_fdร—bar_dispersion_entropy and aggression_ratioร—bar_petrosian_fd were not โ€” so the statement is scoped to the four measured.) The price columns correlate at โ‰ˆ1.0 only because they all track the same BTC price level โ€” which is exactly why the campaign measures orthogonality on derived/ranked series, not raw prices.

Trust basis โ€” every number above

NumberTrust basisWhat it is โ€” and is NOT
871,376 bars; span 2018-01-16 โ†’ 2026-06-24Direct read of live production data (read-only).Ground-truth scale/recency. Not an estimate.
price-level corr โ‰ˆ 1 (measured 0.999976, 0.999975)Metric-calibration anchor (co-moving non-stationary level columns โ†’ โ‰ˆ1; measured values corr(open,vwap)=0.9999758, corr(high,low)=0.9999754 โ€” near-1 from level co-movement, not exact column identity).Confirms what near-maximal correlation looks like for the metric. NOT evidence of feature redundancy โ€” it is a levels artifact.
stamped-four |Pearson| 0.22โ€“0.79Direct real-data computation on the most recent 50k bars.A live Pearson, pairwise sanity anchor. It is NOT the campaign's Spearman max|ฯ|-vs-active-set metric, and NOT the ICP regime-invariance test. It is descriptive (no out-of-sample / no regime stratification), so no leakage gate applies and it is promoted to no gate. The 0.85 WATCH / 0.95 BAN lines are the Spearman max|ฯ| policy thresholds and are deliberately NOT applied to these descriptive Pearson values (they are mentioned here only to pre-empt that misread).
ICP 0/91 (crypto regime-invariance)Type-I-controlled ICP (ฮฑ=0.05); committed + replicated. 0/91 = the deduplicated 91-feature panel (full_grid_aggregate.json); the SPIKE-2026-06-16 replication reports the same finding as 0/92 on its 92-slot panel. Both = STABLE-empty. Leakage gate PASS.The actual Q1 answer. The two live queries above corroborate that the data and features are real and current; they do not re-run ICP itself.

Data provenance & correction (so nothing here misleads)

On ClickHouse access โ€” correcting earlier wording. This loop does have read-only ClickHouse access (the contract grants clickhouse-client --readonly=2 and the read-only probe orthogonality_probe.py). Iterations 1 and 2 did not run live queries because those questions were answerable from committed result files โ€” icp_v3_cross_regime.json and full_grid_aggregate.json โ€” which were themselves produced by earlier capped, read-only ClickHouse runs (the FULL-GRID and SPIKE-2026-06-16 jobs) and are checked into the PR branch.

Their original wording implied access was lacking; that was imprecise and has been corrected in place to make clear that no query was needed this firing โ€” the answers were synthesised from committed artifacts. Erratum banners were added to iterations 1 and 2 to that effect; their findings are unchanged. From this iteration forward, numbers are grounded in live read-only ClickHouse queries, shown verbatim, per the operator's standing instruction.

Read-only & resource-cap proof (this firing)

GuardEvidence
ClickHouse READ-ONLY (R1)Every query ran with --readonly=2 (writes are rejected by the server). 3 read-only SELECTs this firing: (1) scale/recency [Query 1 above], (2) a column-existence check confirming the feature columns exist [not reproduced here], (3) the correlations [Query 2 above]. 0 writes.
Watchdog precheck (R2)load1 = 9.53 โ‰ค 24 before any query.
Resource cap (R2)Each client wrapped in systemd-run --user --scope: CPUQuota=500% (5 cores), MemoryMax=3 GiB, MemorySwapMax=0, nice -n 19 ionice -c3 taskset -c 0-4 (idle IO, 5-core affinity). Server side bounded by max_execution_time=30 and max_memory_usage = 2.0 GB on the count/scale query, 2.5 GB on the correlation query.
Leakage discipline (R7)The live correlations are descriptive (no OOS, no regime split) โ†’ leakage_guard.py not applicable; nothing promoted to a gate. The ICP 0/91 inherits its committed leakage-PASS basis.
Write surfaces (R3)Only this dashboard folder + the PR #543 body. No source/code edits. Forex harness fix deferred to operator (out of scope by decision).

What's next

SSoT: live opendeviationbar_cache.open_deviation_bars (read-only) ยท pre-spec.md v2 (crypto-only) ยท orthogonality_probe.py ยท icp_v3_cross_regime.json + full_grid_aggregate.json (ICP 0/91) ยท SPIKE-2026-06-16-RECONFIRMATION.md ยท audit twin 2026-05-26-forward-orthogonality-prediction/ ยท read-only evaluator ยท do NOT merge PR #543