Worst-of-N basis, grid, and probe read-path

Verdict — plain English

What we set out to check

Whether the pass mark used to judge candidate features (0.50) means today what it meant when it was set. It does not, and the reason is not what anyone expected.

What we found

The pass mark was set by measuring one thing. The probe measures another.

The feature chosen to represent "definitely orthogonal" — kyle_lambda_proxy — was scored 0.494 against one specific partner. The pass mark was then placed at 0.50, just above it.

But the live probe does not score a feature against one partner. It scores it against 112 columns and keeps the worst. Under that rule the same feature reads 0.9996 — a near-perfect duplicate. Its worst partner is intra_kyle_lambda, essentially the same quantity computed a second way.

So the feature that defines the pass mark would fail it. Not on one cell — on all 60 we could read.

What is reassuring

• The original declaration was done correctly. We reproduced its number to within 0.0005, on the same worst cell it recorded. It was not sloppy work.
• The maths engine is fine. We suspected the estimator had drifted after being replaced. It has not — old and new agree to four decimal places. That worry is retired.
• Nothing was tuned to make this come out. No threshold moved. Every constant is untouched.

What went wrong, and why it kept going wrong

Four things drifted apart after the pass mark was set, and nothing was comparing them:

Only the last one really matters, and it is the one nobody had measured.

Two things are wrong with the instrument, today

1. Two cells cannot be read at all — they run out of memory. That is the probe's own production read path, so the live probe cannot read them either.
2. When a cell fails, it is dropped silently — and dropping a cell always improves the score, because the score is a worst-of. The code comments claim a guard against this. There isn't one.

Together those mean a bad day on the database can turn a fail into a pass, and nothing in the output would show it.

The legacy probe — separately, and better news

It uses Spearman correlation and h_norm. No Chatterjee ξ. No R². So essentially none of the above applies to it. We also verified its tie handling matches the textbook implementation exactly, which closes a question that had been open and unresearched.

It is usable and its numbers are believable. It has one measured bias (it counts "moves with price" as redundancy, inflating scores by up to +0.42), two things nobody has measured, and the same silent-cell-dropping hygiene problem.

It runs in 1.5–5 hours depending on parallelism, uses 420 MB, and a lab is built and verified. There is currently no valid legacy baseline at all — every previous result predates the fix that halved its data — so a run produces something that does not exist today.

Honest about our own mistakes

• Two of our own harnesses reported success having done nothing. Both were caught by guards we had added for exactly that reason, and both are recorded rather than quietly fixed.
• We deviated from our own frozen plan by using the wrong estimator on one leg. That deviation is appended to the frozen document, not edited into it.
• Two of five predictions were badly wrong — one by a factor of twenty, and one reversed direction entirely. The reversal is the most important result here.

What we are not saying

We are not saying the pass mark is wrong and should move. We are saying it was calibrated against a different measurement than the one it now governs, and by how much. Choosing what to do about that is a decision, not a bug fix, and it needs pre-registration and sign-off.