CROSS-HOST KV REUSE ON A HYBRID, END TO END — 2026-09-10
=========================================================

WHAT IS UNDER TEST. Two host-resident halves that must BOTH hold for a hybrid
to serve a cached prefix:
  * the recurrent SAVEPOINT, DMA'd host->device at admission;
  * the PREFIX TIER, which keeps radix-evicted KV pages in page-locked host
    RAM and faults them back.
A hit proves the pair crossed the host boundary. Either half missing and
``savepoint_admission`` discards the KV match too and re-prefills from zero.

RIG
  206 GPU 1, container sphyb, /models/Qwen3.5-0.8B (GDN hybrid, 24 layers,
  6 paged / 18 recurrent), arbi-serve @ the served-not-matched counter fix,
  ARBI_SAVEPOINT_ENABLED=1 ARBI_PREFIX_TIER=1.
  Geometry off the boot: KV pool 6,860 pages x 256 = 1,756,160 tok;
  tier arena 1,365 slots x 3,145,728 B = 349,440 tok of reach;
  savepoint row 19,759,104 B, store retains 64; ring 16 (DMA STAGING).

1. NO POOL PRESSURE — savepoint + radix, tier never engages
   4 turns per conversation, own growing history.

     opening   cached% med   TTFT cold -> reuse      saved   discards
     2k             96.4     0.072 -> 0.033 s        53.7%      0
     8k             96.1     0.101 -> 0.042 s        58.8%      0
     32k            99.6     0.393 -> 0.036 s        90.8%      0
     128k           99.8     2.954 -> 0.090 s        96.9%      0
     32k shared x8  99.6     0.395 -> 0.036 s        90.8%      0

   Tier handouts 0 throughout: nothing was evicted, so this is the savepoint
   pair alone. 128k reuse is 32x faster than cold.

2. THE CONTROL — the same suite with ARBI_SAVEPOINT_ENABLED=0
     2k 0.0% | 8k 0.0% | 32k 0.0% | 128k 0.0% | shared 0.0%
     128k TTFT 2.932 -> 2.938 s (-0.2%): no saving at all.
     pair-rule losses 3/3/3/3/7 — one per reuse turn.
   The KV is resident and matched; the missing recurrent half discards it.

3. UNDER PRESSURE — 64 conversations x ~33k tok = 1.2x the pool
     pass  cached%  TTFT med  served      tier rst  evict   sp resume  pair loss
       0      0.0    0.540 s        0          0     2,650      0          0
       1      0.0    0.686 s  196,864      7,412     7,412      6         58
       2     99.3    0.153 s 1,446,144     7,038     7,425     44         17
   arena: 17,485 handouts / 16,120 spills / 1,365 live (saturated, cycling).

   This is the cross-host path doing the work: ~7,000 pages faulted back from
   host RAM per pass. Pass 1 is the transitional state and is worth reading —
   the tier restored 7,412 pages while only 6 savepoints existed yet, so 58
   matches died on the pair rule. By pass 2 both halves are present and the
   pair completes: 99.3% cached, 0.153 s against 0.540 s cold (72% saved).

4. THE CLIFF — 96 conversations x ~33k tok = 1.79x the pool
     pass  cached%  TTFT med  tier rst  evict
       0     93.0*   0.220 s     7,365  11,550     * contaminated: the first 64
       1      0.0    0.640 s         0  12,329       seeds carried over from (3)
       2      0.0    0.642 s         0  12,329
   arena: 53,693 handouts / 52,328 spills — thrashing.

   Tier restores fall to ZERO. The arithmetic:
       pool 1,756,160 + arena 349,440 = 2,105,600 tokens of TOTAL REACH
       64 x 33k = 2.1M ~= reach  -> 99.3%
       96 x 33k = 3.1M  > reach  ->   0.0%
   Past total reach a conversation's pages are gone from BOTH tiers before it
   returns. It is a cliff, not a slope.

VERDICT
  Within reach the cross-host method is excellent and the tier is what
  extends it: 99.3-99.8% cached, 72-97% of TTFT removed, 32x faster reuse at
  128k. The tier adds ~20% of reach on top of the pool (349k on 1.76M) and
  carries ~7,000 restores per pass once pressure exists. Without the savepoint
  half none of it serves — 0.0% at every length.

  The operating rule that follows: size the pool + tier against the WORKING
  SET, not against one conversation. Reach is pool + arena; beyond it the hit
  rate does not degrade gracefully, it goes to zero.

WHAT THIS DOES NOT ESTABLISH
  * One model, one card, TP1, one run per cell — no intervals. The 0.0-vs-99.3
    contrast is far outside any plausible noise; the 96.4-vs-99.8 spread
    across openings is not, and should not be read as an ordering.
  * Pass 0 of section 4 shares seeds with section 3 and is contaminated. It is
    printed rather than dropped because dropping it would hide that the cliff
    is about RETURN distance, not about the first touch.
  * The disk tier was OFF throughout (cold=off). Nothing here says whether it
    moves the cliff.
  * A harness bug was found and fixed mid-run: the body generator built a
    fresh Random per WORD, so every body was one repeated word and 64
    "distinct" conversations collapsed onto ~10 prompts. It read as 99.8%
    cached with zero evictions at 1.2x pool — a plausible-looking number from
    a test that was not testing anything. Section 3 is the re-run.
