128k+ CONTEXTS CACHE AT 99.3% — AND WHAT DECIDES IT — 2026-09-09
================================================================

THE QUESTION. The long-context mix of record (savepoint-observability-and-
long-context-mix-2026-09-09.md) reported the 64k+ band at 0/6 — no reuse turn
resumed anything — on a 27B hybrid. That receipt attributed it to the KV pool:
933 pages x 256 = 238,848 tokens, so two 128k sequences cannot both be
resident, the KV half of a pair is evicted, and savepoint_admission answers a
match missing either half by discarding both.

This run tests that attribution by removing pool scarcity and changing nothing
else.

RIG
  box 206 GPU 0, container sp08, port 8125
  arbi-serve main 32a41cfd2, bind-mounted; image :test-latest
  model /models/Qwen3.5-0.8B -- a GDN HYBRID (linear_num_key_heads 16,
        linear_conv_kernel_dim 4), 24 layers, 6 paged, max_position 262144.
        Chosen because it is the same architecture class as the 27B at a
        fraction of the weights, so the pool-to-context ratio is the only
        term that moves.
  ARBI_SAVEPOINT_ENABLED=1 ARBI_PREFIX_TIER=1, --kv-cache-dtype=auto
        --mtp-n-draft=0 --max-batch=8
  harness tools/savepoint/retention_scenarios.py --scenarios linear
        --mix 128k=2 --turns 4 --ring-slots 16 --page 256

GEOMETRY, off the boot rather than assumed
  pool: 24 model layers (6 paged), 12288 B/tok
  KV: 6868 pages x 256 = 1,758,208 servable tokens
  served context 262,144 (the full model bound)
  savepoint ring 16 slots x 19,759,104 B = 316 MB pinned
  prefix tier arena 1365 slots
  => 13.4x a 128k sequence. Two 128k conversations occupy 15% of the pool.

RESULT — THE BAND CACHES
  s  t  prompt    cached    pct
  -- -- --------- --------- -----
  0  0    126896         0    0.0   cold, correct
  0  1    127431    126720   99.4
  0  2    127452    127232   99.8
  0  3    127473    127232   99.8
  1  0    125979         0    0.0   cold, correct
  1  1    126801    125952   99.3
  1  2    126822         0    0.0   MISS -- see OPEN below
  1  3    130343    126720   97.2

  reuse turns 6, misses 1, cached_pct MEDIAN 99.3 (mean 82.6, dragged by the
  single miss -- the distribution is bimodal, so the median is the statistic)
  TTFT cold median 6.19 s -> reuse median 0.19 s, 96.9% saved
  decode 161.9 tok/s median

WHAT DECIDES IT: POOL HEADROOM, NOT SAVEPOINT COVERAGE
  Same harness, same band, same architecture class, differing in pool ratio:

    model         pages   servable tok   x128k   cached% at 64k+
    ------------- ------- -------------- ------- ----------------
    Qwen3.8-27B      956        244,736    1.87   23.7 (tier on)
                                                   0/6 (tier off, on record)
    Qwen3.5-0.8B    6868      1,758,208   13.4    99.3

  So KV scarcity is the binding constraint at long context, and the earlier
  receipt's attribution was right.

A WRONG INFERENCE, RECORDED BECAUSE IT WAS ACTED ON
  Mid-run, arbi_serve_prefix_cache_recurrent_cap_truncated_total read 2 on a
  PARTIAL pass of this same drive. That counter fires only when a prefix walk
  was bounded by the hybrid recurrent-coverage cap WHILE THE TREE STILL HELD
  FURTHER PAGES, so it was read as: the KV side has the data and the savepoint
  cannot follow, therefore coverage is the ceiling and freeing KV cannot lift
  it. That conclusion was reported.

  The completed pass refutes it. The counter ended at 3 and the band still
  cached 99.3%: the cap trims a page at the margin, it does not bind. A
  counter that fires is not a counter that dominates, and a partial run is not
  a run.

WHAT THE TIER DID HERE: NOTHING, CORRECTLY
  tier_arena handouts = 0, live = 0. With 13.4x headroom nothing was evicted,
  so nothing was demoted and there was nothing to restore. The 99.3% is radix
  + savepoint alone. This run says nothing about the warm tier's value; it
  removes the tier as a variable.

RESOLVED SINCE (the miss)
  * s=1 t=2 was a FULL miss (cached 0) between two 99%+ hits on the same
    session with pool to spare, and this receipt first recorded it as
    unexplained. It is explained, and it is two things at once.

    The TRIGGER was an artifact of this drive. No --max-tokens cap was passed,
    so the harness let the model stop on its own: s=1 t=0 emitted 82,812 output
    tokens over 579 s and s=0 t=2 emitted 116,870 over 765 s. That is not
    traffic, it is a degenerate loop, and between them those replies wrote 68
    savepoints through a 16-slot ring.

    The MECHANISM is real. The ring wrapped past s=1's prompt-end entry before
    t=2 asked for it, so the KV was present and the savepoint was not, and
    savepoint_admission discarded BOTH halves and re-prefilled 127k tokens from
    zero. The engine counted it exactly:
    arbi_serve_savepoint_hybrid_no_savepoint_fallback_total moved by 1, one
    event for the one miss. It is the same pair sink that dominates the 27B
    under scarcity — see pair-sink-caps-long-context-2026-09-10.txt, where it
    fires 1396 times.

    So the drive's own parameters manufactured a ring pressure this rig would
    not otherwise see, and the pressure exposed a real failure mode rather than
    a measurement error.

  * The harness's own conformance gate SHOULD have caught this and reported
    "match: true" instead. It read a labelled prometheus series by bare name,
    so its observed count was 0.0 on every run ever made and its verdict
    passed by construction. Fixed in arbi-serve#2339. Any counter_conformance
    verdict in a receipt written before that fix means nothing.

OPEN
  * expected/actual disagree on 5 of 6 reuse turns (harness expected `full`
    or `prefill_only`, got `prefill_only`). The cached fraction is what this
    receipt reports; the class mismatch is the harness's own prediction model
    and is not resolved here.
  * One model, one card, TP1, one drive per cell. No interval is quoted
    because none was measured -- these are single observations, not an A/B.
