SAVEPOINT DISK TIER — the paired A/B the wiring receipt deferred (2026-09-08)
=============================================================================

Question. `savepoint-tier-wiring-e2e-2026-09-08.txt` proved the tier RUNS and
states in its own scope note that it is "not the measurement of record: there
is no paired tier-off control here". This file is that control. It asks the
only question that decides whether the tier should be ON: what is it WORTH,
and worth against WHAT.

The answer is not the one the wiring receipt anticipated. Against nothing the
tier is a large win. Against simply sizing the pinned ring to the workload it
LOSES, on both latency and decode throughput, at equal hit rate.

INSTRUMENT
----------
Box 206 GPU 1 (RTX 4090, 450 W, no clock pins), GPU 0 idle. 93 GB RAM,
earlyoom inactive. Corsair MP600 CORE XT (QLC) 3.6 TB, ext4 on LVM, 96% full.
Image registry.arbi.work/arbi-serve:test-4a2f15c, arbi_serve bind-mounted from
feat/savepoint-tier @ db4f2392a. Qwen3.8-27B-exl3-4.0bpw, --kv-cache-dtype=tkv,
TKV_BITS=4, vq2_hadamard_ladder234_mtp_20260827, --max-batch=8
--chunk-prefill=2048 --max-batched-tokens=2304.

One container recreated per arm (NOT restarted — env_file resolves at create).
Every arm drove the identical command with a fresh nonce, so no arm reads the
cache another filled:

  tools/savepoint/retention_scenarios.py --scenarios linear --sessions 22
    --turns 4 --filler-reps 120 --max-tokens 300 --think-time-s 12
    --replay-reasoning

EOS HONORED. With --ignore-eos a thinking model never closes its think block,
the turn renders no answer, and every retention arm is flattered.

Arms differ in TWO env vars only, and the boot log states which regime it is
in before a token is served — the null control is read, not assumed:

  A   DISK_DIR=/cache/savepoint-ab-on  RING_SLOTS=32   "flush band: started"
  B   DISK_DIR=            (empty)     RING_SLOTS=32   "disk tier: OFF — no
                                                        offload directory"
  C   DISK_DIR=            (empty)     RING_SLOTS=64   same OFF line
  A2  = A, re-run LAST, to separate the result from arm ORDER

0 request errors in all four arms. An earlier arm-A run was DISCARDED, not
reported: it launched before the engine finished warming and took two 503s,
and a session that loses turn 0 has no prefix to reuse later — an error that
biases toward the tier.

THE RESULT — reuse turns only (t>0), n=66 per arm
--------------------------------------------------
                            cached%   TTFT med   decode tok/s   uncached/turn
  A   tier ON   ring 32       90.6      3.041        32.3            447
  A2  tier ON   ring 32       87.8      1.597        42.8            446
  B   tier OFF  ring 32       23.4     11.802        25.2           5683
  C   tier OFF  ring 64       91.0      0.177        51.5            444

  pinned host: A/A2/B = 4.96 GB (+6 GB disk in A/A2);  C = 9.92 GB, no disk

Two readings, and the second is the one that matters.

  1. AGAINST NOTHING, the tier is decisive. A vs B, identical pinned RAM:
     +67.1 pp cached, TTFT 11.80 s -> 3.04 s. A 32-slot ring against 22
     conversations x 2 savepoints = 44 wanted is a hard retention cliff; the
     tier turns it into a graceful extension.

  2. AGAINST PINNED RAM, the tier loses. A/A2 vs C, equal hit rate
     (90.6 / 87.8 / 91.0 — the same number): TTFT 3.041 and 1.597 against
     0.177 s, and decode 32.3 and 42.8 against 51.5 tok/s. Doubling the ring
     costs 4.96 GB of host RAM and beats the tier on BOTH axes.

WHY — AND IT IS THE WRITE PATH, NOT THE READ PATH
--------------------------------------------------
Only 11-12 resumes came off disk, at restore_us_mean 68.2 / 74.9 ms. That
cannot produce seconds of TTFT across 66 turns, and it says nothing about
decode at all. The cost is the flush:

  A    flush_total=77  flush_bytes_written=11,929,387,008  deferred_no_budget=3
  A2   flush_total=68  flush_bytes_written=10,535,043,072  deferred_no_budget=1250

~11 GB pushed to NVMe WHILE THE ENGINE SERVES. A and C carry the same prefill
load, so prefill work is not the difference; decode is pure compute and drops
37%, so the contention is global rather than prefill-specific.

The A/A2 pair is the corroboration, arrived at by accident and worth more than
the primary contrast: A2 deferred 1250 flushes where A deferred 3, issued 9
fewer, and was FASTER on both axes (1.597 s / 42.8 tok/s vs 3.041 / 32.3).
Throttling the band improved serving. That is the causal direction the
mechanism predicts.

The cause is a calibration error, not a design error. `TierBudget` is built in
`engine/active.py` from `rates.write_bytes_per_s` — the drive's full MEASURED
capability, 2.40-3.04 GB/s on these boots. A budget whose purpose is to
prevent device contention is set to the device's capability rather than to the
serving headroom, so it permits the band to write as fast as the flash accepts.
`deferred_no_budget=3` in arm A is that budget declining to bind.

WHAT THIS DOES NOT SAY
-----------------------
  * Not that the tier is wrong. It is correct, lossless, and its invariant
    holds: evictions_that_waited_for_a_flush=0 in every arm.
  * Not a general magnitude. The tier's value is a function of ring
    OVERSUBSCRIPTION; 44 savepoints against 32 slots is the regime it was
    built for, and the A-vs-B gap shrinks to nothing once the ring fits.
  * The A/A2 spread (3.041 vs 1.597 s) is large. Direction is robust across
    replicates; magnitude is not. Two replicates is not a distribution.
  * arena.warm=0 / arena.cold=11-12 in BOTH tier arms: every restore allocated
    its destination and paid the ~64 ms fault storm. A region returns to the
    pool only when the restored entry is dropped, and with 22 live
    conversations none is. The 36.5 ms restore in the component receipt
    describes a steady-state churn this workload does not produce.
  * spill_gate_reject=25-26 against 68-77 flushes: a quarter of candidates sit
    below the 512-token break-even and are refused. That bounds how much of
    this workload the tier can ever serve.

VERDICT
-------
Merge, and keep it OPT-IN — which is what it already is: an empty
`savepoint_disk_dir` leaves every line of the tier inert, and the boot says so.
Do NOT promote it to a default. The ring is the first lever; the tier is for
when pinned host RAM is genuinely unavailable, and there it is worth a great
deal. Promotion should wait on the band's budget being derived from serving
headroom rather than from drive capability, at which point this A/B should be
re-run — the A2 accident suggests that alone may close most of the gap.
