SAVEPOINT INTEGRATION — four branches as one, and the first measurement of
what the whole subsystem costs
==========================================================================

VERDICT, FIRST, BECAUSE IT IS A FAIL
-------------------------------------
The savepoint subsystem costs **+3.59% TTFT** on a turn that cannot benefit
from it, 95% CI [+2.88%, +4.31%]. The owner's bar is 1%. The CI lies entirely
above the bar, the run could resolve 1% (MDE 1.00%), and the effect reproduced
identically in both replicate pairs. This is a FAIL, not a marginal result.

DECODE IS NOT A PASS EITHER — IT IS UNRESOLVED. The point estimate is -0.98%,
but this run's smallest detectable effect on decode was 2.81%. A CI that
straddles zero at that width is not parity, and it is not reported as parity.
Resolving 1% on decode needs roughly n=380 per arm at the observed 4-6%
per-request spread; this run had 48.

WHY THIS NUMBER DID NOT EXIST BEFORE
------------------------------------
Every A/B this subsystem has run compared one of its own settings against
another — ring 32 against 64, tier on against off, per-row copy against fused,
pool at min_entries against pool at demand. Not one compared the machinery
against NO machinery, so the cumulative cost had never been measured, only the
deltas between its configurations. Each of those landed "within noise" and
none of them resolved. They do not sum: a chain of unresolved results can add
to something that does, which is what happened here.

ARBI_SAVEPOINT_ENABLED is the arm that makes the comparison possible. It is
one boot variable with nothing else in the recipe changed, and it is the
reference arm every future measurement of this subsystem should be reported
against.

THE MEASUREMENT
---------------
Engine: integration/savepoint-four, bind-mounted over
registry.arbi.work/arbi-serve:test-latest on box 206 GPU 1, one engine at a
time. Mount VERIFIED by content digest, not assumed: the engine's own
source_identity digest (b9955e2489da6aa4, 948 files) equals the checkout's.
The image is a different commit, so an unverified mount would have measured
the image's code and the whole run would have been void.

Shape: --max-batch 8, concurrency 1, ONE conversation, fixed 6300-token
prompts, --ignore-eos with max_tokens 512 so decode rate is not confounded by
reply length. Fresh nonce per rep, so TURN 0 CANNOT RESTORE ANYTHING and is
the pure-cost cell. 4 warm-up reps discarded per boot, then 24 measured.
A/B/A/B across four separate boots. n=48 per arm.

  cell                    OFF       ON        ON-OFF     95% CI          MDE
  turn 0 TTFT (cost)      1.2275 s  1.2717 s  +3.59%     [+2.88,+4.31]   1.00%
  turn 0 decode tok/s     114.92    113.79    -0.98%     [-2.94,+0.98]   2.81%
  turn 1 TTFT (reuse)     1.2948 s  0.1736 s  -86.59%    [-86.65,-86.53] 0.09%
  turn 1 decode tok/s     108.27    107.62    -0.60%     [-2.89,+1.70]   3.28%

Per-boot means, as a drift check: OFF 1.2298 / 1.2253, ON 1.2739 / 1.2694.
The arms do not drift into each other.

WHERE THE COST IS NOT
---------------------
A third ON boot with ARBI_PREFIX_TIER=0 read +3.31% [+2.60%, +4.02%] on the
same cell. The prefix-tier arena is therefore NOT the cost; the savepoint
machinery itself is. That matters for what to do next: the arena can stay.

A SECOND COST, NOT IN THE BAR BUT REAL
--------------------------------------
The ON arm's cold-TTFT spread is roughly 14x the OFF arm's (rsd 2.4% against
0.2%). The machinery costs tail stability as well as mean, and a bar stated on
means does not see it.

WHAT THE BRIEF GOT WRONG, STATED BECAUSE IT CHANGES THE READING
---------------------------------------------------------------
The measurement was designed on the premise that at concurrency 1 with one
conversation nothing is ever restored, making the ON arm pure cost. That is
true of turn 0 and false of the run: turn 1 restores, and its TTFT falls 86.6%
(1.295 s -> 0.174 s), a 7.5x win. So the +3.59% above is the cold-turn cost in
ISOLATION, which is the right number to hold against the 1% bar, and it is not
the whole picture of what the subsystem does to this shape.

THE SEAM CROSSED
----------------
  savepoint offload: taking the spill worker arm — the offload worker owns
  this tier's I/O.

The worker arm, named with its reason, never "no shared region has been
published". The out-of-process path is live rather than inert.

EFFECTIVE STORE CAPACITY, READ OFF A LIVE BOOT
-----------------------------------------------
  savepoint store: enabled (retains 64 savepoint(s) x 154927104 B =
    9915334656 B; savepoint_max_bytes=1073741824 is a FLOOR, not a cap —
    unstated, so savepoint_min_entries=64 carries the bound, under
    savepoint_hard_max_bytes; overflow=refuse, chunk_size=2048)

64 entries, 9.23 GiB. The line this replaces printed max_bytes=1073741824 from
raw config, understating the real bound by 9.23x AND in the wrong direction —
an operator dividing 1 GiB by a savepoint's size would have inferred about 7
entries. The byte number is now labelled floor-or-cap and the entry count
leads.

PINNED HOST — THE TARGET WAS MET, AND THE LEDGER SAYS WHAT IT REALLY COSTS
---------------------------------------------------------------------------
  savepoint ring: 16 slot(s) (auto from max_batch=8 x 2 live slot(s)/turn)
    x 154927104 B = 2478833664 B PINNED HOST

2.31 GiB asked, not 9.24. The measured-booking seam then fired at 1.73x:

  WARNING savepoint ring: 2478833664 B asked, but 4294967296 B of resident
    page-locked host memory was taken (1.73x) — the caching host allocator
    serves a pinned request out of a rounded block.

So the ledger books 4.00 GiB, which is what the process actually holds. Both
rows present, both bases now measured through one seam:

  cache.savepoint_ring        4096.0 MiB  (16 slot(s) x 154927104 B)
  cache.prefix_tier_arena     4092.8 MiB  (963 slot(s) x 4456448 B)
  TOTAL                       8188.8 MiB

The arena's own booking hit the other branch — VmRSS moved 0 B across the
page-lock, so the row falls back to the ask and may UNDER-report. That branch
is warned about rather than silent, which is the half of this correction that
had diverged between the two branches before they were hoisted together.

NOTE ON THE DIRECTION. A ledger row is SUBTRACTED from VmRSS to get the
working-set floor, and the floor is subtracted from headroom to get what the
next reservation may take. A row reading high therefore SHRINKS the floor and
GRANTS more. Over-booking is the permissive direction, not the safe one. Both
branches' prose claimed the opposite in four places; the behaviour is
unchanged and the prose now states what the arithmetic does.

NO PER-SERVE PINNED ALLOCATION
-------------------------------
arbi_serve_host_pinned_reserved_bytes{owner="cache.prefix_tier_arena"} read
4.291559424e9 at boot and the identical value after a drive of 457 snapshots
and 484 demotions. Ring likewise flat. The arena is being USED, not
re-allocated, which is what #2323 was for.

THE PREFIX TIER IS A PRECONDITION, NOT A PARALLEL IMPROVEMENT
--------------------------------------------------------------
Same workload, ARBI_PREFIX_TIER on and off:

                              tier=1     tier=0
  pairs_demoted               484        0
  pairs_dropped_unpaired      0          58
  store.evictions_radix       0          58
  flushes_from_band           19         0
  retention flush_bytes       2.94 GB    0
  store.flush_total           22 / 3.41 GB   0 / 0
  restores_from_disk / _ram   1 / 1      0 / 0
  offload arm announced       spill worker   never reached the mover
  ring_wrap_miss_total        0          0
  conformance mismatches      46 / 106   52 / 105

With the prefix tier off every radix eviction drops the paired savepoint, the
disk tier writes zero bytes, and the offload seam is never exercised at all.
Savepoint retention cannot exceed KV retention, so #2323 is a precondition for
#2320 mattering at long context. They are sequential, not parallel.

WHICH RESOURCE BOUND EACH ARM
------------------------------
Stated as a first-class output, because ring-bound, cap-bound and KV-bound
produce identical-looking miss rates and need opposite fixes.

  tier=1: CAP-BOUND (weakly). ring_wrap_miss_total absent (=0) despite a 60%
    overwrite share, so NOT ring-bound. pairs_dropped_unpaired=0 and
    kv_coverage_shortfall=0, so NOT KV-bound. What binds is the store's byte
    cap: resident_bytes == capacity_bytes == 6.66e9, evictions_lru=392.

  tier=0: KV-BOUND. ring_wrap_miss_total flat at 0 while
    pairs_dropped_unpaired climbed 0 -> 58. That asymmetry is the
    discriminator, and it fired.

Neither arm was ring-bound. The ring was never the constraint this workload
hit, which is what the earlier replay was pointing at.

CONFORMANCE — NOT A CLEAN RUN
------------------------------
tier=1: 106 checked, 60 matched, 46 mismatches.
  expected->actual: none->{none 38, prefill_only 18, partial 4}
                    prefill_only->{none 22, prefill_only 22, partial 2}

Roughly 13 of them are the harness's model not knowing a branch shares its
parent's prefix, and roughly 11 are the ring model over-predicting a wrap at a
concurrency where the tool itself declares --ring-slots approximate. The real
pile is the 22 predicted hits that were total losses. One ledger artefact is
visible in the raw rows and is NOT explained: shared_prefix/conv3/turn0
reports cov_prefill=124160 against prompt_tokens=32449.

ABSOLUTE THROUGHPUT IS NOT QUOTABLE HERE
-----------------------------------------
Measured MTP accept rate 0.246-0.305 against the harness's 0.6 floor. Every
figure above is a relative arm comparison; no absolute tok/s from this run
should be carried anywhere.

DENSE — THE PATH NOBODY HAD EXERCISED
--------------------------------------
MiniCPM5-1B-exl3-4bit, a LlamaForCausalLM, with ARBI_SAVEPOINT_ENABLED=1 and
ARBI_SAVEPOINT_DISK_DIR set:

  savepoint store: disabled (no recurrent state, ...) — note this is NOT the
  master switch; ARBI_SAVEPOINT_ENABLED=0 is, and it is still 1 here

No ring line, no cache.savepoint_ring ledger row, two processes in the
container (no offload worker), zero savepoint metric series before or after
traffic, and the savepoint directory was never created. A dense boot is the
null control by construction.

The KV tier works on its own, which for dense is the whole feature: the arena
re-measured its slot size for this model (6291456 B/slot against 4456448 on
the 27B) rather than carrying a constant, booked at 1.00x, and TTFT went
0.093 s -> 0.028 s at 95.6% cached, repeatably, with pinned bytes unchanged
across all traffic.

OPEN, WITH EVIDENCE, NOT CLOSED
--------------------------------
1. THE 1% BAR IS NOT MET on TTFT. +3.59% on cold turns. Not fixed here.
2. THE OFFLOAD WORKER STOPS ANSWERING under the heavy full mix. After 3
   successful spills every later job timed out ("job 13 did not answer within
   30.0s"), repeating to blob_write_failed=61 with flush_total frozen at 3 and
   an orphaned 148 MiB .tmp left on disk. The worker process was alive and
   idle in recv(), so this is a lost message or a lost wakeup, not a crash.
3. BOTH OFFLOAD WORKERS WERE SIGKILLED (exitcode=-9) at the instant cudagraph
   capture ended, and respawned. Unexplained.
4. TWO SAVEPOINT SUBSYSTEMS INITIALISE PER BOOT, each with its own supervisor
   and worker (10:26:53 and 10:27:12 in one process). build_active runs
   several times a boot; this may be benign and is not verified to be.
5. DECODE IS UNRESOLVED at this n, as stated above.

Items 2, 3 and 4 are all on the offload path and all appear only under load
that the conformance arms did not reach. None of them can affect a boot with
ARBI_SAVEPOINT_DISK_DIR unset, which is the default.
