savepoint decode write x reasoning replay -- 2x2, 2026-09-08
====================================================================

box 206 GPU 1 (4090, 450 W, no clock pins), one boot per decode-write
setting, both cells of that setting driven back to back. Engine:
  registry.arbi.work/arbi-serve:test-4a2f15c, arbi_serve bind-mounted
  from savepoint/finish; Qwen3.8-27B-exl3-4.0bpw, --kv-cache-dtype=tkv,
  TKV_BITS=4, vq2_hadamard_ladder234_mtp_20260827 calibration,
  --max-batch=8 --chunk-prefill=2048 --max-batched-tokens=2048
  --prefill-capture=eager --gpu-memory-utilization=1.0
Drive: tools/savepoint/retention_scenarios.py --scenarios linear
  --sessions 2 --turns 3 --filler-reps 60 --max-tokens 1200,
  fresh --nonce per arm so no arm reads the cache another arm filled.
EOS is HONORED here: with --ignore-eos the think block never closes and
the turn renders no answer at all, which flatters the replay arms.

--- D2: replay OFF, decode-write OFF   (baseline)
      s  t   prompt   cached    pct  uncached   ttft_s
      0  0     2733        0    0.0     2733    0.736
      1  0     2736        0    0.0     2736    1.013
      0  1     2850     2560   89.8      290    0.15
      1  1     2896     2560   88.4      336    0.152
      1  2     2974     2816   94.7      158    0.143
      0  2     3449     2816   81.6      633    0.193
      -> cached_mean 88.6%   median_ttft 0.151 s   (reuse turns only)

--- A2: replay OFF, decode-write ON    (#2272 alone)
      s  t   prompt   cached    pct  uncached   ttft_s
      0  0     2733        0    0.0     2733    1.059
      1  0     2736        0    0.0     2736    0.78
      0  1     2850     2560   89.8      290    0.147
      1  1     2851     2560   89.8      291    0.15
      0  2     3365     2816   83.7      549    0.191
      1  2     3194     2816   88.2      378    0.154
      -> cached_mean 87.9%   median_ttft 0.152 s   (reuse turns only)

--- C2: replay ON,  decode-write OFF   (#2289 alone)
      s  t   prompt   cached    pct  uncached   ttft_s
      1  0     2736        0    0.0     2736    0.734
      0  0     2733        0    0.0     2733    1.016
      0  1     3160     2560   81.0      600    0.191
      1  1     3040     2560   84.2      480    0.178
      0  2     4246     3072   72.4     1174    0.305
      1  2     4263     2816   66.1     1447    0.342
      -> cached_mean 75.9%   median_ttft 0.248 s   (reuse turns only)

--- B2: replay ON,  decode-write ON    (both)
      s  t   prompt   cached    pct  uncached   ttft_s
      1  0     2736        0    0.0     2736    0.751
      0  0     2733        0    0.0     2733    1.033
      1  1     3070     2816   91.7      254    0.144
      0  1     3280     3072   93.7      208    0.144
      1  2     4293     4096   95.4      197    0.145
      0  2     4326     4096   94.7      230    0.144
      -> cached_mean 93.9%   median_ttft 0.144 s   (reuse turns only)

Reading (median TTFT over reuse turns):
                     write OFF   write ON
      replay OFF       0.151 s     0.152 s
      replay ON        0.248 s     0.144 s

Neither main effect is positive. The savepoint alone is worth nothing
(0.152 vs 0.151); replay alone is a +64% TTFT regression (0.248 vs
0.151). Only the both-on cell wins, and there it is free: B2 carries a
4326-token prompt at turn 3 against the baseline's 3449, at a LOWER
TTFT. B2's TTFT does not grow with conversation depth at all --
0.144 / 0.144 / 0.145 / 0.144 -- while D2 goes 0.150 -> 0.193 and A2
0.147 -> 0.191 over the same turns.

Mechanism, from the `savepoint probe` line (radix_reach / hit):
  hit == radix_reach in every row of every arm. Nothing declines the
  deeper entry -- a savepoint deeper than the radix walk reached is
  never compared (probe_reach's docstring is the contract). In the
  replay-OFF arms the walk stops at the previous prompt's page floor
  4 times out of 4; with replay it runs 1024-1280 tokens past it into
  the served reply. That is the whole of #2272's 'writes but never
  resumes': the entries were unreachable, not rejected.

Same 2x2 under --ignore-eos --max-tokens 700 (arms A/B/C/D, same box,
same day) reproduces the structure: 0.120 / 0.121 / 0.216 / 0.122 s.


====================================================================
CONCURRENCY, 8 conversations x 5 turns, 15 s think-time  (2026-09-08)
====================================================================

The cost side, which the two-session arms could not measure: a 64-slot
ring cannot be exhausted by two sessions, so `overwrite` stayed 0 and
every retention claim was untested.

Both runs are the SAME invocation (--sessions 8 --turns 5 --filler-reps 60
--max-tokens 1200 --replay-reasoning --think-time-s 15). The first ran
before SamplingMsg carried replay_reasoning, so the gate refused every
decode write -- an accidental but exact decode-write-OFF control.

--- gate broken == decode-write OFF
      turn   n   cached%   ttft_s   decode_tok/s   uncached
       0     8      0.0    2.697         38.9       2738
       1     8     80.4    0.205         43.8        629
       2     8     69.5    0.345         42.6       1325
       3     8     74.8    0.456         41.9       1396
       4     8     80.2    0.333         41.4       1340
      -> cached_mean 76.2%  median_ttft 0.331 s

--- decode-write ON, gate working
      turn   n   cached%   ttft_s   decode_tok/s   uncached
       0     8      0.0    2.708         38.7       2736
       1     8     94.9    0.155         43.9        160
       2     8     96.7    0.146         43.3        144
       3     8     96.2    0.155         42.7        215
       4     8     95.7    0.149         42.1        287
      -> cached_mean 95.9%  median_ttft 0.147 s

Ring, decode-write ON: first_use 64, overwrite 115 -- 179 hand-outs for 40
requests = 4.5 per request. The ring DID wrap, so retention was genuinely
exceeded, and cached% held at 96% anyway: what gets overwritten is the
dominated shallow entries while each turn rewrites its own deepest one.
Headroom is thin (~1.8 rounds at max_batch=8) which is what the
writes_per_turn sizing and its boot warning exist to make visible.

Decode is NOT regressed: 43.9 / 43.3 / 42.7 / 42.1 tok/s across reuse
turns. The savepoint copy runs on a stream forked from compute and is
overlapped -- fold_emit_slot_wait was 672 of 672 samples at <=0.05 ms.
The only stall seen was 1 spill fence of 13, costing 11.8 ms, which is the
host-side landing->ring copy, not a per-write decode tax.
