loader read-slab size sweep -- box 206, 2026-09-08
==================================================

Corsair MP600 CORE XT (QLC) 3.6 TB, ext4 on LVM, O_DIRECT, cold (the file was
written with real bytes via os.urandom and read back with O_DIRECT, so this is
not the trimmed-space read that many published "sequential" figures measure).

Single reader, 4.0 GiB sequential stream -- the loader's own access pattern:

    chunk     best s   median s     GB/s   H2D launches   ring x4 (host AND gpu)
    4 MiB       1.24       1.28     3.46           1024                   16 MiB
    8 MiB       1.02       1.02     4.22            512                   32 MiB
   16 MiB       0.94       0.94     4.57            256                   64 MiB
   32 MiB       0.91       0.91     4.72            128                  128 MiB
   64 MiB       0.90       0.91     4.75             64                  256 MiB
  128 MiB       0.91       0.91     4.71             32                  512 MiB

The curve is flat from 32 MiB up: 4.72 / 4.75 / 4.71 GB/s at 32 / 64 / 128 MiB
is inside run-to-run spread. It turns over below 16 MiB, where per-request
overhead starts to show (4 MiB is 27% slower than the plateau).

So 128 MiB was buying nothing over 64 MiB in read rate while reserving twice
the ring: _RING_BUFS(4) x 128 MiB = 512 MiB of pinned host AND 512 MiB of GPU
scratch, against 256 MiB + 256 MiB at 64 MiB. The GPU half matters twice over,
because it is taken before the KV sizer measures what it can hand out.

WHY THIS IS NOT THE SAME ANSWER AS THE SAVEPOINT TIER
-----------------------------------------------------
A savepoint restore reads ONE 147.75 MiB object, and there the sweep peaks at
16 MiB (4.03 GB/s) with 128 MiB the WORST size measured (2.31 GB/s) -- 1.7x
apart. The optimum moves with total transfer size: a 4 GiB stream amortises
per-slab overhead over 64 slabs, a single 147.75 MiB object cannot. Copying
either constant into the other's path costs real throughput; see
arbicity/arbi-serve#2299.

REPRODUCER
----------
tools/loader/slab_sweep.py <file> <bytes> <reps>   (O_DIRECT, 4096-aligned
buffer, preadv loop; reports best and median over reps)
