[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (317092 > 262144). Running this sequence through the model will result in indexing errors
[i8] basket: 20 prompts x 16384 tok, domains ['code', 'longctx', 'math', 'sci', 'wiki']
compile-cache MISS: xgrammar — xgrammar compiled grammars (re)compiles this boot [/cache/xgrammar]
[TKV autotune-provenance] fp=57911e26f4e47f74 regime=cache-hit cells=8 sweep_src=2772549a0ef713db tile_tokens=4 num_splits=32,64,128,256 min_blocks_per_sm=0 resolved=0 held=0 inherited=0
/opt/venv/lib/python3.12/site-packages/torch/__init__.py:1172: FutureWarning: `torch.distributed.reduce_op` is deprecated, please use `torch.distributed.ReduceOp` instead
  return isinstance(obj, torch.Tensor)
[i8] trellis->path map: 401 linears
JIT compile AFTER serving-ready [cpp_ext]: exl3_i8_gemm cached .so load — a live request paid this compile's latency. This is a boot-warmup coverage gap: extend warmup to pre-compile this kernel/specialization. Counter jit_compile_serving (must-not-fire) at GET /v1/admin/flag_truth.
[i8] prefix cache: server default resolved=True, per-request cache_enabled=False; any cache hit below is refused
driver serving-growth OVER RESERVE: the driver holds 2 MiB more than it did when the boot brackets closed, against the 0 MiB the serving floor's transient.serving_step.driver_growth term held for it. Those bytes are in no pool and no allocator counter -- the live card reaches them as driver.modules_serving and the boot ledger only as driver.residual -- and the KV layout is frozen, so the overage comes out of the free VRAM the floor is holding for one in-flight step (the verify tail, the DFlash context-assemble) and those reserves are the ones that fail first. The reading is on record for this configuration, so the next boot holds it; this process serves the rest of its life short by the overage.
activation admission RE-ARMED (driver residency over reserve): the step budget FELL to 5757 MiB, -642 MiB on the 6399 MiB it was enforcing and 6399 MiB the boot reading armed. The widest slate admission will build is narrower from here — narrower prefill chunks and deferred rows instead of a forward that OOMs after the layout freeze.
[i8] warmup done
  [0] code           REF      16384tok  wall=  5.02s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [0] code           REF2     16384tok  wall=  5.01s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [0] code           STD      16384tok  wall=  5.27s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [0] code           INT8     16384tok  wall=  4.83s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [0] code           INT8NOQ  16384tok  wall=  4.90s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [0] code           INT8NOGDN  16384tok  wall=  4.91s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [0] code           SIMROW   16384tok  wall=  7.53s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [0] code           SIMG128  16384tok  wall=  7.50s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [0] code           SIMG32   16384tok  wall=  7.65s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [1] code           REF      16384tok  wall=  5.05s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [1] code           REF2     16384tok  wall=  5.07s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [1] code           STD      16384tok  wall=  5.32s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [1] code           INT8     16384tok  wall=  4.86s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [1] code           INT8NOQ  16384tok  wall=  4.93s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [1] code           INT8NOGDN  16384tok  wall=  4.93s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [1] code           SIMROW   16384tok  wall=  7.55s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [1] code           SIMG128  16384tok  wall=  7.53s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [1] code           SIMG32   16384tok  wall=  7.67s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [2] code           REF      16384tok  wall=  5.07s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [2] code           REF2     16384tok  wall=  5.07s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [2] code           STD      16384tok  wall=  5.33s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [2] code           INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [2] code           INT8NOQ  16384tok  wall=  4.93s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [2] code           INT8NOGDN  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [2] code           SIMROW   16384tok  wall=  7.57s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [2] code           SIMG128  16384tok  wall=  7.55s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [2] code           SIMG32   16384tok  wall=  7.68s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [3] code           REF      16384tok  wall=  5.08s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [3] code           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [3] code           STD      16384tok  wall=  5.34s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [3] code           INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [3] code           INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [3] code           INT8NOGDN  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [3] code           SIMROW   16384tok  wall=  7.57s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [3] code           SIMG128  16384tok  wall=  7.54s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [3] code           SIMG32   16384tok  wall=  7.68s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [4] wiki           REF      16384tok  wall=  5.09s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [4] wiki           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [4] wiki           STD      16384tok  wall=  5.34s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [4] wiki           INT8     16384tok  wall=  4.86s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [4] wiki           INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [4] wiki           INT8NOGDN  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [4] wiki           SIMROW   16384tok  wall=  7.58s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [4] wiki           SIMG128  16384tok  wall=  7.54s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [4] wiki           SIMG32   16384tok  wall=  7.69s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [5] wiki           REF      16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [5] wiki           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [5] wiki           STD      16384tok  wall=  5.34s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [5] wiki           INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [5] wiki           INT8NOQ  16384tok  wall=  4.93s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [5] wiki           INT8NOGDN  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [5] wiki           SIMROW   16384tok  wall=  7.58s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [5] wiki           SIMG128  16384tok  wall=  7.55s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [5] wiki           SIMG32   16384tok  wall=  7.70s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [6] wiki           REF      16384tok  wall=  5.09s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [6] wiki           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [6] wiki           STD      16384tok  wall=  5.34s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [6] wiki           INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [6] wiki           INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [6] wiki           INT8NOGDN  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [6] wiki           SIMROW   16384tok  wall=  7.57s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [6] wiki           SIMG128  16384tok  wall=  7.54s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [6] wiki           SIMG32   16384tok  wall=  7.69s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [7] wiki           REF      16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [7] wiki           REF2     16384tok  wall=  5.09s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [7] wiki           STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [7] wiki           INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [7] wiki           INT8NOQ  16384tok  wall=  5.01s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [7] wiki           INT8NOGDN  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [7] wiki           SIMROW   16384tok  wall=  7.58s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [7] wiki           SIMG128  16384tok  wall=  7.54s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [7] wiki           SIMG32   16384tok  wall=  7.70s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [8] math           REF      16384tok  wall=  5.18s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [8] math           REF2     16384tok  wall=  5.09s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [8] math           STD      16384tok  wall=  5.34s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [8] math           INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [8] math           INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [8] math           INT8NOGDN  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [8] math           SIMROW   16384tok  wall=  7.67s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [8] math           SIMG128  16384tok  wall=  7.54s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [8] math           SIMG32   16384tok  wall=  7.69s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [9] math           REF      16384tok  wall=  5.09s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [9] math           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [9] math           STD      16384tok  wall=  5.36s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [9] math           INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [9] math           INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [9] math           INT8NOGDN  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [9] math           SIMROW   16384tok  wall=  7.58s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [9] math           SIMG128  16384tok  wall=  7.54s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [9] math           SIMG32   16384tok  wall=  7.69s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [10] math           REF      16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [10] math           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [10] math           STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [10] math           INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [10] math           INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [10] math           INT8NOGDN  16384tok  wall=  4.96s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [10] math           SIMROW   16384tok  wall=  7.58s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [10] math           SIMG128  16384tok  wall=  7.55s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [10] math           SIMG32   16384tok  wall=  7.69s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [11] math           REF      16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [11] math           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [11] math           STD      16384tok  wall=  5.34s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [11] math           INT8     16384tok  wall=  4.86s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [11] math           INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [11] math           INT8NOGDN  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [11] math           SIMROW   16384tok  wall=  7.58s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [11] math           SIMG128  16384tok  wall=  7.55s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [11] math           SIMG32   16384tok  wall=  7.69s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [12] longctx        REF      16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [12] longctx        REF2     16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [12] longctx        STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [12] longctx        INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [12] longctx        INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [12] longctx        INT8NOGDN  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [12] longctx        SIMROW   16384tok  wall=  7.57s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [12] longctx        SIMG128  16384tok  wall=  7.55s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [12] longctx        SIMG32   16384tok  wall=  7.69s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [13] longctx        REF      16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [13] longctx        REF2     16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [13] longctx        STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [13] longctx        INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [13] longctx        INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [13] longctx        INT8NOGDN  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [13] longctx        SIMROW   16384tok  wall=  7.58s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [13] longctx        SIMG128  16384tok  wall=  7.55s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [13] longctx        SIMG32   16384tok  wall=  7.70s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [14] longctx        REF      16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [14] longctx        REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [14] longctx        STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [14] longctx        INT8     16384tok  wall=  4.86s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [14] longctx        INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [14] longctx        INT8NOGDN  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [14] longctx        SIMROW   16384tok  wall=  7.59s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [14] longctx        SIMG128  16384tok  wall=  7.55s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [14] longctx        SIMG32   16384tok  wall=  7.70s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [15] longctx        REF      16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [15] longctx        REF2     16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [15] longctx        STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [15] longctx        INT8     16384tok  wall=  4.86s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [15] longctx        INT8NOQ  16384tok  wall=  4.94s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [15] longctx        INT8NOGDN  16384tok  wall=  4.97s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [15] longctx        SIMROW   16384tok  wall=  7.61s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [15] longctx        SIMG128  16384tok  wall=  7.57s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [15] longctx        SIMG32   16384tok  wall=  7.70s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [16] sci            REF      16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [16] sci            REF2     16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [16] sci            STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [16] sci            INT8     16384tok  wall=  4.85s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [16] sci            INT8NOQ  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [16] sci            INT8NOGDN  16384tok  wall=  4.97s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [16] sci            SIMROW   16384tok  wall=  7.59s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [16] sci            SIMG128  16384tok  wall=  7.60s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [16] sci            SIMG32   16384tok  wall=  7.73s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [17] sci            REF      16384tok  wall=  5.13s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [17] sci            REF2     16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [17] sci            STD      16384tok  wall=  5.37s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [17] sci            INT8     16384tok  wall=  4.88s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [17] sci            INT8NOQ  16384tok  wall=  4.96s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [17] sci            INT8NOGDN  16384tok  wall=  4.96s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [17] sci            SIMROW   16384tok  wall=  7.60s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [17] sci            SIMG128  16384tok  wall=  7.56s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [17] sci            SIMG32   16384tok  wall=  7.72s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [18] sci            REF      16384tok  wall=  5.13s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [18] sci            REF2     16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [18] sci            STD      16384tok  wall=  5.36s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [18] sci            INT8     16384tok  wall=  4.87s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [18] sci            INT8NOQ  16384tok  wall=  4.96s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [18] sci            INT8NOGDN  16384tok  wall=  4.97s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [18] sci            SIMROW   16384tok  wall=  7.60s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [18] sci            SIMG128  16384tok  wall=  7.56s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [18] sci            SIMG32   16384tok  wall=  7.71s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [19] sci            REF      16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [19] sci            REF2     16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [19] sci            STD      16384tok  wall=  5.36s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [19] sci            INT8     16384tok  wall=  4.88s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [19] sci            INT8NOQ  16384tok  wall=  4.95s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [19] sci            INT8NOGDN  16384tok  wall=  4.97s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [19] sci            SIMROW   16384tok  wall=  7.59s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [19] sci            SIMG128  16384tok  wall=  7.56s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [19] sci            SIMG32   16384tok  wall=  7.70s  finish=length  leg=0F/3200S acc=3200x16/0x32

[i8] kernel census: int8 calls=161280 rows=330301440 reconstructs SKIPPED=161280 fallback (non-4bpw) calls=30720
[i8] shapes served by the kernel (K,N)->calls: {(5120, 1024): 15360, (5120, 6144): 15360, (5120, 10240): 7680, (5120, 12288): 7680, (5120, 17408): 61440, (6144, 5120): 23040, (17408, 5120): 30720}
[i8] excluded projections: ['in_proj_qkv', 'in_proj_z', 'out_proj']  excluded calls=30720

  linear class                                           served   excl  int8 vs fp32 legB vs fp32 int8 vs legB
  model.layers.0.linear_attn.in_proj_qkv                    161    322     4.827e-03    1.713e-03    5.132e-03
  model.layers.0.linear_attn.in_proj_z                      322    161     6.316e-03    1.129e-03    6.418e-03
  model.layers.0.linear_attn.out_proj                       322    161     1.249e-02    2.189e-03    1.268e-02
  model.layers.0.mlp.down_proj                              483      0     2.122e-02    2.609e-03    2.137e-02
  model.layers.0.mlp.gate_proj                              483      0     9.121e-03    3.413e-03    9.739e-03
  model.layers.0.mlp.up_proj                                483      0     9.082e-03    3.418e-03    9.702e-03
  model.layers.1.linear_attn.in_proj_qkv                    161    322     1.186e-02    2.166e-03    1.206e-02
  model.layers.1.linear_attn.in_proj_z                      322    161     1.361e-02    1.840e-03    1.373e-02
  model.layers.1.linear_attn.out_proj                       322    161     1.604e-02    2.253e-03    1.619e-02
  model.layers.1.mlp.down_proj                              483      0     1.345e-02    2.322e-03    1.364e-02
  model.layers.1.mlp.gate_proj                              483      0     9.700e-03    3.459e-03    1.030e-02
  model.layers.1.mlp.up_proj                                483      0     1.015e-02    3.488e-03    1.073e-02
  model.layers.10.linear_attn.in_proj_qkv                   161    322     1.304e-02    2.256e-03    1.323e-02
  model.layers.10.linear_attn.in_proj_z                     322    161     1.298e-02    1.838e-03    1.310e-02
  model.layers.10.linear_attn.out_proj                      322    161     1.498e-02    2.124e-03    1.513e-02
  model.layers.10.mlp.down_proj                             483      0     1.268e-02    2.347e-03    1.290e-02
  model.layers.10.mlp.gate_proj                             483      0     1.035e-02    3.578e-03    1.095e-02
  model.layers.10.mlp.up_proj                               483      0     1.075e-02    3.590e-03    1.134e-02
  model.layers.11.mlp.down_proj                             483      0     1.337e-02    2.508e-03    1.360e-02
  model.layers.11.mlp.gate_proj                             483      0     1.027e-02    3.508e-03    1.085e-02
  model.layers.11.mlp.up_proj                               483      0     1.087e-02    3.574e-03    1.144e-02
  model.layers.11.self_attn.k_proj                          483      0     1.329e-02    2.697e-03    1.356e-02
  model.layers.11.self_attn.o_proj                          483      0     1.303e-02    2.370e-03    1.324e-02
  model.layers.11.self_attn.q_proj                          483      0     1.173e-02    2.022e-03    1.190e-02
  model.layers.11.self_attn.v_proj                          483      0     1.036e-02    2.280e-03    1.061e-02
  model.layers.12.linear_attn.in_proj_qkv                   161    322     1.292e-02    2.245e-03    1.311e-02
  model.layers.12.linear_attn.in_proj_z                     322    161     1.366e-02    1.891e-03    1.379e-02
  model.layers.12.linear_attn.out_proj                      322    161     1.586e-02    2.138e-03    1.600e-02
  model.layers.12.mlp.down_proj                             483      0     1.316e-02    2.553e-03    1.341e-02
  model.layers.12.mlp.gate_proj                             483      0     1.003e-02    3.475e-03    1.061e-02
  model.layers.12.mlp.up_proj                               483      0     1.073e-02    3.555e-03    1.130e-02
  model.layers.13.linear_attn.in_proj_qkv                   161    322     1.292e-02    2.241e-03    1.311e-02
  model.layers.13.linear_attn.in_proj_z                     322    161     1.418e-02    1.971e-03    1.432e-02
  model.layers.13.linear_attn.out_proj                      322    161     1.683e-02    2.176e-03    1.697e-02
  model.layers.13.mlp.down_proj                             483      0     1.372e-02    2.566e-03    1.396e-02
  model.layers.13.mlp.gate_proj                             483      0     1.006e-02    3.483e-03    1.065e-02
  model.layers.13.mlp.up_proj                               483      0     1.071e-02    3.558e-03    1.128e-02
  model.layers.14.linear_attn.in_proj_qkv                   161    322     1.271e-02    2.230e-03    1.290e-02
  model.layers.14.linear_attn.in_proj_z                     322    161     1.336e-02    1.881e-03    1.349e-02
  model.layers.14.linear_attn.out_proj                      322    161     1.586e-02    2.216e-03    1.601e-02
  model.layers.14.mlp.down_proj                             483      0     1.299e-02    2.463e-03    1.322e-02
  model.layers.14.mlp.gate_proj                             483      0     9.956e-03    3.503e-03    1.056e-02
  model.layers.14.mlp.up_proj                               483      0     1.064e-02    3.566e-03    1.122e-02
  model.layers.15.mlp.down_proj                             483      0     1.350e-02    2.531e-03    1.374e-02
  model.layers.15.mlp.gate_proj                             483      0     1.010e-02    3.487e-03    1.069e-02
  model.layers.15.mlp.up_proj                               483      0     1.068e-02    3.563e-03    1.126e-02
  model.layers.15.self_attn.k_proj                          483      0     1.453e-02    2.890e-03    1.481e-02
  model.layers.15.self_attn.o_proj                          483      0     1.398e-02    2.119e-03    1.414e-02
  model.layers.15.self_attn.q_proj                          483      0     1.300e-02    2.088e-03    1.317e-02
  model.layers.15.self_attn.v_proj                          483      0     9.817e-03    2.224e-03    1.007e-02
  model.layers.16.linear_attn.in_proj_qkv                   161    322     1.290e-02    2.245e-03    1.309e-02
  model.layers.16.linear_attn.in_proj_z                     322    161     1.355e-02    1.897e-03    1.368e-02
  model.layers.16.linear_attn.out_proj                      322    161     1.636e-02    2.282e-03    1.652e-02
  model.layers.16.mlp.down_proj                             483      0     1.300e-02    2.516e-03    1.324e-02
  model.layers.16.mlp.gate_proj                             483      0     1.005e-02    3.471e-03    1.063e-02
  model.layers.16.mlp.up_proj                               483      0     1.070e-02    3.563e-03    1.128e-02
  model.layers.17.linear_attn.in_proj_qkv                   161    322     1.288e-02    2.224e-03    1.307e-02
  model.layers.17.linear_attn.in_proj_z                     322    161     1.351e-02    1.896e-03    1.364e-02
  model.layers.17.linear_attn.out_proj                      322    161     1.776e-02    2.244e-03    1.790e-02
  model.layers.17.mlp.down_proj                             483      0     1.380e-02    2.453e-03    1.401e-02
  model.layers.17.mlp.gate_proj                             483      0     1.009e-02    3.470e-03    1.067e-02
  model.layers.17.mlp.up_proj                               483      0     1.072e-02    3.567e-03    1.129e-02
  model.layers.18.linear_attn.in_proj_qkv                   161    322     1.206e-02    2.187e-03    1.226e-02
  model.layers.18.linear_attn.in_proj_z                     322    161     1.264e-02    1.787e-03    1.276e-02
  model.layers.18.linear_attn.out_proj                      322    161     1.526e-02    2.229e-03    1.542e-02
  model.layers.18.mlp.down_proj                             483      0     1.458e-02    2.426e-03    1.478e-02
  model.layers.18.mlp.gate_proj                             483      0     1.023e-02    3.425e-03    1.079e-02
  model.layers.18.mlp.up_proj                               483      0     1.059e-02    3.530e-03    1.116e-02
  model.layers.19.mlp.down_proj                             483      0     1.587e-02    2.542e-03    1.608e-02
  model.layers.19.mlp.gate_proj                             483      0     1.007e-02    3.395e-03    1.063e-02
  model.layers.19.mlp.up_proj                               483      0     1.096e-02    3.578e-03    1.153e-02
  model.layers.19.self_attn.k_proj                          483      0     1.411e-02    2.794e-03    1.438e-02
  model.layers.19.self_attn.o_proj                          483      0     1.813e-02    2.164e-03    1.826e-02
  model.layers.19.self_attn.q_proj                          483      0     9.504e-03    1.929e-03    9.697e-03
  model.layers.19.self_attn.v_proj                          483      0     1.045e-02    2.357e-03    1.071e-02
  model.layers.2.linear_attn.in_proj_qkv                    161    322     1.248e-02    2.213e-03    1.268e-02
  model.layers.2.linear_attn.in_proj_z                      322    161     1.399e-02    1.926e-03    1.412e-02
  model.layers.2.linear_attn.out_proj                       322    161     1.795e-02    2.320e-03    1.810e-02
  model.layers.2.mlp.down_proj                              483      0     1.706e-02    2.375e-03    1.722e-02
  model.layers.2.mlp.gate_proj                              483      0     1.029e-02    3.535e-03    1.088e-02
  model.layers.2.mlp.up_proj                                483      0     1.062e-02    3.558e-03    1.120e-02
  model.layers.20.linear_attn.in_proj_qkv                   161    322     1.214e-02    2.136e-03    1.233e-02
  model.layers.20.linear_attn.in_proj_z                     322    161     1.271e-02    1.793e-03    1.284e-02
  model.layers.20.linear_attn.out_proj                      322    161     1.578e-02    2.224e-03    1.594e-02
  model.layers.20.mlp.down_proj                             483      0     1.593e-02    2.583e-03    1.614e-02
  model.layers.20.mlp.gate_proj                             483      0     1.008e-02    3.408e-03    1.064e-02
  model.layers.20.mlp.up_proj                               483      0     1.126e-02    3.603e-03    1.183e-02
  model.layers.21.linear_attn.in_proj_qkv                   161    322     1.199e-02    2.201e-03    1.219e-02
  model.layers.21.linear_attn.in_proj_z                     322    161     1.162e-02    1.672e-03    1.174e-02
  model.layers.21.linear_attn.out_proj                      322    161     1.702e-02    2.231e-03    1.716e-02
  model.layers.21.mlp.down_proj                             483      0     1.721e-02    2.588e-03    1.740e-02
  model.layers.21.mlp.gate_proj                             483      0     1.010e-02    3.421e-03    1.066e-02
  model.layers.21.mlp.up_proj                               483      0     1.122e-02    3.605e-03    1.178e-02
  model.layers.22.linear_attn.in_proj_qkv                   161    322     1.201e-02    2.241e-03    1.222e-02
  model.layers.22.linear_attn.in_proj_z                     322    161     1.204e-02    1.712e-03    1.216e-02
  model.layers.22.linear_attn.out_proj                      322    161     1.579e-02    2.305e-03    1.595e-02
  model.layers.22.mlp.down_proj                             483      0     1.513e-02    2.552e-03    1.533e-02
  model.layers.22.mlp.gate_proj                             483      0     1.052e-02    3.137e-03    1.098e-02
  model.layers.22.mlp.up_proj                               483      0     1.142e-02    3.622e-03    1.197e-02
  model.layers.23.mlp.down_proj                             483      0     1.569e-02    2.559e-03    1.589e-02
  model.layers.23.mlp.gate_proj                             483      0     1.009e-02    3.369e-03    1.064e-02
  model.layers.23.mlp.up_proj                               483      0     1.141e-02    3.621e-03    1.197e-02
  model.layers.23.self_attn.k_proj                          483      0     1.378e-02    2.671e-03    1.404e-02
  model.layers.23.self_attn.o_proj                          483      0     1.682e-02    2.421e-03    1.699e-02
  model.layers.23.self_attn.q_proj                          483      0     9.478e-03    1.963e-03    9.680e-03
  model.layers.23.self_attn.v_proj                          483      0     1.034e-02    2.327e-03    1.060e-02
  model.layers.24.linear_attn.in_proj_qkv                   161    322     1.188e-02    2.253e-03    1.209e-02
  model.layers.24.linear_attn.in_proj_z                     322    161     1.221e-02    1.730e-03    1.234e-02
  model.layers.24.linear_attn.out_proj                      322    161     1.603e-02    2.268e-03    1.619e-02
  model.layers.24.mlp.down_proj                             483      0     1.458e-02    2.617e-03    1.481e-02
  model.layers.24.mlp.gate_proj                             483      0     1.026e-02    3.317e-03    1.078e-02
  model.layers.24.mlp.up_proj                               483      0     1.144e-02    3.613e-03    1.199e-02
  model.layers.25.linear_attn.in_proj_qkv                   161    322     1.142e-02    2.212e-03    1.163e-02
  model.layers.25.linear_attn.in_proj_z                     322    161     1.279e-02    1.788e-03    1.291e-02
  model.layers.25.linear_attn.out_proj                      322    161     1.473e-02    2.175e-03    1.489e-02
  model.layers.25.mlp.down_proj                             483      0     1.393e-02    2.554e-03    1.416e-02
  model.layers.25.mlp.gate_proj                             483      0     1.054e-02    3.322e-03    1.105e-02
  model.layers.25.mlp.up_proj                               483      0     1.130e-02    3.600e-03    1.185e-02
  model.layers.26.linear_attn.in_proj_qkv                   161    322     1.149e-02    2.236e-03    1.171e-02
  model.layers.26.linear_attn.in_proj_z                     322    161     1.260e-02    1.796e-03    1.272e-02
  model.layers.26.linear_attn.out_proj                      322    161     1.492e-02    2.343e-03    1.510e-02
  model.layers.26.mlp.down_proj                             483      0     1.270e-02    2.468e-03    1.294e-02
  model.layers.26.mlp.gate_proj                             483      0     1.112e-02    3.297e-03    1.160e-02
  model.layers.26.mlp.up_proj                               483      0     1.086e-02    3.552e-03    1.143e-02
  model.layers.27.mlp.down_proj                             483      0     1.335e-02    2.575e-03    1.359e-02
  model.layers.27.mlp.gate_proj                             483      0     1.079e-02    3.204e-03    1.126e-02
  model.layers.27.mlp.up_proj                               483      0     1.090e-02    3.555e-03    1.146e-02
  model.layers.27.self_attn.k_proj                          483      0     1.215e-02    2.498e-03    1.240e-02
  model.layers.27.self_attn.o_proj                          483      0     1.707e-02    2.745e-03    1.729e-02
  model.layers.27.self_attn.q_proj                          483      0     1.020e-02    2.062e-03    1.041e-02
  model.layers.27.self_attn.v_proj                          483      0     9.850e-03    2.252e-03    1.011e-02
  model.layers.28.linear_attn.in_proj_qkv                   161    322     1.177e-02    2.217e-03    1.198e-02
  model.layers.28.linear_attn.in_proj_z                     322    161     1.287e-02    1.817e-03    1.299e-02
  model.layers.28.linear_attn.out_proj                      322    161     1.479e-02    2.297e-03    1.497e-02
  model.layers.28.mlp.down_proj                             483      0     1.301e-02    2.600e-03    1.326e-02
  model.layers.28.mlp.gate_proj                             483      0     1.104e-02    3.175e-03    1.149e-02
  model.layers.28.mlp.up_proj                               483      0     1.084e-02    3.537e-03    1.140e-02
  model.layers.29.linear_attn.in_proj_qkv                   161    322     1.204e-02    2.246e-03    1.225e-02
  model.layers.29.linear_attn.in_proj_z                     322    161     1.397e-02    1.967e-03    1.411e-02
  model.layers.29.linear_attn.out_proj                      322    161     1.655e-02    2.350e-03    1.671e-02
  model.layers.29.mlp.down_proj                             483      0     1.348e-02    2.623e-03    1.373e-02
  model.layers.29.mlp.gate_proj                             483      0     1.114e-02    3.188e-03    1.159e-02
  model.layers.29.mlp.up_proj                               483      0     1.081e-02    3.536e-03    1.137e-02
  model.layers.3.mlp.down_proj                              483      0     1.418e-02    2.577e-03    1.441e-02
  model.layers.3.mlp.gate_proj                              483      0     1.018e-02    3.528e-03    1.077e-02
  model.layers.3.mlp.up_proj                                483      0     1.101e-02    3.601e-03    1.158e-02
  model.layers.3.self_attn.k_proj                           483      0     1.249e-02    2.587e-03    1.275e-02
  model.layers.3.self_attn.o_proj                           483      0     1.494e-02    2.175e-03    1.510e-02
  model.layers.3.self_attn.q_proj                           483      0     1.169e-02    2.033e-03    1.186e-02
  model.layers.3.self_attn.v_proj                           483      0     1.263e-02    2.311e-03    1.284e-02
  model.layers.30.linear_attn.in_proj_qkv                   161    322     1.189e-02    2.230e-03    1.210e-02
  model.layers.30.linear_attn.in_proj_z                     322    161     1.349e-02    1.886e-03    1.362e-02
  model.layers.30.linear_attn.out_proj                      322    161     1.622e-02    2.412e-03    1.640e-02
  model.layers.30.mlp.down_proj                             483      0     1.285e-02    2.561e-03    1.310e-02
  model.layers.30.mlp.gate_proj                             483      0     1.171e-02    3.188e-03    1.214e-02
  model.layers.30.mlp.up_proj                               483      0     1.067e-02    3.521e-03    1.123e-02
  model.layers.31.mlp.down_proj                             483      0     1.320e-02    2.596e-03    1.345e-02
  model.layers.31.mlp.gate_proj                             483      0     1.148e-02    3.154e-03    1.191e-02
  model.layers.31.mlp.up_proj                               483      0     1.090e-02    3.518e-03    1.146e-02
  model.layers.31.self_attn.k_proj                          483      0     1.207e-02    2.519e-03    1.233e-02
  model.layers.31.self_attn.o_proj                          483      0     2.214e-02    3.029e-03    2.235e-02
  model.layers.31.self_attn.q_proj                          483      0     1.015e-02    2.074e-03    1.036e-02
  model.layers.31.self_attn.v_proj                          483      0     9.912e-03    2.191e-03    1.014e-02
  model.layers.32.linear_attn.in_proj_qkv                   161    322     1.282e-02    2.287e-03    1.303e-02
  model.layers.32.linear_attn.in_proj_z                     322    161     1.411e-02    1.985e-03    1.425e-02
  model.layers.32.linear_attn.out_proj                      322    161     1.567e-02    2.303e-03    1.585e-02
  model.layers.32.mlp.down_proj                             483      0     1.301e-02    2.618e-03    1.327e-02
  model.layers.32.mlp.gate_proj                             483      0     1.165e-02    3.107e-03    1.206e-02
  model.layers.32.mlp.up_proj                               483      0     1.108e-02    3.522e-03    1.162e-02
  model.layers.33.linear_attn.in_proj_qkv                   161    322     1.259e-02    2.298e-03    1.280e-02
  model.layers.33.linear_attn.in_proj_z                     322    161     1.386e-02    1.988e-03    1.400e-02
  model.layers.33.linear_attn.out_proj                      322    161     1.745e-02    2.367e-03    1.761e-02
  model.layers.33.mlp.down_proj                             483      0     1.342e-02    2.602e-03    1.367e-02
  model.layers.33.mlp.gate_proj                             483      0     1.193e-02    3.114e-03    1.232e-02
  model.layers.33.mlp.up_proj                               483      0     1.124e-02    3.542e-03    1.178e-02
  model.layers.34.linear_attn.in_proj_qkv                   161    322     1.252e-02    2.278e-03    1.273e-02
  model.layers.34.linear_attn.in_proj_z                     322    161     1.343e-02    1.914e-03    1.357e-02
  model.layers.34.linear_attn.out_proj                      322    161     1.609e-02    2.363e-03    1.626e-02
  model.layers.34.mlp.down_proj                             483      0     1.377e-02    2.610e-03    1.401e-02
  model.layers.34.mlp.gate_proj                             483      0     1.207e-02    3.070e-03    1.245e-02
  model.layers.34.mlp.up_proj                               483      0     1.115e-02    3.505e-03    1.169e-02
  model.layers.35.mlp.down_proj                             483      0     1.414e-02    2.562e-03    1.437e-02
  model.layers.35.mlp.gate_proj                             483      0     1.066e-02    3.068e-03    1.109e-02
  model.layers.35.mlp.up_proj                               483      0     1.138e-02    3.583e-03    1.193e-02
  model.layers.35.self_attn.k_proj                          483      0     1.177e-02    2.500e-03    1.203e-02
  model.layers.35.self_attn.o_proj                          483      0     1.997e-02    2.809e-03    2.017e-02
  model.layers.35.self_attn.q_proj                          483      0     1.072e-02    2.117e-03    1.092e-02
  model.layers.35.self_attn.v_proj                          483      0     1.071e-02    2.262e-03    1.094e-02
  model.layers.36.linear_attn.in_proj_qkv                   161    322     1.295e-02    2.276e-03    1.314e-02
  model.layers.36.linear_attn.in_proj_z                     322    161     1.212e-02    1.746e-03    1.224e-02
  model.layers.36.linear_attn.out_proj                      322    161     1.667e-02    2.261e-03    1.682e-02
  model.layers.36.mlp.down_proj                             483      0     1.511e-02    2.627e-03    1.534e-02
  model.layers.36.mlp.gate_proj                             483      0     1.026e-02    3.145e-03    1.074e-02
  model.layers.36.mlp.up_proj                               483      0     1.154e-02    3.603e-03    1.209e-02
  model.layers.37.linear_attn.in_proj_qkv                   161    322     1.223e-02    2.242e-03    1.243e-02
  model.layers.37.linear_attn.in_proj_z                     322    161     1.058e-02    1.597e-03    1.071e-02
  model.layers.37.linear_attn.out_proj                      322    161     1.838e-02    2.199e-03    1.851e-02
  model.layers.37.mlp.down_proj                             483      0     1.541e-02    2.734e-03    1.565e-02
  model.layers.37.mlp.gate_proj                             483      0     1.055e-02    3.235e-03    1.104e-02
  model.layers.37.mlp.up_proj                               483      0     1.172e-02    3.628e-03    1.227e-02
  model.layers.38.linear_attn.in_proj_qkv                   161    322     1.201e-02    2.319e-03    1.223e-02
  model.layers.38.linear_attn.in_proj_z                     322    161     1.100e-02    1.630e-03    1.112e-02
  model.layers.38.linear_attn.out_proj                      322    161     1.656e-02    2.310e-03    1.672e-02
  model.layers.38.mlp.down_proj                             483      0     1.595e-02    2.700e-03    1.617e-02
  model.layers.38.mlp.gate_proj                             483      0     1.069e-02    3.065e-03    1.112e-02
  model.layers.38.mlp.up_proj                               483      0     1.171e-02    3.642e-03    1.226e-02
  model.layers.39.mlp.down_proj                             483      0     1.441e-02    2.655e-03    1.465e-02
  model.layers.39.mlp.gate_proj                             483      0     9.965e-03    3.158e-03    1.045e-02
  model.layers.39.mlp.up_proj                               483      0     1.180e-02    3.648e-03    1.235e-02
  model.layers.39.self_attn.k_proj                          483      0     1.179e-02    2.470e-03    1.205e-02
  model.layers.39.self_attn.o_proj                          483      0     2.077e-02    2.841e-03    2.097e-02
  model.layers.39.self_attn.q_proj                          483      0     1.045e-02    2.093e-03    1.066e-02
  model.layers.39.self_attn.v_proj                          483      0     1.090e-02    2.329e-03    1.114e-02
  model.layers.4.linear_attn.in_proj_qkv                    161    322     1.192e-02    2.310e-03    1.214e-02
  model.layers.4.linear_attn.in_proj_z                      322    161     1.247e-02    1.774e-03    1.260e-02
  model.layers.4.linear_attn.out_proj                       322    161     1.788e-02    2.410e-03    1.804e-02
  model.layers.4.mlp.down_proj                              483      0     1.444e-02    2.610e-03    1.468e-02
  model.layers.4.mlp.gate_proj                              483      0     1.015e-02    3.511e-03    1.074e-02
  model.layers.4.mlp.up_proj                                483      0     1.123e-02    3.616e-03    1.180e-02
  model.layers.40.linear_attn.in_proj_qkv                   161    322     1.226e-02    2.281e-03    1.247e-02
  model.layers.40.linear_attn.in_proj_z                     322    161     1.172e-02    1.686e-03    1.185e-02
  model.layers.40.linear_attn.out_proj                      322    161     1.611e-02    2.266e-03    1.626e-02
  model.layers.40.mlp.down_proj                             483      0     1.485e-02    2.675e-03    1.508e-02
  model.layers.40.mlp.gate_proj                             483      0     1.008e-02    3.132e-03    1.056e-02
  model.layers.40.mlp.up_proj                               483      0     1.164e-02    3.641e-03    1.220e-02
  model.layers.41.linear_attn.in_proj_qkv                   161    322     1.175e-02    2.257e-03    1.197e-02
  model.layers.41.linear_attn.in_proj_z                     322    161     1.225e-02    1.732e-03    1.238e-02
  model.layers.41.linear_attn.out_proj                      322    161     1.646e-02    2.257e-03    1.662e-02
  model.layers.41.mlp.down_proj                             483      0     1.475e-02    2.702e-03    1.500e-02
  model.layers.41.mlp.gate_proj                             483      0     1.085e-02    3.212e-03    1.131e-02
  model.layers.41.mlp.up_proj                               483      0     1.166e-02    3.633e-03    1.222e-02
  model.layers.42.linear_attn.in_proj_qkv                   161    322     1.163e-02    2.277e-03    1.185e-02
  model.layers.42.linear_attn.in_proj_z                     322    161     1.240e-02    1.748e-03    1.253e-02
  model.layers.42.linear_attn.out_proj                      322    161     1.574e-02    2.284e-03    1.591e-02
  model.layers.42.mlp.down_proj                             483      0     1.476e-02    2.644e-03    1.499e-02
  model.layers.42.mlp.gate_proj                             483      0     1.173e-02    3.240e-03    1.217e-02
  model.layers.42.mlp.up_proj                               483      0     1.143e-02    3.584e-03    1.198e-02
  model.layers.43.mlp.down_proj                             483      0     1.381e-02    2.670e-03    1.407e-02
  model.layers.43.mlp.gate_proj                             483      0     1.085e-02    3.218e-03    1.132e-02
  model.layers.43.mlp.up_proj                               483      0     1.154e-02    3.618e-03    1.210e-02
  model.layers.43.self_attn.k_proj                          483      0     1.253e-02    2.452e-03    1.276e-02
  model.layers.43.self_attn.o_proj                          483      0     2.163e-02    3.125e-03    2.186e-02
  model.layers.43.self_attn.q_proj                          483      0     1.062e-02    2.091e-03    1.082e-02
  model.layers.43.self_attn.v_proj                          483      0     1.054e-02    2.252e-03    1.078e-02
  model.layers.44.linear_attn.in_proj_qkv                   161    322     1.206e-02    2.230e-03    1.226e-02
  model.layers.44.linear_attn.in_proj_z                     322    161     1.228e-02    1.698e-03    1.240e-02
  model.layers.44.linear_attn.out_proj                      322    161     1.628e-02    2.298e-03    1.644e-02
  model.layers.44.mlp.down_proj                             483      0     1.414e-02    2.712e-03    1.440e-02
  model.layers.44.mlp.gate_proj                             483      0     1.083e-02    3.259e-03    1.131e-02
  model.layers.44.mlp.up_proj                               483      0     1.171e-02    3.618e-03    1.225e-02
  model.layers.45.linear_attn.in_proj_qkv                   161    322     1.209e-02    2.277e-03    1.230e-02
  model.layers.45.linear_attn.in_proj_z                     322    161     1.265e-02    1.759e-03    1.277e-02
  model.layers.45.linear_attn.out_proj                      322    161     1.814e-02    2.430e-03    1.830e-02
  model.layers.45.mlp.down_proj                             483      0     1.437e-02    2.740e-03    1.463e-02
  model.layers.45.mlp.gate_proj                             483      0     1.082e-02    3.289e-03    1.131e-02
  model.layers.45.mlp.up_proj                               483      0     1.157e-02    3.623e-03    1.212e-02
  model.layers.46.linear_attn.in_proj_qkv                   161    322     1.198e-02    2.245e-03    1.219e-02
  model.layers.46.linear_attn.in_proj_z                     322    161     1.186e-02    1.650e-03    1.197e-02
  model.layers.46.linear_attn.out_proj                      322    161     1.868e-02    2.358e-03    1.883e-02
  model.layers.46.mlp.down_proj                             483      0     1.426e-02    2.688e-03    1.451e-02
  model.layers.46.mlp.gate_proj                             483      0     1.112e-02    3.310e-03    1.161e-02
  model.layers.46.mlp.up_proj                               483      0     1.151e-02    3.600e-03    1.206e-02
  model.layers.47.mlp.down_proj                             483      0     1.426e-02    2.639e-03    1.450e-02
  model.layers.47.mlp.gate_proj                             483      0     1.057e-02    3.335e-03    1.108e-02
  model.layers.47.mlp.up_proj                               483      0     1.177e-02    3.621e-03    1.232e-02
  model.layers.47.self_attn.k_proj                          483      0     1.248e-02    2.462e-03    1.272e-02
  model.layers.47.self_attn.o_proj                          483      0     1.954e-02    2.555e-03    1.971e-02
  model.layers.47.self_attn.q_proj                          483      0     9.680e-03    2.056e-03    9.896e-03
  model.layers.47.self_attn.v_proj                          483      0     1.068e-02    2.234e-03    1.091e-02
  model.layers.48.linear_attn.in_proj_qkv                   161    322     1.274e-02    2.267e-03    1.294e-02
  model.layers.48.linear_attn.in_proj_z                     322    161     1.210e-02    1.652e-03    1.221e-02
  model.layers.48.linear_attn.out_proj                      322    161     1.720e-02    2.416e-03    1.737e-02
  model.layers.48.mlp.down_proj                             483      0     1.479e-02    2.684e-03    1.503e-02
  model.layers.48.mlp.gate_proj                             483      0     1.036e-02    3.332e-03    1.088e-02
  model.layers.48.mlp.up_proj                               483      0     1.170e-02    3.606e-03    1.224e-02
  model.layers.49.linear_attn.in_proj_qkv                   161    322     1.211e-02    2.277e-03    1.232e-02
  model.layers.49.linear_attn.in_proj_z                     322    161     1.156e-02    1.608e-03    1.167e-02
  model.layers.49.linear_attn.out_proj                      322    161     1.910e-02    2.175e-03    1.923e-02
  model.layers.49.mlp.down_proj                             483      0     1.567e-02    2.697e-03    1.590e-02
  model.layers.49.mlp.gate_proj                             483      0     1.007e-02    3.367e-03    1.062e-02
  model.layers.49.mlp.up_proj                               483      0     1.150e-02    3.563e-03    1.204e-02
  model.layers.5.linear_attn.in_proj_qkv                    161    322     1.245e-02    2.270e-03    1.266e-02
  model.layers.5.linear_attn.in_proj_z                      322    161     1.391e-02    1.910e-03    1.404e-02
  model.layers.5.linear_attn.out_proj                       322    161     1.772e-02    2.209e-03    1.785e-02
  model.layers.5.mlp.down_proj                              483      0     1.529e-02    2.519e-03    1.550e-02
  model.layers.5.mlp.gate_proj                              483      0     1.020e-02    3.534e-03    1.079e-02
  model.layers.5.mlp.up_proj                                483      0     1.128e-02    3.622e-03    1.185e-02
  model.layers.50.linear_attn.in_proj_qkv                   161    322     1.169e-02    2.255e-03    1.190e-02
  model.layers.50.linear_attn.in_proj_z                     322    161     1.090e-02    1.493e-03    1.100e-02
  model.layers.50.linear_attn.out_proj                      322    161     1.824e-02    2.348e-03    1.839e-02
  model.layers.50.mlp.down_proj                             483      0     1.517e-02    2.666e-03    1.541e-02
  model.layers.50.mlp.gate_proj                             483      0     9.861e-03    3.280e-03    1.039e-02
  model.layers.50.mlp.up_proj                               483      0     1.087e-02    3.439e-03    1.140e-02
  model.layers.51.mlp.down_proj                             483      0     1.524e-02    2.656e-03    1.548e-02
  model.layers.51.mlp.gate_proj                             483      0     8.699e-03    3.392e-03    9.339e-03
  model.layers.51.mlp.up_proj                               483      0     1.111e-02    3.552e-03    1.166e-02
  model.layers.51.self_attn.k_proj                          483      0     1.103e-02    2.402e-03    1.129e-02
  model.layers.51.self_attn.o_proj                          483      0     1.919e-02    2.299e-03    1.933e-02
  model.layers.51.self_attn.q_proj                          483      0     9.135e-03    2.024e-03    9.356e-03
  model.layers.51.self_attn.v_proj                          483      0     1.024e-02    2.274e-03    1.049e-02
  model.layers.52.linear_attn.in_proj_qkv                   161    322     9.642e-03    2.110e-03    9.868e-03
  model.layers.52.linear_attn.in_proj_z                     322    161     8.558e-03    1.286e-03    8.653e-03
  model.layers.52.linear_attn.out_proj                      322    161     1.877e-02    2.303e-03    1.891e-02
  model.layers.52.mlp.down_proj                             483      0     1.784e-02    2.685e-03    1.804e-02
  model.layers.52.mlp.gate_proj                             483      0     8.139e-03    3.398e-03    8.821e-03
  model.layers.52.mlp.up_proj                               483      0     1.105e-02    3.564e-03    1.161e-02
  model.layers.53.linear_attn.in_proj_qkv                   161    322     9.505e-03    2.128e-03    9.740e-03
  model.layers.53.linear_attn.in_proj_z                     322    161     8.688e-03    1.313e-03    8.786e-03
  model.layers.53.linear_attn.out_proj                      322    161     1.699e-02    2.275e-03    1.714e-02
  model.layers.53.mlp.down_proj                             483      0     1.846e-02    2.768e-03    1.867e-02
  model.layers.53.mlp.gate_proj                             483      0     7.902e-03    3.398e-03    8.601e-03
  model.layers.53.mlp.up_proj                               483      0     1.096e-02    3.568e-03    1.153e-02
  model.layers.54.linear_attn.in_proj_qkv                   161    322     9.920e-03    2.231e-03    1.017e-02
  model.layers.54.linear_attn.in_proj_z                     322    161     8.309e-03    1.278e-03    8.406e-03
  model.layers.54.linear_attn.out_proj                      322    161     1.749e-02    2.127e-03    1.762e-02
  model.layers.54.mlp.down_proj                             483      0     1.815e-02    2.477e-03    1.832e-02
  model.layers.54.mlp.gate_proj                             483      0     7.284e-03    3.349e-03    8.016e-03
  model.layers.54.mlp.up_proj                               483      0     9.161e-03    3.418e-03    9.777e-03
  model.layers.55.mlp.down_proj                             483      0     1.696e-02    2.620e-03    1.716e-02
  model.layers.55.mlp.gate_proj                             483      0     7.234e-03    3.384e-03    7.986e-03
  model.layers.55.mlp.up_proj                               483      0     1.132e-02    3.603e-03    1.188e-02
  model.layers.55.self_attn.k_proj                          483      0     1.085e-02    2.400e-03    1.112e-02
  model.layers.55.self_attn.o_proj                          483      0     1.731e-02    2.199e-03    1.745e-02
  model.layers.55.self_attn.q_proj                          483      0     8.915e-03    2.106e-03    9.160e-03
  model.layers.55.self_attn.v_proj                          483      0     9.482e-03    2.219e-03    9.739e-03
  model.layers.56.linear_attn.in_proj_qkv                   161    322     9.010e-03    2.090e-03    9.249e-03
  model.layers.56.linear_attn.in_proj_z                     322    161     8.724e-03    1.331e-03    8.825e-03
  model.layers.56.linear_attn.out_proj                      322    161     1.741e-02    2.098e-03    1.754e-02
  model.layers.56.mlp.down_proj                             483      0     1.968e-02    2.530e-03    1.984e-02
  model.layers.56.mlp.gate_proj                             483      0     7.328e-03    3.347e-03    8.057e-03
  model.layers.56.mlp.up_proj                               483      0     1.133e-02    3.595e-03    1.188e-02
  model.layers.57.linear_attn.in_proj_qkv                   161    322     8.646e-03    2.053e-03    8.886e-03
  model.layers.57.linear_attn.in_proj_z                     322    161     8.532e-03    1.300e-03    8.631e-03
  model.layers.57.linear_attn.out_proj                      322    161     1.595e-02    2.510e-03    1.615e-02
  model.layers.57.mlp.down_proj                             483      0     1.634e-02    2.796e-03    1.657e-02
  model.layers.57.mlp.gate_proj                             483      0     7.144e-03    3.321e-03    7.878e-03
  model.layers.57.mlp.up_proj                               483      0     1.116e-02    3.600e-03    1.173e-02
  model.layers.58.linear_attn.in_proj_qkv                   161    322     9.203e-03    2.131e-03    9.447e-03
  model.layers.58.linear_attn.in_proj_z                     322    161     8.789e-03    1.313e-03    8.885e-03
  model.layers.58.linear_attn.out_proj                      322    161     1.895e-02    2.411e-03    1.910e-02
  model.layers.58.mlp.down_proj                             483      0     1.666e-02    2.607e-03    1.686e-02
  model.layers.58.mlp.gate_proj                             483      0     7.020e-03    3.313e-03    7.763e-03
  model.layers.58.mlp.up_proj                               483      0     1.044e-02    3.547e-03    1.102e-02
  model.layers.59.mlp.down_proj                             483      0     1.667e-02    2.502e-03    1.685e-02
  model.layers.59.mlp.gate_proj                             483      0     6.604e-03    3.306e-03    7.388e-03
  model.layers.59.mlp.up_proj                               483      0     9.269e-03    3.450e-03    9.890e-03
  model.layers.59.self_attn.k_proj                          483      0     1.016e-02    2.336e-03    1.043e-02
  model.layers.59.self_attn.o_proj                          483      0     1.802e-02    2.288e-03    1.816e-02
  model.layers.59.self_attn.q_proj                          483      0     8.663e-03    2.118e-03    8.920e-03
  model.layers.59.self_attn.v_proj                          483      0     9.077e-03    2.160e-03    9.329e-03
  model.layers.6.linear_attn.in_proj_qkv                    161    322     1.263e-02    2.269e-03    1.283e-02
  model.layers.6.linear_attn.in_proj_z                      322    161     1.427e-02    1.989e-03    1.441e-02
  model.layers.6.linear_attn.out_proj                       322    161     1.682e-02    2.292e-03    1.697e-02
  model.layers.6.mlp.down_proj                              483      0     1.358e-02    2.400e-03    1.379e-02
  model.layers.6.mlp.gate_proj                              483      0     1.045e-02    3.493e-03    1.102e-02
  model.layers.6.mlp.up_proj                                483      0     1.113e-02    3.563e-03    1.168e-02
  model.layers.60.linear_attn.in_proj_qkv                   161    322     7.947e-03    1.973e-03    8.189e-03
  model.layers.60.linear_attn.in_proj_z                     322    161     7.538e-03    1.195e-03    7.633e-03
  model.layers.60.linear_attn.out_proj                      322    161     1.696e-02    2.504e-03    1.714e-02
  model.layers.60.mlp.down_proj                             483      0     1.583e-02    2.704e-03    1.606e-02
  model.layers.60.mlp.gate_proj                             483      0     6.834e-03    3.243e-03    7.564e-03
  model.layers.60.mlp.up_proj                               483      0     9.170e-03    3.388e-03    9.776e-03
  model.layers.61.linear_attn.in_proj_qkv                   161    322     8.878e-03    2.088e-03    9.120e-03
  model.layers.61.linear_attn.in_proj_z                     322    161     7.963e-03    1.246e-03    8.060e-03
  model.layers.61.linear_attn.out_proj                      322    161     1.669e-02    2.348e-03    1.686e-02
  model.layers.61.mlp.down_proj                             483      0     1.498e-02    2.664e-03    1.521e-02
  model.layers.61.mlp.gate_proj                             483      0     6.562e-03    3.195e-03    7.300e-03
  model.layers.61.mlp.up_proj                               483      0     8.694e-03    3.315e-03    9.304e-03
  model.layers.62.linear_attn.in_proj_qkv                   161    322     8.149e-03    1.974e-03    8.384e-03
  model.layers.62.linear_attn.in_proj_z                     322    161     7.410e-03    1.221e-03    7.508e-03
  model.layers.62.linear_attn.out_proj                      322    161     1.516e-02    2.354e-03    1.535e-02
  model.layers.62.mlp.down_proj                             483      0     1.496e-02    2.533e-03    1.518e-02
  model.layers.62.mlp.gate_proj                             483      0     6.266e-03    3.134e-03    7.006e-03
  model.layers.62.mlp.up_proj                               483      0     6.854e-03    3.145e-03    7.542e-03
  model.layers.63.mlp.down_proj                             483      0     1.191e-02    2.011e-03    1.208e-02
  model.layers.63.mlp.gate_proj                             483      0     5.660e-03    3.068e-03    6.439e-03
  model.layers.63.mlp.up_proj                               483      0     4.709e-03    3.050e-03    5.610e-03
  model.layers.63.self_attn.k_proj                          483      0     1.072e-02    2.414e-03    1.098e-02
  model.layers.63.self_attn.o_proj                          483      0     1.293e-02    2.012e-03    1.308e-02
  model.layers.63.self_attn.q_proj                          483      0     7.013e-03    1.939e-03    7.277e-03
  model.layers.63.self_attn.v_proj                          483      0     4.703e-03    1.727e-03    5.007e-03
  model.layers.7.mlp.down_proj                              483      0     1.425e-02    2.578e-03    1.448e-02
  model.layers.7.mlp.gate_proj                              483      0     1.041e-02    3.566e-03    1.100e-02
  model.layers.7.mlp.up_proj                                483      0     1.135e-02    3.626e-03    1.191e-02
  model.layers.7.self_attn.k_proj                           483      0     1.383e-02    2.714e-03    1.410e-02
  model.layers.7.self_attn.o_proj                           483      0     1.329e-02    2.267e-03    1.347e-02
  model.layers.7.self_attn.q_proj                           483      0     1.250e-02    2.079e-03    1.267e-02
  model.layers.7.self_attn.v_proj                           483      0     1.102e-02    2.332e-03    1.126e-02
  model.layers.8.linear_attn.in_proj_qkv                    161    322     1.243e-02    2.248e-03    1.263e-02
  model.layers.8.linear_attn.in_proj_z                      322    161     1.308e-02    1.811e-03    1.320e-02
  model.layers.8.linear_attn.out_proj                       322    161     1.617e-02    2.281e-03    1.632e-02
  model.layers.8.mlp.down_proj                              483      0     1.507e-02    2.557e-03    1.528e-02
  model.layers.8.mlp.gate_proj                              483      0     1.051e-02    3.563e-03    1.109e-02
  model.layers.8.mlp.up_proj                                483      0     1.129e-02    3.619e-03    1.186e-02
  model.layers.9.linear_attn.in_proj_qkv                    161    322     1.309e-02    2.233e-03    1.328e-02
  model.layers.9.linear_attn.in_proj_z                      322    161     1.361e-02    1.895e-03    1.374e-02
  model.layers.9.linear_attn.out_proj                       322    161     1.602e-02    2.267e-03    1.618e-02
  model.layers.9.mlp.down_proj                              483      0     1.345e-02    2.475e-03    1.368e-02
  model.layers.9.mlp.gate_proj                              483      0     1.058e-02    3.577e-03    1.117e-02
  model.layers.9.mlp.up_proj                                483      0     1.119e-02    3.610e-03    1.176e-02

INT8 ARM CENSUS -- per-arm, from the arm's own delta, not a cumulative counter.
  arm          i8 calls       i8 rows   excluded   fallback
  INT8            64000     131072000          0          0
  INT8NOQ         56320     115343360       7680       7680
  INT8NOGDN       40960      83886080      23040      23040
  SIMROW          64000 simulated GEMMs
  SIMG128         64000 simulated GEMMs
  SIMG32          64000 simulated GEMMs

ARM RECEIPTS -- what each arm actually executed, from the engine's
own per-call census.  fused/standalone is the reconstruct variant;
acc16/acc32 is the cuBLAS compute type leg B asked hgemm for.
  arm       legB calls          rows     fused  standalone     acc16     acc32   fp32pin
  REF            64000     131072000     64000           0     64000         0         0
  REF2           64000     131072000     64000           0     64000         0         0
  STD            64000     131072000         0       64000     64000         0         0
  INT8           64000     131072000         0       64000     64000         0         0
  INT8NOQ        64000     131072000         0       64000     64000         0         0
  INT8NOGDN       64000     131072000         0       64000     64000         0         0
  SIMROW         64000     131072000         0       64000     64000         0         0
  SIMG128        64000     131072000         0       64000     64000         0         0
  SIMG32         64000     131072000         0       64000     64000         0         0
  all arms match their expected leg/accumulator signature
  prefill positions aligned: [8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8] per prompt, identical across all 9 arms

THE TAIL over PREFILL positions only -- one per prefill chunk, so
every arm scores the SAME context. Decode positions are reported
separately and are NOT a numerics measure: once a token flips, the
arms are reading different contexts. The control proves it -- pooled
over decode, STD (which contains no int8 at all) has the same max KL
as INT8, so a pooled tail cannot attribute anything to the kernel.

arm            n        mean         p50         p95         p99         MAX     argmax flip
REF          160   0.000e+00   0.000e+00   0.000e+00   0.000e+00   0.000e+00 0/160 =   0.00%
REF2         160   0.000e+00   0.000e+00   0.000e+00   0.000e+00   0.000e+00 0/160 =   0.00%
STD          160   1.571e-02   3.134e-04   8.107e-03   7.665e-01   1.167e+00 4/160 =   2.50%
INT8         160   7.370e-02   9.407e-04   1.852e-02   2.040e+00   8.940e+00 5/160 =   3.12%
INT8NOQ      160   4.343e-02   8.923e-04   1.307e-02   1.917e+00   3.688e+00 6/160 =   3.75%
INT8NOGDN    160   1.745e-02   7.248e-04   3.263e-02   3.448e-01   1.686e+00 4/160 =   2.50%
SIMROW       160   1.078e-02   7.276e-04   1.418e-02   2.130e-01   1.044e+00 6/160 =   3.75%
SIMG128      160   1.541e-02   2.622e-04   9.996e-03   9.515e-01   1.170e+00 2/160 =   1.25%
SIMG32       160   1.176e-02   3.846e-04   8.746e-03   2.687e-01   1.112e+00 3/160 =   1.88%

Tail attribution -- the 10 worst INT8 prefill positions, with the
control's KL at the SAME position. A tail that is positional shows
the control elevated too; a tail that is the kernel's does not.
   prompt  chunk      INT8 KL       STD KL    INT8NOQ KL  INT8NOGDN KL     SIMROW KL    SIMG128 KL     SIMG32 KL  ratio INT8/STD
       16      6    8.940e+00    2.786e-02     1.368e-02     4.685e-02     5.223e-02     1.553e-02     9.456e-03          320.88
        0      7    2.040e+00    1.167e+00     1.917e+00     9.463e-02     2.130e-01     1.170e+00     1.112e+00            1.75
        0      1    3.977e-01    7.665e-01     3.688e+00     1.686e+00     1.044e+00     9.515e-01     1.065e-01            0.52
        2      7    5.191e-02    1.035e-03     4.446e-03     9.323e-03     4.090e-03     4.406e-03     3.556e-03           50.15
        2      3    2.703e-02    3.618e-01     1.011e-01     1.008e-01     1.030e-03     2.425e-02     2.195e-01            0.07
        0      2    2.209e-02    2.338e-03     7.990e-03     2.077e-02     8.927e-03     9.529e-03     1.537e-03            9.45
       18      7    2.161e-02    1.509e-03     4.455e-02     1.286e-02     5.880e-03     4.340e-03     3.533e-03           14.32
       14      3    1.852e-02    1.122e-03     9.526e-03     8.139e-03     4.090e-03     3.531e-03     2.922e-03           16.50
        3      4    1.831e-02    2.425e-04     9.753e-03     1.217e-02     2.784e-02     2.447e-03     1.715e-03           75.51
       17      5    1.685e-02    9.180e-03     1.847e-02     1.033e-02     7.866e-03     3.055e-03     2.398e-02            1.84
  Spearman rank correlation STD vs INT8 across all 160 prefill positions: 0.817

Decode positions (divergence-contaminated, shown for completeness):
arm            n        mean         MAX    greedy agree
REF          160   0.000e+00   0.000e+00 160/160 =  100.0%
REF2         160   0.000e+00   0.000e+00 160/160 =  100.0%
STD          160   7.263e-01   2.358e+01 150/160 =   93.8%
INT8         160   2.863e+00   3.252e+01 131/160 =   81.9%
INT8NOQ      160   2.066e+00   3.244e+01 142/160 =   88.8%
INT8NOGDN    160   1.099e+00   2.363e+01 143/160 =   89.4%
SIMROW       160   2.091e+00   3.487e+01 141/160 =   88.1%
SIMG128      160   7.330e-01   2.356e+01 149/160 =   93.1%
SIMG32       160   7.334e-01   2.365e+01 149/160 =   93.1%

SAMPLER-RELEVANT TAIL @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF          160     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/160     0.00%
REF2         160     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/160     0.00%
STD          160      98.04%  1.587e-02  4.462e-02  4.939e-01          inf 26/160     3.52%
INT8         160      98.01%  2.699e-02  6.755e-02  9.229e-01          inf 40/160     5.62%
INT8NOQ      160      97.36%  2.836e-02  6.122e-02  8.602e-01          inf 42/160     6.48%
INT8NOGDN    160      97.86%  2.402e-02  8.639e-02  6.150e-01          inf 39/160     5.55%
SIMROW       160      97.92%  2.079e-02  7.235e-02  4.378e-01          inf 43/160     3.98%
SIMG128      160      97.71%  1.607e-02  5.231e-02  4.939e-01          inf 31/160     4.22%
SIMG32       160      98.54%  1.817e-02  5.370e-02  4.939e-01          inf 27/160     3.52%

SAMPLER-RELEVANT TAIL @ owner's k~40 (T=1.0, top_p=1.0, top_k=40), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF          160     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/160     0.00%
REF2         160     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/160     0.00%
STD          160      97.42%  1.573e-02  3.820e-02  4.806e-01          inf 86/160     4.61%
INT8         160      95.65%  2.644e-02  6.605e-02  9.047e-01          inf 121/160     7.97%
INT8NOQ      160      95.63%  2.728e-02  5.374e-02  8.576e-01          inf 128/160     8.52%
INT8NOGDN    160      95.93%  2.351e-02  6.839e-02  6.192e-01          inf 112/160     7.34%
SIMROW       160      96.40%  2.008e-02  6.820e-02  4.394e-01          inf 110/160     7.19%
SIMG128      160      97.19%  1.584e-02  4.716e-02  4.807e-01          inf 89/160     6.09%
SIMG32       160      97.56%  1.814e-02  5.237e-02  4.774e-01          inf 99/160     5.55%

FULL-VOCAB numbers below, kept and labelled: they are the right
instrument for GREEDY or beam decode, where every rank matters.

arm         first-prefill-position KL       median          max
REF                         0.000e+00    0.000e+00    0.000e+00
REF2                        0.000e+00    0.000e+00    0.000e+00
STD                         3.479e-04    3.551e-05    3.823e-03
INT8                        6.910e-04    1.047e-04    2.922e-03
INT8NOQ                     7.483e-04    1.369e-04    3.917e-03
INT8NOGDN                   4.351e-04    8.386e-05    2.468e-03
SIMROW                      5.748e-04    1.636e-04    2.633e-03
SIMG128                     2.205e-04    6.399e-05    9.959e-04
SIMG32                      2.706e-04    3.925e-05    1.521e-03

PAIRED TV @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions.
Each row is A scored against B -- B is that ROW's reference, not the
table's.  TV is the expected speculative accept-rate loss, so the
column is read directly in percentage points.  The interval is a
cluster bootstrap over PROMPTS; the IQR is the raw spread over
positions with no floor applied to it.

  A       vs B        n  mean TV pp           95% CI pp    p25 pp    p50 pp    p75 pp    p95 pp    max pp
  REF     REF2      160      0.0000 [  0.0000,  0.0000]    0.0000    0.0000    0.0000    0.0000    0.0000
  REF     STD       160      1.5870 [  0.8665,  2.5456]    0.0000    0.2346    1.9388    4.4625   49.3853
  REF     INT8      160      2.6990 [  1.5135,  4.2872]    0.0000    0.6290    2.7828    6.7547   92.2893
  REF     INT8NOQ   160      2.8355 [  1.4964,  4.9122]    0.0000    1.1316    2.9158    6.1224   86.0196
  REF     INT8NOGDN   160      2.4022 [  1.5460,  3.4839]    0.0000    0.6264    3.0422    8.6388   61.4986
  REF     SIMROW    160      2.0790 [  1.3515,  3.0505]    0.0000    0.4233    2.8217    7.2345   43.7824
  REF     SIMG128   160      1.6069 [  0.9504,  2.5699]    0.0000    0.2346    1.9386    5.2314   49.3853
  REF     SIMG32    160      1.8175 [  1.0522,  2.7781]    0.0000    0.2748    2.2522    5.3703   49.3853
  REF2    STD       160      1.5870 [  0.8665,  2.5456]    0.0000    0.2346    1.9388    4.4625   49.3853
  REF2    INT8      160      2.6990 [  1.5135,  4.2872]    0.0000    0.6290    2.7828    6.7547   92.2893
  REF2    INT8NOQ   160      2.8355 [  1.4964,  4.9122]    0.0000    1.1316    2.9158    6.1224   86.0196
  REF2    INT8NOGDN   160      2.4022 [  1.5460,  3.4839]    0.0000    0.6264    3.0422    8.6388   61.4986
  REF2    SIMROW    160      2.0790 [  1.3515,  3.0505]    0.0000    0.4233    2.8217    7.2345   43.7824
  REF2    SIMG128   160      1.6069 [  0.9504,  2.5699]    0.0000    0.2346    1.9386    5.2314   49.3853
  REF2    SIMG32    160      1.8175 [  1.0522,  2.7781]    0.0000    0.2748    2.2522    5.3703   49.3853
  STD     INT8      160      2.9814 [  1.6831,  4.5573]    0.0000    0.5481    3.1110    8.6776   89.6264
  STD     INT8NOQ   160      2.9179 [  1.6078,  5.0315]    0.0000    1.2940    3.0069    6.7387   86.0196
  STD     INT8NOGDN   160      2.6173 [  1.4739,  4.0597]    0.0000    0.3980    2.8346    7.0228   71.1564
  STD     SIMROW    160      2.7028 [  1.4994,  4.4587]    0.0000    0.5695    2.8896    7.6075   79.0139
  STD     SIMG128   160      1.1467 [  0.8203,  1.5260]    0.0000    0.2899    1.8044    4.3751   20.1813
  STD     SIMG32    160      1.3063 [  0.9018,  1.7437]    0.0000    0.2088    2.0033    5.5589   25.3746
  INT8    INT8NOQ   160      2.9435 [  1.5819,  4.6460]    0.0000    0.5460    2.6446    6.2332   94.0335
  INT8    INT8NOGDN   160      2.9872 [  1.4467,  5.0282]    0.0000    0.3197    3.0214    8.7280   85.1655
  INT8    SIMROW    160      3.1098 [  1.6644,  4.9750]    0.0000    0.9935    3.3582    8.1179   95.3613
  INT8    SIMG128   160      3.1003 [  1.8172,  4.7609]    0.0000    1.1130    3.1893    8.1446   93.4343
  INT8    SIMG32    160      3.1226 [  1.7680,  4.7808]    0.0000    0.8315    2.7932    8.9636   92.1741
  INT8NOQ INT8NOGDN   160      3.0302 [  1.5586,  5.1786]    0.0000    0.7460    2.9592    6.8313   86.0196
  INT8NOQ SIMROW    160      3.0036 [  1.6078,  4.9941]    0.0000    1.0833    2.9339    7.4545   86.0196
  INT8NOQ SIMG128   160      2.7804 [  1.5132,  4.8960]    0.0000    0.7460    2.8896    6.9795   86.0196
  INT8NOQ SIMG32    160      2.6961 [  1.4786,  4.7684]    0.0000    0.7460    2.9593    6.8867   86.0196
  INT8NOGDN SIMROW    160      2.5045 [  1.6265,  3.5357]    0.0000    0.6197    3.3261   10.0694   49.2240
  INT8NOGDN SIMG128   160      2.6892 [  1.6029,  4.0664]    0.0000    0.7065    3.1220    7.5858   68.3610
  INT8NOGDN SIMG32    160      2.7154 [  1.6213,  4.0589]    0.0000    0.6062    2.8896    8.8565   63.5085
  SIMROW  SIMG128   160      2.7238 [  1.5937,  4.4600]    0.0000    0.9071    2.9936    8.0185   79.0139
  SIMROW  SIMG32    160      2.8479 [  1.6246,  4.5091]    0.0000    0.7168    2.9960    8.7358   79.0139
  SIMG128 SIMG32    160      1.3663 [  0.9778,  1.7754]    0.0000    0.2199    1.9370    4.9321   22.5792

  TV distance matrix, mean TV pp (symmetric):
          REF      REF2       STD      INT8   INT8NOQ INT8NOGDN    SIMROW   SIMG128    SIMG32
  REF        0.0000    0.0000    1.5870    2.6990    2.8355    2.4022    2.0790    1.6069    1.8175
  REF2       0.0000    0.0000    1.5870    2.6990    2.8355    2.4022    2.0790    1.6069    1.8175
  STD        1.5870    1.5870    0.0000    2.9814    2.9179    2.6173    2.7028    1.1467    1.3063
  INT8       2.6990    2.6990    2.9814    0.0000    2.9435    2.9872    3.1098    3.1003    3.1226
  INT8NOQ    2.8355    2.8355    2.9179    2.9435    0.0000    3.0302    3.0036    2.7804    2.6961
  INT8NOGDN    2.4022    2.4022    2.6173    2.9872    3.0302    0.0000    2.5045    2.6892    2.7154
  SIMROW     2.0790    2.0790    2.7028    3.1098    3.0036    2.5045    0.0000    2.7238    2.8479
  SIMG128    1.6069    1.6069    1.1467    3.1003    2.7804    2.6892    2.7238    0.0000    1.3663
  SIMG32     1.8175    1.8175    1.3063    3.1226    2.6961    2.7154    2.8479    1.3663    0.0000

  TV distance matrix, median TV pp (symmetric):
          REF      REF2       STD      INT8   INT8NOQ INT8NOGDN    SIMROW   SIMG128    SIMG32
  REF        0.0000    0.0000    0.2346    0.6290    1.1316    0.6264    0.4233    0.2346    0.2748
  REF2       0.0000    0.0000    0.2346    0.6290    1.1316    0.6264    0.4233    0.2346    0.2748
  STD        0.2346    0.2346    0.0000    0.5481    1.2940    0.3980    0.5695    0.2899    0.2088
  INT8       0.6290    0.6290    0.5481    0.0000    0.5460    0.3197    0.9935    1.1130    0.8315
  INT8NOQ    1.1316    1.1316    1.2940    0.5460    0.0000    0.7460    1.0833    0.7460    0.7460
  INT8NOGDN    0.6264    0.6264    0.3980    0.3197    0.7460    0.0000    0.6197    0.7065    0.6062
  SIMROW     0.4233    0.4233    0.5695    0.9935    1.0833    0.6197    0.0000    0.9071    0.7168
  SIMG128    0.2346    0.2346    0.2899    1.1130    0.7460    0.7065    0.9071    0.0000    0.2199
  SIMG32     0.2748    0.2748    0.2088    0.8315    0.7460    0.6062    0.7168    0.2199    0.0000

lm_head CUT -- the divergence entering the head vs leaving it,
prefill positions, relative rms.  'in' is the residual stream after
24 layers; 'out' is the centred logits.  amp = out/in: at 1.0 the
head passes the body's divergence through unchanged, above it the
head adds its own.

  A       vs B        n   rel rms IN  rel rms OUT      amp
  REF     REF2      160   0.0000e+00   0.0000e+00      nan
  REF     STD       160   3.9650e-02   3.6420e-02    0.919
  REF     INT8      160   6.9933e-02   6.3770e-02    0.912
  REF     INT8NOQ   160   6.8672e-02   6.2818e-02    0.915
  REF     INT8NOGDN   160   6.5832e-02   6.0324e-02    0.916
  REF     SIMROW    160   5.8675e-02   5.4017e-02    0.921
  REF     SIMG128   160   4.1139e-02   3.8110e-02    0.926
  REF     SIMG32    160   3.9530e-02   3.6415e-02    0.921
  REF2    STD       160   3.9650e-02   3.6420e-02    0.919
  REF2    INT8      160   6.9933e-02   6.3770e-02    0.912
  REF2    INT8NOQ   160   6.8672e-02   6.2818e-02    0.915
  REF2    INT8NOGDN   160   6.5832e-02   6.0324e-02    0.916
  REF2    SIMROW    160   5.8675e-02   5.4017e-02    0.921
  REF2    SIMG128   160   4.1139e-02   3.8110e-02    0.926
  REF2    SIMG32    160   3.9530e-02   3.6415e-02    0.921
  STD     INT8      160   7.1067e-02   6.5211e-02    0.918
  STD     INT8NOQ   160   7.2067e-02   6.5883e-02    0.914
  STD     INT8NOGDN   160   6.8823e-02   6.3307e-02    0.920
  STD     SIMROW    160   6.8578e-02   6.3376e-02    0.924
  STD     SIMG128   160   3.6019e-02   3.3405e-02    0.927
  STD     SIMG32    160   3.6018e-02   3.2728e-02    0.909
  INT8    INT8NOQ   160   7.6134e-02   6.9297e-02    0.910
  INT8    INT8NOGDN   160   8.7317e-02   8.0300e-02    0.920
  INT8    SIMROW    160   8.2613e-02   7.5635e-02    0.916
  INT8    SIMG128   160   7.3289e-02   6.6979e-02    0.914
  INT8    SIMG32    160   7.3741e-02   6.6681e-02    0.904
  INT8NOQ INT8NOGDN   160   7.1343e-02   6.5424e-02    0.917
  INT8NOQ SIMROW    160   7.1874e-02   6.4991e-02    0.904
  INT8NOQ SIMG128   160   7.3193e-02   6.7246e-02    0.919
  INT8NOQ SIMG32    160   7.4010e-02   6.7819e-02    0.916
  INT8NOGDN SIMROW    160   7.3700e-02   6.6940e-02    0.908
  INT8NOGDN SIMG128   160   7.3968e-02   6.7902e-02    0.918
  INT8NOGDN SIMG32    160   7.2214e-02   6.5859e-02    0.912
  SIMROW  SIMG128   160   6.4859e-02   5.9308e-02    0.914
  SIMROW  SIMG32    160   6.6875e-02   6.1090e-02    0.913
  SIMG128 SIMG32    160   3.9212e-02   3.5806e-02    0.913

PER-TOKEN AMAX -- is the crest factor of the quantised row what
selects the tail?  crest = amax/rms of the int8 GEMM's input row;
under the uniform-residual model the relative noise the row picks
up is crest/(127*sqrt(12)) = crest/440, and qerr is that noise
measured directly by round-tripping the scored row through int8.
arm supplying the statistics: INT8

  statistic                                  p50         p95         p99         max
  crest, scored row (max/linear)       1.174e+01   1.421e+01   1.667e+01   1.723e+01
  crest, any row in chunk              1.896e+01   2.112e+01   2.302e+01   2.330e+01
  qerr, scored row (rel rms)           2.661e-02   3.222e-02   3.759e-02   3.908e-02
  n positions: 160

The 10 worst INT8 positions, with the crest factor at each.  If the
per-token amax is the mechanism, these are the extreme-crest rows.
   prompt  chunk      INT8 KL   crest row  crest chunk        qerr  worst linear
       16      6    8.940e+00      11.362       19.089   2.552e-02  L21.linear_attn.out_proj
        0      7    2.040e+00      12.466       20.076   2.823e-02  L13.linear_attn.out_proj
        0      1    3.977e-01      11.198       18.043   2.551e-02  L8.linear_attn.out_proj
        2      7    5.191e-02      12.006       21.493   2.707e-02  L55.self_attn.o_proj
        2      3    2.703e-02      11.240       20.907   2.523e-02  L2.linear_attn.out_proj
        0      2    2.209e-02      16.673       20.897   3.759e-02  L50.linear_attn.out_proj
       18      7    2.161e-02       9.992       18.422   2.269e-02  L17.linear_attn.out_proj
       14      3    1.852e-02      10.895       18.788   2.467e-02  L13.linear_attn.out_proj
        3      4    1.831e-02      12.089       19.112   2.749e-02  L6.mlp.down_proj
       17      5    1.685e-02      11.056       20.079   2.519e-02  L51.self_attn.o_proj
  ... and the 10 MEDIAN INT8 positions, as the contrast:
       18      1    7.316e-04      11.799       20.011   2.661e-02  L0.mlp.down_proj
        2      4    7.954e-04      10.311       21.506   2.357e-02  L56.linear_attn.out_proj
       11      4    8.191e-04      11.021       20.534   2.501e-02  L0.mlp.down_proj
        5      7    8.734e-04      12.458       19.339   2.806e-02  L22.linear_attn.out_proj
       14      4    8.784e-04      10.265       19.919   2.328e-02  L6.linear_attn.out_proj
        1      4    9.407e-04      12.823       18.771   2.899e-02  L58.linear_attn.out_proj
       11      0    9.818e-04      10.919       18.473   2.477e-02  L2.mlp.down_proj
       11      7    9.858e-04      11.863       19.326   2.689e-02  L54.linear_attn.out_proj
       16      2    9.914e-04      14.086       18.511   3.200e-02  L51.mlp.down_proj
       16      1    1.031e-03      11.255       17.735   2.561e-02  L0.mlp.down_proj

Is the tail MORE NOISE or MORE RESPONSE?  rel rms of the residual
stream entering lm_head, per position, and the INT8/STD ratio.
Flat ratio across the tail => the position amplifies every arm and
int8 is merely 2.4x noisier everywhere.  Ratio spiking on the tail
=> int8 injects locally more noise there, which is a mechanism.
  ratio over ALL 160 positions: p50=2.013 p95=3.197 max=8.513
  ratio at the 10 worst INT8 positions: min=0.239 median=2.147 max=7.908
   prompt  chunk      INT8 KL  relIN INT8   relIN STD    ratio
       16      6    8.940e+00  8.3688e-01  1.0582e-01    7.908
        0      7    2.040e+00  7.0663e-01  4.5197e-01    1.563
        0      1    3.977e-01  3.2760e-01  5.1640e-01    0.634
        2      7    5.191e-02  1.0871e+00  3.0422e-01    3.574
        2      3    2.703e-02  1.5538e-01  6.4976e-01    0.239
        0      2    2.209e-02  1.5306e-01  1.1289e-01    1.356
       18      7    2.161e-02  4.8145e-02  2.0128e-02    2.392
       14      3    1.852e-02  6.7665e-02  2.1165e-02    3.197
        3      4    1.831e-02  4.9491e-02  1.8847e-02    2.626
       17      5    1.685e-02  7.8604e-02  4.1313e-02    1.903
  Spearman(relIN ratio, INT8 KL) = 0.177   Spearman(relIN INT8, INT8 KL) = 0.105

Rank correlation of the crest factor with each arm's KL.  STD is the
NULL CONTROL: it runs no quantiser, so the crest factor cannot cause
its KL.  A crest that ranks STD as well as it ranks INT8 is ranking
hard POSITIONS, not the quantiser, and explains nothing.
  crest statistic                    REF       REF2        STD       INT8    INT8NOQ  INT8NOGDN     SIMROW    SIMG128     SIMG32  INT8 excess
  crest, scored row               -0.035     -0.035     -0.014     -0.006      0.010      0.027      0.070     -0.006     -0.028        0.073
  crest, any row                  -0.200     -0.200      0.147      0.185      0.142      0.161      0.215      0.159      0.155        0.157
  qerr, scored row                -0.032     -0.032     -0.019     -0.008      0.009      0.024      0.067     -0.014     -0.035        0.076

Mean crest of the scored row, by linear class -- the 15 heaviest.
A class well above the Gaussian-rotation expectation (~4.5 for a
128-wide Hadamard over K=5120) is a class the per-token scale is
wasting levels on.
  linear class                                        mean crest         p95         max
  L2.mlp.down_proj                                         8.768      11.681      13.547
  L2.linear_attn.out_proj                                  8.288      11.549      13.587
  L0.mlp.down_proj                                         8.151      11.632      12.463
  L58.linear_attn.out_proj                                 8.064      13.624      17.229
  L4.linear_attn.out_proj                                  7.805      11.044      13.454
  L5.linear_attn.out_proj                                  7.688      11.693      14.842
  L1.linear_attn.out_proj                                  7.294      10.832      13.080
  L17.linear_attn.out_proj                                 7.107      10.029      13.344
  L56.linear_attn.out_proj                                 7.080      11.498      15.215
  L49.linear_attn.in_proj_z                                7.065       7.958       8.256
  L44.linear_attn.in_proj_z                                7.028       7.998       8.611
  L45.linear_attn.in_proj_z                                7.000       7.781       8.154
  L6.linear_attn.out_proj                                  6.999      10.021      11.511
  L61.linear_attn.out_proj                                 6.989      10.141      12.757
  L46.linear_attn.in_proj_z                                6.981       7.745       8.087

saved per-arm prefill logits+hidden to /work/logits_tail.pt
