[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (317092 > 262144). Running this sequence through the model will result in indexing errors
[i8] basket: 6 prompts x 16384 tok, domains ['code', 'longctx', 'math', 'sci', 'wiki']
compile-cache MISS: xgrammar — xgrammar compiled grammars (re)compiles this boot [/cache/xgrammar]
Boot: loading model weights — still running (30s). This phase reports nothing until it completes; it is progressing while this line advances. If it stops advancing, the phase is stuck.
Boot: loading model weights — still running (60s). This phase reports nothing until it completes; it is progressing while this line advances. If it stops advancing, the phase is stuck.
default-pool residency snapshot returned no live ranges — the unpooled-weight fold cannot see what it would move
exl3 kernel-shape pin: SEEDED 8 geometr(ies) by TIMING them in 334 ms (17408x5120xK4=K32N128, 5120x10240xK4=K32N128, 5120x1024xK4=K32N128, 5120x12288xK4=K32N256, 5120x17408xK4=K16N512, 5120x248320xK6=K16N512, 5120x6144xK4=K32N128, 6144x5120xK4=K32N128). This boot's kernel choice is a fresh measurement, so it need not match the previous boot's. Expected ONCE per configuration; if it repeats, the pin is not being recorded (point ARBI_SERVE_EXL3_PIN_CACHE_DIR at a durable volume) and every cross-boot comparison keeps paying for it.
[TKV autotune-provenance] fp=57911e26f4e47f74 regime=cache-hit cells=8 sweep_src=2772549a0ef713db tile_tokens=4,16 num_splits=32,64,128,256 min_blocks_per_sm=0,3 resolved=0 held=0 inherited=0
driver.modules_loaded baseline: SEEDED at 134217728 B (0.125 GiB) → /root/.cache/arbi-serve/budget-cache/69b5561ff6f9c4cf.modules.json. This boot ran UNGATED — it booked its own bracketed growth, so an unregistered pool or a raw cudaMalloc would be inside that number rather than on driver.residual. Every later boot at this configuration is held to it. Expected ONCE per configuration; if it repeats, the budget cache is not persisting (point ARBI_SERVE_BUDGET_CACHE_DIR at a durable volume) and the guard is inert.
/opt/venv/lib/python3.12/site-packages/torch/__init__.py:1172: FutureWarning: `torch.distributed.reduce_op` is deprecated, please use `torch.distributed.ReduceOp` instead
  return isinstance(obj, torch.Tensor)
[i8] trellis->path map: 401 linears
JIT compile AFTER serving-ready [cpp_ext]: exl3_i8_gemm cached .so load — a live request paid this compile's latency. This is a boot-warmup coverage gap: extend warmup to pre-compile this kernel/specialization. Counter jit_compile_serving (must-not-fire) at GET /v1/admin/flag_truth.
[i8] prefix cache: server default resolved=True, per-request cache_enabled=False; any cache hit below is refused
JIT compile AFTER serving-ready [cpp_ext]: arbi_serve_exl3_a_prep_v3 cached .so load — a live request paid this compile's latency. This is a boot-warmup coverage gap: extend warmup to pre-compile this kernel/specialization. Counter jit_compile_serving (must-not-fire) at GET /v1/admin/flag_truth.
[i8] warmup done
JIT compile AFTER serving-ready [cpp_ext]: tkv_unified_splitk_g6_v8_k4_v4_m1_v1_smem_rngfull_gms1_mw8_tt4_sh1_rc4_pf1_ldbypass cached .so load — a live request paid this compile's latency. This is a boot-warmup coverage gap: extend warmup to pre-compile this kernel/specialization. Counter jit_compile_serving (must-not-fire) at GET /v1/admin/flag_truth.
  [0] code           REF     16384tok  wall=  5.17s  finish=length
  [0] code           REF2    16384tok  wall=  5.12s  finish=length
  [0] code           INT8F   16384tok  wall=  3.24s  finish=stop
  [0] code           INT8FO  16384tok  wall=  3.09s  finish=stop
  [0] code           INT8FO32  16384tok  wall=  3.09s  finish=stop
  [1] code           REF     16384tok  wall=  4.95s  finish=stop
  [1] code           REF2    16384tok  wall=  4.95s  finish=stop
  [1] code           INT8F   16384tok  wall=  3.33s  finish=stop
  [1] code           INT8FO  16384tok  wall=  3.16s  finish=stop
  [1] code           INT8FO32  16384tok  wall=  3.16s  finish=stop
  [2] wiki           REF     16384tok  wall=  5.18s  finish=length
  [2] wiki           REF2    16384tok  wall=  5.17s  finish=length
  [2] wiki           INT8F   16384tok  wall=  3.53s  finish=length
  [2] wiki           INT8FO  16384tok  wall=  3.39s  finish=length
  [2] wiki           INT8FO32  16384tok  wall=  3.37s  finish=length
  [3] math           REF     16384tok  wall=  5.17s  finish=length
  [3] math           REF2    16384tok  wall=  5.17s  finish=length
  [3] math           INT8F   16384tok  wall=  3.54s  finish=length
  [3] math           INT8FO  16384tok  wall=  3.37s  finish=length
  [3] math           INT8FO32  16384tok  wall=  3.38s  finish=length
  [4] longctx        REF     16384tok  wall=  5.18s  finish=length
  [4] longctx        REF2    16384tok  wall=  5.17s  finish=length
  [4] longctx        INT8F   16384tok  wall=  3.56s  finish=length
  [4] longctx        INT8FO  16384tok  wall=  3.37s  finish=length
  [4] longctx        INT8FO32  16384tok  wall=  3.38s  finish=length
  [5] sci            REF     16384tok  wall=  5.20s  finish=length
  [5] sci            REF2    16384tok  wall=  5.20s  finish=length
  [5] sci            INT8F   16384tok  wall=  3.55s  finish=length
  [5] sci            INT8FO  16384tok  wall=  3.39s  finish=length
  [5] sci            INT8FO32  16384tok  wall=  3.38s  finish=length

[i8] kernel census: int8 calls=57600 rows=117964800 reconstructs SKIPPED=57600 fallback (non-4bpw) calls=0
[i8] shapes served by the kernel (K,N)->calls: {(5120, 1024): 4608, (5120, 6144): 6912, (5120, 10240): 6912, (5120, 12288): 2304, (5120, 17408): 18432, (6144, 5120): 9216, (17408, 5120): 9216}
[i8] excluded projections: none  excluded calls=0

  linear class                                           served   excl  int8 vs fp32 legB vs fp32 int8 vs legB
  model.layers.0.linear_attn.in_proj_qkv                    147      0             -            -            -
  model.layers.0.linear_attn.in_proj_z                      147      0             -            -            -
  model.layers.0.linear_attn.out_proj                       147      0             -            -            -
  model.layers.0.mlp.down_proj                              147      0             -            -            -
  model.layers.0.mlp.gate_proj                              147      0             -            -            -
  model.layers.0.mlp.up_proj                                147      0             -            -            -
  model.layers.1.linear_attn.in_proj_qkv                    147      0             -            -            -
  model.layers.1.linear_attn.in_proj_z                      147      0             -            -            -
  model.layers.1.linear_attn.out_proj                       147      0             -            -            -
  model.layers.1.mlp.down_proj                              147      0             -            -            -
  model.layers.1.mlp.gate_proj                              147      0             -            -            -
  model.layers.1.mlp.up_proj                                147      0             -            -            -
  model.layers.10.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.10.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.10.linear_attn.out_proj                      147      0             -            -            -
  model.layers.10.mlp.down_proj                             147      0             -            -            -
  model.layers.10.mlp.gate_proj                             147      0             -            -            -
  model.layers.10.mlp.up_proj                               147      0             -            -            -
  model.layers.11.mlp.down_proj                             147      0             -            -            -
  model.layers.11.mlp.gate_proj                             147      0             -            -            -
  model.layers.11.mlp.up_proj                               147      0             -            -            -
  model.layers.11.self_attn.k_proj                          147      0             -            -            -
  model.layers.11.self_attn.o_proj                          147      0             -            -            -
  model.layers.11.self_attn.q_proj                          147      0             -            -            -
  model.layers.11.self_attn.v_proj                          147      0             -            -            -
  model.layers.12.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.12.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.12.linear_attn.out_proj                      147      0             -            -            -
  model.layers.12.mlp.down_proj                             147      0             -            -            -
  model.layers.12.mlp.gate_proj                             147      0             -            -            -
  model.layers.12.mlp.up_proj                               147      0             -            -            -
  model.layers.13.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.13.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.13.linear_attn.out_proj                      147      0             -            -            -
  model.layers.13.mlp.down_proj                             147      0             -            -            -
  model.layers.13.mlp.gate_proj                             147      0             -            -            -
  model.layers.13.mlp.up_proj                               147      0             -            -            -
  model.layers.14.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.14.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.14.linear_attn.out_proj                      147      0             -            -            -
  model.layers.14.mlp.down_proj                             147      0             -            -            -
  model.layers.14.mlp.gate_proj                             147      0             -            -            -
  model.layers.14.mlp.up_proj                               147      0             -            -            -
  model.layers.15.mlp.down_proj                             147      0             -            -            -
  model.layers.15.mlp.gate_proj                             147      0             -            -            -
  model.layers.15.mlp.up_proj                               147      0             -            -            -
  model.layers.15.self_attn.k_proj                          147      0             -            -            -
  model.layers.15.self_attn.o_proj                          147      0             -            -            -
  model.layers.15.self_attn.q_proj                          147      0             -            -            -
  model.layers.15.self_attn.v_proj                          147      0             -            -            -
  model.layers.16.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.16.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.16.linear_attn.out_proj                      147      0             -            -            -
  model.layers.16.mlp.down_proj                             147      0             -            -            -
  model.layers.16.mlp.gate_proj                             147      0             -            -            -
  model.layers.16.mlp.up_proj                               147      0             -            -            -
  model.layers.17.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.17.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.17.linear_attn.out_proj                      147      0             -            -            -
  model.layers.17.mlp.down_proj                             147      0             -            -            -
  model.layers.17.mlp.gate_proj                             147      0             -            -            -
  model.layers.17.mlp.up_proj                               147      0             -            -            -
  model.layers.18.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.18.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.18.linear_attn.out_proj                      147      0             -            -            -
  model.layers.18.mlp.down_proj                             147      0             -            -            -
  model.layers.18.mlp.gate_proj                             147      0             -            -            -
  model.layers.18.mlp.up_proj                               147      0             -            -            -
  model.layers.19.mlp.down_proj                             147      0             -            -            -
  model.layers.19.mlp.gate_proj                             147      0             -            -            -
  model.layers.19.mlp.up_proj                               147      0             -            -            -
  model.layers.19.self_attn.k_proj                          147      0             -            -            -
  model.layers.19.self_attn.o_proj                          147      0             -            -            -
  model.layers.19.self_attn.q_proj                          147      0             -            -            -
  model.layers.19.self_attn.v_proj                          147      0             -            -            -
  model.layers.2.linear_attn.in_proj_qkv                    147      0             -            -            -
  model.layers.2.linear_attn.in_proj_z                      147      0             -            -            -
  model.layers.2.linear_attn.out_proj                       147      0             -            -            -
  model.layers.2.mlp.down_proj                              147      0             -            -            -
  model.layers.2.mlp.gate_proj                              147      0             -            -            -
  model.layers.2.mlp.up_proj                                147      0             -            -            -
  model.layers.20.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.20.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.20.linear_attn.out_proj                      147      0             -            -            -
  model.layers.20.mlp.down_proj                             147      0             -            -            -
  model.layers.20.mlp.gate_proj                             147      0             -            -            -
  model.layers.20.mlp.up_proj                               147      0             -            -            -
  model.layers.21.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.21.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.21.linear_attn.out_proj                      147      0             -            -            -
  model.layers.21.mlp.down_proj                             147      0             -            -            -
  model.layers.21.mlp.gate_proj                             147      0             -            -            -
  model.layers.21.mlp.up_proj                               147      0             -            -            -
  model.layers.22.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.22.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.22.linear_attn.out_proj                      147      0             -            -            -
  model.layers.22.mlp.down_proj                             147      0             -            -            -
  model.layers.22.mlp.gate_proj                             147      0             -            -            -
  model.layers.22.mlp.up_proj                               147      0             -            -            -
  model.layers.23.mlp.down_proj                             147      0             -            -            -
  model.layers.23.mlp.gate_proj                             147      0             -            -            -
  model.layers.23.mlp.up_proj                               147      0             -            -            -
  model.layers.23.self_attn.k_proj                          147      0             -            -            -
  model.layers.23.self_attn.o_proj                          147      0             -            -            -
  model.layers.23.self_attn.q_proj                          147      0             -            -            -
  model.layers.23.self_attn.v_proj                          147      0             -            -            -
  model.layers.24.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.24.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.24.linear_attn.out_proj                      147      0             -            -            -
  model.layers.24.mlp.down_proj                             147      0             -            -            -
  model.layers.24.mlp.gate_proj                             147      0             -            -            -
  model.layers.24.mlp.up_proj                               147      0             -            -            -
  model.layers.25.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.25.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.25.linear_attn.out_proj                      147      0             -            -            -
  model.layers.25.mlp.down_proj                             147      0             -            -            -
  model.layers.25.mlp.gate_proj                             147      0             -            -            -
  model.layers.25.mlp.up_proj                               147      0             -            -            -
  model.layers.26.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.26.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.26.linear_attn.out_proj                      147      0             -            -            -
  model.layers.26.mlp.down_proj                             147      0             -            -            -
  model.layers.26.mlp.gate_proj                             147      0             -            -            -
  model.layers.26.mlp.up_proj                               147      0             -            -            -
  model.layers.27.mlp.down_proj                             147      0             -            -            -
  model.layers.27.mlp.gate_proj                             147      0             -            -            -
  model.layers.27.mlp.up_proj                               147      0             -            -            -
  model.layers.27.self_attn.k_proj                          147      0             -            -            -
  model.layers.27.self_attn.o_proj                          147      0             -            -            -
  model.layers.27.self_attn.q_proj                          147      0             -            -            -
  model.layers.27.self_attn.v_proj                          147      0             -            -            -
  model.layers.28.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.28.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.28.linear_attn.out_proj                      147      0             -            -            -
  model.layers.28.mlp.down_proj                             147      0             -            -            -
  model.layers.28.mlp.gate_proj                             147      0             -            -            -
  model.layers.28.mlp.up_proj                               147      0             -            -            -
  model.layers.29.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.29.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.29.linear_attn.out_proj                      147      0             -            -            -
  model.layers.29.mlp.down_proj                             147      0             -            -            -
  model.layers.29.mlp.gate_proj                             147      0             -            -            -
  model.layers.29.mlp.up_proj                               147      0             -            -            -
  model.layers.3.mlp.down_proj                              147      0             -            -            -
  model.layers.3.mlp.gate_proj                              147      0             -            -            -
  model.layers.3.mlp.up_proj                                147      0             -            -            -
  model.layers.3.self_attn.k_proj                           147      0             -            -            -
  model.layers.3.self_attn.o_proj                           147      0             -            -            -
  model.layers.3.self_attn.q_proj                           147      0             -            -            -
  model.layers.3.self_attn.v_proj                           147      0             -            -            -
  model.layers.30.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.30.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.30.linear_attn.out_proj                      147      0             -            -            -
  model.layers.30.mlp.down_proj                             147      0             -            -            -
  model.layers.30.mlp.gate_proj                             147      0             -            -            -
  model.layers.30.mlp.up_proj                               147      0             -            -            -
  model.layers.31.mlp.down_proj                             147      0             -            -            -
  model.layers.31.mlp.gate_proj                             147      0             -            -            -
  model.layers.31.mlp.up_proj                               147      0             -            -            -
  model.layers.31.self_attn.k_proj                          147      0             -            -            -
  model.layers.31.self_attn.o_proj                          147      0             -            -            -
  model.layers.31.self_attn.q_proj                          147      0             -            -            -
  model.layers.31.self_attn.v_proj                          147      0             -            -            -
  model.layers.32.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.32.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.32.linear_attn.out_proj                      147      0             -            -            -
  model.layers.32.mlp.down_proj                             147      0             -            -            -
  model.layers.32.mlp.gate_proj                             147      0             -            -            -
  model.layers.32.mlp.up_proj                               147      0             -            -            -
  model.layers.33.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.33.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.33.linear_attn.out_proj                      147      0             -            -            -
  model.layers.33.mlp.down_proj                             147      0             -            -            -
  model.layers.33.mlp.gate_proj                             147      0             -            -            -
  model.layers.33.mlp.up_proj                               147      0             -            -            -
  model.layers.34.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.34.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.34.linear_attn.out_proj                      147      0             -            -            -
  model.layers.34.mlp.down_proj                             147      0             -            -            -
  model.layers.34.mlp.gate_proj                             147      0             -            -            -
  model.layers.34.mlp.up_proj                               147      0             -            -            -
  model.layers.35.mlp.down_proj                             147      0             -            -            -
  model.layers.35.mlp.gate_proj                             147      0             -            -            -
  model.layers.35.mlp.up_proj                               147      0             -            -            -
  model.layers.35.self_attn.k_proj                          147      0             -            -            -
  model.layers.35.self_attn.o_proj                          147      0             -            -            -
  model.layers.35.self_attn.q_proj                          147      0             -            -            -
  model.layers.35.self_attn.v_proj                          147      0             -            -            -
  model.layers.36.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.36.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.36.linear_attn.out_proj                      147      0             -            -            -
  model.layers.36.mlp.down_proj                             147      0             -            -            -
  model.layers.36.mlp.gate_proj                             147      0             -            -            -
  model.layers.36.mlp.up_proj                               147      0             -            -            -
  model.layers.37.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.37.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.37.linear_attn.out_proj                      147      0             -            -            -
  model.layers.37.mlp.down_proj                             147      0             -            -            -
  model.layers.37.mlp.gate_proj                             147      0             -            -            -
  model.layers.37.mlp.up_proj                               147      0             -            -            -
  model.layers.38.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.38.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.38.linear_attn.out_proj                      147      0             -            -            -
  model.layers.38.mlp.down_proj                             147      0             -            -            -
  model.layers.38.mlp.gate_proj                             147      0             -            -            -
  model.layers.38.mlp.up_proj                               147      0             -            -            -
  model.layers.39.mlp.down_proj                             147      0             -            -            -
  model.layers.39.mlp.gate_proj                             147      0             -            -            -
  model.layers.39.mlp.up_proj                               147      0             -            -            -
  model.layers.39.self_attn.k_proj                          147      0             -            -            -
  model.layers.39.self_attn.o_proj                          147      0             -            -            -
  model.layers.39.self_attn.q_proj                          147      0             -            -            -
  model.layers.39.self_attn.v_proj                          147      0             -            -            -
  model.layers.4.linear_attn.in_proj_qkv                    147      0             -            -            -
  model.layers.4.linear_attn.in_proj_z                      147      0             -            -            -
  model.layers.4.linear_attn.out_proj                       147      0             -            -            -
  model.layers.4.mlp.down_proj                              147      0             -            -            -
  model.layers.4.mlp.gate_proj                              147      0             -            -            -
  model.layers.4.mlp.up_proj                                147      0             -            -            -
  model.layers.40.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.40.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.40.linear_attn.out_proj                      147      0             -            -            -
  model.layers.40.mlp.down_proj                             147      0             -            -            -
  model.layers.40.mlp.gate_proj                             147      0             -            -            -
  model.layers.40.mlp.up_proj                               147      0             -            -            -
  model.layers.41.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.41.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.41.linear_attn.out_proj                      147      0             -            -            -
  model.layers.41.mlp.down_proj                             147      0             -            -            -
  model.layers.41.mlp.gate_proj                             147      0             -            -            -
  model.layers.41.mlp.up_proj                               147      0             -            -            -
  model.layers.42.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.42.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.42.linear_attn.out_proj                      147      0             -            -            -
  model.layers.42.mlp.down_proj                             147      0             -            -            -
  model.layers.42.mlp.gate_proj                             147      0             -            -            -
  model.layers.42.mlp.up_proj                               147      0             -            -            -
  model.layers.43.mlp.down_proj                             147      0             -            -            -
  model.layers.43.mlp.gate_proj                             147      0             -            -            -
  model.layers.43.mlp.up_proj                               147      0             -            -            -
  model.layers.43.self_attn.k_proj                          147      0             -            -            -
  model.layers.43.self_attn.o_proj                          147      0             -            -            -
  model.layers.43.self_attn.q_proj                          147      0             -            -            -
  model.layers.43.self_attn.v_proj                          147      0             -            -            -
  model.layers.44.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.44.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.44.linear_attn.out_proj                      147      0             -            -            -
  model.layers.44.mlp.down_proj                             147      0             -            -            -
  model.layers.44.mlp.gate_proj                             147      0             -            -            -
  model.layers.44.mlp.up_proj                               147      0             -            -            -
  model.layers.45.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.45.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.45.linear_attn.out_proj                      147      0             -            -            -
  model.layers.45.mlp.down_proj                             147      0             -            -            -
  model.layers.45.mlp.gate_proj                             147      0             -            -            -
  model.layers.45.mlp.up_proj                               147      0             -            -            -
  model.layers.46.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.46.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.46.linear_attn.out_proj                      147      0             -            -            -
  model.layers.46.mlp.down_proj                             147      0             -            -            -
  model.layers.46.mlp.gate_proj                             147      0             -            -            -
  model.layers.46.mlp.up_proj                               147      0             -            -            -
  model.layers.47.mlp.down_proj                             147      0             -            -            -
  model.layers.47.mlp.gate_proj                             147      0             -            -            -
  model.layers.47.mlp.up_proj                               147      0             -            -            -
  model.layers.47.self_attn.k_proj                          147      0             -            -            -
  model.layers.47.self_attn.o_proj                          147      0             -            -            -
  model.layers.47.self_attn.q_proj                          147      0             -            -            -
  model.layers.47.self_attn.v_proj                          147      0             -            -            -
  model.layers.48.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.48.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.48.linear_attn.out_proj                      147      0             -            -            -
  model.layers.48.mlp.down_proj                             147      0             -            -            -
  model.layers.48.mlp.gate_proj                             147      0             -            -            -
  model.layers.48.mlp.up_proj                               147      0             -            -            -
  model.layers.49.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.49.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.49.linear_attn.out_proj                      147      0             -            -            -
  model.layers.49.mlp.down_proj                             147      0             -            -            -
  model.layers.49.mlp.gate_proj                             147      0             -            -            -
  model.layers.49.mlp.up_proj                               147      0             -            -            -
  model.layers.5.linear_attn.in_proj_qkv                    147      0             -            -            -
  model.layers.5.linear_attn.in_proj_z                      147      0             -            -            -
  model.layers.5.linear_attn.out_proj                       147      0             -            -            -
  model.layers.5.mlp.down_proj                              147      0             -            -            -
  model.layers.5.mlp.gate_proj                              147      0             -            -            -
  model.layers.5.mlp.up_proj                                147      0             -            -            -
  model.layers.50.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.50.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.50.linear_attn.out_proj                      147      0             -            -            -
  model.layers.50.mlp.down_proj                             147      0             -            -            -
  model.layers.50.mlp.gate_proj                             147      0             -            -            -
  model.layers.50.mlp.up_proj                               147      0             -            -            -
  model.layers.51.mlp.down_proj                             147      0             -            -            -
  model.layers.51.mlp.gate_proj                             147      0             -            -            -
  model.layers.51.mlp.up_proj                               147      0             -            -            -
  model.layers.51.self_attn.k_proj                          147      0             -            -            -
  model.layers.51.self_attn.o_proj                          147      0             -            -            -
  model.layers.51.self_attn.q_proj                          147      0             -            -            -
  model.layers.51.self_attn.v_proj                          147      0             -            -            -
  model.layers.52.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.52.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.52.linear_attn.out_proj                      147      0             -            -            -
  model.layers.52.mlp.down_proj                             147      0             -            -            -
  model.layers.52.mlp.gate_proj                             147      0             -            -            -
  model.layers.52.mlp.up_proj                               147      0             -            -            -
  model.layers.53.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.53.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.53.linear_attn.out_proj                      147      0             -            -            -
  model.layers.53.mlp.down_proj                             147      0             -            -            -
  model.layers.53.mlp.gate_proj                             147      0             -            -            -
  model.layers.53.mlp.up_proj                               147      0             -            -            -
  model.layers.54.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.54.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.54.linear_attn.out_proj                      147      0             -            -            -
  model.layers.54.mlp.down_proj                             147      0             -            -            -
  model.layers.54.mlp.gate_proj                             147      0             -            -            -
  model.layers.54.mlp.up_proj                               147      0             -            -            -
  model.layers.55.mlp.down_proj                             147      0             -            -            -
  model.layers.55.mlp.gate_proj                             147      0             -            -            -
  model.layers.55.mlp.up_proj                               147      0             -            -            -
  model.layers.55.self_attn.k_proj                          147      0             -            -            -
  model.layers.55.self_attn.o_proj                          147      0             -            -            -
  model.layers.55.self_attn.q_proj                          147      0             -            -            -
  model.layers.55.self_attn.v_proj                          147      0             -            -            -
  model.layers.56.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.56.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.56.linear_attn.out_proj                      147      0             -            -            -
  model.layers.56.mlp.down_proj                             147      0             -            -            -
  model.layers.56.mlp.gate_proj                             147      0             -            -            -
  model.layers.56.mlp.up_proj                               147      0             -            -            -
  model.layers.57.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.57.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.57.linear_attn.out_proj                      147      0             -            -            -
  model.layers.57.mlp.down_proj                             147      0             -            -            -
  model.layers.57.mlp.gate_proj                             147      0             -            -            -
  model.layers.57.mlp.up_proj                               147      0             -            -            -
  model.layers.58.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.58.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.58.linear_attn.out_proj                      147      0             -            -            -
  model.layers.58.mlp.down_proj                             147      0             -            -            -
  model.layers.58.mlp.gate_proj                             147      0             -            -            -
  model.layers.58.mlp.up_proj                               147      0             -            -            -
  model.layers.59.mlp.down_proj                             147      0             -            -            -
  model.layers.59.mlp.gate_proj                             147      0             -            -            -
  model.layers.59.mlp.up_proj                               147      0             -            -            -
  model.layers.59.self_attn.k_proj                          147      0             -            -            -
  model.layers.59.self_attn.o_proj                          147      0             -            -            -
  model.layers.59.self_attn.q_proj                          147      0             -            -            -
  model.layers.59.self_attn.v_proj                          147      0             -            -            -
  model.layers.6.linear_attn.in_proj_qkv                    147      0             -            -            -
  model.layers.6.linear_attn.in_proj_z                      147      0             -            -            -
  model.layers.6.linear_attn.out_proj                       147      0             -            -            -
  model.layers.6.mlp.down_proj                              147      0             -            -            -
  model.layers.6.mlp.gate_proj                              147      0             -            -            -
  model.layers.6.mlp.up_proj                                147      0             -            -            -
  model.layers.60.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.60.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.60.linear_attn.out_proj                      147      0             -            -            -
  model.layers.60.mlp.down_proj                             147      0             -            -            -
  model.layers.60.mlp.gate_proj                             147      0             -            -            -
  model.layers.60.mlp.up_proj                               147      0             -            -            -
  model.layers.61.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.61.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.61.linear_attn.out_proj                      147      0             -            -            -
  model.layers.61.mlp.down_proj                             147      0             -            -            -
  model.layers.61.mlp.gate_proj                             147      0             -            -            -
  model.layers.61.mlp.up_proj                               147      0             -            -            -
  model.layers.62.linear_attn.in_proj_qkv                   147      0             -            -            -
  model.layers.62.linear_attn.in_proj_z                     147      0             -            -            -
  model.layers.62.linear_attn.out_proj                      147      0             -            -            -
  model.layers.62.mlp.down_proj                             147      0             -            -            -
  model.layers.62.mlp.gate_proj                             147      0             -            -            -
  model.layers.62.mlp.up_proj                               147      0             -            -            -
  model.layers.63.mlp.down_proj                             147      0             -            -            -
  model.layers.63.mlp.gate_proj                             147      0             -            -            -
  model.layers.63.mlp.up_proj                               147      0             -            -            -
  model.layers.63.self_attn.k_proj                          147      0             -            -            -
  model.layers.63.self_attn.o_proj                          147      0             -            -            -
  model.layers.63.self_attn.q_proj                          147      0             -            -            -
  model.layers.63.self_attn.v_proj                          147      0             -            -            -
  model.layers.7.mlp.down_proj                              147      0             -            -            -
  model.layers.7.mlp.gate_proj                              147      0             -            -            -
  model.layers.7.mlp.up_proj                                147      0             -            -            -
  model.layers.7.self_attn.k_proj                           147      0             -            -            -
  model.layers.7.self_attn.o_proj                           147      0             -            -            -
  model.layers.7.self_attn.q_proj                           147      0             -            -            -
  model.layers.7.self_attn.v_proj                           147      0             -            -            -
  model.layers.8.linear_attn.in_proj_qkv                    147      0             -            -            -
  model.layers.8.linear_attn.in_proj_z                      147      0             -            -            -
  model.layers.8.linear_attn.out_proj                       147      0             -            -            -
  model.layers.8.mlp.down_proj                              147      0             -            -            -
  model.layers.8.mlp.gate_proj                              147      0             -            -            -
  model.layers.8.mlp.up_proj                                147      0             -            -            -
  model.layers.9.linear_attn.in_proj_qkv                    147      0             -            -            -
  model.layers.9.linear_attn.in_proj_z                      147      0             -            -            -
  model.layers.9.linear_attn.out_proj                       147      0             -            -            -
  model.layers.9.mlp.down_proj                              147      0             -            -            -
  model.layers.9.mlp.gate_proj                              147      0             -            -            -
  model.layers.9.mlp.up_proj                                147      0             -            -            -

THE TAIL over PREFILL positions only -- one per prefill chunk, so
every arm scores the SAME context. Decode positions are reported
separately and are NOT a numerics measure: once a token flips, the
arms are reading different contexts. The control proves it -- pooled
over decode, STD (which contains no int8 at all) has the same max KL
as INT8, so a pooled tail cannot attribute anything to the kernel.

arm            n        mean         p50         p95         p99         MAX     argmax flip
REF           48   0.000e+00   0.000e+00   0.000e+00   0.000e+00   0.000e+00 0/48 =   0.00%
REF2          48   0.000e+00   0.000e+00   0.000e+00   0.000e+00   0.000e+00 0/48 =   0.00%
INT8F         47   4.894e-02   7.970e-04   2.297e-02   2.134e+00   2.134e+00 2/47 =   4.26%
INT8FO        47   4.894e-02   7.970e-04   2.297e-02   2.134e+00   2.134e+00 2/47 =   4.26%
INT8FO32      47   1.021e-02   1.291e-03   3.750e-02   2.061e-01   2.061e-01 2/47 =   4.26%

Decode positions (divergence-contaminated, shown for completeness):
arm            n        mean         MAX    greedy agree
REF           42   0.000e+00   0.000e+00 42/42 =  100.0%
REF2          42   0.000e+00   0.000e+00 42/42 =  100.0%
INT8F         35   4.360e-02   1.442e+00 34/35 =   97.1%
INT8FO        35   4.360e-02   1.442e+00 34/35 =   97.1%
INT8FO32      35   4.895e-02   1.621e+00 34/35 =   97.1%

SAMPLER-RELEVANT TAIL @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF           48     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/ 48     0.00%
REF2          48     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/ 48     0.00%
INT8F         47      95.71%  2.828e-02  7.941e-02  4.876e-01          inf 13/ 47     5.85%
INT8FO        47      95.71%  2.828e-02  7.941e-02  4.876e-01          inf 13/ 47     5.85%
INT8FO32      47      96.93%  2.839e-02  1.009e-01  2.960e-01          inf 16/ 47     6.38%

SAMPLER-RELEVANT TAIL @ owner's k~40 (T=1.0, top_p=1.0, top_k=40), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF           48     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/ 48     0.00%
REF2          48     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/ 48     0.00%
INT8F         47      94.45%  2.822e-02  7.600e-02  4.904e-01          inf 37/ 47     7.18%
INT8FO        47      94.45%  2.822e-02  7.600e-02  4.904e-01          inf 37/ 47     7.18%
INT8FO32      47      95.92%  2.726e-02  9.355e-02  2.856e-01          inf 34/ 47     7.98%

FULL-VOCAB numbers below, kept and labelled: they are the right
instrument for GREEDY or beam decode, where every rank matters.

arm         first-prefill-position KL       median          max
REF                         0.000e+00    0.000e+00    0.000e+00
REF2                        0.000e+00    0.000e+00    0.000e+00
INT8F                       4.836e-04    7.744e-05    2.548e-03
INT8FO                      4.836e-04    7.744e-05    2.548e-03
INT8FO32                    5.305e-04    7.850e-05    2.516e-03

[i8fo] folded GEMMs=39200 of 39200 served  displaced (A fragments already clobbered)=0  unmatched (served the shipped way)=0
[i8f]  fused prep calls=58800  deferred-rotation flushes=0  unmatched=0
