[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (317092 > 262144). Running this sequence through the model will result in indexing errors
[i8] basket: 5 prompts x 2048 tok, domains ['code', 'longctx', 'math', 'sci', 'wiki']
compile-cache MISS: xgrammar — xgrammar compiled grammars (re)compiles this boot [/cache/xgrammar]
VRAM at profile: this card's residency could NOT be split between this process and any other — NVML's per-process walk did not identify us — so the whole 14.26 GiB resident on device 0 is being booked as driver.cuda_context. Any co-tenant's bytes are inside that number and KV is sized that much smaller for this boot.
[TKV autotune-provenance] fp=f4e763cb2f31eb74 regime=cache-hit cells=8 sweep_src=b5332b8dfc1c8389 tile_tokens=4 num_splits=32,64,128,256 min_blocks_per_sm=0 resolved=0 held=0 inherited=0
  return isinstance(obj, torch.Tensor)
[i8] trellis->path map: 401 linears
JIT compile AFTER serving-ready [cpp_ext]: exl3_i8_gemm_k4_cb2 cached .so load — a live request paid this compile's latency. This is a boot-warmup coverage gap: extend warmup to pre-compile this kernel/specialization. Counter jit_compile_serving (must-not-fire) at GET /v1/admin/flag_truth.
[i8] prefix cache: server default resolved=True, per-request cache_enabled=False; any cache hit below is refused
driver serving-growth OVER RESERVE: the driver holds 2 MiB more than it did when the boot brackets closed, against the 0 MiB the serving floor's transient.serving_step.driver_growth term held for it. Those bytes are in no pool and no allocator counter -- the live card reaches them as driver.modules_serving and the boot ledger only as driver.residual -- and the KV layout is frozen, so the overage comes out of the free VRAM the floor is holding for one in-flight step (the verify tail, the DFlash context-assemble) and those reserves are the ones that fail first. The reading is on record for this configuration, so the next boot holds it; this process serves the rest of its life short by the overage.
activation admission RE-ARMED (driver residency over reserve): the step budget FELL to 5741 MiB, -642 MiB on the 6383 MiB it was enforcing and 6383 MiB the boot reading armed. The widest slate admission will build is narrower from here — narrower prefill chunks and deferred rows instead of a forward that OOMs after the layout freeze.
[i8] warmup done
  [0] code           REF       2048tok  wall=  0.84s  finish=length  leg=400F/0S acc=400x16/0x32
  [0] code           INT8      2048tok  wall=  0.82s  finish=length  leg=0F/400S acc=400x16/0x32
  [0] code           INT8G1    2048tok  wall=  0.87s  finish=length  leg=0F/400S acc=400x16/0x32
  [1] wiki           REF       2048tok  wall=  0.82s  finish=length  leg=400F/0S acc=400x16/0x32
  [1] wiki           INT8      2048tok  wall=  0.82s  finish=length  leg=0F/400S acc=400x16/0x32
  [1] wiki           INT8G1    2048tok  wall=  0.87s  finish=length  leg=0F/400S acc=400x16/0x32
  [2] math           REF       2048tok  wall=  0.82s  finish=length  leg=400F/0S acc=400x16/0x32
  [2] math           INT8      2048tok  wall=  0.82s  finish=length  leg=0F/400S acc=400x16/0x32
  [2] math           INT8G1    2048tok  wall=  0.87s  finish=length  leg=0F/400S acc=400x16/0x32
  [3] longctx        REF       2048tok  wall=  0.83s  finish=length  leg=400F/0S acc=400x16/0x32
  [3] longctx        INT8      2048tok  wall=  0.82s  finish=length  leg=0F/400S acc=400x16/0x32
  [3] longctx        INT8G1    2048tok  wall=  0.86s  finish=length  leg=0F/400S acc=400x16/0x32
  [4] sci            REF       2048tok  wall=  0.81s  finish=length  leg=400F/0S acc=400x16/0x32
  [4] sci            INT8      2048tok  wall=  0.80s  finish=length  leg=0F/400S acc=400x16/0x32
  [4] sci            INT8G1    2048tok  wall=  0.86s  finish=length  leg=0F/400S acc=400x16/0x32

[i8] kernel census: int8 calls=4000 rows=8192000 reconstructs SKIPPED=4000 fallback (non-4bpw) calls=0
[i8] shapes served by the kernel (K,N)->calls: {(5120, 1024): 320, (5120, 6144): 480, (5120, 10240): 480, (5120, 12288): 160, (5120, 17408): 1280, (6144, 5120): 640, (17408, 5120): 640}
[i8] excluded projections: none  excluded calls=0

  linear class                                           served   excl  int8 vs fp32 legB vs fp32 int8 vs legB
  model.layers.0.linear_attn.in_proj_qkv                     12      0     4.827e-03    1.713e-03    5.132e-03
  model.layers.0.linear_attn.in_proj_z                       12      0     6.316e-03    1.129e-03    6.418e-03
  model.layers.0.linear_attn.out_proj                        12      0     1.249e-02    2.189e-03    1.268e-02
  model.layers.0.mlp.down_proj                               12      0     2.123e-02    2.610e-03    2.139e-02
  model.layers.0.mlp.gate_proj                               12      0     9.121e-03    3.416e-03    9.741e-03
  model.layers.0.mlp.up_proj                                 12      0     9.089e-03    3.416e-03    9.708e-03
  model.layers.1.linear_attn.in_proj_qkv                     12      0     1.187e-02    2.166e-03    1.207e-02
  model.layers.1.linear_attn.in_proj_z                       12      0     1.361e-02    1.840e-03    1.374e-02
  model.layers.1.linear_attn.out_proj                        12      0     1.607e-02    2.259e-03    1.622e-02
  model.layers.1.mlp.down_proj                               12      0     1.342e-02    2.320e-03    1.362e-02
  model.layers.1.mlp.gate_proj                               12      0     9.697e-03    3.460e-03    1.030e-02
  model.layers.1.mlp.up_proj                                 12      0     1.014e-02    3.489e-03    1.072e-02
  model.layers.10.linear_attn.in_proj_qkv                    12      0     1.305e-02    2.252e-03    1.324e-02
  model.layers.10.linear_attn.in_proj_z                      12      0     1.299e-02    1.839e-03    1.312e-02
  model.layers.10.linear_attn.out_proj                       12      0     1.497e-02    2.122e-03    1.512e-02
  model.layers.10.mlp.down_proj                              12      0     1.269e-02    2.344e-03    1.291e-02
  model.layers.10.mlp.gate_proj                              12      0     1.035e-02    3.580e-03    1.095e-02
  model.layers.10.mlp.up_proj                                12      0     1.075e-02    3.590e-03    1.134e-02
  model.layers.11.mlp.down_proj                              12      0     1.334e-02    2.506e-03    1.357e-02
  model.layers.11.mlp.gate_proj                              12      0     1.027e-02    3.508e-03    1.086e-02
  model.layers.11.mlp.up_proj                                12      0     1.088e-02    3.578e-03    1.145e-02
  model.layers.11.self_attn.k_proj                           12      0     1.328e-02    2.698e-03    1.356e-02
  model.layers.11.self_attn.o_proj                           12      0     1.304e-02    2.371e-03    1.326e-02
  model.layers.11.self_attn.q_proj                           12      0     1.174e-02    2.023e-03    1.192e-02
  model.layers.11.self_attn.v_proj                           12      0     1.037e-02    2.285e-03    1.062e-02
  model.layers.12.linear_attn.in_proj_qkv                    12      0     1.293e-02    2.244e-03    1.312e-02
  model.layers.12.linear_attn.in_proj_z                      12      0     1.367e-02    1.892e-03    1.380e-02
  model.layers.12.linear_attn.out_proj                       12      0     1.581e-02    2.140e-03    1.596e-02
  model.layers.12.mlp.down_proj                              12      0     1.317e-02    2.548e-03    1.341e-02
  model.layers.12.mlp.gate_proj                              12      0     1.002e-02    3.476e-03    1.061e-02
  model.layers.12.mlp.up_proj                                12      0     1.072e-02    3.553e-03    1.130e-02
  model.layers.13.linear_attn.in_proj_qkv                    12      0     1.292e-02    2.242e-03    1.312e-02
  model.layers.13.linear_attn.in_proj_z                      12      0     1.418e-02    1.977e-03    1.432e-02
  model.layers.13.linear_attn.out_proj                       12      0     1.680e-02    2.178e-03    1.695e-02
  model.layers.13.mlp.down_proj                              12      0     1.372e-02    2.565e-03    1.396e-02
  model.layers.13.mlp.gate_proj                              12      0     1.007e-02    3.482e-03    1.065e-02
  model.layers.13.mlp.up_proj                                12      0     1.070e-02    3.556e-03    1.128e-02
  model.layers.14.linear_attn.in_proj_qkv                    12      0     1.270e-02    2.232e-03    1.289e-02
  model.layers.14.linear_attn.in_proj_z                      12      0     1.336e-02    1.881e-03    1.349e-02
  model.layers.14.linear_attn.out_proj                       12      0     1.588e-02    2.213e-03    1.603e-02
  model.layers.14.mlp.down_proj                              12      0     1.300e-02    2.462e-03    1.324e-02
  model.layers.14.mlp.gate_proj                              12      0     9.965e-03    3.501e-03    1.056e-02
  model.layers.14.mlp.up_proj                                12      0     1.064e-02    3.564e-03    1.122e-02
  model.layers.15.mlp.down_proj                              12      0     1.349e-02    2.531e-03    1.373e-02
  model.layers.15.mlp.gate_proj                              12      0     1.010e-02    3.486e-03    1.068e-02
  model.layers.15.mlp.up_proj                                12      0     1.067e-02    3.562e-03    1.125e-02
  model.layers.15.self_attn.k_proj                           12      0     1.455e-02    2.891e-03    1.483e-02
  model.layers.15.self_attn.o_proj                           12      0     1.396e-02    2.117e-03    1.412e-02
  model.layers.15.self_attn.q_proj                           12      0     1.301e-02    2.088e-03    1.318e-02
  model.layers.15.self_attn.v_proj                           12      0     9.798e-03    2.225e-03    1.005e-02
  model.layers.16.linear_attn.in_proj_qkv                    12      0     1.287e-02    2.243e-03    1.307e-02
  model.layers.16.linear_attn.in_proj_z                      12      0     1.354e-02    1.892e-03    1.367e-02
  model.layers.16.linear_attn.out_proj                       12      0     1.636e-02    2.280e-03    1.652e-02
  model.layers.16.mlp.down_proj                              12      0     1.296e-02    2.518e-03    1.321e-02
  model.layers.16.mlp.gate_proj                              12      0     1.005e-02    3.475e-03    1.064e-02
  model.layers.16.mlp.up_proj                                12      0     1.071e-02    3.559e-03    1.128e-02
  model.layers.17.linear_attn.in_proj_qkv                    12      0     1.287e-02    2.223e-03    1.306e-02
  model.layers.17.linear_attn.in_proj_z                      12      0     1.350e-02    1.895e-03    1.364e-02
  model.layers.17.linear_attn.out_proj                       12      0     1.778e-02    2.246e-03    1.792e-02
  model.layers.17.mlp.down_proj                              12      0     1.381e-02    2.456e-03    1.402e-02
  model.layers.17.mlp.gate_proj                              12      0     1.010e-02    3.469e-03    1.068e-02
  model.layers.17.mlp.up_proj                                12      0     1.072e-02    3.565e-03    1.129e-02
  model.layers.18.linear_attn.in_proj_qkv                    12      0     1.206e-02    2.188e-03    1.226e-02
  model.layers.18.linear_attn.in_proj_z                      12      0     1.264e-02    1.787e-03    1.276e-02
  model.layers.18.linear_attn.out_proj                       12      0     1.529e-02    2.227e-03    1.545e-02
  model.layers.18.mlp.down_proj                              12      0     1.459e-02    2.424e-03    1.479e-02
  model.layers.18.mlp.gate_proj                              12      0     1.024e-02    3.424e-03    1.080e-02
  model.layers.18.mlp.up_proj                                12      0     1.058e-02    3.531e-03    1.115e-02
  model.layers.19.mlp.down_proj                              12      0     1.586e-02    2.536e-03    1.606e-02
  model.layers.19.mlp.gate_proj                              12      0     1.007e-02    3.397e-03    1.063e-02
  model.layers.19.mlp.up_proj                                12      0     1.096e-02    3.579e-03    1.154e-02
  model.layers.19.self_attn.k_proj                           12      0     1.406e-02    2.789e-03    1.434e-02
  model.layers.19.self_attn.o_proj                           12      0     1.812e-02    2.164e-03    1.825e-02
  model.layers.19.self_attn.q_proj                           12      0     9.499e-03    1.928e-03    9.693e-03
  model.layers.19.self_attn.v_proj                           12      0     1.047e-02    2.344e-03    1.072e-02
  model.layers.2.linear_attn.in_proj_qkv                     12      0     1.249e-02    2.214e-03    1.269e-02
  model.layers.2.linear_attn.in_proj_z                       12      0     1.398e-02    1.925e-03    1.412e-02
  model.layers.2.linear_attn.out_proj                        12      0     1.798e-02    2.323e-03    1.812e-02
  model.layers.2.mlp.down_proj                               12      0     1.704e-02    2.374e-03    1.721e-02
  model.layers.2.mlp.gate_proj                               12      0     1.030e-02    3.532e-03    1.089e-02
  model.layers.2.mlp.up_proj                                 12      0     1.062e-02    3.557e-03    1.120e-02
  model.layers.20.linear_attn.in_proj_qkv                    12      0     1.213e-02    2.136e-03    1.232e-02
  model.layers.20.linear_attn.in_proj_z                      12      0     1.273e-02    1.792e-03    1.285e-02
  model.layers.20.linear_attn.out_proj                       12      0     1.582e-02    2.229e-03    1.597e-02
  model.layers.20.mlp.down_proj                              12      0     1.593e-02    2.584e-03    1.614e-02
  model.layers.20.mlp.gate_proj                              12      0     1.008e-02    3.405e-03    1.064e-02
  model.layers.20.mlp.up_proj                                12      0     1.127e-02    3.606e-03    1.183e-02
  model.layers.21.linear_attn.in_proj_qkv                    12      0     1.198e-02    2.198e-03    1.218e-02
  model.layers.21.linear_attn.in_proj_z                      12      0     1.161e-02    1.671e-03    1.173e-02
  model.layers.21.linear_attn.out_proj                       12      0     1.699e-02    2.233e-03    1.713e-02
  model.layers.21.mlp.down_proj                              12      0     1.720e-02    2.590e-03    1.739e-02
  model.layers.21.mlp.gate_proj                              12      0     1.010e-02    3.421e-03    1.066e-02
  model.layers.21.mlp.up_proj                                12      0     1.123e-02    3.605e-03    1.180e-02
  model.layers.22.linear_attn.in_proj_qkv                    12      0     1.201e-02    2.236e-03    1.222e-02
  model.layers.22.linear_attn.in_proj_z                      12      0     1.203e-02    1.713e-03    1.215e-02
  model.layers.22.linear_attn.out_proj                       12      0     1.578e-02    2.301e-03    1.595e-02
  model.layers.22.mlp.down_proj                              12      0     1.514e-02    2.549e-03    1.536e-02
  model.layers.22.mlp.gate_proj                              12      0     1.053e-02    3.139e-03    1.099e-02
  model.layers.22.mlp.up_proj                                12      0     1.141e-02    3.619e-03    1.197e-02
  model.layers.23.mlp.down_proj                              12      0     1.569e-02    2.554e-03    1.589e-02
  model.layers.23.mlp.gate_proj                              12      0     1.009e-02    3.372e-03    1.064e-02
  model.layers.23.mlp.up_proj                                12      0     1.141e-02    3.621e-03    1.197e-02
  model.layers.23.self_attn.k_proj                           12      0     1.378e-02    2.666e-03    1.404e-02
  model.layers.23.self_attn.o_proj                           12      0     1.681e-02    2.421e-03    1.698e-02
  model.layers.23.self_attn.q_proj                           12      0     9.469e-03    1.965e-03    9.672e-03
  model.layers.23.self_attn.v_proj                           12      0     1.034e-02    2.328e-03    1.059e-02
  model.layers.24.linear_attn.in_proj_qkv                    12      0     1.188e-02    2.254e-03    1.209e-02
  model.layers.24.linear_attn.in_proj_z                      12      0     1.222e-02    1.731e-03    1.234e-02
  model.layers.24.linear_attn.out_proj                       12      0     1.606e-02    2.272e-03    1.622e-02
  model.layers.24.mlp.down_proj                              12      0     1.456e-02    2.622e-03    1.480e-02
  model.layers.24.mlp.gate_proj                              12      0     1.027e-02    3.317e-03    1.079e-02
  model.layers.24.mlp.up_proj                                12      0     1.144e-02    3.614e-03    1.200e-02
  model.layers.25.linear_attn.in_proj_qkv                    12      0     1.142e-02    2.213e-03    1.163e-02
  model.layers.25.linear_attn.in_proj_z                      12      0     1.278e-02    1.790e-03    1.291e-02
  model.layers.25.linear_attn.out_proj                       12      0     1.473e-02    2.175e-03    1.489e-02
  model.layers.25.mlp.down_proj                              12      0     1.392e-02    2.551e-03    1.415e-02
  model.layers.25.mlp.gate_proj                              12      0     1.054e-02    3.324e-03    1.105e-02
  model.layers.25.mlp.up_proj                                12      0     1.130e-02    3.600e-03    1.186e-02
  model.layers.26.linear_attn.in_proj_qkv                    12      0     1.149e-02    2.236e-03    1.171e-02
  model.layers.26.linear_attn.in_proj_z                      12      0     1.258e-02    1.797e-03    1.271e-02
  model.layers.26.linear_attn.out_proj                       12      0     1.493e-02    2.343e-03    1.511e-02
  model.layers.26.mlp.down_proj                              12      0     1.270e-02    2.467e-03    1.294e-02
  model.layers.26.mlp.gate_proj                              12      0     1.111e-02    3.295e-03    1.159e-02
  model.layers.26.mlp.up_proj                                12      0     1.086e-02    3.547e-03    1.142e-02
  model.layers.27.mlp.down_proj                              12      0     1.336e-02    2.576e-03    1.360e-02
  model.layers.27.mlp.gate_proj                              12      0     1.080e-02    3.203e-03    1.126e-02
  model.layers.27.mlp.up_proj                                12      0     1.090e-02    3.553e-03    1.146e-02
  model.layers.27.self_attn.k_proj                           12      0     1.216e-02    2.498e-03    1.241e-02
  model.layers.27.self_attn.o_proj                           12      0     1.711e-02    2.745e-03    1.732e-02
  model.layers.27.self_attn.q_proj                           12      0     1.021e-02    2.061e-03    1.041e-02
  model.layers.27.self_attn.v_proj                           12      0     9.822e-03    2.244e-03    1.008e-02
  model.layers.28.linear_attn.in_proj_qkv                    12      0     1.178e-02    2.221e-03    1.199e-02
  model.layers.28.linear_attn.in_proj_z                      12      0     1.288e-02    1.818e-03    1.301e-02
  model.layers.28.linear_attn.out_proj                       12      0     1.477e-02    2.295e-03    1.495e-02
  model.layers.28.mlp.down_proj                              12      0     1.301e-02    2.596e-03    1.326e-02
  model.layers.28.mlp.gate_proj                              12      0     1.103e-02    3.174e-03    1.148e-02
  model.layers.28.mlp.up_proj                                12      0     1.084e-02    3.538e-03    1.140e-02
  model.layers.29.linear_attn.in_proj_qkv                    12      0     1.204e-02    2.244e-03    1.224e-02
  model.layers.29.linear_attn.in_proj_z                      12      0     1.396e-02    1.962e-03    1.409e-02
  model.layers.29.linear_attn.out_proj                       12      0     1.654e-02    2.351e-03    1.670e-02
  model.layers.29.mlp.down_proj                              12      0     1.347e-02    2.622e-03    1.372e-02
  model.layers.29.mlp.gate_proj                              12      0     1.114e-02    3.185e-03    1.159e-02
  model.layers.29.mlp.up_proj                                12      0     1.080e-02    3.537e-03    1.136e-02
  model.layers.3.mlp.down_proj                               12      0     1.419e-02    2.578e-03    1.442e-02
  model.layers.3.mlp.gate_proj                               12      0     1.018e-02    3.522e-03    1.077e-02
  model.layers.3.mlp.up_proj                                 12      0     1.100e-02    3.600e-03    1.157e-02
  model.layers.3.self_attn.k_proj                            12      0     1.246e-02    2.588e-03    1.273e-02
  model.layers.3.self_attn.o_proj                            12      0     1.494e-02    2.175e-03    1.510e-02
  model.layers.3.self_attn.q_proj                            12      0     1.169e-02    2.036e-03    1.186e-02
  model.layers.3.self_attn.v_proj                            12      0     1.263e-02    2.305e-03    1.284e-02
  model.layers.30.linear_attn.in_proj_qkv                    12      0     1.190e-02    2.230e-03    1.211e-02
  model.layers.30.linear_attn.in_proj_z                      12      0     1.350e-02    1.891e-03    1.363e-02
  model.layers.30.linear_attn.out_proj                       12      0     1.623e-02    2.415e-03    1.641e-02
  model.layers.30.mlp.down_proj                              12      0     1.285e-02    2.560e-03    1.310e-02
  model.layers.30.mlp.gate_proj                              12      0     1.172e-02    3.189e-03    1.214e-02
  model.layers.30.mlp.up_proj                                12      0     1.067e-02    3.520e-03    1.124e-02
  model.layers.31.mlp.down_proj                              12      0     1.318e-02    2.599e-03    1.343e-02
  model.layers.31.mlp.gate_proj                              12      0     1.148e-02    3.157e-03    1.190e-02
  model.layers.31.mlp.up_proj                                12      0     1.089e-02    3.517e-03    1.145e-02
  model.layers.31.self_attn.k_proj                           12      0     1.206e-02    2.514e-03    1.232e-02
  model.layers.31.self_attn.o_proj                           12      0     2.216e-02    3.029e-03    2.236e-02
  model.layers.31.self_attn.q_proj                           12      0     1.017e-02    2.075e-03    1.038e-02
  model.layers.31.self_attn.v_proj                           12      0     9.911e-03    2.190e-03    1.015e-02
  model.layers.32.linear_attn.in_proj_qkv                    12      0     1.281e-02    2.287e-03    1.301e-02
  model.layers.32.linear_attn.in_proj_z                      12      0     1.411e-02    1.984e-03    1.424e-02
  model.layers.32.linear_attn.out_proj                       12      0     1.565e-02    2.305e-03    1.582e-02
  model.layers.32.mlp.down_proj                              12      0     1.303e-02    2.616e-03    1.329e-02
  model.layers.32.mlp.gate_proj                              12      0     1.165e-02    3.105e-03    1.206e-02
  model.layers.32.mlp.up_proj                                12      0     1.107e-02    3.522e-03    1.162e-02
  model.layers.33.linear_attn.in_proj_qkv                    12      0     1.260e-02    2.297e-03    1.280e-02
  model.layers.33.linear_attn.in_proj_z                      12      0     1.385e-02    1.989e-03    1.400e-02
  model.layers.33.linear_attn.out_proj                       12      0     1.744e-02    2.367e-03    1.760e-02
  model.layers.33.mlp.down_proj                              12      0     1.341e-02    2.604e-03    1.366e-02
  model.layers.33.mlp.gate_proj                              12      0     1.192e-02    3.114e-03    1.232e-02
  model.layers.33.mlp.up_proj                                12      0     1.124e-02    3.539e-03    1.178e-02
  model.layers.34.linear_attn.in_proj_qkv                    12      0     1.252e-02    2.275e-03    1.272e-02
  model.layers.34.linear_attn.in_proj_z                      12      0     1.343e-02    1.914e-03    1.356e-02
  model.layers.34.linear_attn.out_proj                       12      0     1.613e-02    2.365e-03    1.630e-02
  model.layers.34.mlp.down_proj                              12      0     1.377e-02    2.611e-03    1.402e-02
  model.layers.34.mlp.gate_proj                              12      0     1.207e-02    3.072e-03    1.245e-02
  model.layers.34.mlp.up_proj                                12      0     1.115e-02    3.508e-03    1.169e-02
  model.layers.35.mlp.down_proj                              12      0     1.414e-02    2.562e-03    1.437e-02
  model.layers.35.mlp.gate_proj                              12      0     1.065e-02    3.066e-03    1.108e-02
  model.layers.35.mlp.up_proj                                12      0     1.138e-02    3.581e-03    1.193e-02
  model.layers.35.self_attn.k_proj                           12      0     1.174e-02    2.499e-03    1.201e-02
  model.layers.35.self_attn.o_proj                           12      0     1.996e-02    2.805e-03    2.016e-02
  model.layers.35.self_attn.q_proj                           12      0     1.073e-02    2.114e-03    1.093e-02
  model.layers.35.self_attn.v_proj                           12      0     1.069e-02    2.260e-03    1.092e-02
  model.layers.36.linear_attn.in_proj_qkv                    12      0     1.293e-02    2.275e-03    1.314e-02
  model.layers.36.linear_attn.in_proj_z                      12      0     1.209e-02    1.744e-03    1.221e-02
  model.layers.36.linear_attn.out_proj                       12      0     1.664e-02    2.260e-03    1.679e-02
  model.layers.36.mlp.down_proj                              12      0     1.511e-02    2.625e-03    1.534e-02
  model.layers.36.mlp.gate_proj                              12      0     1.026e-02    3.146e-03    1.073e-02
  model.layers.36.mlp.up_proj                                12      0     1.153e-02    3.605e-03    1.208e-02
  model.layers.37.linear_attn.in_proj_qkv                    12      0     1.222e-02    2.241e-03    1.242e-02
  model.layers.37.linear_attn.in_proj_z                      12      0     1.057e-02    1.598e-03    1.069e-02
  model.layers.37.linear_attn.out_proj                       12      0     1.841e-02    2.196e-03    1.854e-02
  model.layers.37.mlp.down_proj                              12      0     1.541e-02    2.734e-03    1.565e-02
  model.layers.37.mlp.gate_proj                              12      0     1.056e-02    3.236e-03    1.104e-02
  model.layers.37.mlp.up_proj                                12      0     1.173e-02    3.629e-03    1.228e-02
  model.layers.38.linear_attn.in_proj_qkv                    12      0     1.201e-02    2.319e-03    1.223e-02
  model.layers.38.linear_attn.in_proj_z                      12      0     1.100e-02    1.629e-03    1.112e-02
  model.layers.38.linear_attn.out_proj                       12      0     1.659e-02    2.312e-03    1.675e-02
  model.layers.38.mlp.down_proj                              12      0     1.600e-02    2.698e-03    1.622e-02
  model.layers.38.mlp.gate_proj                              12      0     1.070e-02    3.068e-03    1.113e-02
  model.layers.38.mlp.up_proj                                12      0     1.171e-02    3.642e-03    1.227e-02
  model.layers.39.mlp.down_proj                              12      0     1.436e-02    2.656e-03    1.461e-02
  model.layers.39.mlp.gate_proj                              12      0     9.969e-03    3.158e-03    1.046e-02
  model.layers.39.mlp.up_proj                                12      0     1.180e-02    3.648e-03    1.235e-02
  model.layers.39.self_attn.k_proj                           12      0     1.179e-02    2.469e-03    1.205e-02
  model.layers.39.self_attn.o_proj                           12      0     2.073e-02    2.839e-03    2.093e-02
  model.layers.39.self_attn.q_proj                           12      0     1.046e-02    2.095e-03    1.066e-02
  model.layers.39.self_attn.v_proj                           12      0     1.092e-02    2.333e-03    1.116e-02
  model.layers.4.linear_attn.in_proj_qkv                     12      0     1.192e-02    2.308e-03    1.214e-02
  model.layers.4.linear_attn.in_proj_z                       12      0     1.245e-02    1.774e-03    1.258e-02
  model.layers.4.linear_attn.out_proj                        12      0     1.792e-02    2.410e-03    1.808e-02
  model.layers.4.mlp.down_proj                               12      0     1.444e-02    2.608e-03    1.467e-02
  model.layers.4.mlp.gate_proj                               12      0     1.014e-02    3.510e-03    1.073e-02
  model.layers.4.mlp.up_proj                                 12      0     1.122e-02    3.616e-03    1.179e-02
  model.layers.40.linear_attn.in_proj_qkv                    12      0     1.225e-02    2.282e-03    1.246e-02
  model.layers.40.linear_attn.in_proj_z                      12      0     1.172e-02    1.687e-03    1.184e-02
  model.layers.40.linear_attn.out_proj                       12      0     1.605e-02    2.268e-03    1.621e-02
  model.layers.40.mlp.down_proj                              12      0     1.482e-02    2.678e-03    1.506e-02
  model.layers.40.mlp.gate_proj                              12      0     1.009e-02    3.135e-03    1.056e-02
  model.layers.40.mlp.up_proj                                12      0     1.164e-02    3.642e-03    1.220e-02
  model.layers.41.linear_attn.in_proj_qkv                    12      0     1.176e-02    2.259e-03    1.198e-02
  model.layers.41.linear_attn.in_proj_z                      12      0     1.224e-02    1.731e-03    1.237e-02
  model.layers.41.linear_attn.out_proj                       12      0     1.650e-02    2.256e-03    1.666e-02
  model.layers.41.mlp.down_proj                              12      0     1.479e-02    2.701e-03    1.504e-02
  model.layers.41.mlp.gate_proj                              12      0     1.085e-02    3.215e-03    1.132e-02
  model.layers.41.mlp.up_proj                                12      0     1.165e-02    3.636e-03    1.220e-02
  model.layers.42.linear_attn.in_proj_qkv                    12      0     1.163e-02    2.276e-03    1.185e-02
  model.layers.42.linear_attn.in_proj_z                      12      0     1.242e-02    1.746e-03    1.254e-02
  model.layers.42.linear_attn.out_proj                       12      0     1.567e-02    2.282e-03    1.583e-02
  model.layers.42.mlp.down_proj                              12      0     1.453e-02    2.664e-03    1.477e-02
  model.layers.42.mlp.gate_proj                              12      0     1.173e-02    3.239e-03    1.217e-02
  model.layers.42.mlp.up_proj                                12      0     1.142e-02    3.582e-03    1.197e-02
  model.layers.43.mlp.down_proj                              12      0     1.385e-02    2.671e-03    1.411e-02
  model.layers.43.mlp.gate_proj                              12      0     1.084e-02    3.217e-03    1.131e-02
  model.layers.43.mlp.up_proj                                12      0     1.150e-02    3.615e-03    1.206e-02
  model.layers.43.self_attn.k_proj                           12      0     1.246e-02    2.466e-03    1.271e-02
  model.layers.43.self_attn.o_proj                           12      0     2.163e-02    3.132e-03    2.185e-02
  model.layers.43.self_attn.q_proj                           12      0     1.058e-02    2.089e-03    1.078e-02
  model.layers.43.self_attn.v_proj                           12      0     1.044e-02    2.263e-03    1.069e-02
  model.layers.44.linear_attn.in_proj_qkv                    12      0     1.204e-02    2.230e-03    1.225e-02
  model.layers.44.linear_attn.in_proj_z                      12      0     1.224e-02    1.700e-03    1.236e-02
  model.layers.44.linear_attn.out_proj                       12      0     1.626e-02    2.298e-03    1.643e-02
  model.layers.44.mlp.down_proj                              12      0     1.413e-02    2.713e-03    1.439e-02
  model.layers.44.mlp.gate_proj                              12      0     1.083e-02    3.261e-03    1.131e-02
  model.layers.44.mlp.up_proj                                12      0     1.171e-02    3.618e-03    1.225e-02
  model.layers.45.linear_attn.in_proj_qkv                    12      0     1.207e-02    2.277e-03    1.228e-02
  model.layers.45.linear_attn.in_proj_z                      12      0     1.264e-02    1.763e-03    1.276e-02
  model.layers.45.linear_attn.out_proj                       12      0     1.812e-02    2.434e-03    1.828e-02
  model.layers.45.mlp.down_proj                              12      0     1.437e-02    2.742e-03    1.463e-02
  model.layers.45.mlp.gate_proj                              12      0     1.082e-02    3.287e-03    1.131e-02
  model.layers.45.mlp.up_proj                                12      0     1.155e-02    3.622e-03    1.211e-02
  model.layers.46.linear_attn.in_proj_qkv                    12      0     1.195e-02    2.245e-03    1.216e-02
  model.layers.46.linear_attn.in_proj_z                      12      0     1.186e-02    1.653e-03    1.197e-02
  model.layers.46.linear_attn.out_proj                       12      0     1.869e-02    2.371e-03    1.884e-02
  model.layers.46.mlp.down_proj                              12      0     1.422e-02    2.687e-03    1.447e-02
  model.layers.46.mlp.gate_proj                              12      0     1.110e-02    3.312e-03    1.158e-02
  model.layers.46.mlp.up_proj                                12      0     1.150e-02    3.600e-03    1.205e-02
  model.layers.47.mlp.down_proj                              12      0     1.426e-02    2.640e-03    1.450e-02
  model.layers.47.mlp.gate_proj                              12      0     1.055e-02    3.335e-03    1.106e-02
  model.layers.47.mlp.up_proj                                12      0     1.175e-02    3.621e-03    1.229e-02
  model.layers.47.self_attn.k_proj                           12      0     1.241e-02    2.473e-03    1.265e-02
  model.layers.47.self_attn.o_proj                           12      0     1.955e-02    2.555e-03    1.971e-02
  model.layers.47.self_attn.q_proj                           12      0     9.651e-03    2.057e-03    9.869e-03
  model.layers.47.self_attn.v_proj                           12      0     1.057e-02    2.230e-03    1.081e-02
  model.layers.48.linear_attn.in_proj_qkv                    12      0     1.271e-02    2.265e-03    1.291e-02
  model.layers.48.linear_attn.in_proj_z                      12      0     1.209e-02    1.652e-03    1.220e-02
  model.layers.48.linear_attn.out_proj                       12      0     1.715e-02    2.419e-03    1.731e-02
  model.layers.48.mlp.down_proj                              12      0     1.482e-02    2.681e-03    1.506e-02
  model.layers.48.mlp.gate_proj                              12      0     1.034e-02    3.333e-03    1.086e-02
  model.layers.48.mlp.up_proj                                12      0     1.169e-02    3.603e-03    1.223e-02
  model.layers.49.linear_attn.in_proj_qkv                    12      0     1.208e-02    2.277e-03    1.229e-02
  model.layers.49.linear_attn.in_proj_z                      12      0     1.151e-02    1.606e-03    1.162e-02
  model.layers.49.linear_attn.out_proj                       12      0     1.911e-02    2.175e-03    1.923e-02
  model.layers.49.mlp.down_proj                              12      0     1.566e-02    2.696e-03    1.589e-02
  model.layers.49.mlp.gate_proj                              12      0     1.006e-02    3.366e-03    1.061e-02
  model.layers.49.mlp.up_proj                                12      0     1.149e-02    3.562e-03    1.203e-02
  model.layers.5.linear_attn.in_proj_qkv                     12      0     1.246e-02    2.271e-03    1.266e-02
  model.layers.5.linear_attn.in_proj_z                       12      0     1.391e-02    1.910e-03    1.404e-02
  model.layers.5.linear_attn.out_proj                        12      0     1.770e-02    2.208e-03    1.783e-02
  model.layers.5.mlp.down_proj                               12      0     1.529e-02    2.522e-03    1.549e-02
  model.layers.5.mlp.gate_proj                               12      0     1.017e-02    3.533e-03    1.077e-02
  model.layers.5.mlp.up_proj                                 12      0     1.128e-02    3.622e-03    1.184e-02
  model.layers.50.linear_attn.in_proj_qkv                    12      0     1.166e-02    2.255e-03    1.188e-02
  model.layers.50.linear_attn.in_proj_z                      12      0     1.085e-02    1.487e-03    1.095e-02
  model.layers.50.linear_attn.out_proj                       12      0     1.816e-02    2.350e-03    1.831e-02
  model.layers.50.mlp.down_proj                              12      0     1.519e-02    2.670e-03    1.542e-02
  model.layers.50.mlp.gate_proj                              12      0     9.845e-03    3.282e-03    1.038e-02
  model.layers.50.mlp.up_proj                                12      0     1.086e-02    3.437e-03    1.140e-02
  model.layers.51.mlp.down_proj                              12      0     1.523e-02    2.655e-03    1.546e-02
  model.layers.51.mlp.gate_proj                              12      0     8.697e-03    3.392e-03    9.334e-03
  model.layers.51.mlp.up_proj                                12      0     1.112e-02    3.553e-03    1.167e-02
  model.layers.51.self_attn.k_proj                           12      0     1.103e-02    2.407e-03    1.130e-02
  model.layers.51.self_attn.o_proj                           12      0     1.923e-02    2.302e-03    1.936e-02
  model.layers.51.self_attn.q_proj                           12      0     9.138e-03    2.022e-03    9.359e-03
  model.layers.51.self_attn.v_proj                           12      0     1.024e-02    2.279e-03    1.049e-02
  model.layers.52.linear_attn.in_proj_qkv                    12      0     9.636e-03    2.113e-03    9.866e-03
  model.layers.52.linear_attn.in_proj_z                      12      0     8.545e-03    1.285e-03    8.642e-03
  model.layers.52.linear_attn.out_proj                       12      0     1.874e-02    2.302e-03    1.888e-02
  model.layers.52.mlp.down_proj                              12      0     1.785e-02    2.686e-03    1.805e-02
  model.layers.52.mlp.gate_proj                              12      0     8.147e-03    3.398e-03    8.827e-03
  model.layers.52.mlp.up_proj                                12      0     1.106e-02    3.565e-03    1.162e-02
  model.layers.53.linear_attn.in_proj_qkv                    12      0     9.508e-03    2.128e-03    9.743e-03
  model.layers.53.linear_attn.in_proj_z                      12      0     8.683e-03    1.312e-03    8.782e-03
  model.layers.53.linear_attn.out_proj                       12      0     1.691e-02    2.275e-03    1.707e-02
  model.layers.53.mlp.down_proj                              12      0     1.847e-02    2.768e-03    1.867e-02
  model.layers.53.mlp.gate_proj                              12      0     7.921e-03    3.402e-03    8.621e-03
  model.layers.53.mlp.up_proj                                12      0     1.097e-02    3.568e-03    1.154e-02
  model.layers.54.linear_attn.in_proj_qkv                    12      0     9.918e-03    2.234e-03    1.017e-02
  model.layers.54.linear_attn.in_proj_z                      12      0     8.291e-03    1.279e-03    8.388e-03
  model.layers.54.linear_attn.out_proj                       12      0     1.747e-02    2.135e-03    1.760e-02
  model.layers.54.mlp.down_proj                              12      0     1.823e-02    2.478e-03    1.839e-02
  model.layers.54.mlp.gate_proj                              12      0     7.279e-03    3.353e-03    8.014e-03
  model.layers.54.mlp.up_proj                                12      0     9.170e-03    3.420e-03    9.788e-03
  model.layers.55.mlp.down_proj                              12      0     1.696e-02    2.628e-03    1.716e-02
  model.layers.55.mlp.gate_proj                              12      0     7.240e-03    3.387e-03    7.994e-03
  model.layers.55.mlp.up_proj                                12      0     1.131e-02    3.603e-03    1.187e-02
  model.layers.55.self_attn.k_proj                           12      0     1.086e-02    2.399e-03    1.112e-02
  model.layers.55.self_attn.o_proj                           12      0     1.727e-02    2.197e-03    1.741e-02
  model.layers.55.self_attn.q_proj                           12      0     8.911e-03    2.105e-03    9.156e-03
  model.layers.55.self_attn.v_proj                           12      0     9.493e-03    2.221e-03    9.748e-03
  model.layers.56.linear_attn.in_proj_qkv                    12      0     9.005e-03    2.094e-03    9.245e-03
  model.layers.56.linear_attn.in_proj_z                      12      0     8.721e-03    1.329e-03    8.821e-03
  model.layers.56.linear_attn.out_proj                       12      0     1.732e-02    2.108e-03    1.745e-02
  model.layers.56.mlp.down_proj                              12      0     1.966e-02    2.529e-03    1.983e-02
  model.layers.56.mlp.gate_proj                              12      0     7.331e-03    3.347e-03    8.060e-03
  model.layers.56.mlp.up_proj                                12      0     1.133e-02    3.598e-03    1.189e-02
  model.layers.57.linear_attn.in_proj_qkv                    12      0     8.637e-03    2.052e-03    8.878e-03
  model.layers.57.linear_attn.in_proj_z                      12      0     8.528e-03    1.300e-03    8.627e-03
  model.layers.57.linear_attn.out_proj                       12      0     1.596e-02    2.503e-03    1.615e-02
  model.layers.57.mlp.down_proj                              12      0     1.632e-02    2.800e-03    1.656e-02
  model.layers.57.mlp.gate_proj                              12      0     7.140e-03    3.324e-03    7.876e-03
  model.layers.57.mlp.up_proj                                12      0     1.115e-02    3.599e-03    1.172e-02
  model.layers.58.linear_attn.in_proj_qkv                    12      0     9.195e-03    2.133e-03    9.439e-03
  model.layers.58.linear_attn.in_proj_z                      12      0     8.769e-03    1.312e-03    8.868e-03
  model.layers.58.linear_attn.out_proj                       12      0     1.887e-02    2.408e-03    1.902e-02
  model.layers.58.mlp.down_proj                              12      0     1.671e-02    2.600e-03    1.692e-02
  model.layers.58.mlp.gate_proj                              12      0     7.023e-03    3.315e-03    7.767e-03
  model.layers.58.mlp.up_proj                                12      0     1.044e-02    3.551e-03    1.103e-02
  model.layers.59.mlp.down_proj                              12      0     1.666e-02    2.500e-03    1.685e-02
  model.layers.59.mlp.gate_proj                              12      0     6.615e-03    3.308e-03    7.396e-03
  model.layers.59.mlp.up_proj                                12      0     9.282e-03    3.449e-03    9.900e-03
  model.layers.59.self_attn.k_proj                           12      0     1.021e-02    2.339e-03    1.048e-02
  model.layers.59.self_attn.o_proj                           12      0     1.797e-02    2.281e-03    1.812e-02
  model.layers.59.self_attn.q_proj                           12      0     8.669e-03    2.119e-03    8.923e-03
  model.layers.59.self_attn.v_proj                           12      0     9.121e-03    2.155e-03    9.374e-03
  model.layers.6.linear_attn.in_proj_qkv                     12      0     1.263e-02    2.268e-03    1.284e-02
  model.layers.6.linear_attn.in_proj_z                       12      0     1.426e-02    1.993e-03    1.440e-02
  model.layers.6.linear_attn.out_proj                        12      0     1.682e-02    2.297e-03    1.698e-02
  model.layers.6.mlp.down_proj                               12      0     1.356e-02    2.399e-03    1.377e-02
  model.layers.6.mlp.gate_proj                               12      0     1.046e-02    3.491e-03    1.102e-02
  model.layers.6.mlp.up_proj                                 12      0     1.113e-02    3.563e-03    1.168e-02
  model.layers.60.linear_attn.in_proj_qkv                    12      0     7.945e-03    1.976e-03    8.186e-03
  model.layers.60.linear_attn.in_proj_z                      12      0     7.519e-03    1.195e-03    7.613e-03
  model.layers.60.linear_attn.out_proj                       12      0     1.690e-02    2.503e-03    1.709e-02
  model.layers.60.mlp.down_proj                              12      0     1.588e-02    2.707e-03    1.610e-02
  model.layers.60.mlp.gate_proj                              12      0     6.827e-03    3.241e-03    7.557e-03
  model.layers.60.mlp.up_proj                                12      0     9.185e-03    3.390e-03    9.792e-03
  model.layers.61.linear_attn.in_proj_qkv                    12      0     8.884e-03    2.090e-03    9.126e-03
  model.layers.61.linear_attn.in_proj_z                      12      0     7.958e-03    1.246e-03    8.054e-03
  model.layers.61.linear_attn.out_proj                       12      0     1.672e-02    2.346e-03    1.688e-02
  model.layers.61.mlp.down_proj                              12      0     1.499e-02    2.662e-03    1.522e-02
  model.layers.61.mlp.gate_proj                              12      0     6.562e-03    3.199e-03    7.300e-03
  model.layers.61.mlp.up_proj                                12      0     8.706e-03    3.317e-03    9.314e-03
  model.layers.62.linear_attn.in_proj_qkv                    12      0     8.143e-03    1.972e-03    8.378e-03
  model.layers.62.linear_attn.in_proj_z                      12      0     7.405e-03    1.221e-03    7.506e-03
  model.layers.62.linear_attn.out_proj                       12      0     1.516e-02    2.358e-03    1.534e-02
  model.layers.62.mlp.down_proj                              12      0     1.495e-02    2.536e-03    1.516e-02
  model.layers.62.mlp.gate_proj                              12      0     6.269e-03    3.135e-03    7.008e-03
  model.layers.62.mlp.up_proj                                12      0     6.864e-03    3.147e-03    7.549e-03
  model.layers.63.mlp.down_proj                              12      0     1.190e-02    2.008e-03    1.207e-02
  model.layers.63.mlp.gate_proj                              12      0     5.647e-03    3.070e-03    6.427e-03
  model.layers.63.mlp.up_proj                                12      0     4.713e-03    3.051e-03    5.616e-03
  model.layers.63.self_attn.k_proj                           12      0     1.073e-02    2.414e-03    1.099e-02
  model.layers.63.self_attn.o_proj                           12      0     1.299e-02    2.013e-03    1.315e-02
  model.layers.63.self_attn.q_proj                           12      0     7.009e-03    1.937e-03    7.272e-03
  model.layers.63.self_attn.v_proj                           12      0     4.678e-03    1.730e-03    4.987e-03
  model.layers.7.mlp.down_proj                               12      0     1.419e-02    2.576e-03    1.443e-02
  model.layers.7.mlp.gate_proj                               12      0     1.042e-02    3.567e-03    1.101e-02
  model.layers.7.mlp.up_proj                                 12      0     1.136e-02    3.626e-03    1.192e-02
  model.layers.7.self_attn.k_proj                            12      0     1.386e-02    2.708e-03    1.412e-02
  model.layers.7.self_attn.o_proj                            12      0     1.324e-02    2.268e-03    1.343e-02
  model.layers.7.self_attn.q_proj                            12      0     1.249e-02    2.078e-03    1.267e-02
  model.layers.7.self_attn.v_proj                            12      0     1.102e-02    2.326e-03    1.126e-02
  model.layers.8.linear_attn.in_proj_qkv                     12      0     1.245e-02    2.246e-03    1.265e-02
  model.layers.8.linear_attn.in_proj_z                       12      0     1.305e-02    1.811e-03    1.317e-02
  model.layers.8.linear_attn.out_proj                        12      0     1.615e-02    2.282e-03    1.631e-02
  model.layers.8.mlp.down_proj                               12      0     1.507e-02    2.555e-03    1.529e-02
  model.layers.8.mlp.gate_proj                               12      0     1.051e-02    3.565e-03    1.110e-02
  model.layers.8.mlp.up_proj                                 12      0     1.130e-02    3.618e-03    1.187e-02
  model.layers.9.linear_attn.in_proj_qkv                     12      0     1.310e-02    2.236e-03    1.329e-02
  model.layers.9.linear_attn.in_proj_z                       12      0     1.360e-02    1.895e-03    1.373e-02
  model.layers.9.linear_attn.out_proj                        12      0     1.599e-02    2.267e-03    1.615e-02
  model.layers.9.mlp.down_proj                               12      0     1.345e-02    2.472e-03    1.368e-02
  model.layers.9.mlp.gate_proj                               12      0     1.059e-02    3.573e-03    1.118e-02
  model.layers.9.mlp.up_proj                                 12      0     1.120e-02    3.610e-03    1.177e-02
[i8] wrote per-class op-level errors to /cache/classerr_27b.json

OP-LEVEL error against an fp32 oracle, per linear class, on each
arm's OWN activation -- upstream of the model's amplifier, and
paired by class.  Ratios are arm / INT8; below 1.0 is better.
  arm           classes  median rel err  median ratio   better/worse       p05       p95
  INT8              400      1.1951e-02        1.0000      0/400        1.0000    1.0000
  INT8G1            400      8.5070e-03        0.6800    400/0          0.5305    0.8672
  A sign test on `better/worse` is the check that cannot be carried by
  one class: INT8 is its own row and must read exactly 1.0000 with
  every class on the boundary, or the pairing is not by class.
[i8] per-arm census (int8 kernel calls): {'REF': 0, 'INT8': 2000, 'INT8G1': 2000}

ARM RECEIPTS -- what each arm actually executed, from the engine's
own per-call census.  fused/standalone is the reconstruct variant;
acc16/acc32 is the cuBLAS compute type leg B asked hgemm for.
  arm       legB calls          rows     fused  standalone     acc16     acc32   fp32pin
  REF             2000       4096000      2000           0      2000         0         0
  INT8            2000       4096000         0        2000      2000         0         0
  INT8G1          2000       4096000         0        2000      2000         0         0
  all arms match their expected leg/accumulator signature
  prefill positions aligned: [1, 1, 1, 1, 1] per prompt, identical across all 3 arms

THE TAIL over PREFILL positions only -- one per prefill chunk, so
every arm scores the SAME context. Decode positions are reported
separately and are NOT a numerics measure: once a token flips, the
arms are reading different contexts. The control proves it -- pooled
over decode, STD (which contains no int8 at all) has the same max KL
as INT8, so a pooled tail cannot attribute anything to the kernel.

arm            n        mean         p50         p95         p99         MAX     argmax flip
REF            5   0.000e+00   0.000e+00   0.000e+00   0.000e+00   0.000e+00 0/5 =   0.00%
INT8           5   8.630e-04   1.731e-04   3.815e-03   3.815e-03   3.815e-03 0/5 =   0.00%
INT8G1         5   2.306e-04   2.661e-05   1.057e-03   1.057e-03   1.057e-03 0/5 =   0.00%

Decode positions (divergence-contaminated, shown for completeness):
arm            n        mean         MAX    greedy agree
REF           40   0.000e+00   0.000e+00 40/40 =  100.0%
INT8          40   1.616e-03   1.150e-02 40/40 =  100.0%
INT8G1        40   8.782e-04   7.746e-03 40/40 =  100.0%

SAMPLER-RELEVANT TAIL @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF            5     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/  5     0.00%
INT8           5     100.00%  6.476e-03  3.238e-02  3.238e-02    3.198e-03 0/  5     2.50%
INT8G1         5     100.00%  2.935e-03  1.467e-02  1.467e-02    6.154e-04 1/  5     2.50%

SAMPLER-RELEVANT TAIL @ owner's k~40 (T=1.0, top_p=1.0, top_k=40), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF            5     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/  5     0.00%
INT8           5      98.06%  7.910e-03  3.518e-02  3.518e-02          inf 5/  5     2.50%
INT8G1         5      97.54%  3.293e-03  1.520e-02  1.520e-02          inf 5/  5     0.00%

FULL-VOCAB numbers below, kept and labelled: they are the right
instrument for GREEDY or beam decode, where every rank matters.

arm         first-prefill-position KL       median          max
REF                         0.000e+00    0.000e+00    0.000e+00
INT8                        8.630e-04    1.731e-04    3.815e-03
INT8G1                      2.306e-04    2.661e-05    1.057e-03

PAIRED TV @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions.
Each row is A scored against B -- B is that ROW's reference, not the
table's.  TV is the expected speculative accept-rate loss, so the
column is read directly in percentage points.  The interval is a
cluster bootstrap over PROMPTS; the IQR is the raw spread over
positions with no floor applied to it.

  A       vs B        n  mean TV pp           95% CI pp    p25 pp    p50 pp    p75 pp    p95 pp    max pp
  INT8    REF         5      0.6476 [  0.0000,  1.9427]    0.0000    0.0000    0.0000    3.2379    3.2379

lm_head CUT -- the divergence entering the head vs leaving it,
prefill positions, relative rms.  'in' is the residual stream after
24 layers; 'out' is the centred logits.  amp = out/in: at 1.0 the
head passes the body's divergence through unchanged, above it the
head adds its own.

  A       vs B        n   rel rms IN  rel rms OUT      amp
  INT8    REF         5   4.5344e-02   4.1785e-02    0.922

PER-TOKEN AMAX -- is the crest factor of the quantised row what
selects the tail?  crest = amax/rms of the int8 GEMM's input row;
under the uniform-residual model the relative noise the row picks
up is crest/(127*sqrt(12)) = crest/440, and qerr is that noise
measured directly by round-tripping the scored row through int8.
arm supplying the statistics: INT8

  statistic                                  p50         p95         p99         max
  crest, scored row (max/linear)       1.134e+01   1.267e+01   1.267e+01   1.267e+01
  crest, any row in chunk              1.849e+01   1.913e+01   1.913e+01   1.913e+01
  qerr, scored row (rel rms)           2.581e-02   2.864e-02   2.864e-02   2.864e-02
  n positions: 5

What the per-token scale is paying for, over the whole census.
crest_g is the crest a per-128-group scale would leave --
sqrt(mean_b amax_b^2)/rms -- and its ratio to crest is the noise
reduction available.  conc is max/median of the per-128-block rms:
at 1 the row is flat and grouping buys nothing.
  statistic                                  p50         p95         p99         max
  crest_g (per-128 scale)                  2.623       2.890       2.955       3.031
  crest / crest_g (noise cut)              2.082       3.917       5.284       7.783
  block rms concentration                  3.168      10.600      21.681      39.114
  n (class, position) samples: 2000

The 10 worst INT8 positions, with the crest factor at each.  If the
per-token amax is the mechanism, these are the extreme-crest rows.
   prompt  chunk      INT8 KL   crest row  crest chunk        qerr  worst linear
        3      0    3.815e-03      10.952       18.495   2.486e-02  L6.linear_attn.out_proj
        4      0    1.886e-04      11.626       17.762   2.660e-02  L3.mlp.down_proj
        2      0    1.731e-04      11.220       18.872   2.527e-02  L2.linear_attn.out_proj
        1      0    1.021e-04      11.341       19.134   2.581e-02  L1.mlp.down_proj
        0      0    3.621e-05      12.668       18.448   2.864e-02  L14.linear_attn.out_proj
  ... and the 10 MEDIAN INT8 positions, as the contrast:
        2      0    1.731e-04      11.220       18.872   2.527e-02  L2.linear_attn.out_proj
        4      0    1.886e-04      11.626       17.762   2.660e-02  L3.mlp.down_proj
        3      0    3.815e-03      10.952       18.495   2.486e-02  L6.linear_attn.out_proj

Rank correlation of the crest factor with each arm's KL.  STD is the
NULL CONTROL: it runs no quantiser, so the crest factor cannot cause
its KL.  A crest that ranks STD as well as it ranks INT8 is ranking
hard POSITIONS, not the quantiser, and explains nothing.
  crest statistic                    REF       INT8     INT8G1  INT8 excess
  crest, scored row               -0.400     -0.700     -0.600
  crest, any row                  -0.400     -0.200     -0.100
  qerr, scored row                -0.400     -0.700     -0.600

Mean crest of the scored row, by linear class -- the 15 heaviest.
A class well above the Gaussian-rotation expectation (~4.5 for a
128-wide Hadamard over K=5120) is a class the per-token scale is
wasting levels on.
  linear class                                        mean crest         p95         max
  L0.mlp.down_proj                                         9.118      11.152      11.152
  L5.mlp.down_proj                                         8.532      10.262      10.262
  L1.mlp.down_proj                                         8.194      11.341      11.341
  L14.linear_attn.out_proj                                 8.101      12.668      12.668
  L60.mlp.down_proj                                        7.970       9.410       9.410
  L2.mlp.down_proj                                         7.951       9.155       9.155
  L45.linear_attn.out_proj                                 7.905       9.695       9.695
  L2.linear_attn.out_proj                                  7.845      11.220      11.220
  L37.linear_attn.out_proj                                 7.479       8.742       8.742
  L49.linear_attn.out_proj                                 7.467       9.430       9.430
  L6.linear_attn.out_proj                                  7.461      10.952      10.952
  L17.linear_attn.out_proj                                 7.446       8.525       8.525
  L6.mlp.down_proj                                         7.412      10.300      10.300
  L19.self_attn.o_proj                                     7.311       9.311       9.311
  L4.linear_attn.out_proj                                  7.295       8.398       8.398
EXIT=0
