[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (317092 > 262144). Running this sequence through the model will result in indexing errors
[i8] basket: 1 prompts x 16384 tok, domains ['sci']
compile-cache MISS: xgrammar — xgrammar compiled grammars (re)compiles this boot [/cache/xgrammar]
[TKV autotune-provenance] fp=57911e26f4e47f74 regime=cache-hit cells=8 sweep_src=2772549a0ef713db tile_tokens=4 num_splits=32,64,128,256 min_blocks_per_sm=0 resolved=0 held=0 inherited=0
/opt/venv/lib/python3.12/site-packages/torch/__init__.py:1172: FutureWarning: `torch.distributed.reduce_op` is deprecated, please use `torch.distributed.ReduceOp` instead
  return isinstance(obj, torch.Tensor)
[i8] trellis->path map: 401 linears
[i8] layer trace: 64 decoder layers hooked
JIT compile AFTER serving-ready [cpp_ext]: exl3_i8_gemm cached .so load — a live request paid this compile's latency. This is a boot-warmup coverage gap: extend warmup to pre-compile this kernel/specialization. Counter jit_compile_serving (must-not-fire) at GET /v1/admin/flag_truth.
[i8] prefix cache: server default resolved=True, per-request cache_enabled=False; any cache hit below is refused
driver serving-growth OVER RESERVE: the driver holds 2 MiB more than it did when the boot brackets closed, against the 0 MiB the serving floor's transient.serving_step.driver_growth term held for it. Those bytes are in no pool and no allocator counter -- the live card reaches them as driver.modules_serving and the boot ledger only as driver.residual -- and the KV layout is frozen, so the overage comes out of the free VRAM the floor is holding for one in-flight step (the verify tail, the DFlash context-assemble) and those reserves are the ones that fail first. The reading is on record for this configuration, so the next boot holds it; this process serves the rest of its life short by the overage.
activation admission RE-ARMED (driver residency over reserve): the step budget FELL to 5741 MiB, -642 MiB on the 6383 MiB it was enforcing and 6383 MiB the boot reading armed. The widest slate admission will build is narrower from here — narrower prefill chunks and deferred rows instead of a forward that OOMs after the layout freeze.
[i8] warmup done
  [0] sci            REF      16384tok  wall=  5.14s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [0] sci            REF2     16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [0] sci            STD      16384tok  wall=  5.41s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [0] sci            SIMROW   16384tok  wall=  7.67s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [0] sci            INT8     16384tok  wall= 12.93s  finish=length  leg=0F/3200S acc=3200x16/0x32

[i8] kernel census: int8 calls=3200 rows=6553600 reconstructs SKIPPED=3200 fallback (non-4bpw) calls=0
[i8] shapes served by the kernel (K,N)->calls: {(5120, 1024): 256, (5120, 6144): 384, (5120, 10240): 384, (5120, 12288): 128, (5120, 17408): 1024, (6144, 5120): 512, (17408, 5120): 512}
[i8] excluded projections: none  excluded calls=0

  linear class                                           served   excl  int8 vs fp32 legB vs fp32 int8 vs legB
  model.layers.0.linear_attn.in_proj_qkv                      9      0     4.827e-03    1.713e-03    5.132e-03
  model.layers.0.linear_attn.in_proj_z                        9      0     6.316e-03    1.129e-03    6.418e-03
  model.layers.0.linear_attn.out_proj                         9      0     1.249e-02    2.189e-03    1.268e-02
  model.layers.0.mlp.down_proj                                9      0     2.122e-02    2.609e-03    2.137e-02
  model.layers.0.mlp.gate_proj                                9      0     9.121e-03    3.413e-03    9.739e-03
  model.layers.0.mlp.up_proj                                  9      0     9.082e-03    3.418e-03    9.702e-03
  model.layers.1.linear_attn.in_proj_qkv                      9      0     1.186e-02    2.166e-03    1.206e-02
  model.layers.1.linear_attn.in_proj_z                        9      0     1.361e-02    1.840e-03    1.373e-02
  model.layers.1.linear_attn.out_proj                         9      0     1.604e-02    2.253e-03    1.619e-02
  model.layers.1.mlp.down_proj                                9      0     1.345e-02    2.322e-03    1.364e-02
  model.layers.1.mlp.gate_proj                                9      0     9.700e-03    3.459e-03    1.030e-02
  model.layers.1.mlp.up_proj                                  9      0     1.015e-02    3.488e-03    1.073e-02
  model.layers.10.linear_attn.in_proj_qkv                     9      0     1.304e-02    2.256e-03    1.323e-02
  model.layers.10.linear_attn.in_proj_z                       9      0     1.298e-02    1.838e-03    1.310e-02
  model.layers.10.linear_attn.out_proj                        9      0     1.498e-02    2.124e-03    1.513e-02
  model.layers.10.mlp.down_proj                               9      0     1.268e-02    2.347e-03    1.290e-02
  model.layers.10.mlp.gate_proj                               9      0     1.035e-02    3.578e-03    1.095e-02
  model.layers.10.mlp.up_proj                                 9      0     1.075e-02    3.590e-03    1.134e-02
  model.layers.11.mlp.down_proj                               9      0     1.337e-02    2.508e-03    1.360e-02
  model.layers.11.mlp.gate_proj                               9      0     1.027e-02    3.508e-03    1.085e-02
  model.layers.11.mlp.up_proj                                 9      0     1.087e-02    3.574e-03    1.144e-02
  model.layers.11.self_attn.k_proj                            9      0     1.329e-02    2.697e-03    1.356e-02
  model.layers.11.self_attn.o_proj                            9      0     1.303e-02    2.370e-03    1.324e-02
  model.layers.11.self_attn.q_proj                            9      0     1.173e-02    2.022e-03    1.190e-02
  model.layers.11.self_attn.v_proj                            9      0     1.036e-02    2.280e-03    1.061e-02
  model.layers.12.linear_attn.in_proj_qkv                     9      0     1.292e-02    2.245e-03    1.311e-02
  model.layers.12.linear_attn.in_proj_z                       9      0     1.366e-02    1.891e-03    1.379e-02
  model.layers.12.linear_attn.out_proj                        9      0     1.586e-02    2.138e-03    1.600e-02
  model.layers.12.mlp.down_proj                               9      0     1.316e-02    2.553e-03    1.341e-02
  model.layers.12.mlp.gate_proj                               9      0     1.003e-02    3.475e-03    1.061e-02
  model.layers.12.mlp.up_proj                                 9      0     1.073e-02    3.555e-03    1.130e-02
  model.layers.13.linear_attn.in_proj_qkv                     9      0     1.292e-02    2.241e-03    1.311e-02
  model.layers.13.linear_attn.in_proj_z                       9      0     1.418e-02    1.971e-03    1.432e-02
  model.layers.13.linear_attn.out_proj                        9      0     1.683e-02    2.176e-03    1.697e-02
  model.layers.13.mlp.down_proj                               9      0     1.372e-02    2.566e-03    1.396e-02
  model.layers.13.mlp.gate_proj                               9      0     1.006e-02    3.483e-03    1.065e-02
  model.layers.13.mlp.up_proj                                 9      0     1.071e-02    3.558e-03    1.128e-02
  model.layers.14.linear_attn.in_proj_qkv                     9      0     1.271e-02    2.230e-03    1.290e-02
  model.layers.14.linear_attn.in_proj_z                       9      0     1.336e-02    1.881e-03    1.349e-02
  model.layers.14.linear_attn.out_proj                        9      0     1.586e-02    2.216e-03    1.601e-02
  model.layers.14.mlp.down_proj                               9      0     1.299e-02    2.463e-03    1.322e-02
  model.layers.14.mlp.gate_proj                               9      0     9.956e-03    3.503e-03    1.056e-02
  model.layers.14.mlp.up_proj                                 9      0     1.064e-02    3.566e-03    1.122e-02
  model.layers.15.mlp.down_proj                               9      0     1.350e-02    2.531e-03    1.374e-02
  model.layers.15.mlp.gate_proj                               9      0     1.010e-02    3.487e-03    1.069e-02
  model.layers.15.mlp.up_proj                                 9      0     1.068e-02    3.563e-03    1.126e-02
  model.layers.15.self_attn.k_proj                            9      0     1.453e-02    2.890e-03    1.481e-02
  model.layers.15.self_attn.o_proj                            9      0     1.398e-02    2.119e-03    1.414e-02
  model.layers.15.self_attn.q_proj                            9      0     1.300e-02    2.088e-03    1.317e-02
  model.layers.15.self_attn.v_proj                            9      0     9.817e-03    2.224e-03    1.007e-02
  model.layers.16.linear_attn.in_proj_qkv                     9      0     1.290e-02    2.245e-03    1.309e-02
  model.layers.16.linear_attn.in_proj_z                       9      0     1.355e-02    1.897e-03    1.368e-02
  model.layers.16.linear_attn.out_proj                        9      0     1.636e-02    2.282e-03    1.652e-02
  model.layers.16.mlp.down_proj                               9      0     1.300e-02    2.516e-03    1.324e-02
  model.layers.16.mlp.gate_proj                               9      0     1.005e-02    3.471e-03    1.063e-02
  model.layers.16.mlp.up_proj                                 9      0     1.070e-02    3.563e-03    1.128e-02
  model.layers.17.linear_attn.in_proj_qkv                     9      0     1.288e-02    2.224e-03    1.307e-02
  model.layers.17.linear_attn.in_proj_z                       9      0     1.351e-02    1.896e-03    1.364e-02
  model.layers.17.linear_attn.out_proj                        9      0     1.776e-02    2.244e-03    1.790e-02
  model.layers.17.mlp.down_proj                               9      0     1.380e-02    2.453e-03    1.401e-02
  model.layers.17.mlp.gate_proj                               9      0     1.009e-02    3.470e-03    1.067e-02
  model.layers.17.mlp.up_proj                                 9      0     1.072e-02    3.567e-03    1.129e-02
  model.layers.18.linear_attn.in_proj_qkv                     9      0     1.206e-02    2.187e-03    1.226e-02
  model.layers.18.linear_attn.in_proj_z                       9      0     1.264e-02    1.787e-03    1.276e-02
  model.layers.18.linear_attn.out_proj                        9      0     1.526e-02    2.229e-03    1.542e-02
  model.layers.18.mlp.down_proj                               9      0     1.458e-02    2.426e-03    1.478e-02
  model.layers.18.mlp.gate_proj                               9      0     1.023e-02    3.425e-03    1.079e-02
  model.layers.18.mlp.up_proj                                 9      0     1.059e-02    3.530e-03    1.116e-02
  model.layers.19.mlp.down_proj                               9      0     1.587e-02    2.542e-03    1.608e-02
  model.layers.19.mlp.gate_proj                               9      0     1.007e-02    3.395e-03    1.063e-02
  model.layers.19.mlp.up_proj                                 9      0     1.096e-02    3.578e-03    1.153e-02
  model.layers.19.self_attn.k_proj                            9      0     1.411e-02    2.794e-03    1.438e-02
  model.layers.19.self_attn.o_proj                            9      0     1.813e-02    2.164e-03    1.826e-02
  model.layers.19.self_attn.q_proj                            9      0     9.504e-03    1.929e-03    9.697e-03
  model.layers.19.self_attn.v_proj                            9      0     1.045e-02    2.357e-03    1.071e-02
  model.layers.2.linear_attn.in_proj_qkv                      9      0     1.248e-02    2.213e-03    1.268e-02
  model.layers.2.linear_attn.in_proj_z                        9      0     1.399e-02    1.926e-03    1.412e-02
  model.layers.2.linear_attn.out_proj                         9      0     1.795e-02    2.320e-03    1.810e-02
  model.layers.2.mlp.down_proj                                9      0     1.706e-02    2.375e-03    1.722e-02
  model.layers.2.mlp.gate_proj                                9      0     1.029e-02    3.535e-03    1.088e-02
  model.layers.2.mlp.up_proj                                  9      0     1.062e-02    3.558e-03    1.120e-02
  model.layers.20.linear_attn.in_proj_qkv                     9      0     1.214e-02    2.136e-03    1.233e-02
  model.layers.20.linear_attn.in_proj_z                       9      0     1.271e-02    1.793e-03    1.284e-02
  model.layers.20.linear_attn.out_proj                        9      0     1.578e-02    2.224e-03    1.594e-02
  model.layers.20.mlp.down_proj                               9      0     1.593e-02    2.583e-03    1.614e-02
  model.layers.20.mlp.gate_proj                               9      0     1.008e-02    3.408e-03    1.064e-02
  model.layers.20.mlp.up_proj                                 9      0     1.126e-02    3.603e-03    1.183e-02
  model.layers.21.linear_attn.in_proj_qkv                     9      0     1.199e-02    2.201e-03    1.219e-02
  model.layers.21.linear_attn.in_proj_z                       9      0     1.162e-02    1.672e-03    1.174e-02
  model.layers.21.linear_attn.out_proj                        9      0     1.702e-02    2.231e-03    1.716e-02
  model.layers.21.mlp.down_proj                               9      0     1.721e-02    2.588e-03    1.740e-02
  model.layers.21.mlp.gate_proj                               9      0     1.010e-02    3.421e-03    1.066e-02
  model.layers.21.mlp.up_proj                                 9      0     1.122e-02    3.605e-03    1.178e-02
  model.layers.22.linear_attn.in_proj_qkv                     9      0     1.201e-02    2.241e-03    1.222e-02
  model.layers.22.linear_attn.in_proj_z                       9      0     1.204e-02    1.712e-03    1.216e-02
  model.layers.22.linear_attn.out_proj                        9      0     1.579e-02    2.305e-03    1.595e-02
  model.layers.22.mlp.down_proj                               9      0     1.513e-02    2.552e-03    1.533e-02
  model.layers.22.mlp.gate_proj                               9      0     1.052e-02    3.137e-03    1.098e-02
  model.layers.22.mlp.up_proj                                 9      0     1.142e-02    3.622e-03    1.197e-02
  model.layers.23.mlp.down_proj                               9      0     1.569e-02    2.559e-03    1.589e-02
  model.layers.23.mlp.gate_proj                               9      0     1.009e-02    3.369e-03    1.064e-02
  model.layers.23.mlp.up_proj                                 9      0     1.141e-02    3.621e-03    1.197e-02
  model.layers.23.self_attn.k_proj                            9      0     1.378e-02    2.671e-03    1.404e-02
  model.layers.23.self_attn.o_proj                            9      0     1.682e-02    2.421e-03    1.699e-02
  model.layers.23.self_attn.q_proj                            9      0     9.478e-03    1.963e-03    9.680e-03
  model.layers.23.self_attn.v_proj                            9      0     1.034e-02    2.327e-03    1.060e-02
  model.layers.24.linear_attn.in_proj_qkv                     9      0     1.188e-02    2.253e-03    1.209e-02
  model.layers.24.linear_attn.in_proj_z                       9      0     1.221e-02    1.730e-03    1.234e-02
  model.layers.24.linear_attn.out_proj                        9      0     1.603e-02    2.268e-03    1.619e-02
  model.layers.24.mlp.down_proj                               9      0     1.458e-02    2.617e-03    1.481e-02
  model.layers.24.mlp.gate_proj                               9      0     1.026e-02    3.317e-03    1.078e-02
  model.layers.24.mlp.up_proj                                 9      0     1.144e-02    3.613e-03    1.199e-02
  model.layers.25.linear_attn.in_proj_qkv                     9      0     1.142e-02    2.212e-03    1.163e-02
  model.layers.25.linear_attn.in_proj_z                       9      0     1.279e-02    1.788e-03    1.291e-02
  model.layers.25.linear_attn.out_proj                        9      0     1.473e-02    2.175e-03    1.489e-02
  model.layers.25.mlp.down_proj                               9      0     1.393e-02    2.554e-03    1.416e-02
  model.layers.25.mlp.gate_proj                               9      0     1.054e-02    3.322e-03    1.105e-02
  model.layers.25.mlp.up_proj                                 9      0     1.130e-02    3.600e-03    1.185e-02
  model.layers.26.linear_attn.in_proj_qkv                     9      0     1.149e-02    2.236e-03    1.171e-02
  model.layers.26.linear_attn.in_proj_z                       9      0     1.260e-02    1.796e-03    1.272e-02
  model.layers.26.linear_attn.out_proj                        9      0     1.492e-02    2.343e-03    1.510e-02
  model.layers.26.mlp.down_proj                               9      0     1.270e-02    2.468e-03    1.294e-02
  model.layers.26.mlp.gate_proj                               9      0     1.112e-02    3.297e-03    1.160e-02
  model.layers.26.mlp.up_proj                                 9      0     1.086e-02    3.552e-03    1.143e-02
  model.layers.27.mlp.down_proj                               9      0     1.335e-02    2.575e-03    1.359e-02
  model.layers.27.mlp.gate_proj                               9      0     1.079e-02    3.204e-03    1.126e-02
  model.layers.27.mlp.up_proj                                 9      0     1.090e-02    3.555e-03    1.146e-02
  model.layers.27.self_attn.k_proj                            9      0     1.215e-02    2.498e-03    1.240e-02
  model.layers.27.self_attn.o_proj                            9      0     1.707e-02    2.745e-03    1.729e-02
  model.layers.27.self_attn.q_proj                            9      0     1.020e-02    2.062e-03    1.041e-02
  model.layers.27.self_attn.v_proj                            9      0     9.850e-03    2.252e-03    1.011e-02
  model.layers.28.linear_attn.in_proj_qkv                     9      0     1.177e-02    2.217e-03    1.198e-02
  model.layers.28.linear_attn.in_proj_z                       9      0     1.287e-02    1.817e-03    1.299e-02
  model.layers.28.linear_attn.out_proj                        9      0     1.479e-02    2.297e-03    1.497e-02
  model.layers.28.mlp.down_proj                               9      0     1.301e-02    2.600e-03    1.326e-02
  model.layers.28.mlp.gate_proj                               9      0     1.104e-02    3.175e-03    1.149e-02
  model.layers.28.mlp.up_proj                                 9      0     1.084e-02    3.537e-03    1.140e-02
  model.layers.29.linear_attn.in_proj_qkv                     9      0     1.204e-02    2.246e-03    1.225e-02
  model.layers.29.linear_attn.in_proj_z                       9      0     1.397e-02    1.967e-03    1.411e-02
  model.layers.29.linear_attn.out_proj                        9      0     1.655e-02    2.350e-03    1.671e-02
  model.layers.29.mlp.down_proj                               9      0     1.348e-02    2.623e-03    1.373e-02
  model.layers.29.mlp.gate_proj                               9      0     1.114e-02    3.188e-03    1.159e-02
  model.layers.29.mlp.up_proj                                 9      0     1.081e-02    3.536e-03    1.137e-02
  model.layers.3.mlp.down_proj                                9      0     1.418e-02    2.577e-03    1.441e-02
  model.layers.3.mlp.gate_proj                                9      0     1.018e-02    3.528e-03    1.077e-02
  model.layers.3.mlp.up_proj                                  9      0     1.101e-02    3.601e-03    1.158e-02
  model.layers.3.self_attn.k_proj                             9      0     1.249e-02    2.587e-03    1.275e-02
  model.layers.3.self_attn.o_proj                             9      0     1.494e-02    2.175e-03    1.510e-02
  model.layers.3.self_attn.q_proj                             9      0     1.169e-02    2.033e-03    1.186e-02
  model.layers.3.self_attn.v_proj                             9      0     1.263e-02    2.311e-03    1.284e-02
  model.layers.30.linear_attn.in_proj_qkv                     9      0     1.189e-02    2.230e-03    1.210e-02
  model.layers.30.linear_attn.in_proj_z                       9      0     1.349e-02    1.886e-03    1.362e-02
  model.layers.30.linear_attn.out_proj                        9      0     1.622e-02    2.412e-03    1.640e-02
  model.layers.30.mlp.down_proj                               9      0     1.285e-02    2.561e-03    1.310e-02
  model.layers.30.mlp.gate_proj                               9      0     1.171e-02    3.188e-03    1.214e-02
  model.layers.30.mlp.up_proj                                 9      0     1.067e-02    3.521e-03    1.123e-02
  model.layers.31.mlp.down_proj                               9      0     1.320e-02    2.596e-03    1.345e-02
  model.layers.31.mlp.gate_proj                               9      0     1.148e-02    3.154e-03    1.191e-02
  model.layers.31.mlp.up_proj                                 9      0     1.090e-02    3.518e-03    1.146e-02
  model.layers.31.self_attn.k_proj                            9      0     1.207e-02    2.519e-03    1.233e-02
  model.layers.31.self_attn.o_proj                            9      0     2.214e-02    3.029e-03    2.235e-02
  model.layers.31.self_attn.q_proj                            9      0     1.015e-02    2.074e-03    1.036e-02
  model.layers.31.self_attn.v_proj                            9      0     9.912e-03    2.191e-03    1.014e-02
  model.layers.32.linear_attn.in_proj_qkv                     9      0     1.282e-02    2.287e-03    1.303e-02
  model.layers.32.linear_attn.in_proj_z                       9      0     1.411e-02    1.985e-03    1.425e-02
  model.layers.32.linear_attn.out_proj                        9      0     1.567e-02    2.303e-03    1.585e-02
  model.layers.32.mlp.down_proj                               9      0     1.301e-02    2.618e-03    1.327e-02
  model.layers.32.mlp.gate_proj                               9      0     1.165e-02    3.107e-03    1.206e-02
  model.layers.32.mlp.up_proj                                 9      0     1.108e-02    3.522e-03    1.162e-02
  model.layers.33.linear_attn.in_proj_qkv                     9      0     1.259e-02    2.298e-03    1.280e-02
  model.layers.33.linear_attn.in_proj_z                       9      0     1.386e-02    1.988e-03    1.400e-02
  model.layers.33.linear_attn.out_proj                        9      0     1.745e-02    2.367e-03    1.761e-02
  model.layers.33.mlp.down_proj                               9      0     1.342e-02    2.602e-03    1.367e-02
  model.layers.33.mlp.gate_proj                               9      0     1.193e-02    3.114e-03    1.232e-02
  model.layers.33.mlp.up_proj                                 9      0     1.124e-02    3.542e-03    1.178e-02
  model.layers.34.linear_attn.in_proj_qkv                     9      0     1.252e-02    2.278e-03    1.273e-02
  model.layers.34.linear_attn.in_proj_z                       9      0     1.343e-02    1.914e-03    1.357e-02
  model.layers.34.linear_attn.out_proj                        9      0     1.609e-02    2.363e-03    1.626e-02
  model.layers.34.mlp.down_proj                               9      0     1.377e-02    2.610e-03    1.401e-02
  model.layers.34.mlp.gate_proj                               9      0     1.207e-02    3.070e-03    1.245e-02
  model.layers.34.mlp.up_proj                                 9      0     1.115e-02    3.505e-03    1.169e-02
  model.layers.35.mlp.down_proj                               9      0     1.414e-02    2.562e-03    1.437e-02
  model.layers.35.mlp.gate_proj                               9      0     1.066e-02    3.068e-03    1.109e-02
  model.layers.35.mlp.up_proj                                 9      0     1.138e-02    3.583e-03    1.193e-02
  model.layers.35.self_attn.k_proj                            9      0     1.177e-02    2.500e-03    1.203e-02
  model.layers.35.self_attn.o_proj                            9      0     1.997e-02    2.809e-03    2.017e-02
  model.layers.35.self_attn.q_proj                            9      0     1.072e-02    2.117e-03    1.092e-02
  model.layers.35.self_attn.v_proj                            9      0     1.071e-02    2.262e-03    1.094e-02
  model.layers.36.linear_attn.in_proj_qkv                     9      0     1.295e-02    2.276e-03    1.314e-02
  model.layers.36.linear_attn.in_proj_z                       9      0     1.212e-02    1.746e-03    1.224e-02
  model.layers.36.linear_attn.out_proj                        9      0     1.667e-02    2.261e-03    1.682e-02
  model.layers.36.mlp.down_proj                               9      0     1.511e-02    2.627e-03    1.534e-02
  model.layers.36.mlp.gate_proj                               9      0     1.026e-02    3.145e-03    1.074e-02
  model.layers.36.mlp.up_proj                                 9      0     1.154e-02    3.603e-03    1.209e-02
  model.layers.37.linear_attn.in_proj_qkv                     9      0     1.223e-02    2.242e-03    1.243e-02
  model.layers.37.linear_attn.in_proj_z                       9      0     1.058e-02    1.597e-03    1.071e-02
  model.layers.37.linear_attn.out_proj                        9      0     1.838e-02    2.199e-03    1.851e-02
  model.layers.37.mlp.down_proj                               9      0     1.541e-02    2.734e-03    1.565e-02
  model.layers.37.mlp.gate_proj                               9      0     1.055e-02    3.235e-03    1.104e-02
  model.layers.37.mlp.up_proj                                 9      0     1.172e-02    3.628e-03    1.227e-02
  model.layers.38.linear_attn.in_proj_qkv                     9      0     1.201e-02    2.319e-03    1.223e-02
  model.layers.38.linear_attn.in_proj_z                       9      0     1.100e-02    1.630e-03    1.112e-02
  model.layers.38.linear_attn.out_proj                        9      0     1.656e-02    2.310e-03    1.672e-02
  model.layers.38.mlp.down_proj                               9      0     1.595e-02    2.700e-03    1.617e-02
  model.layers.38.mlp.gate_proj                               9      0     1.069e-02    3.065e-03    1.112e-02
  model.layers.38.mlp.up_proj                                 9      0     1.171e-02    3.642e-03    1.226e-02
  model.layers.39.mlp.down_proj                               9      0     1.441e-02    2.655e-03    1.465e-02
  model.layers.39.mlp.gate_proj                               9      0     9.965e-03    3.158e-03    1.045e-02
  model.layers.39.mlp.up_proj                                 9      0     1.180e-02    3.648e-03    1.235e-02
  model.layers.39.self_attn.k_proj                            9      0     1.179e-02    2.470e-03    1.205e-02
  model.layers.39.self_attn.o_proj                            9      0     2.077e-02    2.841e-03    2.097e-02
  model.layers.39.self_attn.q_proj                            9      0     1.045e-02    2.093e-03    1.066e-02
  model.layers.39.self_attn.v_proj                            9      0     1.090e-02    2.329e-03    1.114e-02
  model.layers.4.linear_attn.in_proj_qkv                      9      0     1.192e-02    2.310e-03    1.214e-02
  model.layers.4.linear_attn.in_proj_z                        9      0     1.247e-02    1.774e-03    1.260e-02
  model.layers.4.linear_attn.out_proj                         9      0     1.788e-02    2.410e-03    1.804e-02
  model.layers.4.mlp.down_proj                                9      0     1.444e-02    2.610e-03    1.468e-02
  model.layers.4.mlp.gate_proj                                9      0     1.015e-02    3.511e-03    1.074e-02
  model.layers.4.mlp.up_proj                                  9      0     1.123e-02    3.616e-03    1.180e-02
  model.layers.40.linear_attn.in_proj_qkv                     9      0     1.226e-02    2.281e-03    1.247e-02
  model.layers.40.linear_attn.in_proj_z                       9      0     1.172e-02    1.686e-03    1.185e-02
  model.layers.40.linear_attn.out_proj                        9      0     1.611e-02    2.266e-03    1.626e-02
  model.layers.40.mlp.down_proj                               9      0     1.485e-02    2.675e-03    1.508e-02
  model.layers.40.mlp.gate_proj                               9      0     1.008e-02    3.132e-03    1.056e-02
  model.layers.40.mlp.up_proj                                 9      0     1.164e-02    3.641e-03    1.220e-02
  model.layers.41.linear_attn.in_proj_qkv                     9      0     1.175e-02    2.257e-03    1.197e-02
  model.layers.41.linear_attn.in_proj_z                       9      0     1.225e-02    1.732e-03    1.238e-02
  model.layers.41.linear_attn.out_proj                        9      0     1.646e-02    2.257e-03    1.662e-02
  model.layers.41.mlp.down_proj                               9      0     1.475e-02    2.702e-03    1.500e-02
  model.layers.41.mlp.gate_proj                               9      0     1.085e-02    3.212e-03    1.131e-02
  model.layers.41.mlp.up_proj                                 9      0     1.166e-02    3.633e-03    1.222e-02
  model.layers.42.linear_attn.in_proj_qkv                     9      0     1.163e-02    2.277e-03    1.185e-02
  model.layers.42.linear_attn.in_proj_z                       9      0     1.240e-02    1.748e-03    1.253e-02
  model.layers.42.linear_attn.out_proj                        9      0     1.574e-02    2.284e-03    1.591e-02
  model.layers.42.mlp.down_proj                               9      0     1.476e-02    2.644e-03    1.499e-02
  model.layers.42.mlp.gate_proj                               9      0     1.173e-02    3.240e-03    1.217e-02
  model.layers.42.mlp.up_proj                                 9      0     1.143e-02    3.584e-03    1.198e-02
  model.layers.43.mlp.down_proj                               9      0     1.381e-02    2.670e-03    1.407e-02
  model.layers.43.mlp.gate_proj                               9      0     1.085e-02    3.218e-03    1.132e-02
  model.layers.43.mlp.up_proj                                 9      0     1.154e-02    3.618e-03    1.210e-02
  model.layers.43.self_attn.k_proj                            9      0     1.253e-02    2.452e-03    1.276e-02
  model.layers.43.self_attn.o_proj                            9      0     2.163e-02    3.125e-03    2.186e-02
  model.layers.43.self_attn.q_proj                            9      0     1.062e-02    2.091e-03    1.082e-02
  model.layers.43.self_attn.v_proj                            9      0     1.054e-02    2.252e-03    1.078e-02
  model.layers.44.linear_attn.in_proj_qkv                     9      0     1.206e-02    2.230e-03    1.226e-02
  model.layers.44.linear_attn.in_proj_z                       9      0     1.228e-02    1.698e-03    1.240e-02
  model.layers.44.linear_attn.out_proj                        9      0     1.628e-02    2.298e-03    1.644e-02
  model.layers.44.mlp.down_proj                               9      0     1.414e-02    2.712e-03    1.440e-02
  model.layers.44.mlp.gate_proj                               9      0     1.083e-02    3.259e-03    1.131e-02
  model.layers.44.mlp.up_proj                                 9      0     1.171e-02    3.618e-03    1.225e-02
  model.layers.45.linear_attn.in_proj_qkv                     9      0     1.209e-02    2.277e-03    1.230e-02
  model.layers.45.linear_attn.in_proj_z                       9      0     1.265e-02    1.759e-03    1.277e-02
  model.layers.45.linear_attn.out_proj                        9      0     1.814e-02    2.430e-03    1.830e-02
  model.layers.45.mlp.down_proj                               9      0     1.437e-02    2.740e-03    1.463e-02
  model.layers.45.mlp.gate_proj                               9      0     1.082e-02    3.289e-03    1.131e-02
  model.layers.45.mlp.up_proj                                 9      0     1.157e-02    3.623e-03    1.212e-02
  model.layers.46.linear_attn.in_proj_qkv                     9      0     1.198e-02    2.245e-03    1.219e-02
  model.layers.46.linear_attn.in_proj_z                       9      0     1.186e-02    1.650e-03    1.197e-02
  model.layers.46.linear_attn.out_proj                        9      0     1.868e-02    2.358e-03    1.883e-02
  model.layers.46.mlp.down_proj                               9      0     1.426e-02    2.688e-03    1.451e-02
  model.layers.46.mlp.gate_proj                               9      0     1.112e-02    3.310e-03    1.161e-02
  model.layers.46.mlp.up_proj                                 9      0     1.151e-02    3.600e-03    1.206e-02
  model.layers.47.mlp.down_proj                               9      0     1.426e-02    2.639e-03    1.450e-02
  model.layers.47.mlp.gate_proj                               9      0     1.057e-02    3.335e-03    1.108e-02
  model.layers.47.mlp.up_proj                                 9      0     1.177e-02    3.621e-03    1.232e-02
  model.layers.47.self_attn.k_proj                            9      0     1.248e-02    2.462e-03    1.272e-02
  model.layers.47.self_attn.o_proj                            9      0     1.954e-02    2.555e-03    1.971e-02
  model.layers.47.self_attn.q_proj                            9      0     9.680e-03    2.056e-03    9.896e-03
  model.layers.47.self_attn.v_proj                            9      0     1.068e-02    2.234e-03    1.091e-02
  model.layers.48.linear_attn.in_proj_qkv                     9      0     1.274e-02    2.267e-03    1.294e-02
  model.layers.48.linear_attn.in_proj_z                       9      0     1.210e-02    1.652e-03    1.221e-02
  model.layers.48.linear_attn.out_proj                        9      0     1.720e-02    2.416e-03    1.737e-02
  model.layers.48.mlp.down_proj                               9      0     1.479e-02    2.684e-03    1.503e-02
  model.layers.48.mlp.gate_proj                               9      0     1.036e-02    3.332e-03    1.088e-02
  model.layers.48.mlp.up_proj                                 9      0     1.170e-02    3.606e-03    1.224e-02
  model.layers.49.linear_attn.in_proj_qkv                     9      0     1.211e-02    2.277e-03    1.232e-02
  model.layers.49.linear_attn.in_proj_z                       9      0     1.156e-02    1.608e-03    1.167e-02
  model.layers.49.linear_attn.out_proj                        9      0     1.910e-02    2.175e-03    1.923e-02
  model.layers.49.mlp.down_proj                               9      0     1.567e-02    2.697e-03    1.590e-02
  model.layers.49.mlp.gate_proj                               9      0     1.007e-02    3.367e-03    1.062e-02
  model.layers.49.mlp.up_proj                                 9      0     1.150e-02    3.563e-03    1.204e-02
  model.layers.5.linear_attn.in_proj_qkv                      9      0     1.245e-02    2.270e-03    1.266e-02
  model.layers.5.linear_attn.in_proj_z                        9      0     1.391e-02    1.910e-03    1.404e-02
  model.layers.5.linear_attn.out_proj                         9      0     1.772e-02    2.209e-03    1.785e-02
  model.layers.5.mlp.down_proj                                9      0     1.529e-02    2.519e-03    1.550e-02
  model.layers.5.mlp.gate_proj                                9      0     1.020e-02    3.534e-03    1.079e-02
  model.layers.5.mlp.up_proj                                  9      0     1.128e-02    3.622e-03    1.185e-02
  model.layers.50.linear_attn.in_proj_qkv                     9      0     1.169e-02    2.255e-03    1.190e-02
  model.layers.50.linear_attn.in_proj_z                       9      0     1.090e-02    1.493e-03    1.100e-02
  model.layers.50.linear_attn.out_proj                        9      0     1.824e-02    2.348e-03    1.839e-02
  model.layers.50.mlp.down_proj                               9      0     1.517e-02    2.666e-03    1.541e-02
  model.layers.50.mlp.gate_proj                               9      0     9.861e-03    3.280e-03    1.039e-02
  model.layers.50.mlp.up_proj                                 9      0     1.087e-02    3.439e-03    1.140e-02
  model.layers.51.mlp.down_proj                               9      0     1.524e-02    2.656e-03    1.548e-02
  model.layers.51.mlp.gate_proj                               9      0     8.699e-03    3.392e-03    9.339e-03
  model.layers.51.mlp.up_proj                                 9      0     1.111e-02    3.552e-03    1.166e-02
  model.layers.51.self_attn.k_proj                            9      0     1.103e-02    2.402e-03    1.129e-02
  model.layers.51.self_attn.o_proj                            9      0     1.919e-02    2.299e-03    1.933e-02
  model.layers.51.self_attn.q_proj                            9      0     9.135e-03    2.024e-03    9.356e-03
  model.layers.51.self_attn.v_proj                            9      0     1.024e-02    2.274e-03    1.049e-02
  model.layers.52.linear_attn.in_proj_qkv                     9      0     9.642e-03    2.110e-03    9.868e-03
  model.layers.52.linear_attn.in_proj_z                       9      0     8.558e-03    1.286e-03    8.653e-03
  model.layers.52.linear_attn.out_proj                        9      0     1.877e-02    2.303e-03    1.891e-02
  model.layers.52.mlp.down_proj                               9      0     1.784e-02    2.685e-03    1.804e-02
  model.layers.52.mlp.gate_proj                               9      0     8.139e-03    3.398e-03    8.821e-03
  model.layers.52.mlp.up_proj                                 9      0     1.105e-02    3.564e-03    1.161e-02
  model.layers.53.linear_attn.in_proj_qkv                     9      0     9.505e-03    2.128e-03    9.740e-03
  model.layers.53.linear_attn.in_proj_z                       9      0     8.688e-03    1.313e-03    8.786e-03
  model.layers.53.linear_attn.out_proj                        9      0     1.699e-02    2.275e-03    1.714e-02
  model.layers.53.mlp.down_proj                               9      0     1.846e-02    2.768e-03    1.867e-02
  model.layers.53.mlp.gate_proj                               9      0     7.902e-03    3.398e-03    8.601e-03
  model.layers.53.mlp.up_proj                                 9      0     1.096e-02    3.568e-03    1.153e-02
  model.layers.54.linear_attn.in_proj_qkv                     9      0     9.920e-03    2.231e-03    1.017e-02
  model.layers.54.linear_attn.in_proj_z                       9      0     8.309e-03    1.278e-03    8.406e-03
  model.layers.54.linear_attn.out_proj                        9      0     1.749e-02    2.127e-03    1.762e-02
  model.layers.54.mlp.down_proj                               9      0     1.815e-02    2.477e-03    1.832e-02
  model.layers.54.mlp.gate_proj                               9      0     7.284e-03    3.349e-03    8.016e-03
  model.layers.54.mlp.up_proj                                 9      0     9.161e-03    3.418e-03    9.777e-03
  model.layers.55.mlp.down_proj                               9      0     1.696e-02    2.620e-03    1.716e-02
  model.layers.55.mlp.gate_proj                               9      0     7.234e-03    3.384e-03    7.986e-03
  model.layers.55.mlp.up_proj                                 9      0     1.132e-02    3.603e-03    1.188e-02
  model.layers.55.self_attn.k_proj                            9      0     1.085e-02    2.400e-03    1.112e-02
  model.layers.55.self_attn.o_proj                            9      0     1.731e-02    2.199e-03    1.745e-02
  model.layers.55.self_attn.q_proj                            9      0     8.915e-03    2.106e-03    9.160e-03
  model.layers.55.self_attn.v_proj                            9      0     9.482e-03    2.219e-03    9.739e-03
  model.layers.56.linear_attn.in_proj_qkv                     9      0     9.010e-03    2.090e-03    9.249e-03
  model.layers.56.linear_attn.in_proj_z                       9      0     8.724e-03    1.331e-03    8.825e-03
  model.layers.56.linear_attn.out_proj                        9      0     1.741e-02    2.098e-03    1.754e-02
  model.layers.56.mlp.down_proj                               9      0     1.968e-02    2.530e-03    1.984e-02
  model.layers.56.mlp.gate_proj                               9      0     7.328e-03    3.347e-03    8.057e-03
  model.layers.56.mlp.up_proj                                 9      0     1.133e-02    3.595e-03    1.188e-02
  model.layers.57.linear_attn.in_proj_qkv                     9      0     8.646e-03    2.053e-03    8.886e-03
  model.layers.57.linear_attn.in_proj_z                       9      0     8.532e-03    1.300e-03    8.631e-03
  model.layers.57.linear_attn.out_proj                        9      0     1.595e-02    2.510e-03    1.615e-02
  model.layers.57.mlp.down_proj                               9      0     1.634e-02    2.796e-03    1.657e-02
  model.layers.57.mlp.gate_proj                               9      0     7.144e-03    3.321e-03    7.878e-03
  model.layers.57.mlp.up_proj                                 9      0     1.116e-02    3.600e-03    1.173e-02
  model.layers.58.linear_attn.in_proj_qkv                     9      0     9.203e-03    2.131e-03    9.447e-03
  model.layers.58.linear_attn.in_proj_z                       9      0     8.789e-03    1.313e-03    8.885e-03
  model.layers.58.linear_attn.out_proj                        9      0     1.895e-02    2.411e-03    1.910e-02
  model.layers.58.mlp.down_proj                               9      0     1.666e-02    2.607e-03    1.686e-02
  model.layers.58.mlp.gate_proj                               9      0     7.020e-03    3.313e-03    7.763e-03
  model.layers.58.mlp.up_proj                                 9      0     1.044e-02    3.547e-03    1.102e-02
  model.layers.59.mlp.down_proj                               9      0     1.667e-02    2.502e-03    1.685e-02
  model.layers.59.mlp.gate_proj                               9      0     6.604e-03    3.306e-03    7.388e-03
  model.layers.59.mlp.up_proj                                 9      0     9.269e-03    3.450e-03    9.890e-03
  model.layers.59.self_attn.k_proj                            9      0     1.016e-02    2.336e-03    1.043e-02
  model.layers.59.self_attn.o_proj                            9      0     1.802e-02    2.288e-03    1.816e-02
  model.layers.59.self_attn.q_proj                            9      0     8.663e-03    2.118e-03    8.920e-03
  model.layers.59.self_attn.v_proj                            9      0     9.077e-03    2.160e-03    9.329e-03
  model.layers.6.linear_attn.in_proj_qkv                      9      0     1.263e-02    2.269e-03    1.283e-02
  model.layers.6.linear_attn.in_proj_z                        9      0     1.427e-02    1.989e-03    1.441e-02
  model.layers.6.linear_attn.out_proj                         9      0     1.682e-02    2.292e-03    1.697e-02
  model.layers.6.mlp.down_proj                                9      0     1.358e-02    2.400e-03    1.379e-02
  model.layers.6.mlp.gate_proj                                9      0     1.045e-02    3.493e-03    1.102e-02
  model.layers.6.mlp.up_proj                                  9      0     1.113e-02    3.563e-03    1.168e-02
  model.layers.60.linear_attn.in_proj_qkv                     9      0     7.947e-03    1.973e-03    8.189e-03
  model.layers.60.linear_attn.in_proj_z                       9      0     7.538e-03    1.195e-03    7.633e-03
  model.layers.60.linear_attn.out_proj                        9      0     1.696e-02    2.504e-03    1.714e-02
  model.layers.60.mlp.down_proj                               9      0     1.583e-02    2.704e-03    1.606e-02
  model.layers.60.mlp.gate_proj                               9      0     6.834e-03    3.243e-03    7.564e-03
  model.layers.60.mlp.up_proj                                 9      0     9.170e-03    3.388e-03    9.776e-03
  model.layers.61.linear_attn.in_proj_qkv                     9      0     8.878e-03    2.088e-03    9.120e-03
  model.layers.61.linear_attn.in_proj_z                       9      0     7.963e-03    1.246e-03    8.060e-03
  model.layers.61.linear_attn.out_proj                        9      0     1.669e-02    2.348e-03    1.686e-02
  model.layers.61.mlp.down_proj                               9      0     1.498e-02    2.664e-03    1.521e-02
  model.layers.61.mlp.gate_proj                               9      0     6.562e-03    3.195e-03    7.300e-03
  model.layers.61.mlp.up_proj                                 9      0     8.694e-03    3.315e-03    9.304e-03
  model.layers.62.linear_attn.in_proj_qkv                     9      0     8.149e-03    1.974e-03    8.384e-03
  model.layers.62.linear_attn.in_proj_z                       9      0     7.410e-03    1.221e-03    7.508e-03
  model.layers.62.linear_attn.out_proj                        9      0     1.516e-02    2.354e-03    1.535e-02
  model.layers.62.mlp.down_proj                               9      0     1.496e-02    2.533e-03    1.518e-02
  model.layers.62.mlp.gate_proj                               9      0     6.266e-03    3.134e-03    7.006e-03
  model.layers.62.mlp.up_proj                                 9      0     6.854e-03    3.145e-03    7.542e-03
  model.layers.63.mlp.down_proj                               9      0     1.191e-02    2.011e-03    1.208e-02
  model.layers.63.mlp.gate_proj                               9      0     5.660e-03    3.068e-03    6.439e-03
  model.layers.63.mlp.up_proj                                 9      0     4.709e-03    3.050e-03    5.610e-03
  model.layers.63.self_attn.k_proj                            9      0     1.072e-02    2.414e-03    1.098e-02
  model.layers.63.self_attn.o_proj                            9      0     1.293e-02    2.012e-03    1.308e-02
  model.layers.63.self_attn.q_proj                            9      0     7.013e-03    1.939e-03    7.277e-03
  model.layers.63.self_attn.v_proj                            9      0     4.703e-03    1.727e-03    5.007e-03
  model.layers.7.mlp.down_proj                                9      0     1.425e-02    2.578e-03    1.448e-02
  model.layers.7.mlp.gate_proj                                9      0     1.041e-02    3.566e-03    1.100e-02
  model.layers.7.mlp.up_proj                                  9      0     1.135e-02    3.626e-03    1.191e-02
  model.layers.7.self_attn.k_proj                             9      0     1.383e-02    2.714e-03    1.410e-02
  model.layers.7.self_attn.o_proj                             9      0     1.329e-02    2.267e-03    1.347e-02
  model.layers.7.self_attn.q_proj                             9      0     1.250e-02    2.079e-03    1.267e-02
  model.layers.7.self_attn.v_proj                             9      0     1.102e-02    2.332e-03    1.126e-02
  model.layers.8.linear_attn.in_proj_qkv                      9      0     1.243e-02    2.248e-03    1.263e-02
  model.layers.8.linear_attn.in_proj_z                        9      0     1.308e-02    1.811e-03    1.320e-02
  model.layers.8.linear_attn.out_proj                         9      0     1.617e-02    2.281e-03    1.632e-02
  model.layers.8.mlp.down_proj                                9      0     1.507e-02    2.557e-03    1.528e-02
  model.layers.8.mlp.gate_proj                                9      0     1.051e-02    3.563e-03    1.109e-02
  model.layers.8.mlp.up_proj                                  9      0     1.129e-02    3.619e-03    1.186e-02
  model.layers.9.linear_attn.in_proj_qkv                      9      0     1.309e-02    2.233e-03    1.328e-02
  model.layers.9.linear_attn.in_proj_z                        9      0     1.361e-02    1.895e-03    1.374e-02
  model.layers.9.linear_attn.out_proj                         9      0     1.602e-02    2.267e-03    1.618e-02
  model.layers.9.mlp.down_proj                                9      0     1.345e-02    2.475e-03    1.368e-02
  model.layers.9.mlp.gate_proj                                9      0     1.058e-02    3.577e-03    1.117e-02
  model.layers.9.mlp.up_proj                                  9      0     1.119e-02    3.610e-03    1.176e-02
[i8] per-arm census (int8 kernel calls): {'REF': 0, 'REF2': 0, 'STD': 0, 'SIMROW': 0, 'INT8': 3200}
[i8] STD control: fused-reconstruct override consulted 3200 times

ARM RECEIPTS -- what each arm actually executed, from the engine's
own per-call census.  fused/standalone is the reconstruct variant;
acc16/acc32 is the cuBLAS compute type leg B asked hgemm for.
  arm       legB calls          rows     fused  standalone     acc16     acc32   fp32pin
  REF             3200       6553600      3200           0      3200         0         0
  REF2            3200       6553600      3200           0      3200         0         0
  STD             3200       6553600         0        3200      3200         0         0
  SIMROW          3200       6553600         0        3200      3200         0         0
  INT8            3200       6553600         0        3200      3200         0         0
  all arms match their expected leg/accumulator signature
  prefill positions aligned: [8] per prompt, identical across all 5 arms
[i8] null control: REF2 vs REF max KL is exactly 0

THE TAIL over PREFILL positions only -- one per prefill chunk, so
every arm scores the SAME context. Decode positions are reported
separately and are NOT a numerics measure: once a token flips, the
arms are reading different contexts. The control proves it -- pooled
over decode, STD (which contains no int8 at all) has the same max KL
as INT8, so a pooled tail cannot attribute anything to the kernel.

arm            n        mean         p50         p95         p99         MAX     argmax flip
REF            8   0.000e+00   0.000e+00   0.000e+00   0.000e+00   0.000e+00 0/8 =   0.00%
REF2           8   0.000e+00   0.000e+00   0.000e+00   0.000e+00   0.000e+00 0/8 =   0.00%
STD            8   3.863e-03   5.652e-04   2.786e-02   2.786e-02   2.786e-02 0/8 =   0.00%
SIMROW         8   8.770e-03   4.692e-03   5.223e-02   5.223e-02   5.223e-02 0/8 =   0.00%
INT8           8   1.120e+00   2.387e-03   8.940e+00   8.940e+00   8.940e+00 1/8 =  12.50%

Tail attribution -- the 10 worst INT8 prefill positions, with the
control's KL at the SAME position. A tail that is positional shows
the control elevated too; a tail that is the kernel's does not.
   prompt  chunk      INT8 KL       STD KL     SIMROW KL  ratio INT8/STD
        0      6    8.940e+00    2.786e-02     5.223e-02          320.88
        0      7    1.156e-02    8.139e-04     4.692e-03           14.21
        0      3    2.825e-03    1.018e-03     7.363e-03            2.78
        0      4    2.387e-03    5.652e-04     5.537e-03            4.22
        0      1    1.031e-03    5.318e-04     2.382e-04            1.94
        0      2    9.914e-04    5.195e-05     4.302e-05           19.09
        0      0    7.360e-05    5.943e-05     6.153e-05            1.24
        0      5    4.626e-07    1.371e-06     1.420e-06            0.34
  Spearman rank correlation STD vs INT8 across all 8 prefill positions: 0.952

Decode positions (divergence-contaminated, shown for completeness):
arm            n        mean         MAX    greedy agree
REF            8   0.000e+00   0.000e+00 8/8 =  100.0%
REF2           8   0.000e+00   0.000e+00 8/8 =  100.0%
STD            8   8.944e-04   2.984e-03 8/8 =  100.0%
SIMROW         8   2.461e-03   7.529e-03 8/8 =  100.0%
INT8           8   3.574e+00   2.855e+01 7/8 =   87.5%

SAMPLER-RELEVANT TAIL @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF            8     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/  8     0.00%
REF2           8     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/  8     0.00%
STD            8      94.43%  1.256e-02  3.560e-02  3.560e-02          inf 4/  8     1.56%
SIMROW         8      90.03%  2.779e-02  8.610e-02  8.610e-02          inf 3/  8     1.56%
INT8           8      88.78%  1.317e-01  9.229e-01  9.229e-01          inf 3/  8    20.31%

SAMPLER-RELEVANT TAIL @ owner's k~40 (T=1.0, top_p=1.0, top_k=40), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF            8     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/  8     0.00%
REF2           8     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/  8     0.00%
STD            8      97.87%  9.587e-03  3.820e-02  3.820e-02          inf 5/  8     6.25%
SIMROW         8      96.94%  2.595e-02  7.853e-02  7.853e-02          inf 6/  8     6.25%
INT8           8      91.15%  1.288e-01  9.047e-01  9.047e-01          inf 8/  8    23.44%

FULL-VOCAB numbers below, kept and labelled: they are the right
instrument for GREEDY or beam decode, where every rank matters.

arm         first-prefill-position KL       median          max
REF                         0.000e+00    0.000e+00    0.000e+00
REF2                        0.000e+00    0.000e+00    0.000e+00
STD                         5.943e-05    5.943e-05    5.943e-05
SIMROW                      6.153e-05    6.153e-05    6.153e-05
INT8                        7.360e-05    7.360e-05    7.360e-05

saved layer trace for prompt 16 to /scratch/trace16.pt

LAYER TRACE, prompt 16 -- rel rms against REF of the residual
stream leaving each decoder layer, at the token the KL is scored
at.  A STEP at one layer is a defect with an address; smooth
multiplicative growth with no step is the model amplifying.
  per-chunk full-vocab KL vs REF on this prompt:
  chunk           REF        REF2         STD      SIMROW        INT8
      0     0.000e+00   0.000e+00   5.943e-05   6.153e-05   7.360e-05
      1     0.000e+00   0.000e+00   5.318e-04   2.382e-04   1.031e-03
      2     0.000e+00   0.000e+00   5.195e-05   4.302e-05   9.914e-04
      3     0.000e+00   0.000e+00   1.018e-03   7.363e-03   2.825e-03
      4     0.000e+00   0.000e+00   5.652e-04   5.537e-03   2.387e-03
      5     0.000e+00   0.000e+00   1.371e-06   1.420e-06   4.626e-07
      6     0.000e+00   0.000e+00   2.786e-02   5.223e-02   8.940e+00
      7     0.000e+00   0.000e+00   8.139e-04   4.692e-03   1.156e-02
  worst chunk for INT8: c6

  chunk 0: rel rms after layer L, vs REF
  layer        REF2         STD      SIMROW        INT8     ratio
      0   0.000e+00   8.725e-03   2.074e-02   2.337e-02       nan  linear_attn
      1   0.000e+00   1.550e-02   2.324e-02   2.548e-02       nan  linear_attn
      2   0.000e+00   2.111e-02   2.279e-02   1.879e-02       nan  linear_attn
      3   0.000e+00   1.177e-02   2.961e-02   3.201e-02       nan  self_attn
      4   0.000e+00   1.948e-02   3.022e-02   3.459e-02       nan  linear_attn
      5   0.000e+00   1.927e-02   3.104e-02   4.179e-02       nan  linear_attn
      6   0.000e+00   1.124e-02   2.680e-02   2.996e-02       nan  linear_attn
      7   0.000e+00   1.471e-02   2.984e-02   3.399e-02       nan  self_attn
      8   0.000e+00   1.191e-02   2.831e-02   3.565e-02       nan  linear_attn
      9   0.000e+00   8.817e-03   2.618e-02   3.329e-02       nan  linear_attn
     10   0.000e+00   1.240e-02   2.758e-02   3.644e-02       nan  linear_attn
     11   0.000e+00   1.244e-02   2.696e-02   3.360e-02       nan  self_attn
     12   0.000e+00   1.389e-02   2.860e-02   3.648e-02       nan  linear_attn
     13   0.000e+00   1.545e-02   3.175e-02   3.872e-02       nan  linear_attn
     14   0.000e+00   1.429e-02   2.937e-02   3.515e-02       nan  linear_attn
     15   0.000e+00   1.653e-02   3.695e-02   4.196e-02       nan  self_attn
     16   0.000e+00   1.591e-02   3.397e-02   3.860e-02       nan  linear_attn
     17   0.000e+00   1.285e-02   2.895e-02   3.370e-02       nan  linear_attn
     18   0.000e+00   1.021e-02   2.022e-02   2.337e-02       nan  linear_attn
     19   0.000e+00   1.169e-02   2.668e-02   3.162e-02       nan  self_attn
     20   0.000e+00   1.146e-02   2.786e-02   3.235e-02       nan  linear_attn
     21   0.000e+00   1.313e-02   2.907e-02   3.292e-02       nan  linear_attn
     22   0.000e+00   1.245e-02   3.008e-02   3.104e-02       nan  linear_attn
     23   0.000e+00   1.340e-02   3.316e-02   3.699e-02       nan  self_attn
     24   0.000e+00   1.345e-02   3.518e-02   3.707e-02       nan  linear_attn
     25   0.000e+00   1.422e-02   3.119e-02   3.778e-02       nan  linear_attn
     26   0.000e+00   1.261e-02   2.874e-02   3.076e-02       nan  linear_attn
     27   0.000e+00   1.532e-02   3.225e-02   3.573e-02       nan  self_attn
     28   0.000e+00   1.728e-02   3.528e-02   4.105e-02       nan  linear_attn
     29   0.000e+00   1.550e-02   3.454e-02   3.718e-02       nan  linear_attn
     30   0.000e+00   1.527e-02   3.426e-02   3.807e-02       nan  linear_attn
     31   0.000e+00   1.692e-02   3.652e-02   4.158e-02       nan  self_attn
     32   0.000e+00   2.046e-02   3.899e-02   4.314e-02       nan  linear_attn
     33   0.000e+00   2.099e-02   4.155e-02   4.528e-02       nan  linear_attn
     34   0.000e+00   2.034e-02   4.037e-02   4.299e-02       nan  linear_attn
     35   0.000e+00   1.979e-02   3.741e-02   4.102e-02       nan  self_attn
     36   0.000e+00   2.114e-02   4.068e-02   4.601e-02       nan  linear_attn
     37   0.000e+00   2.253e-02   4.596e-02   4.938e-02       nan  linear_attn
     38   0.000e+00   2.346e-02   4.494e-02   4.740e-02       nan  linear_attn
     39   0.000e+00   2.172e-02   3.818e-02   4.250e-02       nan  self_attn
     40   0.000e+00   2.436e-02   4.511e-02   4.853e-02       nan  linear_attn
     41   0.000e+00   2.566e-02   4.570e-02   4.809e-02       nan  linear_attn
     42   0.000e+00   2.017e-02   4.118e-02   4.653e-02       nan  linear_attn
     43   0.000e+00   2.789e-02   5.656e-02   6.366e-02       nan  self_attn
     44   0.000e+00   2.393e-02   5.347e-02   5.940e-02       nan  linear_attn
     45   0.000e+00   2.541e-02   5.627e-02   6.064e-02       nan  linear_attn
     46   0.000e+00   2.645e-02   4.910e-02   5.793e-02       nan  linear_attn
     47   0.000e+00   2.372e-02   5.197e-02   5.316e-02       nan  self_attn
     48   0.000e+00   2.221e-02   4.804e-02   5.554e-02       nan  linear_attn
     49   0.000e+00   1.979e-02   3.739e-02   4.374e-02       nan  linear_attn
     50   0.000e+00   1.901e-02   3.287e-02   3.559e-02       nan  linear_attn
     51   0.000e+00   2.165e-02   3.778e-02   4.158e-02       nan  self_attn
     52   0.000e+00   1.781e-02   3.866e-02   4.152e-02       nan  linear_attn
     53   0.000e+00   2.185e-02   3.634e-02   4.430e-02       nan  linear_attn
     54   0.000e+00   1.736e-02   3.429e-02   3.634e-02       nan  linear_attn
     55   0.000e+00   1.990e-02   3.611e-02   3.917e-02       nan  self_attn
     56   0.000e+00   2.071e-02   3.236e-02   4.073e-02       nan  linear_attn
     57   0.000e+00   2.325e-02   4.010e-02   4.687e-02       nan  linear_attn
     58   0.000e+00   2.168e-02   3.199e-02   3.911e-02       nan  linear_attn
     59   0.000e+00   1.672e-02   2.852e-02   3.733e-02       nan  self_attn
     60   0.000e+00   2.207e-02   3.134e-02   4.511e-02       nan  linear_attn
     61   0.000e+00   1.951e-02   2.816e-02   3.396e-02       nan  linear_attn
     62   0.000e+00   2.914e-02   3.171e-02   4.600e-02       nan  linear_attn
     63   0.000e+00   2.906e-02   3.771e-02   3.735e-02       nan  self_attn

  chunk 5: rel rms after layer L, vs REF
  layer        REF2         STD      SIMROW        INT8     ratio
      0   0.000e+00   1.041e-02   1.844e-02   2.106e-02       nan  linear_attn
      1   0.000e+00   1.632e-02   2.368e-02   2.740e-02       nan  linear_attn
      2   0.000e+00   9.332e-03   1.002e-02   1.091e-02       nan  linear_attn
      3   0.000e+00   1.479e-02   2.798e-02   3.113e-02       nan  self_attn
      4   0.000e+00   1.538e-02   3.080e-02   3.345e-02       nan  linear_attn
      5   0.000e+00   1.590e-02   2.913e-02   3.199e-02       nan  linear_attn
      6   0.000e+00   1.268e-02   2.434e-02   2.728e-02       nan  linear_attn
      7   0.000e+00   1.652e-02   3.437e-02   3.922e-02       nan  self_attn
      8   0.000e+00   1.569e-02   3.250e-02   3.742e-02       nan  linear_attn
      9   0.000e+00   1.645e-02   3.332e-02   3.765e-02       nan  linear_attn
     10   0.000e+00   1.411e-02   3.200e-02   3.631e-02       nan  linear_attn
     11   0.000e+00   1.383e-02   3.048e-02   3.919e-02       nan  self_attn
     12   0.000e+00   1.475e-02   3.152e-02   3.921e-02       nan  linear_attn
     13   0.000e+00   1.531e-02   3.113e-02   3.779e-02       nan  linear_attn
     14   0.000e+00   1.594e-02   3.452e-02   4.045e-02       nan  linear_attn
     15   0.000e+00   1.659e-02   3.640e-02   4.364e-02       nan  self_attn
     16   0.000e+00   1.660e-02   3.463e-02   4.219e-02       nan  linear_attn
     17   0.000e+00   1.661e-02   3.644e-02   4.354e-02       nan  linear_attn
     18   0.000e+00   1.375e-02   2.649e-02   3.009e-02       nan  linear_attn
     19   0.000e+00   1.786e-02   3.490e-02   3.674e-02       nan  self_attn
     20   0.000e+00   1.603e-02   3.796e-02   3.723e-02       nan  linear_attn
     21   0.000e+00   2.011e-02   3.748e-02   4.286e-02       nan  linear_attn
     22   0.000e+00   1.967e-02   3.721e-02   4.171e-02       nan  linear_attn
     23   0.000e+00   2.076e-02   3.688e-02   4.144e-02       nan  self_attn
     24   0.000e+00   2.000e-02   3.738e-02   4.276e-02       nan  linear_attn
     25   0.000e+00   2.012e-02   3.529e-02   3.959e-02       nan  linear_attn
     26   0.000e+00   1.781e-02   3.589e-02   4.020e-02       nan  linear_attn
     27   0.000e+00   1.888e-02   4.121e-02   4.452e-02       nan  self_attn
     28   0.000e+00   2.063e-02   4.384e-02   4.798e-02       nan  linear_attn
     29   0.000e+00   1.902e-02   4.222e-02   4.545e-02       nan  linear_attn
     30   0.000e+00   1.883e-02   4.299e-02   4.734e-02       nan  linear_attn
     31   0.000e+00   1.978e-02   4.529e-02   5.050e-02       nan  self_attn
     32   0.000e+00   2.235e-02   4.898e-02   5.034e-02       nan  linear_attn
     33   0.000e+00   2.033e-02   4.891e-02   5.124e-02       nan  linear_attn
     34   0.000e+00   1.979e-02   4.275e-02   4.485e-02       nan  linear_attn
     35   0.000e+00   1.964e-02   4.832e-02   4.817e-02       nan  self_attn
     36   0.000e+00   2.034e-02   4.935e-02   4.807e-02       nan  linear_attn
     37   0.000e+00   2.126e-02   4.771e-02   4.963e-02       nan  linear_attn
     38   0.000e+00   2.042e-02   4.507e-02   4.694e-02       nan  linear_attn
     39   0.000e+00   1.966e-02   4.357e-02   4.437e-02       nan  self_attn
     40   0.000e+00   1.913e-02   4.403e-02   4.521e-02       nan  linear_attn
     41   0.000e+00   2.149e-02   4.839e-02   4.770e-02       nan  linear_attn
     42   0.000e+00   2.090e-02   4.238e-02   4.291e-02       nan  linear_attn
     43   0.000e+00   1.828e-02   5.178e-02   4.872e-02       nan  self_attn
     44   0.000e+00   2.006e-02   4.871e-02   4.910e-02       nan  linear_attn
     45   0.000e+00   1.842e-02   4.376e-02   4.500e-02       nan  linear_attn
     46   0.000e+00   1.829e-02   4.010e-02   4.353e-02       nan  linear_attn
     47   0.000e+00   1.991e-02   4.436e-02   4.499e-02       nan  self_attn
     48   0.000e+00   2.315e-02   4.195e-02   4.696e-02       nan  linear_attn
     49   0.000e+00   1.812e-02   3.613e-02   3.824e-02       nan  linear_attn
     50   0.000e+00   1.540e-02   3.000e-02   3.423e-02       nan  linear_attn
     51   0.000e+00   1.618e-02   3.524e-02   3.798e-02       nan  self_attn
     52   0.000e+00   1.636e-02   3.919e-02   3.463e-02       nan  linear_attn
     53   0.000e+00   1.545e-02   3.405e-02   3.572e-02       nan  linear_attn
     54   0.000e+00   1.501e-02   2.966e-02   3.281e-02       nan  linear_attn
     55   0.000e+00   1.713e-02   3.426e-02   3.962e-02       nan  self_attn
     56   0.000e+00   1.890e-02   3.807e-02   3.914e-02       nan  linear_attn
     57   0.000e+00   1.994e-02   3.857e-02   4.373e-02       nan  linear_attn
     58   0.000e+00   1.770e-02   3.636e-02   5.194e-02       nan  linear_attn
     59   0.000e+00   1.522e-02   3.054e-02   3.237e-02       nan  self_attn
     60   0.000e+00   1.796e-02   3.491e-02   3.855e-02       nan  linear_attn
     61   0.000e+00   1.729e-02   3.209e-02   3.774e-02       nan  linear_attn
     62   0.000e+00   1.563e-02   3.266e-02   3.277e-02       nan  linear_attn
     63   0.000e+00   2.090e-02   4.362e-02   6.301e-02       nan  self_attn

  chunk 6: rel rms after layer L, vs REF
  layer        REF2         STD      SIMROW        INT8     ratio
      0   0.000e+00   1.309e-02   2.407e-02   2.297e-02       nan  linear_attn
      1   0.000e+00   1.127e-02   2.050e-02   2.610e-02       nan  linear_attn
      2   0.000e+00   7.867e-03   1.343e-02   2.427e-02       nan  linear_attn
      3   0.000e+00   1.249e-02   2.945e-02   2.894e-02       nan  self_attn
      4   0.000e+00   1.495e-02   2.999e-02   3.616e-02       nan  linear_attn
      5   0.000e+00   1.744e-02   3.380e-02   3.908e-02       nan  linear_attn
      6   0.000e+00   1.482e-02   3.122e-02   3.794e-02       nan  linear_attn
      7   0.000e+00   1.674e-02   3.687e-02   4.222e-02       nan  self_attn
      8   0.000e+00   1.600e-02   3.467e-02   4.138e-02       nan  linear_attn
      9   0.000e+00   1.495e-02   3.346e-02   3.740e-02       nan  linear_attn
     10   0.000e+00   1.340e-02   2.894e-02   3.334e-02       nan  linear_attn
     11   0.000e+00   1.617e-02   3.960e-02   4.308e-02       nan  self_attn
     12   0.000e+00   1.472e-02   3.260e-02   3.774e-02       nan  linear_attn
     13   0.000e+00   1.455e-02   3.316e-02   3.982e-02       nan  linear_attn
     14   0.000e+00   1.374e-02   3.187e-02   3.517e-02       nan  linear_attn
     15   0.000e+00   1.446e-02   3.251e-02   3.794e-02       nan  self_attn
     16   0.000e+00   1.451e-02   3.178e-02   3.616e-02       nan  linear_attn
     17   0.000e+00   1.554e-02   3.662e-02   3.852e-02       nan  linear_attn
     18   0.000e+00   1.477e-02   3.084e-02   3.584e-02       nan  linear_attn
     19   0.000e+00   1.470e-02   3.164e-02   3.603e-02       nan  self_attn
     20   0.000e+00   1.481e-02   2.857e-02   3.148e-02       nan  linear_attn
     21   0.000e+00   1.616e-02   3.447e-02   3.978e-02       nan  linear_attn
     22   0.000e+00   1.424e-02   3.443e-02   3.765e-02       nan  linear_attn
     23   0.000e+00   1.507e-02   3.424e-02   3.812e-02       nan  self_attn
     24   0.000e+00   1.571e-02   3.537e-02   4.027e-02       nan  linear_attn
     25   0.000e+00   1.527e-02   3.405e-02   4.018e-02       nan  linear_attn
     26   0.000e+00   1.536e-02   3.358e-02   3.928e-02       nan  linear_attn
     27   0.000e+00   1.740e-02   3.998e-02   4.611e-02       nan  self_attn
     28   0.000e+00   2.001e-02   4.339e-02   4.816e-02       nan  linear_attn
     29   0.000e+00   1.829e-02   3.888e-02   4.424e-02       nan  linear_attn
     30   0.000e+00   2.050e-02   3.510e-02   4.370e-02       nan  linear_attn
     31   0.000e+00   2.386e-02   4.306e-02   5.176e-02       nan  self_attn
     32   0.000e+00   2.215e-02   4.113e-02   5.035e-02       nan  linear_attn
     33   0.000e+00   2.643e-02   4.675e-02   6.222e-02       nan  linear_attn
     34   0.000e+00   3.682e-02   4.712e-02   8.082e-02       nan  linear_attn
     35   0.000e+00   6.778e-02   5.597e-02   1.540e-01       nan  self_attn
     36   0.000e+00   6.521e-02   5.539e-02   1.547e-01       nan  linear_attn
     37   0.000e+00   6.758e-02   5.395e-02   1.578e-01       nan  linear_attn
     38   0.000e+00   7.343e-02   6.011e-02   1.698e-01       nan  linear_attn
     39   0.000e+00   9.330e-02   8.473e-02   2.089e-01       nan  self_attn
     40   0.000e+00   8.745e-02   7.769e-02   2.025e-01       nan  linear_attn
     41   0.000e+00   8.709e-02   8.858e-02   2.088e-01       nan  linear_attn
     42   0.000e+00   7.826e-02   7.944e-02   1.796e-01       nan  linear_attn
     43   0.000e+00   9.709e-02   1.062e-01   1.954e-01       nan  self_attn
     44   0.000e+00   9.266e-02   1.091e-01   1.887e-01       nan  linear_attn
     45   0.000e+00   8.477e-02   1.006e-01   1.865e-01       nan  linear_attn
     46   0.000e+00   9.827e-02   1.159e-01   2.283e-01       nan  linear_attn
     47   0.000e+00   9.936e-02   1.469e-01   2.805e-01       nan  self_attn
     48   0.000e+00   9.428e-02   1.344e-01   4.140e-01       nan  linear_attn
     49   0.000e+00   1.048e-01   1.407e-01   5.638e-01       nan  linear_attn
     50   0.000e+00   1.191e-01   1.439e-01   7.070e-01       nan  linear_attn
     51   0.000e+00   1.017e-01   1.123e-01   7.308e-01       nan  self_attn
     52   0.000e+00   9.491e-02   1.037e-01   6.947e-01       nan  linear_attn
     53   0.000e+00   9.770e-02   9.933e-02   7.521e-01       nan  linear_attn
     54   0.000e+00   1.161e-01   1.119e-01   8.309e-01       nan  linear_attn
     55   0.000e+00   1.081e-01   1.223e-01   7.901e-01       nan  self_attn
     56   0.000e+00   9.248e-02   9.610e-02   6.860e-01       nan  linear_attn
     57   0.000e+00   6.786e-02   8.301e-02   8.499e-01       nan  linear_attn
     58   0.000e+00   1.275e-01   8.523e-02   9.230e-01       nan  linear_attn
     59   0.000e+00   8.650e-02   7.808e-02   6.447e-01       nan  self_attn
     60   0.000e+00   1.180e-01   1.238e-01   8.824e-01       nan  linear_attn
     61   0.000e+00   1.266e-01   1.070e-01   9.617e-01       nan  linear_attn
     62   0.000e+00   1.004e-01   1.069e-01   8.205e-01       nan  linear_attn
     63   0.000e+00   7.856e-02   8.295e-02   6.734e-01       nan  self_attn

PER-CALL op error on the traced prompt.  Both sides are handed the
arm's OWN activation, so this is what the op did to the input it
was given: an op error that is ordinary at the worst chunk leaves
only the input as the carrier.
  arm INT8: 3200 scored calls
  chunk  calls   p50 i8-legB   max i8-legB   p50 i8-f32   max i8-f32  p50 legB-f32                        worst class
      0    400     1.258e-02     2.273e-02    1.230e-02    2.253e-02     2.434e-03   model.layers.31.self_attn.o_proj
      1    400     1.280e-02     2.292e-02    1.254e-02    2.271e-02     2.460e-03   model.layers.31.self_attn.o_proj
      2    400     1.270e-02     2.178e-02    1.240e-02    2.156e-02     2.432e-03   model.layers.43.self_attn.o_proj
      3    400     1.300e-02     2.243e-02    1.275e-02    2.228e-02     2.436e-03 model.layers.58.linear_attn.out_proj
      4    400     1.243e-02     2.163e-02    1.215e-02    2.142e-02     2.422e-03   model.layers.31.self_attn.o_proj
      5    400     1.271e-02     2.246e-02    1.240e-02    2.225e-02     2.439e-03   model.layers.31.self_attn.o_proj
      6    400     1.285e-02     2.234e-02    1.254e-02    2.213e-02     2.456e-03   model.layers.31.self_attn.o_proj
      7    400     1.285e-02     2.216e-02    1.256e-02    2.201e-02     2.432e-03 model.layers.58.linear_attn.out_proj
  the 10 calls with the largest int8-vs-fp32 relative error:
    c1 model.layers.31.self_attn.o_proj         i8/legB=2.292e-02 i8/f32=2.271e-02 legB/f32=3.078e-03 M=2048 K=6144 N=5120
    c0 model.layers.31.self_attn.o_proj         i8/legB=2.273e-02 i8/f32=2.253e-02 legB/f32=3.038e-03 M=2048 K=6144 N=5120
    c3 model.layers.58.linear_attn.out_proj     i8/legB=2.243e-02 i8/f32=2.228e-02 legB/f32=2.619e-03 M=2048 K=6144 N=5120
    c5 model.layers.31.self_attn.o_proj         i8/legB=2.246e-02 i8/f32=2.225e-02 legB/f32=3.060e-03 M=2048 K=6144 N=5120
    c6 model.layers.31.self_attn.o_proj         i8/legB=2.234e-02 i8/f32=2.213e-02 legB/f32=3.068e-03 M=2048 K=6144 N=5120
    c7 model.layers.58.linear_attn.out_proj     i8/legB=2.216e-02 i8/f32=2.201e-02 legB/f32=2.574e-03 M=2048 K=6144 N=5120
    c5 model.layers.58.linear_attn.out_proj     i8/legB=2.211e-02 i8/f32=2.196e-02 legB/f32=2.555e-03 M=2048 K=6144 N=5120
    c3 model.layers.31.self_attn.o_proj         i8/legB=2.214e-02 i8/f32=2.193e-02 legB/f32=3.032e-03 M=2048 K=6144 N=5120
    c7 model.layers.31.self_attn.o_proj         i8/legB=2.198e-02 i8/f32=2.176e-02 legB/f32=3.095e-03 M=2048 K=6144 N=5120
    c3 model.layers.43.self_attn.o_proj         i8/legB=2.183e-02 i8/f32=2.161e-02 legB/f32=3.083e-03 M=2048 K=6144 N=5120

PAIRED TV @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions.
Each row is A scored against B -- B is that ROW's reference, not the
table's.  TV is the expected speculative accept-rate loss, so the
column is read directly in percentage points.  The interval is a
cluster bootstrap over PROMPTS; the IQR is the raw spread over
positions with no floor applied to it.

  A       vs B        n  mean TV pp           95% CI pp    p25 pp    p50 pp    p75 pp    p95 pp    max pp
  REF     REF2        8      0.0000 [     nan,     nan]    0.0000    0.0000    0.0000    0.0000    0.0000
  REF     STD         8      1.2563 [     nan,     nan]    0.0000    1.8602    2.3977    3.5596    3.5596
  REF     SIMROW      8      2.7789 [     nan,     nan]    0.0000    3.6859    5.2558    8.6105    8.6105
  REF     INT8        8     13.1708 [     nan,     nan]    0.8577    2.4094    5.9471   92.2893   92.2893
  REF2    STD         8      1.2563 [     nan,     nan]    0.0000    1.8602    2.3977    3.5596    3.5596
  REF2    SIMROW      8      2.7789 [     nan,     nan]    0.0000    3.6859    5.2558    8.6105    8.6105
  REF2    INT8        8     13.1708 [     nan,     nan]    0.8577    2.4094    5.9471   92.2893   92.2893
  STD     SIMROW      8      2.9865 [     nan,     nan]    0.0000    3.9001    5.4922    8.1641    8.1641
  STD     INT8        8     12.6914 [     nan,     nan]    0.2342    1.1341    6.4074   89.6264   89.6264
  SIMROW  INT8        8     14.4211 [     nan,     nan]    1.1341    4.8256    6.9938   95.3613   95.3613

  TV distance matrix, mean TV pp (symmetric):
          REF      REF2       STD    SIMROW      INT8
  REF        0.0000    0.0000    1.2563    2.7789   13.1708
  REF2       0.0000    0.0000    1.2563    2.7789   13.1708
  STD        1.2563    1.2563    0.0000    2.9865   12.6914
  SIMROW     2.7789    2.7789    2.9865    0.0000   14.4211
  INT8      13.1708   13.1708   12.6914   14.4211    0.0000

  TV distance matrix, median TV pp (symmetric):
          REF      REF2       STD    SIMROW      INT8
  REF        0.0000    0.0000    1.8602    3.6859    2.4094
  REF2       0.0000    0.0000    1.8602    3.6859    2.4094
  STD        1.8602    1.8602    0.0000    3.9001    1.1341
  SIMROW     3.6859    3.6859    3.9001    0.0000    4.8256
  INT8       2.4094    2.4094    1.1341    4.8256    0.0000

lm_head CUT -- the divergence entering the head vs leaving it,
prefill positions, relative rms.  'in' is the residual stream after
24 layers; 'out' is the centred logits.  amp = out/in: at 1.0 the
head passes the body's divergence through unchanged, above it the
head adds its own.

  A       vs B        n   rel rms IN  rel rms OUT      amp
  REF     REF2        8   0.0000e+00   0.0000e+00      nan
  REF     STD         8   3.0632e-02   2.6879e-02    0.877
  REF     SIMROW      8   4.5449e-02   4.0073e-02    0.882
  REF     INT8        8   1.4908e-01   1.3846e-01    0.929
  REF2    STD         8   3.0632e-02   2.6879e-02    0.877
  REF2    SIMROW      8   4.5449e-02   4.0073e-02    0.882
  REF2    INT8        8   1.4908e-01   1.3846e-01    0.929
  STD     SIMROW      8   4.8306e-02   4.1570e-02    0.861
  STD     INT8        8   1.4618e-01   1.3600e-01    0.930
  SIMROW  INT8        8   1.5798e-01   1.4638e-01    0.927

PER-TOKEN AMAX -- is the crest factor of the quantised row what
selects the tail?  crest = amax/rms of the int8 GEMM's input row;
under the uniform-residual model the relative noise the row picks
up is crest/(127*sqrt(12)) = crest/440, and qerr is that noise
measured directly by round-tripping the scored row through int8.
arm supplying the statistics: INT8

  statistic                                  p50         p95         p99         max
  crest, scored row (max/linear)       1.210e+01   1.470e+01   1.470e+01   1.470e+01
  crest, any row in chunk              1.837e+01   1.919e+01   1.919e+01   1.919e+01
  qerr, scored row (rel rms)           2.747e-02   3.376e-02   3.376e-02   3.376e-02
  n positions: 8

What the per-token scale is paying for, over the whole census.
crest_g is the crest a per-128-group scale would leave --
sqrt(mean_b amax_b^2)/rms -- and its ratio to crest is the noise
reduction available.  conc is max/median of the per-128-block rms:
at 1 the row is flat and grouping buys nothing.
  statistic                                  p50         p95         p99         max
  crest_g (per-128 scale)                  2.593       2.879       2.942       3.001
  crest / crest_g (noise cut)              2.201       3.998       5.435       7.930
  block rms concentration                  3.410      11.428      23.087      39.813
  n (class, position) samples: 3200
  heaviest-block STABILITY per class (share of positions whose argmax block is that class's modal block):
    p05=0.125 p50=0.625 p95=1.000 over 400 classes
    a static outlier split needs this near 1.0; at 1/nblocks (~0.025 for K=5120) the heavy block is a different block every token and no static partition exists

The 10 worst INT8 positions, with the crest factor at each.  If the
per-token amax is the mechanism, these are the extreme-crest rows.
   prompt  chunk      INT8 KL   crest row  crest chunk        qerr  worst linear
        0      6    8.940e+00      11.362       19.089   2.552e-02  L21.linear_attn.out_proj
        0      7    1.156e-02      11.528       19.187   2.625e-02  L0.mlp.down_proj
        0      3    2.825e-03      12.099       17.630   2.747e-02  L18.linear_attn.out_proj
        0      4    2.387e-03      13.264       18.142   3.011e-02  L21.linear_attn.out_proj
        0      1    1.031e-03      11.255       17.735   2.561e-02  L0.mlp.down_proj
        0      2    9.914e-04      14.086       18.511   3.200e-02  L51.mlp.down_proj
        0      0    7.360e-05      11.625       17.800   2.654e-02  L3.mlp.down_proj
        0      5    4.626e-07      14.697       18.374   3.376e-02  L54.linear_attn.out_proj
  ... and the 10 MEDIAN INT8 positions, as the contrast:
        0      6    8.940e+00      11.362       19.089   2.552e-02  L21.linear_attn.out_proj

Is the tail MORE NOISE or MORE RESPONSE?  rel rms of the residual
stream entering lm_head, per position, and the INT8/STD ratio.
Flat ratio across the tail => the position amplifies every arm and
int8 is merely 2.4x noisier everywhere.  Ratio spiking on the tail
=> int8 injects locally more noise there, which is a mechanism.
  ratio over ALL 8 positions: p50=2.208 p95=7.908 max=7.908
  ratio at the 10 worst INT8 positions: min=1.556 median=2.071 max=7.908
   prompt  chunk      INT8 KL  relIN INT8   relIN STD    ratio
        0      6    8.940e+00  8.3688e-01  1.0582e-01    7.908
        0      7    1.156e-02  3.5898e-02  1.6255e-02    2.208
        0      3    2.825e-03  5.2579e-02  2.2964e-02    2.290
        0      4    2.387e-03  5.2518e-02  2.1120e-02    2.487
        0      1    1.031e-03  3.3852e-02  1.8768e-02    1.804
        0      2    9.914e-04  2.6260e-02  1.3719e-02    1.914
        0      0    7.360e-05  4.3991e-02  2.8267e-02    1.556
        0      5    4.626e-07  3.5188e-02  1.8196e-02    1.934
  Spearman(relIN ratio, INT8 KL) = 0.738   Spearman(relIN INT8, INT8 KL) = 0.619

Rank correlation of the crest factor with each arm's KL.  STD is the
NULL CONTROL: it runs no quantiser, so the crest factor cannot cause
its KL.  A crest that ranks STD as well as it ranks INT8 is ranking
hard POSITIONS, not the quantiser, and explains nothing.
  crest statistic                    REF       REF2        STD     SIMROW       INT8  INT8 excess
  crest, scored row                0.024      0.024     -0.595     -0.524     -0.571       -0.405
  crest, any row                   0.714      0.714      0.071     -0.024      0.310        0.476
  qerr, scored row                -0.095     -0.095     -0.690     -0.619     -0.667       -0.524

Mean crest of the scored row, by linear class -- the 15 heaviest.
A class well above the Gaussian-rotation expectation (~4.5 for a
128-wide Hadamard over K=5120) is a class the per-token scale is
wasting levels on.
  linear class                                        mean crest         p95         max
  L0.mlp.down_proj                                         9.829      12.463      12.463
  L2.mlp.down_proj                                         8.519       9.804       9.804
  L4.linear_attn.out_proj                                  8.333       9.779       9.779
  L48.linear_attn.out_proj                                 8.241       9.864       9.864
  L54.linear_attn.out_proj                                 8.238      14.697      14.697
  L21.linear_attn.out_proj                                 8.153      13.264      13.264
  L37.linear_attn.out_proj                                 7.929      11.826      11.826
  L2.linear_attn.out_proj                                  7.897       9.804       9.804
  L3.mlp.down_proj                                         7.620      11.625      11.625
  L52.mlp.down_proj                                        7.604      10.970      10.970
  L5.linear_attn.out_proj                                  7.540      10.546      10.546
  L50.linear_attn.out_proj                                 7.539       8.887       8.887
  L13.linear_attn.out_proj                                 7.529       9.901       9.901
  L19.self_attn.o_proj                                     7.487      10.478      10.478
  L1.linear_attn.out_proj                                  7.433      10.649      10.649
