[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (317092 > 262144). Running this sequence through the model will result in indexing errors
[i8] basket: 5 prompts x 2048 tok, domains ['code', 'longctx', 'math', 'sci', 'wiki']
Boot: loading model weights — still running (30s). This phase reports nothing until it completes; it is progressing while this line advances. If it stops advancing, the phase is stuck.
Boot: loading model weights — still running (60s). This phase reports nothing until it completes; it is progressing while this line advances. If it stops advancing, the phase is stuck.
Boot: loading model weights — still running (90s). This phase reports nothing until it completes; it is progressing while this line advances. If it stops advancing, the phase is stuck.
default-pool residency snapshot returned no live ranges — the unpooled-weight fold cannot see what it would move
[TKV autotune-provenance] fp=57911e26f4e47f74 regime=cache-hit cells=8 sweep_src=2772549a0ef713db tile_tokens=4 num_splits=8,16,32,64,128,256 min_blocks_per_sm=0,3 resolved=0 held=0 inherited=0
activation profile: the decode probe's FIRST call cost 12098 allocator events / 1 MiB / 981.7 ms against 12102 / 1 MiB / 44.9 ms warm — a first-call kernel search inside the probe, not a step. The reported peak AND device time are the warm ones (3 passes, the last is what is reported); the cold numbers are logged so the difference is visible.
driver.modules_loaded baseline: SEEDED at 150994944 B (0.141 GiB) → /root/.cache/arbi-serve/budget-cache/84169d322e956268.modules.json. This boot ran UNGATED — it booked its own bracketed growth, so an unregistered pool or a raw cudaMalloc would be inside that number rather than on driver.residual. Every later boot at this configuration is held to it. Expected ONCE per configuration; if it repeats, the budget cache is not persisting (point ARBI_SERVE_BUDGET_CACHE_DIR at a durable volume) and the guard is inert.
/opt/venv/lib/python3.12/site-packages/torch/__init__.py:1172: FutureWarning: `torch.distributed.reduce_op` is deprecated, please use `torch.distributed.ReduceOp` instead
  return isinstance(obj, torch.Tensor)
[i8] trellis->path map: 401 linears
JIT compile AFTER serving-ready [cpp_ext]: exl3_i8_gemm_k4 cached .so load — a live request paid this compile's latency. This is a boot-warmup coverage gap: extend warmup to pre-compile this kernel/specialization. Counter jit_compile_serving (must-not-fire) at GET /v1/admin/flag_truth.
[i8] prefix cache: server default resolved=True, per-request cache_enabled=False; any cache hit below is refused
[i8] warmup done
JIT compile AFTER serving-ready [cpp_ext]: tkv_unified_splitk_g6_v8_k4_v4_m1_v1_smem_rngfull_lbmb3_gms1_mw8_tt4_sh1_rc4_pf1_ldbypass cached .so load — a live request paid this compile's latency. This is a boot-warmup coverage gap: extend warmup to pre-compile this kernel/specialization. Counter jit_compile_serving (must-not-fire) at GET /v1/admin/flag_truth.
activation admission RE-ARMED (JIT compile after serving-ready): the step budget FELL to 5622 MiB, -540 MiB on the 6162 MiB it was enforcing and 6162 MiB the boot reading armed. The widest slate admission will build is narrower from here — narrower prefill chunks and deferred rows instead of a forward that OOMs after the layout freeze.
  [0] code           REF       2048tok  wall=  0.87s  finish=length  leg=400F/0S acc=400x16/0x32
  [0] code           INT8      2048tok  wall=  0.81s  finish=length  leg=0F/400S acc=400x16/0x32
  [0] code           INT8G     2048tok  wall=  0.97s  finish=length  leg=0F/400S acc=400x16/0x32
  [0] code           INT8G256   2048tok  wall=  0.94s  finish=length  leg=0F/400S acc=400x16/0x32
  [0] code           INT8GN    2048tok  wall=  0.94s  finish=length  leg=0F/400S acc=400x16/0x32
  [1] wiki           REF       2048tok  wall=  0.82s  finish=length  leg=400F/0S acc=400x16/0x32
  [1] wiki           INT8      2048tok  wall=  0.80s  finish=length  leg=0F/400S acc=400x16/0x32
  [1] wiki           INT8G     2048tok  wall=  0.96s  finish=length  leg=0F/400S acc=400x16/0x32
  [1] wiki           INT8G256   2048tok  wall=  0.91s  finish=length  leg=0F/400S acc=400x16/0x32
  [1] wiki           INT8GN    2048tok  wall=  0.93s  finish=length  leg=0F/400S acc=400x16/0x32
  [2] math           REF       2048tok  wall=  0.82s  finish=length  leg=400F/0S acc=400x16/0x32
  [2] math           INT8      2048tok  wall=  0.82s  finish=length  leg=0F/400S acc=400x16/0x32
  [2] math           INT8G     2048tok  wall=  0.97s  finish=length  leg=0F/400S acc=400x16/0x32
  [2] math           INT8G256   2048tok  wall=  0.92s  finish=length  leg=0F/400S acc=400x16/0x32
  [2] math           INT8GN    2048tok  wall=  0.93s  finish=length  leg=0F/400S acc=400x16/0x32
  [3] longctx        REF       2048tok  wall=  0.82s  finish=length  leg=400F/0S acc=400x16/0x32
  [3] longctx        INT8      2048tok  wall=  0.80s  finish=length  leg=0F/400S acc=400x16/0x32
  [3] longctx        INT8G     2048tok  wall=  0.95s  finish=length  leg=0F/400S acc=400x16/0x32
  [3] longctx        INT8G256   2048tok  wall=  0.92s  finish=length  leg=0F/400S acc=400x16/0x32
  [3] longctx        INT8GN    2048tok  wall=  0.93s  finish=length  leg=0F/400S acc=400x16/0x32
  [4] sci            REF       2048tok  wall=  0.83s  finish=length  leg=400F/0S acc=400x16/0x32
  [4] sci            INT8      2048tok  wall=  0.82s  finish=length  leg=0F/400S acc=400x16/0x32
  [4] sci            INT8G     2048tok  wall=  0.97s  finish=length  leg=0F/400S acc=400x16/0x32
  [4] sci            INT8G256   2048tok  wall=  0.93s  finish=length  leg=0F/400S acc=400x16/0x32
  [4] sci            INT8GN    2048tok  wall=  0.93s  finish=length  leg=0F/400S acc=400x16/0x32

[i8] kernel census: int8 calls=8000 rows=16384000 reconstructs SKIPPED=8000 fallback (non-4bpw) calls=0
[i8] shapes served by the kernel (K,N)->calls: {(5120, 1024): 640, (5120, 6144): 960, (5120, 10240): 960, (5120, 12288): 320, (5120, 17408): 2560, (6144, 5120): 1280, (17408, 5120): 1280}
[i8] excluded projections: none  excluded calls=0

  linear class                                           served   excl  int8 vs fp32 legB vs fp32 int8 vs legB
  model.layers.0.linear_attn.in_proj_qkv                     24      0     4.827e-03    1.713e-03    5.132e-03
  model.layers.0.linear_attn.in_proj_z                       24      0     6.316e-03    1.129e-03    6.418e-03
  model.layers.0.linear_attn.out_proj                        24      0     1.249e-02    2.189e-03    1.268e-02
  model.layers.0.mlp.down_proj                               24      0     2.122e-02    2.609e-03    2.137e-02
  model.layers.0.mlp.gate_proj                               24      0     9.121e-03    3.413e-03    9.739e-03
  model.layers.0.mlp.up_proj                                 24      0     9.082e-03    3.418e-03    9.702e-03
  model.layers.1.linear_attn.in_proj_qkv                     24      0     1.186e-02    2.166e-03    1.206e-02
  model.layers.1.linear_attn.in_proj_z                       24      0     1.361e-02    1.840e-03    1.373e-02
  model.layers.1.linear_attn.out_proj                        24      0     1.604e-02    2.254e-03    1.619e-02
  model.layers.1.mlp.down_proj                               24      0     1.344e-02    2.322e-03    1.364e-02
  model.layers.1.mlp.gate_proj                               24      0     9.700e-03    3.460e-03    1.030e-02
  model.layers.1.mlp.up_proj                                 24      0     1.015e-02    3.488e-03    1.073e-02
  model.layers.10.linear_attn.in_proj_qkv                    24      0     1.304e-02    2.255e-03    1.324e-02
  model.layers.10.linear_attn.in_proj_z                      24      0     1.298e-02    1.836e-03    1.311e-02
  model.layers.10.linear_attn.out_proj                       24      0     1.497e-02    2.120e-03    1.512e-02
  model.layers.10.mlp.down_proj                              24      0     1.268e-02    2.348e-03    1.289e-02
  model.layers.10.mlp.gate_proj                              24      0     1.035e-02    3.581e-03    1.095e-02
  model.layers.10.mlp.up_proj                                24      0     1.076e-02    3.588e-03    1.134e-02
  model.layers.11.mlp.down_proj                              24      0     1.333e-02    2.506e-03    1.356e-02
  model.layers.11.mlp.gate_proj                              24      0     1.027e-02    3.507e-03    1.085e-02
  model.layers.11.mlp.up_proj                                24      0     1.088e-02    3.576e-03    1.145e-02
  model.layers.11.self_attn.k_proj                           24      0     1.330e-02    2.705e-03    1.357e-02
  model.layers.11.self_attn.o_proj                           24      0     1.304e-02    2.370e-03    1.326e-02
  model.layers.11.self_attn.q_proj                           24      0     1.174e-02    2.022e-03    1.191e-02
  model.layers.11.self_attn.v_proj                           24      0     1.037e-02    2.283e-03    1.062e-02
  model.layers.12.linear_attn.in_proj_qkv                    24      0     1.292e-02    2.244e-03    1.312e-02
  model.layers.12.linear_attn.in_proj_z                      24      0     1.365e-02    1.892e-03    1.378e-02
  model.layers.12.linear_attn.out_proj                       24      0     1.583e-02    2.138e-03    1.597e-02
  model.layers.12.mlp.down_proj                              24      0     1.316e-02    2.550e-03    1.341e-02
  model.layers.12.mlp.gate_proj                              24      0     1.002e-02    3.472e-03    1.061e-02
  model.layers.12.mlp.up_proj                                24      0     1.072e-02    3.553e-03    1.129e-02
  model.layers.13.linear_attn.in_proj_qkv                    24      0     1.292e-02    2.241e-03    1.312e-02
  model.layers.13.linear_attn.in_proj_z                      24      0     1.418e-02    1.970e-03    1.431e-02
  model.layers.13.linear_attn.out_proj                       24      0     1.680e-02    2.175e-03    1.693e-02
  model.layers.13.mlp.down_proj                              24      0     1.373e-02    2.565e-03    1.396e-02
  model.layers.13.mlp.gate_proj                              24      0     1.007e-02    3.483e-03    1.066e-02
  model.layers.13.mlp.up_proj                                24      0     1.071e-02    3.557e-03    1.128e-02
  model.layers.14.linear_attn.in_proj_qkv                    24      0     1.272e-02    2.232e-03    1.291e-02
  model.layers.14.linear_attn.in_proj_z                      24      0     1.334e-02    1.883e-03    1.348e-02
  model.layers.14.linear_attn.out_proj                       24      0     1.586e-02    2.214e-03    1.601e-02
  model.layers.14.mlp.down_proj                              24      0     1.300e-02    2.463e-03    1.323e-02
  model.layers.14.mlp.gate_proj                              24      0     9.963e-03    3.503e-03    1.056e-02
  model.layers.14.mlp.up_proj                                24      0     1.064e-02    3.565e-03    1.122e-02
  model.layers.15.mlp.down_proj                              24      0     1.349e-02    2.537e-03    1.373e-02
  model.layers.15.mlp.gate_proj                              24      0     1.010e-02    3.488e-03    1.068e-02
  model.layers.15.mlp.up_proj                                24      0     1.068e-02    3.562e-03    1.125e-02
  model.layers.15.self_attn.k_proj                           24      0     1.452e-02    2.890e-03    1.481e-02
  model.layers.15.self_attn.o_proj                           24      0     1.398e-02    2.117e-03    1.414e-02
  model.layers.15.self_attn.q_proj                           24      0     1.302e-02    2.087e-03    1.318e-02
  model.layers.15.self_attn.v_proj                           24      0     9.832e-03    2.228e-03    1.008e-02
  model.layers.16.linear_attn.in_proj_qkv                    24      0     1.289e-02    2.246e-03    1.308e-02
  model.layers.16.linear_attn.in_proj_z                      24      0     1.353e-02    1.895e-03    1.366e-02
  model.layers.16.linear_attn.out_proj                       24      0     1.636e-02    2.275e-03    1.651e-02
  model.layers.16.mlp.down_proj                              24      0     1.298e-02    2.519e-03    1.322e-02
  model.layers.16.mlp.gate_proj                              24      0     1.006e-02    3.472e-03    1.064e-02
  model.layers.16.mlp.up_proj                                24      0     1.070e-02    3.563e-03    1.128e-02
  model.layers.17.linear_attn.in_proj_qkv                    24      0     1.287e-02    2.222e-03    1.306e-02
  model.layers.17.linear_attn.in_proj_z                      24      0     1.352e-02    1.896e-03    1.366e-02
  model.layers.17.linear_attn.out_proj                       24      0     1.781e-02    2.245e-03    1.796e-02
  model.layers.17.mlp.down_proj                              24      0     1.381e-02    2.455e-03    1.403e-02
  model.layers.17.mlp.gate_proj                              24      0     1.010e-02    3.471e-03    1.068e-02
  model.layers.17.mlp.up_proj                                24      0     1.072e-02    3.565e-03    1.129e-02
  model.layers.18.linear_attn.in_proj_qkv                    24      0     1.206e-02    2.189e-03    1.225e-02
  model.layers.18.linear_attn.in_proj_z                      24      0     1.264e-02    1.788e-03    1.276e-02
  model.layers.18.linear_attn.out_proj                       24      0     1.527e-02    2.227e-03    1.543e-02
  model.layers.18.mlp.down_proj                              24      0     1.461e-02    2.424e-03    1.481e-02
  model.layers.18.mlp.gate_proj                              24      0     1.024e-02    3.422e-03    1.079e-02
  model.layers.18.mlp.up_proj                                24      0     1.058e-02    3.532e-03    1.116e-02
  model.layers.19.mlp.down_proj                              24      0     1.585e-02    2.536e-03    1.605e-02
  model.layers.19.mlp.gate_proj                              24      0     1.008e-02    3.395e-03    1.063e-02
  model.layers.19.mlp.up_proj                                24      0     1.096e-02    3.581e-03    1.153e-02
  model.layers.19.self_attn.k_proj                           24      0     1.407e-02    2.788e-03    1.434e-02
  model.layers.19.self_attn.o_proj                           24      0     1.814e-02    2.166e-03    1.826e-02
  model.layers.19.self_attn.q_proj                           24      0     9.500e-03    1.927e-03    9.692e-03
  model.layers.19.self_attn.v_proj                           24      0     1.047e-02    2.352e-03    1.073e-02
  model.layers.2.linear_attn.in_proj_qkv                     24      0     1.249e-02    2.213e-03    1.268e-02
  model.layers.2.linear_attn.in_proj_z                       24      0     1.398e-02    1.926e-03    1.411e-02
  model.layers.2.linear_attn.out_proj                        24      0     1.794e-02    2.322e-03    1.808e-02
  model.layers.2.mlp.down_proj                               24      0     1.706e-02    2.372e-03    1.723e-02
  model.layers.2.mlp.gate_proj                               24      0     1.029e-02    3.530e-03    1.088e-02
  model.layers.2.mlp.up_proj                                 24      0     1.061e-02    3.561e-03    1.120e-02
  model.layers.20.linear_attn.in_proj_qkv                    24      0     1.215e-02    2.138e-03    1.234e-02
  model.layers.20.linear_attn.in_proj_z                      24      0     1.272e-02    1.791e-03    1.285e-02
  model.layers.20.linear_attn.out_proj                       24      0     1.578e-02    2.229e-03    1.594e-02
  model.layers.20.mlp.down_proj                              24      0     1.592e-02    2.582e-03    1.613e-02
  model.layers.20.mlp.gate_proj                              24      0     1.009e-02    3.407e-03    1.065e-02
  model.layers.20.mlp.up_proj                                24      0     1.128e-02    3.604e-03    1.184e-02
  model.layers.21.linear_attn.in_proj_qkv                    24      0     1.198e-02    2.201e-03    1.218e-02
  model.layers.21.linear_attn.in_proj_z                      24      0     1.162e-02    1.671e-03    1.174e-02
  model.layers.21.linear_attn.out_proj                       24      0     1.703e-02    2.236e-03    1.717e-02
  model.layers.21.mlp.down_proj                              24      0     1.719e-02    2.592e-03    1.739e-02
  model.layers.21.mlp.gate_proj                              24      0     1.010e-02    3.421e-03    1.066e-02
  model.layers.21.mlp.up_proj                                24      0     1.124e-02    3.609e-03    1.180e-02
  model.layers.22.linear_attn.in_proj_qkv                    24      0     1.201e-02    2.240e-03    1.221e-02
  model.layers.22.linear_attn.in_proj_z                      24      0     1.202e-02    1.714e-03    1.214e-02
  model.layers.22.linear_attn.out_proj                       24      0     1.580e-02    2.304e-03    1.597e-02
  model.layers.22.mlp.down_proj                              24      0     1.511e-02    2.552e-03    1.533e-02
  model.layers.22.mlp.gate_proj                              24      0     1.054e-02    3.140e-03    1.100e-02
  model.layers.22.mlp.up_proj                                24      0     1.140e-02    3.621e-03    1.196e-02
  model.layers.23.mlp.down_proj                              24      0     1.568e-02    2.553e-03    1.589e-02
  model.layers.23.mlp.gate_proj                              24      0     1.009e-02    3.367e-03    1.063e-02
  model.layers.23.mlp.up_proj                                24      0     1.141e-02    3.621e-03    1.197e-02
  model.layers.23.self_attn.k_proj                           24      0     1.382e-02    2.675e-03    1.408e-02
  model.layers.23.self_attn.o_proj                           24      0     1.681e-02    2.423e-03    1.698e-02
  model.layers.23.self_attn.q_proj                           24      0     9.476e-03    1.964e-03    9.678e-03
  model.layers.23.self_attn.v_proj                           24      0     1.034e-02    2.335e-03    1.061e-02
  model.layers.24.linear_attn.in_proj_qkv                    24      0     1.188e-02    2.253e-03    1.210e-02
  model.layers.24.linear_attn.in_proj_z                      24      0     1.223e-02    1.734e-03    1.235e-02
  model.layers.24.linear_attn.out_proj                       24      0     1.604e-02    2.270e-03    1.620e-02
  model.layers.24.mlp.down_proj                              24      0     1.459e-02    2.620e-03    1.482e-02
  model.layers.24.mlp.gate_proj                              24      0     1.026e-02    3.316e-03    1.078e-02
  model.layers.24.mlp.up_proj                                24      0     1.144e-02    3.612e-03    1.199e-02
  model.layers.25.linear_attn.in_proj_qkv                    24      0     1.141e-02    2.212e-03    1.162e-02
  model.layers.25.linear_attn.in_proj_z                      24      0     1.279e-02    1.788e-03    1.291e-02
  model.layers.25.linear_attn.out_proj                       24      0     1.475e-02    2.178e-03    1.491e-02
  model.layers.25.mlp.down_proj                              24      0     1.391e-02    2.553e-03    1.414e-02
  model.layers.25.mlp.gate_proj                              24      0     1.054e-02    3.323e-03    1.105e-02
  model.layers.25.mlp.up_proj                                24      0     1.130e-02    3.601e-03    1.185e-02
  model.layers.26.linear_attn.in_proj_qkv                    24      0     1.151e-02    2.236e-03    1.172e-02
  model.layers.26.linear_attn.in_proj_z                      24      0     1.259e-02    1.796e-03    1.272e-02
  model.layers.26.linear_attn.out_proj                       24      0     1.494e-02    2.342e-03    1.512e-02
  model.layers.26.mlp.down_proj                              24      0     1.268e-02    2.467e-03    1.292e-02
  model.layers.26.mlp.gate_proj                              24      0     1.112e-02    3.295e-03    1.160e-02
  model.layers.26.mlp.up_proj                                24      0     1.085e-02    3.549e-03    1.142e-02
  model.layers.27.mlp.down_proj                              24      0     1.335e-02    2.577e-03    1.360e-02
  model.layers.27.mlp.gate_proj                              24      0     1.080e-02    3.204e-03    1.126e-02
  model.layers.27.mlp.up_proj                                24      0     1.090e-02    3.553e-03    1.147e-02
  model.layers.27.self_attn.k_proj                           24      0     1.215e-02    2.501e-03    1.240e-02
  model.layers.27.self_attn.o_proj                           24      0     1.711e-02    2.742e-03    1.733e-02
  model.layers.27.self_attn.q_proj                           24      0     1.021e-02    2.059e-03    1.042e-02
  model.layers.27.self_attn.v_proj                           24      0     9.828e-03    2.250e-03    1.009e-02
  model.layers.28.linear_attn.in_proj_qkv                    24      0     1.179e-02    2.219e-03    1.200e-02
  model.layers.28.linear_attn.in_proj_z                      24      0     1.290e-02    1.818e-03    1.302e-02
  model.layers.28.linear_attn.out_proj                       24      0     1.477e-02    2.298e-03    1.495e-02
  model.layers.28.mlp.down_proj                              24      0     1.301e-02    2.596e-03    1.327e-02
  model.layers.28.mlp.gate_proj                              24      0     1.104e-02    3.175e-03    1.149e-02
  model.layers.28.mlp.up_proj                                24      0     1.084e-02    3.538e-03    1.141e-02
  model.layers.29.linear_attn.in_proj_qkv                    24      0     1.205e-02    2.246e-03    1.226e-02
  model.layers.29.linear_attn.in_proj_z                      24      0     1.395e-02    1.964e-03    1.408e-02
  model.layers.29.linear_attn.out_proj                       24      0     1.658e-02    2.351e-03    1.674e-02
  model.layers.29.mlp.down_proj                              24      0     1.346e-02    2.623e-03    1.371e-02
  model.layers.29.mlp.gate_proj                              24      0     1.115e-02    3.186e-03    1.159e-02
  model.layers.29.mlp.up_proj                                24      0     1.080e-02    3.540e-03    1.137e-02
  model.layers.3.mlp.down_proj                               24      0     1.415e-02    2.578e-03    1.438e-02
  model.layers.3.mlp.gate_proj                               24      0     1.017e-02    3.527e-03    1.076e-02
  model.layers.3.mlp.up_proj                                 24      0     1.100e-02    3.601e-03    1.158e-02
  model.layers.3.self_attn.k_proj                            24      0     1.247e-02    2.581e-03    1.273e-02
  model.layers.3.self_attn.o_proj                            24      0     1.495e-02    2.177e-03    1.511e-02
  model.layers.3.self_attn.q_proj                            24      0     1.169e-02    2.033e-03    1.187e-02
  model.layers.3.self_attn.v_proj                            24      0     1.263e-02    2.305e-03    1.285e-02
  model.layers.30.linear_attn.in_proj_qkv                    24      0     1.190e-02    2.231e-03    1.211e-02
  model.layers.30.linear_attn.in_proj_z                      24      0     1.349e-02    1.888e-03    1.362e-02
  model.layers.30.linear_attn.out_proj                       24      0     1.621e-02    2.413e-03    1.639e-02
  model.layers.30.mlp.down_proj                              24      0     1.287e-02    2.561e-03    1.312e-02
  model.layers.30.mlp.gate_proj                              24      0     1.172e-02    3.189e-03    1.214e-02
  model.layers.30.mlp.up_proj                                24      0     1.067e-02    3.520e-03    1.123e-02
  model.layers.31.mlp.down_proj                              24      0     1.317e-02    2.594e-03    1.342e-02
  model.layers.31.mlp.gate_proj                              24      0     1.148e-02    3.154e-03    1.191e-02
  model.layers.31.mlp.up_proj                                24      0     1.090e-02    3.518e-03    1.145e-02
  model.layers.31.self_attn.k_proj                           24      0     1.207e-02    2.525e-03    1.233e-02
  model.layers.31.self_attn.o_proj                           24      0     2.213e-02    3.026e-03    2.233e-02
  model.layers.31.self_attn.q_proj                           24      0     1.016e-02    2.073e-03    1.037e-02
  model.layers.31.self_attn.v_proj                           24      0     9.914e-03    2.186e-03    1.014e-02
  model.layers.32.linear_attn.in_proj_qkv                    24      0     1.282e-02    2.288e-03    1.302e-02
  model.layers.32.linear_attn.in_proj_z                      24      0     1.412e-02    1.983e-03    1.425e-02
  model.layers.32.linear_attn.out_proj                       24      0     1.564e-02    2.301e-03    1.581e-02
  model.layers.32.mlp.down_proj                              24      0     1.303e-02    2.618e-03    1.329e-02
  model.layers.32.mlp.gate_proj                              24      0     1.166e-02    3.107e-03    1.206e-02
  model.layers.32.mlp.up_proj                                24      0     1.108e-02    3.524e-03    1.163e-02
  model.layers.33.linear_attn.in_proj_qkv                    24      0     1.261e-02    2.298e-03    1.282e-02
  model.layers.33.linear_attn.in_proj_z                      24      0     1.385e-02    1.990e-03    1.399e-02
  model.layers.33.linear_attn.out_proj                       24      0     1.747e-02    2.364e-03    1.763e-02
  model.layers.33.mlp.down_proj                              24      0     1.343e-02    2.602e-03    1.368e-02
  model.layers.33.mlp.gate_proj                              24      0     1.191e-02    3.112e-03    1.231e-02
  model.layers.33.mlp.up_proj                                24      0     1.124e-02    3.540e-03    1.178e-02
  model.layers.34.linear_attn.in_proj_qkv                    24      0     1.252e-02    2.276e-03    1.273e-02
  model.layers.34.linear_attn.in_proj_z                      24      0     1.341e-02    1.911e-03    1.355e-02
  model.layers.34.linear_attn.out_proj                       24      0     1.617e-02    2.364e-03    1.634e-02
  model.layers.34.mlp.down_proj                              24      0     1.379e-02    2.611e-03    1.404e-02
  model.layers.34.mlp.gate_proj                              24      0     1.206e-02    3.070e-03    1.244e-02
  model.layers.34.mlp.up_proj                                24      0     1.115e-02    3.506e-03    1.169e-02
  model.layers.35.mlp.down_proj                              24      0     1.417e-02    2.560e-03    1.441e-02
  model.layers.35.mlp.gate_proj                              24      0     1.066e-02    3.067e-03    1.109e-02
  model.layers.35.mlp.up_proj                                24      0     1.138e-02    3.579e-03    1.193e-02
  model.layers.35.self_attn.k_proj                           24      0     1.175e-02    2.504e-03    1.202e-02
  model.layers.35.self_attn.o_proj                           24      0     1.991e-02    2.804e-03    2.011e-02
  model.layers.35.self_attn.q_proj                           24      0     1.072e-02    2.115e-03    1.093e-02
  model.layers.35.self_attn.v_proj                           24      0     1.073e-02    2.262e-03    1.096e-02
  model.layers.36.linear_attn.in_proj_qkv                    24      0     1.294e-02    2.274e-03    1.314e-02
  model.layers.36.linear_attn.in_proj_z                      24      0     1.210e-02    1.745e-03    1.223e-02
  model.layers.36.linear_attn.out_proj                       24      0     1.662e-02    2.261e-03    1.678e-02
  model.layers.36.mlp.down_proj                              24      0     1.508e-02    2.627e-03    1.531e-02
  model.layers.36.mlp.gate_proj                              24      0     1.025e-02    3.143e-03    1.072e-02
  model.layers.36.mlp.up_proj                                24      0     1.154e-02    3.605e-03    1.209e-02
  model.layers.37.linear_attn.in_proj_qkv                    24      0     1.223e-02    2.241e-03    1.243e-02
  model.layers.37.linear_attn.in_proj_z                      24      0     1.059e-02    1.597e-03    1.071e-02
  model.layers.37.linear_attn.out_proj                       24      0     1.839e-02    2.197e-03    1.852e-02
  model.layers.37.mlp.down_proj                              24      0     1.540e-02    2.731e-03    1.564e-02
  model.layers.37.mlp.gate_proj                              24      0     1.056e-02    3.237e-03    1.104e-02
  model.layers.37.mlp.up_proj                                24      0     1.173e-02    3.628e-03    1.228e-02
  model.layers.38.linear_attn.in_proj_qkv                    24      0     1.200e-02    2.317e-03    1.222e-02
  model.layers.38.linear_attn.in_proj_z                      24      0     1.099e-02    1.628e-03    1.111e-02
  model.layers.38.linear_attn.out_proj                       24      0     1.656e-02    2.310e-03    1.672e-02
  model.layers.38.mlp.down_proj                              24      0     1.595e-02    2.699e-03    1.617e-02
  model.layers.38.mlp.gate_proj                              24      0     1.069e-02    3.068e-03    1.112e-02
  model.layers.38.mlp.up_proj                                24      0     1.171e-02    3.638e-03    1.226e-02
  model.layers.39.mlp.down_proj                              24      0     1.439e-02    2.656e-03    1.463e-02
  model.layers.39.mlp.gate_proj                              24      0     9.970e-03    3.161e-03    1.046e-02
  model.layers.39.mlp.up_proj                                24      0     1.179e-02    3.647e-03    1.234e-02
  model.layers.39.self_attn.k_proj                           24      0     1.178e-02    2.463e-03    1.204e-02
  model.layers.39.self_attn.o_proj                           24      0     2.078e-02    2.837e-03    2.097e-02
  model.layers.39.self_attn.q_proj                           24      0     1.046e-02    2.097e-03    1.067e-02
  model.layers.39.self_attn.v_proj                           24      0     1.091e-02    2.328e-03    1.116e-02
  model.layers.4.linear_attn.in_proj_qkv                     24      0     1.192e-02    2.308e-03    1.214e-02
  model.layers.4.linear_attn.in_proj_z                       24      0     1.247e-02    1.771e-03    1.259e-02
  model.layers.4.linear_attn.out_proj                        24      0     1.789e-02    2.411e-03    1.806e-02
  model.layers.4.mlp.down_proj                               24      0     1.443e-02    2.603e-03    1.467e-02
  model.layers.4.mlp.gate_proj                               24      0     1.013e-02    3.511e-03    1.072e-02
  model.layers.4.mlp.up_proj                                 24      0     1.123e-02    3.618e-03    1.180e-02
  model.layers.40.linear_attn.in_proj_qkv                    24      0     1.224e-02    2.282e-03    1.245e-02
  model.layers.40.linear_attn.in_proj_z                      24      0     1.173e-02    1.688e-03    1.185e-02
  model.layers.40.linear_attn.out_proj                       24      0     1.607e-02    2.268e-03    1.623e-02
  model.layers.40.mlp.down_proj                              24      0     1.478e-02    2.675e-03    1.501e-02
  model.layers.40.mlp.gate_proj                              24      0     1.008e-02    3.134e-03    1.055e-02
  model.layers.40.mlp.up_proj                                24      0     1.164e-02    3.641e-03    1.220e-02
  model.layers.41.linear_attn.in_proj_qkv                    24      0     1.176e-02    2.256e-03    1.197e-02
  model.layers.41.linear_attn.in_proj_z                      24      0     1.226e-02    1.733e-03    1.238e-02
  model.layers.41.linear_attn.out_proj                       24      0     1.649e-02    2.255e-03    1.664e-02
  model.layers.41.mlp.down_proj                              24      0     1.477e-02    2.705e-03    1.502e-02
  model.layers.41.mlp.gate_proj                              24      0     1.085e-02    3.211e-03    1.132e-02
  model.layers.41.mlp.up_proj                                24      0     1.167e-02    3.636e-03    1.223e-02
  model.layers.42.linear_attn.in_proj_qkv                    24      0     1.162e-02    2.276e-03    1.184e-02
  model.layers.42.linear_attn.in_proj_z                      24      0     1.242e-02    1.747e-03    1.254e-02
  model.layers.42.linear_attn.out_proj                       24      0     1.568e-02    2.272e-03    1.584e-02
  model.layers.42.mlp.down_proj                              24      0     1.467e-02    2.652e-03    1.490e-02
  model.layers.42.mlp.gate_proj                              24      0     1.173e-02    3.240e-03    1.217e-02
  model.layers.42.mlp.up_proj                                24      0     1.143e-02    3.586e-03    1.198e-02
  model.layers.43.mlp.down_proj                              24      0     1.386e-02    2.670e-03    1.411e-02
  model.layers.43.mlp.gate_proj                              24      0     1.084e-02    3.218e-03    1.131e-02
  model.layers.43.mlp.up_proj                                24      0     1.153e-02    3.615e-03    1.208e-02
  model.layers.43.self_attn.k_proj                           24      0     1.250e-02    2.453e-03    1.274e-02
  model.layers.43.self_attn.o_proj                           24      0     2.162e-02    3.131e-03    2.185e-02
  model.layers.43.self_attn.q_proj                           24      0     1.059e-02    2.086e-03    1.079e-02
  model.layers.43.self_attn.v_proj                           24      0     1.050e-02    2.259e-03    1.074e-02
  model.layers.44.linear_attn.in_proj_qkv                    24      0     1.203e-02    2.229e-03    1.224e-02
  model.layers.44.linear_attn.in_proj_z                      24      0     1.227e-02    1.697e-03    1.239e-02
  model.layers.44.linear_attn.out_proj                       24      0     1.629e-02    2.299e-03    1.645e-02
  model.layers.44.mlp.down_proj                              24      0     1.413e-02    2.713e-03    1.439e-02
  model.layers.44.mlp.gate_proj                              24      0     1.082e-02    3.260e-03    1.130e-02
  model.layers.44.mlp.up_proj                                24      0     1.171e-02    3.618e-03    1.225e-02
  model.layers.45.linear_attn.in_proj_qkv                    24      0     1.207e-02    2.277e-03    1.229e-02
  model.layers.45.linear_attn.in_proj_z                      24      0     1.264e-02    1.761e-03    1.276e-02
  model.layers.45.linear_attn.out_proj                       24      0     1.809e-02    2.431e-03    1.825e-02
  model.layers.45.mlp.down_proj                              24      0     1.435e-02    2.745e-03    1.461e-02
  model.layers.45.mlp.gate_proj                              24      0     1.081e-02    3.287e-03    1.130e-02
  model.layers.45.mlp.up_proj                                24      0     1.157e-02    3.622e-03    1.212e-02
  model.layers.46.linear_attn.in_proj_qkv                    24      0     1.197e-02    2.246e-03    1.218e-02
  model.layers.46.linear_attn.in_proj_z                      24      0     1.186e-02    1.650e-03    1.197e-02
  model.layers.46.linear_attn.out_proj                       24      0     1.870e-02    2.364e-03    1.885e-02
  model.layers.46.mlp.down_proj                              24      0     1.426e-02    2.686e-03    1.451e-02
  model.layers.46.mlp.gate_proj                              24      0     1.112e-02    3.310e-03    1.160e-02
  model.layers.46.mlp.up_proj                                24      0     1.150e-02    3.598e-03    1.205e-02
  model.layers.47.mlp.down_proj                              24      0     1.427e-02    2.639e-03    1.451e-02
  model.layers.47.mlp.gate_proj                              24      0     1.056e-02    3.336e-03    1.107e-02
  model.layers.47.mlp.up_proj                                24      0     1.177e-02    3.624e-03    1.232e-02
  model.layers.47.self_attn.k_proj                           24      0     1.243e-02    2.468e-03    1.268e-02
  model.layers.47.self_attn.o_proj                           24      0     1.958e-02    2.554e-03    1.975e-02
  model.layers.47.self_attn.q_proj                           24      0     9.645e-03    2.054e-03    9.860e-03
  model.layers.47.self_attn.v_proj                           24      0     1.061e-02    2.234e-03    1.084e-02
  model.layers.48.linear_attn.in_proj_qkv                    24      0     1.272e-02    2.266e-03    1.292e-02
  model.layers.48.linear_attn.in_proj_z                      24      0     1.209e-02    1.649e-03    1.220e-02
  model.layers.48.linear_attn.out_proj                       24      0     1.710e-02    2.416e-03    1.727e-02
  model.layers.48.mlp.down_proj                              24      0     1.481e-02    2.681e-03    1.505e-02
  model.layers.48.mlp.gate_proj                              24      0     1.035e-02    3.332e-03    1.087e-02
  model.layers.48.mlp.up_proj                                24      0     1.169e-02    3.605e-03    1.223e-02
  model.layers.49.linear_attn.in_proj_qkv                    24      0     1.209e-02    2.281e-03    1.230e-02
  model.layers.49.linear_attn.in_proj_z                      24      0     1.152e-02    1.601e-03    1.163e-02
  model.layers.49.linear_attn.out_proj                       24      0     1.905e-02    2.170e-03    1.918e-02
  model.layers.49.mlp.down_proj                              24      0     1.568e-02    2.694e-03    1.591e-02
  model.layers.49.mlp.gate_proj                              24      0     1.006e-02    3.369e-03    1.061e-02
  model.layers.49.mlp.up_proj                                24      0     1.149e-02    3.564e-03    1.203e-02
  model.layers.5.linear_attn.in_proj_qkv                     24      0     1.246e-02    2.271e-03    1.266e-02
  model.layers.5.linear_attn.in_proj_z                       24      0     1.391e-02    1.909e-03    1.404e-02
  model.layers.5.linear_attn.out_proj                        24      0     1.771e-02    2.206e-03    1.785e-02
  model.layers.5.mlp.down_proj                               24      0     1.528e-02    2.518e-03    1.548e-02
  model.layers.5.mlp.gate_proj                               24      0     1.018e-02    3.535e-03    1.077e-02
  model.layers.5.mlp.up_proj                                 24      0     1.129e-02    3.621e-03    1.185e-02
  model.layers.50.linear_attn.in_proj_qkv                    24      0     1.167e-02    2.256e-03    1.188e-02
  model.layers.50.linear_attn.in_proj_z                      24      0     1.087e-02    1.486e-03    1.097e-02
  model.layers.50.linear_attn.out_proj                       24      0     1.808e-02    2.348e-03    1.823e-02
  model.layers.50.mlp.down_proj                              24      0     1.518e-02    2.665e-03    1.541e-02
  model.layers.50.mlp.gate_proj                              24      0     9.856e-03    3.283e-03    1.039e-02
  model.layers.50.mlp.up_proj                                24      0     1.087e-02    3.439e-03    1.140e-02
  model.layers.51.mlp.down_proj                              24      0     1.525e-02    2.653e-03    1.548e-02
  model.layers.51.mlp.gate_proj                              24      0     8.703e-03    3.392e-03    9.342e-03
  model.layers.51.mlp.up_proj                                24      0     1.112e-02    3.553e-03    1.167e-02
  model.layers.51.self_attn.k_proj                           24      0     1.104e-02    2.409e-03    1.130e-02
  model.layers.51.self_attn.o_proj                           24      0     1.923e-02    2.297e-03    1.937e-02
  model.layers.51.self_attn.q_proj                           24      0     9.125e-03    2.022e-03    9.347e-03
  model.layers.51.self_attn.v_proj                           24      0     1.025e-02    2.269e-03    1.050e-02
  model.layers.52.linear_attn.in_proj_qkv                    24      0     9.626e-03    2.112e-03    9.855e-03
  model.layers.52.linear_attn.in_proj_z                      24      0     8.540e-03    1.286e-03    8.638e-03
  model.layers.52.linear_attn.out_proj                       24      0     1.872e-02    2.304e-03    1.886e-02
  model.layers.52.mlp.down_proj                              24      0     1.783e-02    2.683e-03    1.803e-02
  model.layers.52.mlp.gate_proj                              24      0     8.150e-03    3.400e-03    8.832e-03
  model.layers.52.mlp.up_proj                                24      0     1.107e-02    3.565e-03    1.163e-02
  model.layers.53.linear_attn.in_proj_qkv                    24      0     9.506e-03    2.129e-03    9.742e-03
  model.layers.53.linear_attn.in_proj_z                      24      0     8.685e-03    1.312e-03    8.785e-03
  model.layers.53.linear_attn.out_proj                       24      0     1.688e-02    2.284e-03    1.704e-02
  model.layers.53.mlp.down_proj                              24      0     1.846e-02    2.770e-03    1.866e-02
  model.layers.53.mlp.gate_proj                              24      0     7.914e-03    3.405e-03    8.616e-03
  model.layers.53.mlp.up_proj                                24      0     1.098e-02    3.567e-03    1.154e-02
  model.layers.54.linear_attn.in_proj_qkv                    24      0     9.930e-03    2.233e-03    1.018e-02
  model.layers.54.linear_attn.in_proj_z                      24      0     8.296e-03    1.277e-03    8.394e-03
  model.layers.54.linear_attn.out_proj                       24      0     1.746e-02    2.133e-03    1.759e-02
  model.layers.54.mlp.down_proj                              24      0     1.830e-02    2.512e-03    1.847e-02
  model.layers.54.mlp.gate_proj                              24      0     7.284e-03    3.355e-03    8.020e-03
  model.layers.54.mlp.up_proj                                24      0     9.182e-03    3.420e-03    9.799e-03
  model.layers.55.mlp.down_proj                              24      0     1.699e-02    2.628e-03    1.720e-02
  model.layers.55.mlp.gate_proj                              24      0     7.234e-03    3.388e-03    7.989e-03
  model.layers.55.mlp.up_proj                                24      0     1.132e-02    3.604e-03    1.188e-02
  model.layers.55.self_attn.k_proj                           24      0     1.087e-02    2.402e-03    1.113e-02
  model.layers.55.self_attn.o_proj                           24      0     1.732e-02    2.198e-03    1.746e-02
  model.layers.55.self_attn.q_proj                           24      0     8.896e-03    2.104e-03    9.143e-03
  model.layers.55.self_attn.v_proj                           24      0     9.491e-03    2.219e-03    9.745e-03
  model.layers.56.linear_attn.in_proj_qkv                    24      0     8.996e-03    2.092e-03    9.236e-03
  model.layers.56.linear_attn.in_proj_z                      24      0     8.713e-03    1.329e-03    8.815e-03
  model.layers.56.linear_attn.out_proj                       24      0     1.730e-02    2.108e-03    1.743e-02
  model.layers.56.mlp.down_proj                              24      0     1.973e-02    2.530e-03    1.989e-02
  model.layers.56.mlp.gate_proj                              24      0     7.323e-03    3.347e-03    8.052e-03
  model.layers.56.mlp.up_proj                                24      0     1.132e-02    3.599e-03    1.188e-02
  model.layers.57.linear_attn.in_proj_qkv                    24      0     8.642e-03    2.054e-03    8.885e-03
  model.layers.57.linear_attn.in_proj_z                      24      0     8.515e-03    1.299e-03    8.613e-03
  model.layers.57.linear_attn.out_proj                       24      0     1.590e-02    2.507e-03    1.610e-02
  model.layers.57.mlp.down_proj                              24      0     1.637e-02    2.798e-03    1.661e-02
  model.layers.57.mlp.gate_proj                              24      0     7.134e-03    3.326e-03    7.872e-03
  model.layers.57.mlp.up_proj                                24      0     1.116e-02    3.599e-03    1.173e-02
  model.layers.58.linear_attn.in_proj_qkv                    24      0     9.192e-03    2.133e-03    9.437e-03
  model.layers.58.linear_attn.in_proj_z                      24      0     8.763e-03    1.313e-03    8.861e-03
  model.layers.58.linear_attn.out_proj                       24      0     1.892e-02    2.407e-03    1.908e-02
  model.layers.58.mlp.down_proj                              24      0     1.657e-02    2.620e-03    1.677e-02
  model.layers.58.mlp.gate_proj                              24      0     7.016e-03    3.317e-03    7.760e-03
  model.layers.58.mlp.up_proj                                24      0     1.044e-02    3.550e-03    1.103e-02
  model.layers.59.mlp.down_proj                              24      0     1.664e-02    2.527e-03    1.683e-02
  model.layers.59.mlp.gate_proj                              24      0     6.606e-03    3.308e-03    7.388e-03
  model.layers.59.mlp.up_proj                                24      0     9.285e-03    3.448e-03    9.903e-03
  model.layers.59.self_attn.k_proj                           24      0     1.020e-02    2.349e-03    1.046e-02
  model.layers.59.self_attn.o_proj                           24      0     1.804e-02    2.288e-03    1.819e-02
  model.layers.59.self_attn.q_proj                           24      0     8.661e-03    2.117e-03    8.916e-03
  model.layers.59.self_attn.v_proj                           24      0     9.120e-03    2.162e-03    9.376e-03
  model.layers.6.linear_attn.in_proj_qkv                     24      0     1.264e-02    2.270e-03    1.285e-02
  model.layers.6.linear_attn.in_proj_z                       24      0     1.425e-02    1.992e-03    1.439e-02
  model.layers.6.linear_attn.out_proj                        24      0     1.683e-02    2.298e-03    1.698e-02
  model.layers.6.mlp.down_proj                               24      0     1.354e-02    2.403e-03    1.376e-02
  model.layers.6.mlp.gate_proj                               24      0     1.046e-02    3.490e-03    1.102e-02
  model.layers.6.mlp.up_proj                                 24      0     1.113e-02    3.560e-03    1.169e-02
  model.layers.60.linear_attn.in_proj_qkv                    24      0     7.948e-03    1.975e-03    8.189e-03
  model.layers.60.linear_attn.in_proj_z                      24      0     7.501e-03    1.194e-03    7.596e-03
  model.layers.60.linear_attn.out_proj                       24      0     1.692e-02    2.505e-03    1.710e-02
  model.layers.60.mlp.down_proj                              24      0     1.584e-02    2.710e-03    1.607e-02
  model.layers.60.mlp.gate_proj                              24      0     6.830e-03    3.245e-03    7.562e-03
  model.layers.60.mlp.up_proj                                24      0     9.185e-03    3.392e-03    9.790e-03
  model.layers.61.linear_attn.in_proj_qkv                    24      0     8.884e-03    2.091e-03    9.125e-03
  model.layers.61.linear_attn.in_proj_z                      24      0     7.949e-03    1.247e-03    8.046e-03
  model.layers.61.linear_attn.out_proj                       24      0     1.666e-02    2.347e-03    1.682e-02
  model.layers.61.mlp.down_proj                              24      0     1.498e-02    2.662e-03    1.522e-02
  model.layers.61.mlp.gate_proj                              24      0     6.566e-03    3.199e-03    7.304e-03
  model.layers.61.mlp.up_proj                                24      0     8.718e-03    3.320e-03    9.330e-03
  model.layers.62.linear_attn.in_proj_qkv                    24      0     8.142e-03    1.974e-03    8.379e-03
  model.layers.62.linear_attn.in_proj_z                      24      0     7.383e-03    1.219e-03    7.483e-03
  model.layers.62.linear_attn.out_proj                       24      0     1.517e-02    2.360e-03    1.535e-02
  model.layers.62.mlp.down_proj                              24      0     1.491e-02    2.530e-03    1.512e-02
  model.layers.62.mlp.gate_proj                              24      0     6.260e-03    3.133e-03    6.999e-03
  model.layers.62.mlp.up_proj                                24      0     6.868e-03    3.148e-03    7.554e-03
  model.layers.63.mlp.down_proj                              24      0     1.188e-02    2.006e-03    1.205e-02
  model.layers.63.mlp.gate_proj                              24      0     5.645e-03    3.069e-03    6.425e-03
  model.layers.63.mlp.up_proj                                24      0     4.720e-03    3.052e-03    5.620e-03
  model.layers.63.self_attn.k_proj                           24      0     1.075e-02    2.411e-03    1.101e-02
  model.layers.63.self_attn.o_proj                           24      0     1.295e-02    2.015e-03    1.310e-02
  model.layers.63.self_attn.q_proj                           24      0     7.002e-03    1.936e-03    7.265e-03
  model.layers.63.self_attn.v_proj                           24      0     4.688e-03    1.728e-03    4.996e-03
  model.layers.7.mlp.down_proj                               24      0     1.422e-02    2.582e-03    1.446e-02
  model.layers.7.mlp.gate_proj                               24      0     1.041e-02    3.565e-03    1.100e-02
  model.layers.7.mlp.up_proj                                 24      0     1.135e-02    3.628e-03    1.192e-02
  model.layers.7.self_attn.k_proj                            24      0     1.384e-02    2.716e-03    1.410e-02
  model.layers.7.self_attn.o_proj                            24      0     1.326e-02    2.271e-03    1.346e-02
  model.layers.7.self_attn.q_proj                            24      0     1.250e-02    2.077e-03    1.267e-02
  model.layers.7.self_attn.v_proj                            24      0     1.103e-02    2.331e-03    1.128e-02
  model.layers.8.linear_attn.in_proj_qkv                     24      0     1.244e-02    2.249e-03    1.264e-02
  model.layers.8.linear_attn.in_proj_z                       24      0     1.306e-02    1.814e-03    1.318e-02
  model.layers.8.linear_attn.out_proj                        24      0     1.616e-02    2.279e-03    1.632e-02
  model.layers.8.mlp.down_proj                               24      0     1.507e-02    2.553e-03    1.528e-02
  model.layers.8.mlp.gate_proj                               24      0     1.050e-02    3.562e-03    1.109e-02
  model.layers.8.mlp.up_proj                                 24      0     1.129e-02    3.616e-03    1.186e-02
  model.layers.9.linear_attn.in_proj_qkv                     24      0     1.310e-02    2.236e-03    1.329e-02
  model.layers.9.linear_attn.in_proj_z                       24      0     1.360e-02    1.892e-03    1.373e-02
  model.layers.9.linear_attn.out_proj                        24      0     1.603e-02    2.266e-03    1.619e-02
  model.layers.9.mlp.down_proj                               24      0     1.344e-02    2.469e-03    1.366e-02
  model.layers.9.mlp.gate_proj                               24      0     1.058e-02    3.571e-03    1.117e-02
  model.layers.9.mlp.up_proj                                 24      0     1.119e-02    3.608e-03    1.175e-02

OP-LEVEL error against an fp32 oracle, per linear class, on each
arm's OWN activation -- upstream of the model's amplifier, and
paired by class.  Ratios are arm / INT8; below 1.0 is better.
  arm           classes  median rel err  median ratio   better/worse       p05       p95
  INT8              400      1.1973e-02        1.0000      0/400        1.0000    1.0000
  INT8G             400      8.5041e-03        0.6800    400/0          0.5310    0.8677
  INT8G256          400      8.8052e-03        0.7053    400/0          0.5545    0.8929
  INT8GN            400      1.1970e-02        1.0001    187/213        0.9981    1.0022
  A sign test on `better/worse` is the check that cannot be carried by
  one class: INT8 is its own row and must read exactly 1.0000 with
  every class on the boundary, or the pairing is not by class.
[i8] per-arm census (int8 kernel calls): {'REF': 0, 'INT8': 2000, 'INT8G': 2000, 'INT8G256': 2000, 'INT8GN': 2000}

ARM RECEIPTS -- what each arm actually executed, from the engine's
own per-call census.  fused/standalone is the reconstruct variant;
acc16/acc32 is the cuBLAS compute type leg B asked hgemm for.
  arm       legB calls          rows     fused  standalone     acc16     acc32   fp32pin
  REF             2000       4096000      2000           0      2000         0         0
  INT8            2000       4096000         0        2000      2000         0         0
  INT8G           2000       4096000         0        2000      2000         0         0
  INT8G256        2000       4096000         0        2000      2000         0         0
  INT8GN          2000       4096000         0        2000      2000         0         0
  all arms match their expected leg/accumulator signature
  prefill positions aligned: [1, 1, 1, 1, 1] per prompt, identical across all 5 arms

THE TAIL over PREFILL positions only -- one per prefill chunk, so
every arm scores the SAME context. Decode positions are reported
separately and are NOT a numerics measure: once a token flips, the
arms are reading different contexts. The control proves it -- pooled
over decode, STD (which contains no int8 at all) has the same max KL
as INT8, so a pooled tail cannot attribute anything to the kernel.

arm            n        mean         p50         p95         p99         MAX     argmax flip
REF            5   0.000e+00   0.000e+00   0.000e+00   0.000e+00   0.000e+00 0/5 =   0.00%
INT8           5   8.335e-04   3.453e-05   4.044e-03   4.044e-03   4.044e-03 0/5 =   0.00%
INT8G          5   4.730e-04   7.326e-05   2.167e-03   2.167e-03   2.167e-03 0/5 =   0.00%
INT8G256       5   3.846e-04   2.586e-05   1.783e-03   1.783e-03   1.783e-03 0/5 =   0.00%
INT8GN         5   5.314e-04   1.115e-04   2.259e-03   2.259e-03   2.259e-03 0/5 =   0.00%

Decode positions (divergence-contaminated, shown for completeness):
arm            n        mean         MAX    greedy agree
REF           40   0.000e+00   0.000e+00 40/40 =  100.0%
INT8          40   2.678e-03   7.068e-02 40/40 =  100.0%
INT8G         40   1.302e-03   1.891e-02 40/40 =  100.0%
INT8G256      40   1.206e-03   1.240e-02 40/40 =  100.0%
INT8GN        40   2.277e-03   5.430e-02 40/40 =  100.0%

SAMPLER-RELEVANT TAIL @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF            5     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/  5     0.00%
INT8           5     100.00%  7.035e-03  3.517e-02  3.517e-02    3.797e-03 0/  5     0.00%
INT8G          5     100.00%  5.170e-03  2.585e-02  2.585e-02    2.024e-03 1/  5     5.00%
INT8G256       5     100.00%  5.360e-03  2.680e-02  2.680e-02    1.750e-03 1/  5     5.00%
INT8GN         5     100.00%  4.610e-03  2.305e-02  2.305e-02    1.524e-03 1/  5     5.00%

SAMPLER-RELEVANT TAIL @ owner's k~40 (T=1.0, top_p=1.0, top_k=40), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF            5     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/  5     0.00%
INT8           5      96.63%  7.157e-03  3.476e-02  3.476e-02          inf 2/  5     5.00%
INT8G          5      95.70%  5.737e-03  2.601e-02  2.601e-02          inf 5/  5     5.00%
INT8G256       5      96.67%  5.648e-03  2.679e-02  2.679e-02          inf 4/  5     5.00%
INT8GN         5      96.68%  5.677e-03  2.426e-02  2.426e-02          inf 4/  5     7.50%

FULL-VOCAB numbers below, kept and labelled: they are the right
instrument for GREEDY or beam decode, where every rank matters.

arm         first-prefill-position KL       median          max
REF                         0.000e+00    0.000e+00    0.000e+00
INT8                        8.335e-04    3.453e-05    4.044e-03
INT8G                       4.730e-04    7.326e-05    2.167e-03
INT8G256                    3.846e-04    2.586e-05    1.783e-03
INT8GN                      5.314e-04    1.115e-04    2.259e-03

PAIRED TV @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions.
Each row is A scored against B -- B is that ROW's reference, not the
table's.  TV is the expected speculative accept-rate loss, so the
column is read directly in percentage points.  The interval is a
cluster bootstrap over PROMPTS; the IQR is the raw spread over
positions with no floor applied to it.

  A       vs B        n  mean TV pp           95% CI pp    p25 pp    p50 pp    p75 pp    p95 pp    max pp
  REF     INT8        5      0.7035 [  0.0000,  2.1104]    0.0000    0.0000    0.0000    3.5174    3.5174
  REF     INT8G       5      0.5170 [  0.0000,  1.5510]    0.0000    0.0000    0.0000    2.5850    2.5850
  REF     INT8G256     5      0.5360 [  0.0000,  1.6081]    0.0000    0.0000    0.0000    2.6801    2.6801
  REF     INT8GN      5      0.4610 [  0.0000,  1.3830]    0.0000    0.0000    0.0000    2.3050    2.3050
  INT8    INT8G       5      0.3411 [  0.0000,  1.0232]    0.0000    0.0000    0.0000    1.7054    1.7054
  INT8    INT8G256     5      0.3663 [  0.0000,  1.0990]    0.0000    0.0000    0.0000    1.8317    1.8317
  INT8    INT8GN      5      0.6918 [  0.0000,  2.0755]    0.0000    0.0000    0.0000    3.4591    3.4591
  INT8G   INT8G256     5      0.2158 [  0.0000,  0.6475]    0.0000    0.0000    0.0000    1.0792    1.0792
  INT8G   INT8GN      5      0.5055 [  0.0000,  1.5165]    0.0000    0.0000    0.0000    2.5274    2.5274
  INT8G256 INT8GN      5      0.4114 [  0.0000,  1.2343]    0.0000    0.0000    0.0000    2.0571    2.0571

  TV distance matrix, mean TV pp (symmetric):
          REF      INT8     INT8G  INT8G256    INT8GN
  REF        0.0000    0.7035    0.5170    0.5360    0.4610
  INT8       0.7035    0.0000    0.3411    0.3663    0.6918
  INT8G      0.5170    0.3411    0.0000    0.2158    0.5055
  INT8G256    0.5360    0.3663    0.2158    0.0000    0.4114
  INT8GN     0.4610    0.6918    0.5055    0.4114    0.0000

  TV distance matrix, median TV pp (symmetric):
          REF      INT8     INT8G  INT8G256    INT8GN
  REF        0.0000    0.0000    0.0000    0.0000    0.0000
  INT8       0.0000    0.0000    0.0000    0.0000    0.0000
  INT8G      0.0000    0.0000    0.0000    0.0000    0.0000
  INT8G256    0.0000    0.0000    0.0000    0.0000    0.0000
  INT8GN     0.0000    0.0000    0.0000    0.0000    0.0000

lm_head CUT -- the divergence entering the head vs leaving it,
prefill positions, relative rms.  'in' is the residual stream after
24 layers; 'out' is the centred logits.  amp = out/in: at 1.0 the
head passes the body's divergence through unchanged, above it the
head adds its own.

  A       vs B        n   rel rms IN  rel rms OUT      amp
  REF     INT8        5   4.7284e-02   4.2916e-02    0.908
  REF     INT8G       5   3.3018e-02   3.0876e-02    0.935
  REF     INT8G256     5   3.7253e-02   3.4803e-02    0.934
  REF     INT8GN      5   4.6732e-02   4.4051e-02    0.943
  INT8    INT8G       5   4.1388e-02   3.7687e-02    0.911
  INT8    INT8G256     5   4.3610e-02   3.9288e-02    0.901
  INT8    INT8GN      5   4.8930e-02   4.4789e-02    0.915
  INT8G   INT8G256     5   2.9814e-02   2.7520e-02    0.923
  INT8G   INT8GN      5   4.0335e-02   3.7854e-02    0.938
  INT8G256 INT8GN      5   4.1332e-02   3.8453e-02    0.930

PER-TOKEN AMAX -- is the crest factor of the quantised row what
selects the tail?  crest = amax/rms of the int8 GEMM's input row;
under the uniform-residual model the relative noise the row picks
up is crest/(127*sqrt(12)) = crest/440, and qerr is that noise
measured directly by round-tripping the scored row through int8.
arm supplying the statistics: INT8

  statistic                                  p50         p95         p99         max
  crest, scored row (max/linear)       1.136e+01   1.260e+01   1.260e+01   1.260e+01
  crest, any row in chunk              1.850e+01   1.909e+01   1.909e+01   1.909e+01
  qerr, scored row (rel rms)           2.590e-02   2.844e-02   2.844e-02   2.844e-02
  n positions: 5

What the per-token scale is paying for, over the whole census.
crest_g is the crest a per-128-group scale would leave --
sqrt(mean_b amax_b^2)/rms -- and its ratio to crest is the noise
reduction available.  conc is max/median of the per-128-block rms:
at 1 the row is flat and grouping buys nothing.
  statistic                                  p50         p95         p99         max
  crest_g (per-128 scale)                  2.621       2.891       2.954       3.031
  crest / crest_g (noise cut)              2.095       3.917       5.287       7.811
  block rms concentration                  3.161      10.624      21.538      39.597
  n (class, position) samples: 2000

The 10 worst INT8 positions, with the crest factor at each.  If the
per-token amax is the mechanism, these are the extreme-crest rows.
   prompt  chunk      INT8 KL   crest row  crest chunk        qerr  worst linear
        3      0    4.044e-03      10.934       18.497   2.474e-02  L6.linear_attn.out_proj
        0      0    4.701e-05      12.600       18.574   2.844e-02  L14.linear_attn.out_proj
        4      0    3.453e-05      11.634       17.648   2.659e-02  L3.mlp.down_proj
        1      0    2.484e-05      11.355       19.090   2.590e-02  L1.mlp.down_proj
        2      0    1.736e-05      11.185       17.200   2.517e-02  L2.linear_attn.out_proj
  ... and the 10 MEDIAN INT8 positions, as the contrast:
        4      0    3.453e-05      11.634       17.648   2.659e-02  L3.mlp.down_proj
        0      0    4.701e-05      12.600       18.574   2.844e-02  L14.linear_attn.out_proj
        3      0    4.044e-03      10.934       18.497   2.474e-02  L6.linear_attn.out_proj

Rank correlation of the crest factor with each arm's KL.  STD is the
NULL CONTROL: it runs no quantiser, so the crest factor cannot cause
its KL.  A crest that ranks STD as well as it ranks INT8 is ranking
hard POSITIONS, not the quantiser, and explains nothing.
  crest statistic                    REF       INT8      INT8G   INT8G256     INT8GN  INT8 excess
  crest, scored row               -0.400      0.000     -0.400     -0.300     -0.300
  crest, any row                  -0.600      0.300      0.500      0.700      0.200
  qerr, scored row                -0.400      0.000     -0.400     -0.300     -0.300

Mean crest of the scored row, by linear class -- the 15 heaviest.
A class well above the Gaussian-rotation expectation (~4.5 for a
128-wide Hadamard over K=5120) is a class the per-token scale is
wasting levels on.
  linear class                                        mean crest         p95         max
  L0.mlp.down_proj                                         9.124      11.151      11.151
  L5.mlp.down_proj                                         8.539      10.196      10.196
  L1.mlp.down_proj                                         8.187      11.355      11.355
  L14.linear_attn.out_proj                                 8.114      12.600      12.600
  L60.mlp.down_proj                                        7.996       9.458       9.458
  L2.mlp.down_proj                                         7.947       9.158       9.158
  L2.linear_attn.out_proj                                  7.850      11.185      11.185
  L45.linear_attn.out_proj                                 7.739       9.712       9.712
  L6.linear_attn.out_proj                                  7.482      10.934      10.934
  L17.linear_attn.out_proj                                 7.403       8.486       8.486
  L6.mlp.down_proj                                         7.398      10.311      10.311
  L49.linear_attn.out_proj                                 7.391       9.524       9.524
  L4.linear_attn.out_proj                                  7.335       8.433       8.433
  L37.linear_attn.out_proj                                 7.325       8.676       8.676
  L19.self_attn.o_proj                                     7.264       9.256       9.256
