[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (317092 > 262144). Running this sequence through the model will result in indexing errors
[i8] basket: 20 prompts x 16384 tok, domains ['code', 'longctx', 'math', 'sci', 'wiki']
boot: this card's residency could NOT be split between this process and any other — neither NVML's per-process walk nor a pre-context mark was available — so driver.cuda_context is being booked from the DEVICE-WIDE reading (397 MiB). If anything else is resident on this device, its bytes are inside that number and the KV pool is sized that much smaller for the life of this boot.
compile-cache MISS: xgrammar — xgrammar compiled grammars (re)compiles this boot [/cache/xgrammar]
VRAM at profile: this card's residency could NOT be split between this process and any other — NVML's per-process walk did not identify us — so the whole 14.26 GiB resident on device 0 is being booked as driver.cuda_context. Any co-tenant's bytes are inside that number and KV is sized that much smaller for this boot.
[TKV autotune-provenance] fp=57911e26f4e47f74 regime=cache-hit cells=8 sweep_src=2772549a0ef713db tile_tokens=4 num_splits=32,64,128,256 min_blocks_per_sm=0,3 resolved=0 held=0 inherited=0
/opt/venv/lib/python3.12/site-packages/torch/__init__.py:1172: FutureWarning: `torch.distributed.reduce_op` is deprecated, please use `torch.distributed.ReduceOp` instead
  return isinstance(obj, torch.Tensor)
[i8] trellis->path map: 401 linears
JIT compile AFTER serving-ready [cpp_ext]: exl3_i8_gemm cached .so load — a live request paid this compile's latency. This is a boot-warmup coverage gap: extend warmup to pre-compile this kernel/specialization. Counter jit_compile_serving (must-not-fire) at GET /v1/admin/flag_truth.
[i8] prefix cache: server default resolved=True, per-request cache_enabled=False; any cache hit below is refused
activation admission RE-ARMED (JIT compile after serving-ready): the step budget FELL to 6397 MiB, -2 MiB on the 6399 MiB it was enforcing and 6399 MiB the boot reading armed. The widest slate admission will build is narrower from here — narrower prefill chunks and deferred rows instead of a forward that OOMs after the layout freeze.
driver serving-growth OVER RESERVE: the driver holds 2 MiB more than it did when the boot brackets closed, against the 0 MiB the serving floor's transient.serving_step.driver_growth term held for it. Those bytes are in no pool and no allocator counter -- the live card reaches them as driver.modules_serving and the boot ledger only as driver.residual -- and the KV layout is frozen, so the overage comes out of the free VRAM the floor is holding for one in-flight step (the verify tail, the DFlash context-assemble) and those reserves are the ones that fail first. The reading is on record for this configuration, so the next boot holds it; this process serves the rest of its life short by the overage.
activation admission RE-ARMED (driver residency over reserve): the step budget FELL to 5751 MiB, -646 MiB on the 6397 MiB it was enforcing and 6399 MiB the boot reading armed. The widest slate admission will build is narrower from here — narrower prefill chunks and deferred rows instead of a forward that OOMs after the layout freeze.
[i8] warmup done
JIT compile AFTER serving-ready [cpp_ext]: tkv_unified_splitk_g6_v8_k4_v4_m1_v1_smem_rngfull_lbmb3_gms1_mw8_tt4_sh1_rc4_pf1_ldbypass cached .so load — a live request paid this compile's latency. This is a boot-warmup coverage gap: extend warmup to pre-compile this kernel/specialization. Counter jit_compile_serving (must-not-fire) at GET /v1/admin/flag_truth.
  [0] code           REF      16384tok  wall=  5.06s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [0] code           REF2     16384tok  wall=  5.04s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [0] code           STD      16384tok  wall=  5.28s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [0] code           REF32    16384tok  wall=  6.80s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [0] code           STD32    16384tok  wall=  7.05s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [0] code           INT8     16384tok  wall=  3.77s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [1] code           REF      16384tok  wall=  5.05s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [1] code           REF2     16384tok  wall=  5.05s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [1] code           STD      16384tok  wall=  5.34s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [1] code           REF32    16384tok  wall=  6.85s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [1] code           STD32    16384tok  wall=  7.08s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [1] code           INT8     16384tok  wall=  3.80s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [2] code           REF      16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [2] code           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [2] code           STD      16384tok  wall=  5.34s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [2] code           REF32    16384tok  wall=  6.85s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [2] code           STD32    16384tok  wall=  7.09s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [2] code           INT8     16384tok  wall=  3.79s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [3] code           REF      16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [3] code           REF2     16384tok  wall=  5.09s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [3] code           STD      16384tok  wall=  5.36s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [3] code           REF32    16384tok  wall=  6.86s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [3] code           STD32    16384tok  wall=  7.10s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [3] code           INT8     16384tok  wall=  3.79s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [4] wiki           REF      16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [4] wiki           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [4] wiki           STD      16384tok  wall=  5.34s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [4] wiki           REF32    16384tok  wall=  6.86s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [4] wiki           STD32    16384tok  wall=  7.10s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [4] wiki           INT8     16384tok  wall=  3.80s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [5] wiki           REF      16384tok  wall=  5.09s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [5] wiki           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [5] wiki           STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [5] wiki           REF32    16384tok  wall=  6.86s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [5] wiki           STD32    16384tok  wall=  7.10s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [5] wiki           INT8     16384tok  wall=  3.79s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [6] wiki           REF      16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [6] wiki           REF2     16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [6] wiki           STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [6] wiki           REF32    16384tok  wall=  6.85s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [6] wiki           STD32    16384tok  wall=  7.11s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [6] wiki           INT8     16384tok  wall=  3.79s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [7] wiki           REF      16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [7] wiki           REF2     16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [7] wiki           STD      16384tok  wall=  5.36s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [7] wiki           REF32    16384tok  wall=  6.87s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [7] wiki           STD32    16384tok  wall=  7.12s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [7] wiki           INT8     16384tok  wall=  3.82s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [8] math           REF      16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [8] math           REF2     16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [8] math           STD      16384tok  wall=  5.36s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [8] math           REF32    16384tok  wall=  6.87s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [8] math           STD32    16384tok  wall=  7.10s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [8] math           INT8     16384tok  wall=  3.80s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [9] math           REF      16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [9] math           REF2     16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [9] math           STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [9] math           REF32    16384tok  wall=  6.86s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [9] math           STD32    16384tok  wall=  7.10s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [9] math           INT8     16384tok  wall=  3.79s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [10] math           REF      16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [10] math           REF2     16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [10] math           STD      16384tok  wall=  5.35s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [10] math           REF32    16384tok  wall=  6.87s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [10] math           STD32    16384tok  wall=  7.10s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [10] math           INT8     16384tok  wall=  3.80s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [11] math           REF      16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [11] math           REF2     16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [11] math           STD      16384tok  wall=  5.44s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [11] math           REF32    16384tok  wall=  6.87s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [11] math           STD32    16384tok  wall=  7.11s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [11] math           INT8     16384tok  wall=  3.80s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [12] longctx        REF      16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [12] longctx        REF2     16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [12] longctx        STD      16384tok  wall=  5.36s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [12] longctx        REF32    16384tok  wall=  6.88s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [12] longctx        STD32    16384tok  wall=  7.11s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [12] longctx        INT8     16384tok  wall=  3.80s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [13] longctx        REF      16384tok  wall=  5.13s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [13] longctx        REF2     16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [13] longctx        STD      16384tok  wall=  5.36s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [13] longctx        REF32    16384tok  wall=  6.87s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [13] longctx        STD32    16384tok  wall=  7.11s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [13] longctx        INT8     16384tok  wall=  3.80s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [14] longctx        REF      16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [14] longctx        REF2     16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [14] longctx        STD      16384tok  wall=  5.37s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [14] longctx        REF32    16384tok  wall=  6.87s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [14] longctx        STD32    16384tok  wall=  7.11s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [14] longctx        INT8     16384tok  wall=  3.79s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [15] longctx        REF      16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [15] longctx        REF2     16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [15] longctx        STD      16384tok  wall=  5.37s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [15] longctx        REF32    16384tok  wall=  6.87s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [15] longctx        STD32    16384tok  wall=  7.12s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [15] longctx        INT8     16384tok  wall=  3.82s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [16] sci            REF      16384tok  wall=  5.13s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [16] sci            REF2     16384tok  wall=  5.14s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [16] sci            STD      16384tok  wall=  5.39s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [16] sci            REF32    16384tok  wall=  6.89s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [16] sci            STD32    16384tok  wall=  7.11s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [16] sci            INT8     16384tok  wall=  3.82s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [17] sci            REF      16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [17] sci            REF2     16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [17] sci            STD      16384tok  wall=  5.38s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [17] sci            REF32    16384tok  wall=  6.88s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [17] sci            STD32    16384tok  wall=  7.11s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [17] sci            INT8     16384tok  wall=  3.80s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [18] sci            REF      16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [18] sci            REF2     16384tok  wall=  5.12s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [18] sci            STD      16384tok  wall=  5.38s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [18] sci            REF32    16384tok  wall=  6.87s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [18] sci            STD32    16384tok  wall=  7.10s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [18] sci            INT8     16384tok  wall=  3.80s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [19] sci            REF      16384tok  wall=  5.10s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [19] sci            REF2     16384tok  wall=  5.11s  finish=length  leg=3200F/0S acc=3200x16/0x32
  [19] sci            STD      16384tok  wall=  5.36s  finish=length  leg=0F/3200S acc=3200x16/0x32
  [19] sci            REF32    16384tok  wall=  6.86s  finish=length  leg=3200F/0S acc=0x16/3200x32
  [19] sci            STD32    16384tok  wall=  7.10s  finish=length  leg=0F/3200S acc=0x16/3200x32
  [19] sci            INT8     16384tok  wall=  3.80s  finish=length  leg=0F/3200S acc=3200x16/0x32

[i8] kernel census: int8 calls=64000 rows=131072000 reconstructs SKIPPED=64000 fallback (non-4bpw) calls=0
[i8] shapes served by the kernel (K,N)->calls: {(5120, 1024): 5120, (5120, 6144): 7680, (5120, 10240): 7680, (5120, 12288): 2560, (5120, 17408): 20480, (6144, 5120): 10240, (17408, 5120): 10240}
[i8] excluded projections: none  excluded calls=0

  linear class                                           served   excl  int8 vs fp32 legB vs fp32 int8 vs legB
  model.layers.0.linear_attn.in_proj_qkv                    161      0     4.827e-03    1.713e-03    5.132e-03
  model.layers.0.linear_attn.in_proj_z                      161      0     6.316e-03    1.129e-03    6.418e-03
  model.layers.0.linear_attn.out_proj                       161      0     1.249e-02    2.189e-03    1.268e-02
  model.layers.0.mlp.down_proj                              161      0     2.122e-02    2.609e-03    2.137e-02
  model.layers.0.mlp.gate_proj                              161      0     9.121e-03    3.413e-03    9.739e-03
  model.layers.0.mlp.up_proj                                161      0     9.082e-03    3.418e-03    9.702e-03
  model.layers.1.linear_attn.in_proj_qkv                    161      0     1.186e-02    2.166e-03    1.206e-02
  model.layers.1.linear_attn.in_proj_z                      161      0     1.361e-02    1.840e-03    1.373e-02
  model.layers.1.linear_attn.out_proj                       161      0     1.604e-02    2.254e-03    1.619e-02
  model.layers.1.mlp.down_proj                              161      0     1.344e-02    2.322e-03    1.364e-02
  model.layers.1.mlp.gate_proj                              161      0     9.700e-03    3.460e-03    1.030e-02
  model.layers.1.mlp.up_proj                                161      0     1.015e-02    3.488e-03    1.073e-02
  model.layers.10.linear_attn.in_proj_qkv                   161      0     1.304e-02    2.255e-03    1.324e-02
  model.layers.10.linear_attn.in_proj_z                     161      0     1.298e-02    1.836e-03    1.311e-02
  model.layers.10.linear_attn.out_proj                      161      0     1.497e-02    2.120e-03    1.512e-02
  model.layers.10.mlp.down_proj                             161      0     1.268e-02    2.348e-03    1.289e-02
  model.layers.10.mlp.gate_proj                             161      0     1.035e-02    3.581e-03    1.095e-02
  model.layers.10.mlp.up_proj                               161      0     1.076e-02    3.588e-03    1.134e-02
  model.layers.11.mlp.down_proj                             161      0     1.333e-02    2.506e-03    1.356e-02
  model.layers.11.mlp.gate_proj                             161      0     1.027e-02    3.507e-03    1.085e-02
  model.layers.11.mlp.up_proj                               161      0     1.088e-02    3.576e-03    1.145e-02
  model.layers.11.self_attn.k_proj                          161      0     1.330e-02    2.705e-03    1.357e-02
  model.layers.11.self_attn.o_proj                          161      0     1.304e-02    2.370e-03    1.326e-02
  model.layers.11.self_attn.q_proj                          161      0     1.174e-02    2.022e-03    1.191e-02
  model.layers.11.self_attn.v_proj                          161      0     1.037e-02    2.283e-03    1.062e-02
  model.layers.12.linear_attn.in_proj_qkv                   161      0     1.292e-02    2.244e-03    1.312e-02
  model.layers.12.linear_attn.in_proj_z                     161      0     1.365e-02    1.892e-03    1.378e-02
  model.layers.12.linear_attn.out_proj                      161      0     1.583e-02    2.138e-03    1.597e-02
  model.layers.12.mlp.down_proj                             161      0     1.316e-02    2.550e-03    1.341e-02
  model.layers.12.mlp.gate_proj                             161      0     1.002e-02    3.472e-03    1.061e-02
  model.layers.12.mlp.up_proj                               161      0     1.072e-02    3.553e-03    1.129e-02
  model.layers.13.linear_attn.in_proj_qkv                   161      0     1.292e-02    2.241e-03    1.312e-02
  model.layers.13.linear_attn.in_proj_z                     161      0     1.418e-02    1.970e-03    1.431e-02
  model.layers.13.linear_attn.out_proj                      161      0     1.680e-02    2.175e-03    1.693e-02
  model.layers.13.mlp.down_proj                             161      0     1.373e-02    2.565e-03    1.396e-02
  model.layers.13.mlp.gate_proj                             161      0     1.007e-02    3.483e-03    1.066e-02
  model.layers.13.mlp.up_proj                               161      0     1.071e-02    3.557e-03    1.128e-02
  model.layers.14.linear_attn.in_proj_qkv                   161      0     1.272e-02    2.232e-03    1.291e-02
  model.layers.14.linear_attn.in_proj_z                     161      0     1.334e-02    1.883e-03    1.348e-02
  model.layers.14.linear_attn.out_proj                      161      0     1.586e-02    2.214e-03    1.601e-02
  model.layers.14.mlp.down_proj                             161      0     1.300e-02    2.463e-03    1.323e-02
  model.layers.14.mlp.gate_proj                             161      0     9.963e-03    3.503e-03    1.056e-02
  model.layers.14.mlp.up_proj                               161      0     1.064e-02    3.565e-03    1.122e-02
  model.layers.15.mlp.down_proj                             161      0     1.349e-02    2.537e-03    1.373e-02
  model.layers.15.mlp.gate_proj                             161      0     1.010e-02    3.488e-03    1.068e-02
  model.layers.15.mlp.up_proj                               161      0     1.068e-02    3.562e-03    1.125e-02
  model.layers.15.self_attn.k_proj                          161      0     1.452e-02    2.890e-03    1.481e-02
  model.layers.15.self_attn.o_proj                          161      0     1.398e-02    2.117e-03    1.414e-02
  model.layers.15.self_attn.q_proj                          161      0     1.302e-02    2.087e-03    1.318e-02
  model.layers.15.self_attn.v_proj                          161      0     9.832e-03    2.228e-03    1.008e-02
  model.layers.16.linear_attn.in_proj_qkv                   161      0     1.289e-02    2.246e-03    1.308e-02
  model.layers.16.linear_attn.in_proj_z                     161      0     1.353e-02    1.895e-03    1.366e-02
  model.layers.16.linear_attn.out_proj                      161      0     1.636e-02    2.275e-03    1.651e-02
  model.layers.16.mlp.down_proj                             161      0     1.298e-02    2.519e-03    1.322e-02
  model.layers.16.mlp.gate_proj                             161      0     1.006e-02    3.472e-03    1.064e-02
  model.layers.16.mlp.up_proj                               161      0     1.070e-02    3.563e-03    1.128e-02
  model.layers.17.linear_attn.in_proj_qkv                   161      0     1.287e-02    2.222e-03    1.306e-02
  model.layers.17.linear_attn.in_proj_z                     161      0     1.352e-02    1.896e-03    1.366e-02
  model.layers.17.linear_attn.out_proj                      161      0     1.781e-02    2.245e-03    1.796e-02
  model.layers.17.mlp.down_proj                             161      0     1.381e-02    2.455e-03    1.403e-02
  model.layers.17.mlp.gate_proj                             161      0     1.010e-02    3.471e-03    1.068e-02
  model.layers.17.mlp.up_proj                               161      0     1.072e-02    3.565e-03    1.129e-02
  model.layers.18.linear_attn.in_proj_qkv                   161      0     1.206e-02    2.189e-03    1.225e-02
  model.layers.18.linear_attn.in_proj_z                     161      0     1.264e-02    1.788e-03    1.276e-02
  model.layers.18.linear_attn.out_proj                      161      0     1.527e-02    2.227e-03    1.543e-02
  model.layers.18.mlp.down_proj                             161      0     1.461e-02    2.424e-03    1.481e-02
  model.layers.18.mlp.gate_proj                             161      0     1.024e-02    3.422e-03    1.079e-02
  model.layers.18.mlp.up_proj                               161      0     1.058e-02    3.532e-03    1.116e-02
  model.layers.19.mlp.down_proj                             161      0     1.585e-02    2.536e-03    1.605e-02
  model.layers.19.mlp.gate_proj                             161      0     1.008e-02    3.395e-03    1.063e-02
  model.layers.19.mlp.up_proj                               161      0     1.096e-02    3.581e-03    1.153e-02
  model.layers.19.self_attn.k_proj                          161      0     1.407e-02    2.788e-03    1.434e-02
  model.layers.19.self_attn.o_proj                          161      0     1.814e-02    2.166e-03    1.826e-02
  model.layers.19.self_attn.q_proj                          161      0     9.500e-03    1.927e-03    9.692e-03
  model.layers.19.self_attn.v_proj                          161      0     1.047e-02    2.352e-03    1.073e-02
  model.layers.2.linear_attn.in_proj_qkv                    161      0     1.249e-02    2.213e-03    1.268e-02
  model.layers.2.linear_attn.in_proj_z                      161      0     1.398e-02    1.926e-03    1.411e-02
  model.layers.2.linear_attn.out_proj                       161      0     1.794e-02    2.322e-03    1.808e-02
  model.layers.2.mlp.down_proj                              161      0     1.706e-02    2.372e-03    1.723e-02
  model.layers.2.mlp.gate_proj                              161      0     1.029e-02    3.530e-03    1.088e-02
  model.layers.2.mlp.up_proj                                161      0     1.061e-02    3.561e-03    1.120e-02
  model.layers.20.linear_attn.in_proj_qkv                   161      0     1.215e-02    2.138e-03    1.234e-02
  model.layers.20.linear_attn.in_proj_z                     161      0     1.272e-02    1.791e-03    1.285e-02
  model.layers.20.linear_attn.out_proj                      161      0     1.578e-02    2.229e-03    1.594e-02
  model.layers.20.mlp.down_proj                             161      0     1.592e-02    2.582e-03    1.613e-02
  model.layers.20.mlp.gate_proj                             161      0     1.009e-02    3.407e-03    1.065e-02
  model.layers.20.mlp.up_proj                               161      0     1.128e-02    3.604e-03    1.184e-02
  model.layers.21.linear_attn.in_proj_qkv                   161      0     1.198e-02    2.201e-03    1.218e-02
  model.layers.21.linear_attn.in_proj_z                     161      0     1.162e-02    1.671e-03    1.174e-02
  model.layers.21.linear_attn.out_proj                      161      0     1.703e-02    2.236e-03    1.717e-02
  model.layers.21.mlp.down_proj                             161      0     1.719e-02    2.592e-03    1.739e-02
  model.layers.21.mlp.gate_proj                             161      0     1.010e-02    3.421e-03    1.066e-02
  model.layers.21.mlp.up_proj                               161      0     1.124e-02    3.609e-03    1.180e-02
  model.layers.22.linear_attn.in_proj_qkv                   161      0     1.201e-02    2.240e-03    1.221e-02
  model.layers.22.linear_attn.in_proj_z                     161      0     1.202e-02    1.714e-03    1.214e-02
  model.layers.22.linear_attn.out_proj                      161      0     1.580e-02    2.304e-03    1.597e-02
  model.layers.22.mlp.down_proj                             161      0     1.511e-02    2.552e-03    1.533e-02
  model.layers.22.mlp.gate_proj                             161      0     1.054e-02    3.140e-03    1.100e-02
  model.layers.22.mlp.up_proj                               161      0     1.140e-02    3.621e-03    1.196e-02
  model.layers.23.mlp.down_proj                             161      0     1.568e-02    2.553e-03    1.589e-02
  model.layers.23.mlp.gate_proj                             161      0     1.009e-02    3.367e-03    1.063e-02
  model.layers.23.mlp.up_proj                               161      0     1.141e-02    3.621e-03    1.197e-02
  model.layers.23.self_attn.k_proj                          161      0     1.382e-02    2.675e-03    1.408e-02
  model.layers.23.self_attn.o_proj                          161      0     1.681e-02    2.423e-03    1.698e-02
  model.layers.23.self_attn.q_proj                          161      0     9.476e-03    1.964e-03    9.678e-03
  model.layers.23.self_attn.v_proj                          161      0     1.034e-02    2.335e-03    1.061e-02
  model.layers.24.linear_attn.in_proj_qkv                   161      0     1.188e-02    2.253e-03    1.210e-02
  model.layers.24.linear_attn.in_proj_z                     161      0     1.223e-02    1.734e-03    1.235e-02
  model.layers.24.linear_attn.out_proj                      161      0     1.604e-02    2.270e-03    1.620e-02
  model.layers.24.mlp.down_proj                             161      0     1.459e-02    2.620e-03    1.482e-02
  model.layers.24.mlp.gate_proj                             161      0     1.026e-02    3.316e-03    1.078e-02
  model.layers.24.mlp.up_proj                               161      0     1.144e-02    3.612e-03    1.199e-02
  model.layers.25.linear_attn.in_proj_qkv                   161      0     1.141e-02    2.212e-03    1.162e-02
  model.layers.25.linear_attn.in_proj_z                     161      0     1.279e-02    1.788e-03    1.291e-02
  model.layers.25.linear_attn.out_proj                      161      0     1.475e-02    2.178e-03    1.491e-02
  model.layers.25.mlp.down_proj                             161      0     1.391e-02    2.553e-03    1.414e-02
  model.layers.25.mlp.gate_proj                             161      0     1.054e-02    3.323e-03    1.105e-02
  model.layers.25.mlp.up_proj                               161      0     1.130e-02    3.601e-03    1.185e-02
  model.layers.26.linear_attn.in_proj_qkv                   161      0     1.151e-02    2.236e-03    1.172e-02
  model.layers.26.linear_attn.in_proj_z                     161      0     1.259e-02    1.796e-03    1.272e-02
  model.layers.26.linear_attn.out_proj                      161      0     1.494e-02    2.342e-03    1.512e-02
  model.layers.26.mlp.down_proj                             161      0     1.268e-02    2.467e-03    1.292e-02
  model.layers.26.mlp.gate_proj                             161      0     1.112e-02    3.295e-03    1.160e-02
  model.layers.26.mlp.up_proj                               161      0     1.085e-02    3.549e-03    1.142e-02
  model.layers.27.mlp.down_proj                             161      0     1.335e-02    2.577e-03    1.360e-02
  model.layers.27.mlp.gate_proj                             161      0     1.080e-02    3.204e-03    1.126e-02
  model.layers.27.mlp.up_proj                               161      0     1.090e-02    3.553e-03    1.147e-02
  model.layers.27.self_attn.k_proj                          161      0     1.215e-02    2.501e-03    1.240e-02
  model.layers.27.self_attn.o_proj                          161      0     1.711e-02    2.742e-03    1.733e-02
  model.layers.27.self_attn.q_proj                          161      0     1.021e-02    2.059e-03    1.042e-02
  model.layers.27.self_attn.v_proj                          161      0     9.828e-03    2.250e-03    1.009e-02
  model.layers.28.linear_attn.in_proj_qkv                   161      0     1.179e-02    2.219e-03    1.200e-02
  model.layers.28.linear_attn.in_proj_z                     161      0     1.290e-02    1.818e-03    1.302e-02
  model.layers.28.linear_attn.out_proj                      161      0     1.477e-02    2.298e-03    1.495e-02
  model.layers.28.mlp.down_proj                             161      0     1.301e-02    2.596e-03    1.327e-02
  model.layers.28.mlp.gate_proj                             161      0     1.104e-02    3.175e-03    1.149e-02
  model.layers.28.mlp.up_proj                               161      0     1.084e-02    3.538e-03    1.141e-02
  model.layers.29.linear_attn.in_proj_qkv                   161      0     1.205e-02    2.246e-03    1.226e-02
  model.layers.29.linear_attn.in_proj_z                     161      0     1.395e-02    1.964e-03    1.408e-02
  model.layers.29.linear_attn.out_proj                      161      0     1.658e-02    2.351e-03    1.674e-02
  model.layers.29.mlp.down_proj                             161      0     1.346e-02    2.623e-03    1.371e-02
  model.layers.29.mlp.gate_proj                             161      0     1.115e-02    3.186e-03    1.159e-02
  model.layers.29.mlp.up_proj                               161      0     1.080e-02    3.540e-03    1.137e-02
  model.layers.3.mlp.down_proj                              161      0     1.415e-02    2.578e-03    1.438e-02
  model.layers.3.mlp.gate_proj                              161      0     1.017e-02    3.527e-03    1.076e-02
  model.layers.3.mlp.up_proj                                161      0     1.100e-02    3.601e-03    1.158e-02
  model.layers.3.self_attn.k_proj                           161      0     1.247e-02    2.581e-03    1.273e-02
  model.layers.3.self_attn.o_proj                           161      0     1.495e-02    2.177e-03    1.511e-02
  model.layers.3.self_attn.q_proj                           161      0     1.169e-02    2.033e-03    1.187e-02
  model.layers.3.self_attn.v_proj                           161      0     1.263e-02    2.305e-03    1.285e-02
  model.layers.30.linear_attn.in_proj_qkv                   161      0     1.190e-02    2.231e-03    1.211e-02
  model.layers.30.linear_attn.in_proj_z                     161      0     1.349e-02    1.888e-03    1.362e-02
  model.layers.30.linear_attn.out_proj                      161      0     1.621e-02    2.413e-03    1.639e-02
  model.layers.30.mlp.down_proj                             161      0     1.287e-02    2.561e-03    1.312e-02
  model.layers.30.mlp.gate_proj                             161      0     1.172e-02    3.189e-03    1.214e-02
  model.layers.30.mlp.up_proj                               161      0     1.067e-02    3.520e-03    1.123e-02
  model.layers.31.mlp.down_proj                             161      0     1.317e-02    2.594e-03    1.342e-02
  model.layers.31.mlp.gate_proj                             161      0     1.148e-02    3.154e-03    1.191e-02
  model.layers.31.mlp.up_proj                               161      0     1.090e-02    3.518e-03    1.145e-02
  model.layers.31.self_attn.k_proj                          161      0     1.207e-02    2.525e-03    1.233e-02
  model.layers.31.self_attn.o_proj                          161      0     2.213e-02    3.026e-03    2.233e-02
  model.layers.31.self_attn.q_proj                          161      0     1.016e-02    2.073e-03    1.037e-02
  model.layers.31.self_attn.v_proj                          161      0     9.914e-03    2.186e-03    1.014e-02
  model.layers.32.linear_attn.in_proj_qkv                   161      0     1.282e-02    2.288e-03    1.302e-02
  model.layers.32.linear_attn.in_proj_z                     161      0     1.412e-02    1.983e-03    1.425e-02
  model.layers.32.linear_attn.out_proj                      161      0     1.564e-02    2.301e-03    1.581e-02
  model.layers.32.mlp.down_proj                             161      0     1.303e-02    2.618e-03    1.329e-02
  model.layers.32.mlp.gate_proj                             161      0     1.166e-02    3.107e-03    1.206e-02
  model.layers.32.mlp.up_proj                               161      0     1.108e-02    3.524e-03    1.163e-02
  model.layers.33.linear_attn.in_proj_qkv                   161      0     1.261e-02    2.298e-03    1.282e-02
  model.layers.33.linear_attn.in_proj_z                     161      0     1.385e-02    1.990e-03    1.399e-02
  model.layers.33.linear_attn.out_proj                      161      0     1.747e-02    2.364e-03    1.763e-02
  model.layers.33.mlp.down_proj                             161      0     1.343e-02    2.602e-03    1.368e-02
  model.layers.33.mlp.gate_proj                             161      0     1.191e-02    3.112e-03    1.231e-02
  model.layers.33.mlp.up_proj                               161      0     1.124e-02    3.540e-03    1.178e-02
  model.layers.34.linear_attn.in_proj_qkv                   161      0     1.252e-02    2.276e-03    1.273e-02
  model.layers.34.linear_attn.in_proj_z                     161      0     1.341e-02    1.911e-03    1.355e-02
  model.layers.34.linear_attn.out_proj                      161      0     1.617e-02    2.364e-03    1.634e-02
  model.layers.34.mlp.down_proj                             161      0     1.379e-02    2.611e-03    1.404e-02
  model.layers.34.mlp.gate_proj                             161      0     1.206e-02    3.070e-03    1.244e-02
  model.layers.34.mlp.up_proj                               161      0     1.115e-02    3.506e-03    1.169e-02
  model.layers.35.mlp.down_proj                             161      0     1.417e-02    2.560e-03    1.441e-02
  model.layers.35.mlp.gate_proj                             161      0     1.066e-02    3.067e-03    1.109e-02
  model.layers.35.mlp.up_proj                               161      0     1.138e-02    3.579e-03    1.193e-02
  model.layers.35.self_attn.k_proj                          161      0     1.175e-02    2.504e-03    1.202e-02
  model.layers.35.self_attn.o_proj                          161      0     1.991e-02    2.804e-03    2.011e-02
  model.layers.35.self_attn.q_proj                          161      0     1.072e-02    2.115e-03    1.093e-02
  model.layers.35.self_attn.v_proj                          161      0     1.073e-02    2.262e-03    1.096e-02
  model.layers.36.linear_attn.in_proj_qkv                   161      0     1.294e-02    2.274e-03    1.314e-02
  model.layers.36.linear_attn.in_proj_z                     161      0     1.210e-02    1.745e-03    1.223e-02
  model.layers.36.linear_attn.out_proj                      161      0     1.662e-02    2.261e-03    1.678e-02
  model.layers.36.mlp.down_proj                             161      0     1.508e-02    2.627e-03    1.531e-02
  model.layers.36.mlp.gate_proj                             161      0     1.025e-02    3.143e-03    1.072e-02
  model.layers.36.mlp.up_proj                               161      0     1.154e-02    3.605e-03    1.209e-02
  model.layers.37.linear_attn.in_proj_qkv                   161      0     1.223e-02    2.241e-03    1.243e-02
  model.layers.37.linear_attn.in_proj_z                     161      0     1.059e-02    1.597e-03    1.071e-02
  model.layers.37.linear_attn.out_proj                      161      0     1.839e-02    2.197e-03    1.852e-02
  model.layers.37.mlp.down_proj                             161      0     1.540e-02    2.731e-03    1.564e-02
  model.layers.37.mlp.gate_proj                             161      0     1.056e-02    3.237e-03    1.104e-02
  model.layers.37.mlp.up_proj                               161      0     1.173e-02    3.628e-03    1.228e-02
  model.layers.38.linear_attn.in_proj_qkv                   161      0     1.200e-02    2.317e-03    1.222e-02
  model.layers.38.linear_attn.in_proj_z                     161      0     1.099e-02    1.628e-03    1.111e-02
  model.layers.38.linear_attn.out_proj                      161      0     1.656e-02    2.310e-03    1.672e-02
  model.layers.38.mlp.down_proj                             161      0     1.595e-02    2.699e-03    1.617e-02
  model.layers.38.mlp.gate_proj                             161      0     1.069e-02    3.068e-03    1.112e-02
  model.layers.38.mlp.up_proj                               161      0     1.171e-02    3.638e-03    1.226e-02
  model.layers.39.mlp.down_proj                             161      0     1.439e-02    2.656e-03    1.463e-02
  model.layers.39.mlp.gate_proj                             161      0     9.970e-03    3.161e-03    1.046e-02
  model.layers.39.mlp.up_proj                               161      0     1.179e-02    3.647e-03    1.234e-02
  model.layers.39.self_attn.k_proj                          161      0     1.178e-02    2.463e-03    1.204e-02
  model.layers.39.self_attn.o_proj                          161      0     2.078e-02    2.837e-03    2.097e-02
  model.layers.39.self_attn.q_proj                          161      0     1.046e-02    2.097e-03    1.067e-02
  model.layers.39.self_attn.v_proj                          161      0     1.091e-02    2.328e-03    1.116e-02
  model.layers.4.linear_attn.in_proj_qkv                    161      0     1.192e-02    2.308e-03    1.214e-02
  model.layers.4.linear_attn.in_proj_z                      161      0     1.247e-02    1.771e-03    1.259e-02
  model.layers.4.linear_attn.out_proj                       161      0     1.789e-02    2.411e-03    1.806e-02
  model.layers.4.mlp.down_proj                              161      0     1.443e-02    2.603e-03    1.467e-02
  model.layers.4.mlp.gate_proj                              161      0     1.013e-02    3.511e-03    1.072e-02
  model.layers.4.mlp.up_proj                                161      0     1.123e-02    3.618e-03    1.180e-02
  model.layers.40.linear_attn.in_proj_qkv                   161      0     1.224e-02    2.282e-03    1.245e-02
  model.layers.40.linear_attn.in_proj_z                     161      0     1.173e-02    1.688e-03    1.185e-02
  model.layers.40.linear_attn.out_proj                      161      0     1.607e-02    2.268e-03    1.623e-02
  model.layers.40.mlp.down_proj                             161      0     1.478e-02    2.675e-03    1.501e-02
  model.layers.40.mlp.gate_proj                             161      0     1.008e-02    3.134e-03    1.055e-02
  model.layers.40.mlp.up_proj                               161      0     1.164e-02    3.641e-03    1.220e-02
  model.layers.41.linear_attn.in_proj_qkv                   161      0     1.176e-02    2.256e-03    1.197e-02
  model.layers.41.linear_attn.in_proj_z                     161      0     1.226e-02    1.733e-03    1.238e-02
  model.layers.41.linear_attn.out_proj                      161      0     1.649e-02    2.255e-03    1.664e-02
  model.layers.41.mlp.down_proj                             161      0     1.477e-02    2.705e-03    1.502e-02
  model.layers.41.mlp.gate_proj                             161      0     1.085e-02    3.211e-03    1.132e-02
  model.layers.41.mlp.up_proj                               161      0     1.167e-02    3.636e-03    1.223e-02
  model.layers.42.linear_attn.in_proj_qkv                   161      0     1.162e-02    2.276e-03    1.184e-02
  model.layers.42.linear_attn.in_proj_z                     161      0     1.242e-02    1.747e-03    1.254e-02
  model.layers.42.linear_attn.out_proj                      161      0     1.568e-02    2.272e-03    1.584e-02
  model.layers.42.mlp.down_proj                             161      0     1.467e-02    2.652e-03    1.490e-02
  model.layers.42.mlp.gate_proj                             161      0     1.173e-02    3.240e-03    1.217e-02
  model.layers.42.mlp.up_proj                               161      0     1.143e-02    3.586e-03    1.198e-02
  model.layers.43.mlp.down_proj                             161      0     1.386e-02    2.670e-03    1.411e-02
  model.layers.43.mlp.gate_proj                             161      0     1.084e-02    3.218e-03    1.131e-02
  model.layers.43.mlp.up_proj                               161      0     1.153e-02    3.615e-03    1.208e-02
  model.layers.43.self_attn.k_proj                          161      0     1.250e-02    2.453e-03    1.274e-02
  model.layers.43.self_attn.o_proj                          161      0     2.162e-02    3.131e-03    2.185e-02
  model.layers.43.self_attn.q_proj                          161      0     1.059e-02    2.086e-03    1.079e-02
  model.layers.43.self_attn.v_proj                          161      0     1.050e-02    2.259e-03    1.074e-02
  model.layers.44.linear_attn.in_proj_qkv                   161      0     1.203e-02    2.229e-03    1.224e-02
  model.layers.44.linear_attn.in_proj_z                     161      0     1.227e-02    1.697e-03    1.239e-02
  model.layers.44.linear_attn.out_proj                      161      0     1.629e-02    2.299e-03    1.645e-02
  model.layers.44.mlp.down_proj                             161      0     1.413e-02    2.713e-03    1.439e-02
  model.layers.44.mlp.gate_proj                             161      0     1.082e-02    3.260e-03    1.130e-02
  model.layers.44.mlp.up_proj                               161      0     1.171e-02    3.618e-03    1.225e-02
  model.layers.45.linear_attn.in_proj_qkv                   161      0     1.207e-02    2.277e-03    1.229e-02
  model.layers.45.linear_attn.in_proj_z                     161      0     1.264e-02    1.761e-03    1.276e-02
  model.layers.45.linear_attn.out_proj                      161      0     1.809e-02    2.431e-03    1.825e-02
  model.layers.45.mlp.down_proj                             161      0     1.435e-02    2.745e-03    1.461e-02
  model.layers.45.mlp.gate_proj                             161      0     1.081e-02    3.287e-03    1.130e-02
  model.layers.45.mlp.up_proj                               161      0     1.157e-02    3.622e-03    1.212e-02
  model.layers.46.linear_attn.in_proj_qkv                   161      0     1.197e-02    2.246e-03    1.218e-02
  model.layers.46.linear_attn.in_proj_z                     161      0     1.186e-02    1.650e-03    1.197e-02
  model.layers.46.linear_attn.out_proj                      161      0     1.870e-02    2.364e-03    1.885e-02
  model.layers.46.mlp.down_proj                             161      0     1.426e-02    2.686e-03    1.451e-02
  model.layers.46.mlp.gate_proj                             161      0     1.112e-02    3.310e-03    1.160e-02
  model.layers.46.mlp.up_proj                               161      0     1.150e-02    3.598e-03    1.205e-02
  model.layers.47.mlp.down_proj                             161      0     1.427e-02    2.639e-03    1.451e-02
  model.layers.47.mlp.gate_proj                             161      0     1.056e-02    3.336e-03    1.107e-02
  model.layers.47.mlp.up_proj                               161      0     1.177e-02    3.624e-03    1.232e-02
  model.layers.47.self_attn.k_proj                          161      0     1.243e-02    2.468e-03    1.268e-02
  model.layers.47.self_attn.o_proj                          161      0     1.958e-02    2.554e-03    1.975e-02
  model.layers.47.self_attn.q_proj                          161      0     9.645e-03    2.054e-03    9.860e-03
  model.layers.47.self_attn.v_proj                          161      0     1.061e-02    2.234e-03    1.084e-02
  model.layers.48.linear_attn.in_proj_qkv                   161      0     1.272e-02    2.266e-03    1.292e-02
  model.layers.48.linear_attn.in_proj_z                     161      0     1.209e-02    1.649e-03    1.220e-02
  model.layers.48.linear_attn.out_proj                      161      0     1.710e-02    2.416e-03    1.727e-02
  model.layers.48.mlp.down_proj                             161      0     1.481e-02    2.681e-03    1.505e-02
  model.layers.48.mlp.gate_proj                             161      0     1.035e-02    3.332e-03    1.087e-02
  model.layers.48.mlp.up_proj                               161      0     1.169e-02    3.605e-03    1.223e-02
  model.layers.49.linear_attn.in_proj_qkv                   161      0     1.209e-02    2.281e-03    1.230e-02
  model.layers.49.linear_attn.in_proj_z                     161      0     1.152e-02    1.601e-03    1.163e-02
  model.layers.49.linear_attn.out_proj                      161      0     1.905e-02    2.170e-03    1.918e-02
  model.layers.49.mlp.down_proj                             161      0     1.568e-02    2.694e-03    1.591e-02
  model.layers.49.mlp.gate_proj                             161      0     1.006e-02    3.369e-03    1.061e-02
  model.layers.49.mlp.up_proj                               161      0     1.149e-02    3.564e-03    1.203e-02
  model.layers.5.linear_attn.in_proj_qkv                    161      0     1.246e-02    2.271e-03    1.266e-02
  model.layers.5.linear_attn.in_proj_z                      161      0     1.391e-02    1.909e-03    1.404e-02
  model.layers.5.linear_attn.out_proj                       161      0     1.771e-02    2.206e-03    1.785e-02
  model.layers.5.mlp.down_proj                              161      0     1.528e-02    2.518e-03    1.548e-02
  model.layers.5.mlp.gate_proj                              161      0     1.018e-02    3.535e-03    1.077e-02
  model.layers.5.mlp.up_proj                                161      0     1.129e-02    3.621e-03    1.185e-02
  model.layers.50.linear_attn.in_proj_qkv                   161      0     1.167e-02    2.256e-03    1.188e-02
  model.layers.50.linear_attn.in_proj_z                     161      0     1.087e-02    1.486e-03    1.097e-02
  model.layers.50.linear_attn.out_proj                      161      0     1.808e-02    2.348e-03    1.823e-02
  model.layers.50.mlp.down_proj                             161      0     1.518e-02    2.665e-03    1.541e-02
  model.layers.50.mlp.gate_proj                             161      0     9.856e-03    3.283e-03    1.039e-02
  model.layers.50.mlp.up_proj                               161      0     1.087e-02    3.439e-03    1.140e-02
  model.layers.51.mlp.down_proj                             161      0     1.525e-02    2.653e-03    1.548e-02
  model.layers.51.mlp.gate_proj                             161      0     8.703e-03    3.392e-03    9.342e-03
  model.layers.51.mlp.up_proj                               161      0     1.112e-02    3.553e-03    1.167e-02
  model.layers.51.self_attn.k_proj                          161      0     1.104e-02    2.409e-03    1.130e-02
  model.layers.51.self_attn.o_proj                          161      0     1.923e-02    2.297e-03    1.937e-02
  model.layers.51.self_attn.q_proj                          161      0     9.125e-03    2.022e-03    9.347e-03
  model.layers.51.self_attn.v_proj                          161      0     1.025e-02    2.269e-03    1.050e-02
  model.layers.52.linear_attn.in_proj_qkv                   161      0     9.626e-03    2.112e-03    9.855e-03
  model.layers.52.linear_attn.in_proj_z                     161      0     8.540e-03    1.286e-03    8.638e-03
  model.layers.52.linear_attn.out_proj                      161      0     1.872e-02    2.304e-03    1.886e-02
  model.layers.52.mlp.down_proj                             161      0     1.783e-02    2.683e-03    1.803e-02
  model.layers.52.mlp.gate_proj                             161      0     8.150e-03    3.400e-03    8.832e-03
  model.layers.52.mlp.up_proj                               161      0     1.107e-02    3.565e-03    1.163e-02
  model.layers.53.linear_attn.in_proj_qkv                   161      0     9.506e-03    2.129e-03    9.742e-03
  model.layers.53.linear_attn.in_proj_z                     161      0     8.685e-03    1.312e-03    8.785e-03
  model.layers.53.linear_attn.out_proj                      161      0     1.688e-02    2.284e-03    1.704e-02
  model.layers.53.mlp.down_proj                             161      0     1.846e-02    2.770e-03    1.866e-02
  model.layers.53.mlp.gate_proj                             161      0     7.914e-03    3.405e-03    8.616e-03
  model.layers.53.mlp.up_proj                               161      0     1.098e-02    3.567e-03    1.154e-02
  model.layers.54.linear_attn.in_proj_qkv                   161      0     9.930e-03    2.233e-03    1.018e-02
  model.layers.54.linear_attn.in_proj_z                     161      0     8.296e-03    1.277e-03    8.394e-03
  model.layers.54.linear_attn.out_proj                      161      0     1.746e-02    2.133e-03    1.759e-02
  model.layers.54.mlp.down_proj                             161      0     1.830e-02    2.512e-03    1.847e-02
  model.layers.54.mlp.gate_proj                             161      0     7.284e-03    3.355e-03    8.020e-03
  model.layers.54.mlp.up_proj                               161      0     9.182e-03    3.420e-03    9.799e-03
  model.layers.55.mlp.down_proj                             161      0     1.699e-02    2.628e-03    1.720e-02
  model.layers.55.mlp.gate_proj                             161      0     7.234e-03    3.388e-03    7.989e-03
  model.layers.55.mlp.up_proj                               161      0     1.132e-02    3.604e-03    1.188e-02
  model.layers.55.self_attn.k_proj                          161      0     1.087e-02    2.402e-03    1.113e-02
  model.layers.55.self_attn.o_proj                          161      0     1.732e-02    2.198e-03    1.746e-02
  model.layers.55.self_attn.q_proj                          161      0     8.896e-03    2.104e-03    9.143e-03
  model.layers.55.self_attn.v_proj                          161      0     9.491e-03    2.219e-03    9.745e-03
  model.layers.56.linear_attn.in_proj_qkv                   161      0     8.996e-03    2.092e-03    9.236e-03
  model.layers.56.linear_attn.in_proj_z                     161      0     8.713e-03    1.329e-03    8.815e-03
  model.layers.56.linear_attn.out_proj                      161      0     1.730e-02    2.108e-03    1.743e-02
  model.layers.56.mlp.down_proj                             161      0     1.973e-02    2.530e-03    1.989e-02
  model.layers.56.mlp.gate_proj                             161      0     7.323e-03    3.347e-03    8.052e-03
  model.layers.56.mlp.up_proj                               161      0     1.132e-02    3.599e-03    1.188e-02
  model.layers.57.linear_attn.in_proj_qkv                   161      0     8.642e-03    2.054e-03    8.885e-03
  model.layers.57.linear_attn.in_proj_z                     161      0     8.515e-03    1.299e-03    8.613e-03
  model.layers.57.linear_attn.out_proj                      161      0     1.590e-02    2.507e-03    1.610e-02
  model.layers.57.mlp.down_proj                             161      0     1.637e-02    2.798e-03    1.661e-02
  model.layers.57.mlp.gate_proj                             161      0     7.134e-03    3.326e-03    7.872e-03
  model.layers.57.mlp.up_proj                               161      0     1.116e-02    3.599e-03    1.173e-02
  model.layers.58.linear_attn.in_proj_qkv                   161      0     9.192e-03    2.133e-03    9.437e-03
  model.layers.58.linear_attn.in_proj_z                     161      0     8.763e-03    1.313e-03    8.861e-03
  model.layers.58.linear_attn.out_proj                      161      0     1.892e-02    2.407e-03    1.908e-02
  model.layers.58.mlp.down_proj                             161      0     1.657e-02    2.620e-03    1.677e-02
  model.layers.58.mlp.gate_proj                             161      0     7.016e-03    3.317e-03    7.760e-03
  model.layers.58.mlp.up_proj                               161      0     1.044e-02    3.550e-03    1.103e-02
  model.layers.59.mlp.down_proj                             161      0     1.664e-02    2.527e-03    1.683e-02
  model.layers.59.mlp.gate_proj                             161      0     6.606e-03    3.308e-03    7.388e-03
  model.layers.59.mlp.up_proj                               161      0     9.285e-03    3.448e-03    9.903e-03
  model.layers.59.self_attn.k_proj                          161      0     1.020e-02    2.349e-03    1.046e-02
  model.layers.59.self_attn.o_proj                          161      0     1.804e-02    2.288e-03    1.819e-02
  model.layers.59.self_attn.q_proj                          161      0     8.661e-03    2.117e-03    8.916e-03
  model.layers.59.self_attn.v_proj                          161      0     9.120e-03    2.162e-03    9.376e-03
  model.layers.6.linear_attn.in_proj_qkv                    161      0     1.264e-02    2.270e-03    1.285e-02
  model.layers.6.linear_attn.in_proj_z                      161      0     1.425e-02    1.992e-03    1.439e-02
  model.layers.6.linear_attn.out_proj                       161      0     1.683e-02    2.298e-03    1.698e-02
  model.layers.6.mlp.down_proj                              161      0     1.354e-02    2.403e-03    1.376e-02
  model.layers.6.mlp.gate_proj                              161      0     1.046e-02    3.490e-03    1.102e-02
  model.layers.6.mlp.up_proj                                161      0     1.113e-02    3.560e-03    1.169e-02
  model.layers.60.linear_attn.in_proj_qkv                   161      0     7.948e-03    1.975e-03    8.189e-03
  model.layers.60.linear_attn.in_proj_z                     161      0     7.501e-03    1.194e-03    7.596e-03
  model.layers.60.linear_attn.out_proj                      161      0     1.692e-02    2.505e-03    1.710e-02
  model.layers.60.mlp.down_proj                             161      0     1.584e-02    2.710e-03    1.607e-02
  model.layers.60.mlp.gate_proj                             161      0     6.830e-03    3.245e-03    7.562e-03
  model.layers.60.mlp.up_proj                               161      0     9.185e-03    3.392e-03    9.790e-03
  model.layers.61.linear_attn.in_proj_qkv                   161      0     8.884e-03    2.091e-03    9.125e-03
  model.layers.61.linear_attn.in_proj_z                     161      0     7.949e-03    1.247e-03    8.046e-03
  model.layers.61.linear_attn.out_proj                      161      0     1.666e-02    2.347e-03    1.682e-02
  model.layers.61.mlp.down_proj                             161      0     1.498e-02    2.662e-03    1.522e-02
  model.layers.61.mlp.gate_proj                             161      0     6.566e-03    3.199e-03    7.304e-03
  model.layers.61.mlp.up_proj                               161      0     8.718e-03    3.320e-03    9.330e-03
  model.layers.62.linear_attn.in_proj_qkv                   161      0     8.142e-03    1.974e-03    8.379e-03
  model.layers.62.linear_attn.in_proj_z                     161      0     7.383e-03    1.219e-03    7.483e-03
  model.layers.62.linear_attn.out_proj                      161      0     1.517e-02    2.360e-03    1.535e-02
  model.layers.62.mlp.down_proj                             161      0     1.491e-02    2.530e-03    1.512e-02
  model.layers.62.mlp.gate_proj                             161      0     6.260e-03    3.133e-03    6.999e-03
  model.layers.62.mlp.up_proj                               161      0     6.868e-03    3.148e-03    7.554e-03
  model.layers.63.mlp.down_proj                             161      0     1.188e-02    2.006e-03    1.205e-02
  model.layers.63.mlp.gate_proj                             161      0     5.645e-03    3.069e-03    6.425e-03
  model.layers.63.mlp.up_proj                               161      0     4.720e-03    3.052e-03    5.620e-03
  model.layers.63.self_attn.k_proj                          161      0     1.075e-02    2.411e-03    1.101e-02
  model.layers.63.self_attn.o_proj                          161      0     1.295e-02    2.015e-03    1.310e-02
  model.layers.63.self_attn.q_proj                          161      0     7.002e-03    1.936e-03    7.265e-03
  model.layers.63.self_attn.v_proj                          161      0     4.688e-03    1.728e-03    4.996e-03
  model.layers.7.mlp.down_proj                              161      0     1.422e-02    2.582e-03    1.446e-02
  model.layers.7.mlp.gate_proj                              161      0     1.041e-02    3.565e-03    1.100e-02
  model.layers.7.mlp.up_proj                                161      0     1.135e-02    3.628e-03    1.192e-02
  model.layers.7.self_attn.k_proj                           161      0     1.384e-02    2.716e-03    1.410e-02
  model.layers.7.self_attn.o_proj                           161      0     1.326e-02    2.271e-03    1.346e-02
  model.layers.7.self_attn.q_proj                           161      0     1.250e-02    2.077e-03    1.267e-02
  model.layers.7.self_attn.v_proj                           161      0     1.103e-02    2.331e-03    1.128e-02
  model.layers.8.linear_attn.in_proj_qkv                    161      0     1.244e-02    2.249e-03    1.264e-02
  model.layers.8.linear_attn.in_proj_z                      161      0     1.306e-02    1.814e-03    1.318e-02
  model.layers.8.linear_attn.out_proj                       161      0     1.616e-02    2.279e-03    1.632e-02
  model.layers.8.mlp.down_proj                              161      0     1.507e-02    2.553e-03    1.528e-02
  model.layers.8.mlp.gate_proj                              161      0     1.050e-02    3.562e-03    1.109e-02
  model.layers.8.mlp.up_proj                                161      0     1.129e-02    3.616e-03    1.186e-02
  model.layers.9.linear_attn.in_proj_qkv                    161      0     1.310e-02    2.236e-03    1.329e-02
  model.layers.9.linear_attn.in_proj_z                      161      0     1.360e-02    1.892e-03    1.373e-02
  model.layers.9.linear_attn.out_proj                       161      0     1.603e-02    2.266e-03    1.619e-02
  model.layers.9.mlp.down_proj                              161      0     1.344e-02    2.469e-03    1.366e-02
  model.layers.9.mlp.gate_proj                              161      0     1.058e-02    3.571e-03    1.117e-02
  model.layers.9.mlp.up_proj                                161      0     1.119e-02    3.608e-03    1.175e-02

ARM RECEIPTS -- what each arm actually executed, from the engine's
own per-call census.  fused/standalone is the reconstruct variant;
acc16/acc32 is the cuBLAS compute type leg B asked hgemm for.
  arm       legB calls          rows     fused  standalone     acc16     acc32   fp32pin
  REF            64000     131072000     64000           0     64000         0         0
  REF2           64000     131072000     64000           0     64000         0         0
  STD            64000     131072000         0       64000     64000         0         0
  REF32          64000     131072000     64000           0         0     64000         0
  STD32          64000     131072000         0       64000         0     64000         0
  INT8           64000     131072000         0       64000     64000         0         0
  all arms match their expected leg/accumulator signature
  prefill positions aligned: [8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8] per prompt, identical across all 6 arms

THE TAIL over PREFILL positions only -- one per prefill chunk, so
every arm scores the SAME context. Decode positions are reported
separately and are NOT a numerics measure: once a token flips, the
arms are reading different contexts. The control proves it -- pooled
over decode, STD (which contains no int8 at all) has the same max KL
as INT8, so a pooled tail cannot attribute anything to the kernel.

arm            n        mean         p50         p95         p99         MAX     argmax flip
REF          160   0.000e+00   0.000e+00   0.000e+00   0.000e+00   0.000e+00 0/160 =   0.00%
REF2         160   0.000e+00   0.000e+00   0.000e+00   0.000e+00   0.000e+00 0/160 =   0.00%
STD          160   2.678e-03   2.592e-04   6.751e-03   9.228e-02   1.028e-01 3/160 =   1.88%
REF32        160   4.815e-03   2.572e-04   3.607e-03   1.463e-01   4.852e-01 1/160 =   0.62%
STD32        160   7.319e-03   2.765e-04   4.596e-03   2.482e-01   7.456e-01 3/160 =   1.88%
INT8         160   3.858e-02   7.787e-04   2.124e-02   1.798e+00   2.380e+00 5/160 =   3.12%

Tail attribution -- the 10 worst INT8 prefill positions, with the
control's KL at the SAME position. A tail that is positional shows
the control elevated too; a tail that is the kernel's does not.
   prompt  chunk      INT8 KL       STD KL  ratio INT8/STD
       17      2    2.380e+00    9.228e-02           25.79
       16      6    1.798e+00    5.397e-02           33.31
        0      7    1.290e+00    2.025e-02           63.69
        0      1    1.967e-01    1.028e-01            1.91
        2      3    1.691e-01    3.030e-02            5.58
        0      6    4.997e-02    2.669e-03           18.73
        0      5    3.829e-02    2.080e-02            1.84
       18      7    2.124e-02    9.101e-03            2.33
       16      4    1.666e-02    1.936e-03            8.61
       16      3    1.210e-02    1.996e-03            6.06
  Spearman rank correlation STD vs INT8 across all 160 prefill positions: 0.782

Decode positions (divergence-contaminated, shown for completeness):
arm            n        mean         MAX    greedy agree
REF          160   0.000e+00   0.000e+00 160/160 =  100.0%
REF2         160   0.000e+00   0.000e+00 160/160 =  100.0%
STD          160   9.462e-04   1.236e-02 160/160 =  100.0%
REF32        160   9.351e-03   1.340e+00 159/160 =   99.4%
STD32        160   8.384e-03   1.171e+00 159/160 =   99.4%
INT8         160   2.075e+00   3.248e+01 145/160 =   90.6%

SAMPLER-RELEVANT TAIL @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF          160     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/160     0.00%
REF2         160     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/160     0.00%
STD          160      98.85%  1.297e-02  4.743e-02  2.118e-01          inf 34/160     4.06%
REF32        160      98.64%  1.255e-02  3.622e-02  3.807e-01          inf 28/160     2.42%
STD32        160      98.58%  1.481e-02  4.212e-02  5.362e-01          inf 34/160     3.44%
INT8         160      97.67%  3.103e-02  7.303e-02  8.303e-01          inf 46/160     5.86%

SAMPLER-RELEVANT TAIL @ owner's k~40 (T=1.0, top_p=1.0, top_k=40), prefill positions only.
TV is the headline: under a drafter that tracks the target, speculative
decoding's expected accept rate is sum_t min(p_ref, p_arm), so TV IS the
expected accept-rate loss.
arm            n  setOverlap    TV mean     TV p95     TV max  truncKL p95  bndryChurn  drawDiff
REF          160     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/160     0.00%
REF2         160     100.00%  0.000e+00  0.000e+00  0.000e+00    0.000e+00 0/160     0.00%
STD          160      97.92%  1.255e-02  4.033e-02  2.122e-01          inf 86/160     5.47%
REF32        160      97.69%  1.222e-02  3.114e-02  3.831e-01          inf 94/160     4.38%
STD32        160      97.65%  1.416e-02  3.671e-02  5.296e-01          inf 94/160     4.69%
INT8         160      95.14%  3.023e-02  5.118e-02  8.100e-01          inf 119/160     8.28%

FULL-VOCAB numbers below, kept and labelled: they are the right
instrument for GREEDY or beam decode, where every rank matters.

arm         first-prefill-position KL       median          max
REF                         0.000e+00    0.000e+00    0.000e+00
REF2                        0.000e+00    0.000e+00    0.000e+00
STD                         2.764e-04    4.700e-05    1.187e-03
REF32                       2.152e-04    1.766e-05    9.786e-04
STD32                       1.695e-04    1.800e-05    8.996e-04
INT8                        5.912e-04    5.895e-05    4.044e-03

PAIRED TV @ canonical (T=1.0, top_p=0.95, top_k=20), prefill positions.
Each row is A scored against B -- B is that ROW's reference, not the
table's.  TV is the expected speculative accept-rate loss, so the
column is read directly in percentage points.  The interval is a
cluster bootstrap over PROMPTS; the IQR is the raw spread over
positions with no floor applied to it.

  A       vs B        n  mean TV pp           95% CI pp    p25 pp    p50 pp    p75 pp    p95 pp    max pp
  REF     REF2      160      0.0000 [  0.0000,  0.0000]    0.0000    0.0000    0.0000    0.0000    0.0000
  REF     STD       160      1.2967 [  0.9400,  1.7017]    0.0000    0.2396    1.8733    4.7426   21.1789
  REF     REF32     160      1.2549 [  0.7746,  1.8564]    0.0000    0.0000    1.5348    3.6216   38.0686
  REF     STD32     160      1.4807 [  0.8281,  2.3426]    0.0000    0.3561    1.7132    4.2125   53.6235
  REF     INT8      160      3.1030 [  1.5306,  4.9355]    0.0000    0.8311    2.7924    7.3030   83.0319
  REF2    STD       160      1.2967 [  0.9400,  1.7017]    0.0000    0.2396    1.8733    4.7426   21.1789
  REF2    REF32     160      1.2549 [  0.7746,  1.8564]    0.0000    0.0000    1.5348    3.6216   38.0686
  REF2    STD32     160      1.4807 [  0.8281,  2.3426]    0.0000    0.3561    1.7132    4.2125   53.6235
  REF2    INT8      160      3.1030 [  1.5306,  4.9355]    0.0000    0.8311    2.7924    7.3030   83.0319
  STD     REF32     160      1.4194 [  0.8956,  2.0452]    0.0000    0.0000    2.0474    4.3657   35.0068
  STD     STD32     160      1.6923 [  0.9523,  2.5681]    0.0000    0.0000    2.1214    4.7426   47.7078
  STD     INT8      160      3.1685 [  1.6172,  5.0165]    0.0000    0.7205    2.9080    6.7930   84.9011
  REF32   STD32     160      0.9807 [  0.6832,  1.2962]    0.0000    0.0000    1.5083    3.4942   16.5443
  REF32   INT8      160      3.2346 [  1.5487,  5.1455]    0.0000    0.8444    2.9507    6.0087   83.8953
  STD32   INT8      160      3.1992 [  1.3678,  5.3380]    0.0000    0.6683    2.7780    6.6936   83.9256

  TV distance matrix, mean TV pp (symmetric):
          REF      REF2       STD     REF32     STD32      INT8
  REF        0.0000    0.0000    1.2967    1.2549    1.4807    3.1030
  REF2       0.0000    0.0000    1.2967    1.2549    1.4807    3.1030
  STD        1.2967    1.2967    0.0000    1.4194    1.6923    3.1685
  REF32      1.2549    1.2549    1.4194    0.0000    0.9807    3.2346
  STD32      1.4807    1.4807    1.6923    0.9807    0.0000    3.1992
  INT8       3.1030    3.1030    3.1685    3.2346    3.1992    0.0000

  TV distance matrix, median TV pp (symmetric):
          REF      REF2       STD     REF32     STD32      INT8
  REF        0.0000    0.0000    0.2396    0.0000    0.3561    0.8311
  REF2       0.0000    0.0000    0.2396    0.0000    0.3561    0.8311
  STD        0.2396    0.2396    0.0000    0.0000    0.0000    0.7205
  REF32      0.0000    0.0000    0.0000    0.0000    0.0000    0.8444
  STD32      0.3561    0.3561    0.0000    0.0000    0.0000    0.6683
  INT8       0.8311    0.8311    0.7205    0.8444    0.6683    0.0000

lm_head CUT -- the divergence entering the head vs leaving it,
prefill positions, relative rms.  'in' is the residual stream after
24 layers; 'out' is the centred logits.  amp = out/in: at 1.0 the
head passes the body's divergence through unchanged, above it the
head adds its own.

  A       vs B        n   rel rms IN  rel rms OUT      amp
  REF     REF2      160   0.0000e+00   0.0000e+00      nan
  REF     STD       160   3.1924e-02   2.9820e-02    0.934
  REF     REF32     160   3.2759e-02   3.0788e-02    0.940
  REF     STD32     160   3.6577e-02   3.4241e-02    0.936
  REF     INT8      160   7.5245e-02   6.8547e-02    0.911
  REF2    STD       160   3.1924e-02   2.9820e-02    0.934
  REF2    REF32     160   3.2759e-02   3.0788e-02    0.940
  REF2    STD32     160   3.6577e-02   3.4241e-02    0.936
  REF2    INT8      160   7.5245e-02   6.8547e-02    0.911
  STD     REF32     160   3.6385e-02   3.4466e-02    0.947
  STD     STD32     160   3.7614e-02   3.5109e-02    0.933
  STD     INT8      160   7.4082e-02   6.7508e-02    0.911
  REF32   STD32     160   3.0519e-02   2.8512e-02    0.934
  REF32   INT8      160   7.3216e-02   6.6572e-02    0.909
  STD32   INT8      160   7.4280e-02   6.7385e-02    0.907

saved per-arm prefill logits+hidden to /work/logits_p20.pt
