# The exl3 GEMM's grid width (force_num_sms), per geometry, at the SERVED slates.
#   tools/exl3_grid_width_record.py, RTX 4090 (128 SMs), box 100, sole tenant.
#
# PREDICTION, REGISTERED BEFORE THE RUN (2026-09-02):
#   P1  5120x1024xK4 -- the k_proj/v_proj pair -- clears the 1% target at the
#       verify slates {48, 64}, by more than 5%, with the optimum in 48..80.
#       (The earlier deferred measurement put it at +10.3..+23.1% with the
#       optimum at 64, but that was scored over a wider row set including M=1.)
#   P2  No other 27B geometry clears the 1% target: every other one was inside
#       +-0.4% of the default when this was first measured.
#   P3  The NULL arm -- the default width entered twice -- lands inside +-1%.
#   P4  The winning width does not cost the decode band (M = 1,2,4,8) more
#       than 1%. Least confident of the four: nothing measured it before.
#
# ---------------------------------------------------------------------------
# OUTCOME. Every number below: RTX 4090 (128 SMs, sm_89), box 100, sole tenant
# verified at both ends, clocks idling at 210-285 MHz against a 3105 MHz max,
# 12 whole races per geometry, arms interleaved and reshuffled per round, L2
# scrubbed before every sample, roofline guard on every cell.
#
# P1 HELD, larger than predicted and in both currencies.
#   5120x1024xK4 (k_proj / v_proj, 34 linears) at the shipped family K32N128:
#     force_num_sms=64  verify(48+64) +23.23% [23.03, 23.47]  12/12
#                       decode(1,2,4,8) +13.45% FASTER, not a cost
#                       null arm +0.11%
#     cross-check under the OTHER scrub bias (scrub=write): +21.32%
#       [20.79, 21.79], same winner. No sign flip, so the reading is not the
#       scrub's.
#     the shipped table records +23.48% [23.24, 23.73] for this cell: a LATER
#       whole run of the same sweep over all ten linears, quoted there because
#       that is the run whose per-width times the end-to-end census below is
#       computed from. Two independent 12-race runs, intervals 0.25pp apart.
#     tied within the null band: {64} at 7 widths, {56, 64} at 11 -- the
#       recorded value is one of a tied pair, not a resolved optimum.
#
#   MECHANISM, and it is why this geometry and no other. At the shipped family
#   the deepest legal tile is 32 rows, so a 48- or 64-row slate runs
#   ceil(M/32) = 2 row-tiles x (1024/128) = 8 column-tiles = 16 threadblocks on
#   a 128-SM card. The default heuristic leaves seven eighths of the machine
#   idle; a wider grid splits the reduction and fills it. Every other geometry
#   is wide enough that the default grid already covers the card.
#
# P2 HELD, and more strongly than stated. The claim under test was "every
#   other geometry is within +-0.4% of default". At the served slates the
#   default is not merely equal, it WINS: the best non-default width loses by
#   4.85 to 11.53% and taxes decode by up to 9.73%.
#
#     17408x5120xK4  best 112  verify -10.67 [-10.80,-10.55]  decode  +9.73
#     5120x10240xK4  best 112  verify -11.53 [-11.64,-11.42]  decode  +8.43
#     10240x5120xK4  best 112  verify  -8.81 [ -8.90, -8.70]  decode  +8.97
#     5120x6144xK4   best 112  verify  -9.24 [ -9.50, -8.93]  decode  +7.86
#     6144x5120xK4   best 112  verify  -6.20 [ -6.53, -5.88]  decode  +5.90
#     5120x12288xK4  best 112  verify  -5.95 [ -6.12, -5.78]  decode  -0.38
#     5120x17408xK4  best 112  verify  -4.85 [ -4.95, -4.74]  decode  +0.09
#     5120x248320xK6 best  96  verify  +0.49 [ +0.45, +0.53]  decode  +0.20
#                              REFUSED: 12/12 but under the 1% target
#
#   So a hardcoded 64 would have been right for one geometry of nine and a
#   regression on the rest. That is the whole argument for a recorded
#   per-geometry value rather than a constant.
#
# P3 HELD. The null arm -- the default width entered twice under two names --
#   landed inside +-0.19% on every cell of every run. Every margin above is
#   two orders of magnitude outside it.
#
# P4 HELD, in the opposite direction from the one it guarded against. The
#   recorded width does not tax decode; it is 13.45% faster there. The guard
#   still earned its place: it is what refused 112 on the five geometries
#   above, where the decode cost is 5.9-9.7%.
#
# THE FAMILY DEPENDENCE, MEASURED. 5120x1024 raced at each of its four legal
# families, 11 widths each, 12 races. The WIN survives every family and the
# winning VALUE does not, which is why the recorded width is gated on the
# family beside it and dropped when tools/exl3_pin_force.py moves one:
#
#     family    best  verify margin over that family's own default  decode
#     K32N128     64  +23.54% [23.29, 23.82]   tied {56, 64}        -14.06%
#     K16N128     64  +22.38% [21.97, 22.81]   tied {64, 80}         -7.79%
#     K32N256     32  +52.64% [52.50, 52.77]   tied {32}            -37.86%
#     K16N512     16  +78.68% [78.66, 78.71]   tied {16}            -59.10%
#
# INVARIANCE. Pinning a different constant is invariance-NEUTRAL, measured not
# argued: row 0 is byte-identical over M = 1, 4, 16, 32, 48, 56, 64, 72, 96,
# 128, 160, 256 at widths 0, 32, 64 and 96 alike, on k_proj and gate_proj. The
# recorder re-runs that check per recorded cell and refuses a width that fails
# it. Separately: at a FIXED shape and a FIXED row count the widths 32..112
# each differ from the default in the emitted bits (2.4e-4 on fp16 output) and
# 128 is byte-identical to 0 -- which is what makes the width part of the
# numerics contract rather than a scheduling knob.
#
# ---------------------------------------------------------------------------
# THE SAME SWEEP AT K=5 AND K=6. Two more checkpoints, same harness, same
# conditions. Five widths recorded in total, and every one of them is a
# NARROW-OUTPUT geometry -- which is the mechanism above restated: a small
# output width times a shallow row tile is a grid that cannot fill a 128-SM
# card at a verify slate.
#
#   Qwen3.8-27B-DFlash2-exl3-6.0bpw (all K=6)
#     5120x1024xK6   K32N128   64  +20.16% [19.96, 20.42]  decode -13.62%  tied {64}
#     5120x1280xK6   K32N128   96  +13.64% [13.32, 13.92]  decode  -7.87%  tied {64,80,96}
#     5120x256xK6    K16N128   16  +68.05% [67.86, 68.25]  decode -58.16%  tied {16}
#         -- NOT recorded: its FAMILY is refused (11/12), and a width cannot be
#            recorded without one. The loader gates them together, so a width
#            without a family would apply to whatever the boot raced.
#     the other five geometries: default wins, best candidate loses 1.0-9.6%
#
#   Step-Audio-2-mini-exl3-4.15bpw (K=4/5/6 in one checkpoint)
#     3584x512xK5    K16N128   32  +54.96% [54.68, 55.22]  decode -39.10%  tied {32}
#     3584x512xK6    K16N128   32  +55.55% [55.19, 55.93]  decode -38.66%  tied {32}
#     the other six geometries: refused; three of them lose 9.3-10.0%
#
#   DOES THE WIDTH TRANSFER ACROSS K? On the only paired cell that can answer
#   it -- 3584x512 at K=5 and K=6, one checkpoint, one geometry, two widths --
#   YES: same recorded value (32), margins 54.96% and 55.55% with overlapping
#   intervals, decode -39.10% and -38.66%. Same answer the family gives (see
#   receipt_pin_per_k.txt). The width is set by the GRID, and K changes the
#   trellis stride rather than the grid.
#
# ---------------------------------------------------------------------------
# WHAT IT IS WORTH END TO END, census-weighted over the served 27B's 409
# linears (counts from the checkpoint's own trellis census, not assumed):
#
#   leg-A trellis GEMM, one 48-row plus one 64-row verify slate, whole model
#     default grid   84762.5 us
#     pinned grid    83973.1 us     -0.931%   (1.0094x)
#   of which the whole delta is 5120x1024xK4: 3362.9 us -> 2573.5 us.
#
#   The same geometry is 4.5% of the model's decode-band leg-A GEMM, so the
#   -13.45% there is about -0.60% of decode.
#
#   SO: ~0.9% of leg-A GEMM at the verify slates and ~0.6% at decode, and it
#   costs nothing anywhere -- the five refused-with-a-tax geometries keep the
#   kernel's own grid. A 23% number on one geometry is not a 23% anything
#   else, and the two get confused whenever only the first is quoted.
#
# ---------------------------------------------------------------------------
# A LEVER THIS RUN FOUND AND DID NOT TAKE. _PIN_SELECT_ROWS is (1, 16, 32) and
# never samples 48 or 64, so the FAMILY is chosen at rows the served recipes
# do not run. Re-racing the families at the real slates picks K16N128 for
# every one of the nine geometries, unanimously, by 13-38%. Census-weighted
# against the shipped per-geometry pin:
#
#     one family everywhere    decode band     verify slates
#     K16N128                     +49.38%          -14.35%
#     K32N256                      -5.72%          +25.52%
#     K32N128                     +12.29%           -1.63%
#     K16N512                     +10.05%          +68.57%
#     (shipped per-geometry pin is the baseline in both columns)
#
# So the 14.35% verify win is bought with a 49.38% decode loss, and the 5.72%
# decode win with a 25.52% verify loss. The shipped pin is not the optimum at
# either end and it is the only one of the four that is not terrible at one of
# them. A per-token debit compounds and there is no exchange rate against a
# verify win, so this is REPORTED and not taken; taking it needs an instrument
# that prices tokens/second, not GEMM microseconds, and the row set is in the
# pin cache key so moving it retires every recorded entry.
#
# That is also why the grid width is the lever worth having here: on the
# geometries where it fires it is faster at BOTH ends, so it needs no
# exchange rate at all.
#
# ---------------------------------------------------------------------------
# INCIDENTAL, AND IT IS NOT THIS CHANGE'S TO FIX. Running the live-GPU
# invariance suite against the SERVED 27B (rather than the 0.8B fixture it
# defaults to) fails one test that has nothing to do with the grid width:
#
#   test_unpinned_is_invariant_through_the_gate_bound
#     the UNPINNED dispatch is NOT row-count invariant through 8 rows on
#     Qwen3.8-27B-exl3-4.0bpw: row 0 moves at 2 rows on in_proj_qkv,
#     out_proj, gate_proj, down_proj, mtp.fc, mtp q_proj and mtp down_proj --
#     7 of the 10 linears the suite walks.
#
# engine/exl3_shape_pin.UNPINNED_INVARIANT_MAX_ROWS is 8 and it gates whether
# the pin is applied AT ALL: a boot whose widest leg-A slate is <= 8 rows
# declines the pin on the claim that the unpinned dispatch already provides
# the invariance. compose.serve.yaml documents --max-batch 1 as a supported
# single-stream recipe, which at k=7 is exactly an 8-row slate. On this
# checkpoint that decline hands out a guarantee the kernel does not honour.
#
# Reproduced with this change's call-site edit REVERTED to the literal 0 it
# replaces, so it is not this branch's: the unpinned path never carries the
# width attribute, and getattr(..., 0) on an absent attribute is the literal.
# Filed here rather than fixed because closing it means every boot pins, which
# costs decode 3.76-3.86% of leg A's weight pass (#1861) -- a trade whoever
# owns that gate should make deliberately, not as a side effect of a grid
# width landing.
