# Shape sweep on the NemotronH Lightning-30B-A3B MoE checkpoint (#1861).
#   /mnt/k8scache/models/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-exl3-4bpw
#   RTX 4090 (sm_89), registry.arbi.work/arbi-serve:test-latest,
#   build narrowed to I8_ONLY_BITS=6 I8_ONLY_CB=2 -- this checkpoint's
#   dense linears are ALL K=6 and ALL mul1, so the narrowed build reaches
#   every one of them and refuses anything else by name.
#
# WHAT IT SHOWS. Every shape whose TILE_N is 64 or 128 agrees, bit for
# bit, at every row count, on every geometry. The five 256-wide shapes
# (5, 8, 9, 11, 13) are correct on N=4096 and N=256 and WRONG on N=2688,
# N=3712 and N=10368 -- the three widths that are 128- but not
# 256-aligned. dim3 grid(N / TN, ...) truncates and the last 128 columns
# are never written, so the relative error is sqrt(128/N) over whatever
# the output buffer held (2.18e-01 = sqrt(128/2688)) or nan. It is
# INTERMITTENT: the same (shape, M) cell sometimes reads back plausible,
# which is what makes this a silent wrong answer rather than a failure.
#
# THE COUNT, per linear, over the 48 non-null-decode shapes:
#   TN in {64, 128}   43 shapes x 7 row counts = 301 cells, 0 bad,
#                     and bit-identical to each other throughout
#   TN == 256          5 shapes x 7 row counts =  35 cells
#                       N % 256 == 0   : 0 bad  (q_proj, k_proj, v_proj)
#                       N % 256 == 128 : 22-27 bad, and the rest are the
#                                        buffer's old contents reading back
#                                        plausible
# 2408 clean cells across the eight dense linears.
#
# NO OTHER CHECKPOINT HERE COULD HAVE CAUGHT THIS. Counting every exl3
# trellis on this machine by (bits, N % 256 == 128):
#   Qwen3.8-27B-exl3-4.0bpw          409 linears, none
#   Qwen3.8-27B-DFlash2-exl3-6.0bpw   47 linears, none
#   Step-Audio-2-mini-exl3-4.15bpw   197 linears, none
#   NVIDIA-Nemotron-3.5-Lightning...  117 dense, ALL FOUR load-bearing
#                                     geometries (2688, 3712, 10368) plus
#                                     both routed ones (1920)
# So the 256-wide arms have been swept clean for the whole life of this
# effort, on three checkpoints that cannot express the defect.
#
# This is #1914's claim, on real weights, on the checkpoint that has the
# geometries. #1914 is therefore a prerequisite for RACING a shape id on
# this checkpoint; it is not one for serving at the default shape 27.

$ python3 sweep_shapes.py --layer 0 --linears in_proj out_proj
  in_proj    k=2688   n=10368  M=1     s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s5=7.75e-02 s6=1.29e-02 s7=1.29e-02 s8=7.75e-02 s9=7.75e-02 s10=1.29e-02 s11=7.75e-02 s12=1.29e-02 s13=nan s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: False
  in_proj    k=2688   n=10368  M=17    s0=1.32e-02 s1=1.32e-02 s2=1.32e-02 s3=1.32e-02 s4=1.32e-02 s5=nan s6=1.32e-02 s7=1.32e-02 s8=nan s9=nan s10=1.32e-02 s11=nan s12=1.32e-02 s13=nan s20=1.32e-02 s21=1.32e-02 s22=1.32e-02 s23=1.32e-02 s24=1.32e-02 s25=1.32e-02 s26=1.32e-02 s27=1.32e-02 s30=1.32e-02 s31=1.32e-02 s32=1.32e-02 s33=1.32e-02 s34=1.32e-02 s35=1.32e-02 s36=1.32e-02 s37=1.32e-02 s50=1.32e-02 s51=1.32e-02 s52=1.32e-02 s53=1.32e-02 s54=1.32e-02 s60=1.32e-02 s61=1.32e-02 s62=1.32e-02 s63=1.32e-02 s64=1.32e-02 s65=1.32e-02 s66=1.32e-02 s67=1.32e-02 s68=1.32e-02 s70=1.32e-02 s71=1.32e-02 s72=1.32e-02 s73=1.32e-02  bit-identical across all shapes: False
  in_proj    k=2688   n=10368  M=63    s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s5=nan s6=1.29e-02 s7=1.29e-02 s8=1.29e-02 s9=nan s10=1.29e-02 s11=nan s12=1.29e-02 s13=nan s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: False
  in_proj    k=2688   n=10368  M=256   s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s5=nan s6=1.30e-02 s7=1.30e-02 s8=1.30e-02 s9=nan s10=1.30e-02 s11=nan s12=1.30e-02 s13=nan s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: False
  in_proj    k=2688   n=10368  M=1000  s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s5=1.30e-02 s6=1.30e-02 s7=1.30e-02 s8=1.30e-02 s9=1.30e-02 s10=1.30e-02 s11=1.30e-02 s12=1.30e-02 s13=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  in_proj    k=2688   n=10368  M=2048  s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s5=nan s6=1.30e-02 s7=1.30e-02 s8=1.30e-02 s9=nan s10=1.30e-02 s11=nan s12=1.30e-02 s13=nan s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: False
  in_proj    k=2688   n=10368  M=8192  s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s5=1.30e-02 s6=1.30e-02 s7=1.30e-02 s8=1.30e-02 s9=1.30e-02 s10=1.30e-02 s11=1.30e-02 s12=1.30e-02 s13=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  out_proj   k=4096   n=2688   M=1     s0=1.52e-02 s1=1.52e-02 s2=1.52e-02 s3=1.52e-02 s4=1.52e-02 s5=2.24e-01 s6=1.52e-02 s7=1.52e-02 s8=2.24e-01 s9=2.24e-01 s10=1.52e-02 s11=2.24e-01 s12=1.52e-02 s13=2.24e-01 s20=1.52e-02 s21=1.52e-02 s22=1.52e-02 s23=1.52e-02 s24=1.52e-02 s25=1.52e-02 s26=1.52e-02 s27=1.52e-02 s30=1.52e-02 s31=1.52e-02 s32=1.52e-02 s33=1.52e-02 s34=1.52e-02 s35=1.52e-02 s36=1.52e-02 s37=1.52e-02 s50=1.52e-02 s51=1.52e-02 s52=1.52e-02 s53=1.52e-02 s54=1.52e-02 s60=1.52e-02 s61=1.52e-02 s62=1.52e-02 s63=1.52e-02 s64=1.52e-02 s65=1.52e-02 s66=1.52e-02 s67=1.52e-02 s68=1.52e-02 s70=1.52e-02 s71=1.52e-02 s72=1.52e-02 s73=1.52e-02  bit-identical across all shapes: False
  out_proj   k=4096   n=2688   M=17    s0=1.53e-02 s1=1.53e-02 s2=1.53e-02 s3=1.53e-02 s4=1.53e-02 s5=2.24e-01 s6=1.53e-02 s7=1.53e-02 s8=2.24e-01 s9=nan s10=1.53e-02 s11=2.24e-01 s12=1.53e-02 s13=2.24e-01 s20=1.53e-02 s21=1.53e-02 s22=1.53e-02 s23=1.53e-02 s24=1.53e-02 s25=1.53e-02 s26=1.53e-02 s27=1.53e-02 s30=1.53e-02 s31=1.53e-02 s32=1.53e-02 s33=1.53e-02 s34=1.53e-02 s35=1.53e-02 s36=1.53e-02 s37=1.53e-02 s50=1.53e-02 s51=1.53e-02 s52=1.53e-02 s53=1.53e-02 s54=1.53e-02 s60=1.53e-02 s61=1.53e-02 s62=1.53e-02 s63=1.53e-02 s64=1.53e-02 s65=1.53e-02 s66=1.53e-02 s67=1.53e-02 s68=1.53e-02 s70=1.53e-02 s71=1.53e-02 s72=1.53e-02 s73=1.53e-02  bit-identical across all shapes: False
  out_proj   k=4096   n=2688   M=63    s0=1.53e-02 s1=1.53e-02 s2=1.53e-02 s3=1.53e-02 s4=1.53e-02 s5=nan s6=1.53e-02 s7=1.53e-02 s8=1.53e-02 s9=2.20e-01 s10=1.53e-02 s11=nan s12=1.53e-02 s13=2.20e-01 s20=1.53e-02 s21=1.53e-02 s22=1.53e-02 s23=1.53e-02 s24=1.53e-02 s25=1.53e-02 s26=1.53e-02 s27=1.53e-02 s30=1.53e-02 s31=1.53e-02 s32=1.53e-02 s33=1.53e-02 s34=1.53e-02 s35=1.53e-02 s36=1.53e-02 s37=1.53e-02 s50=1.53e-02 s51=1.53e-02 s52=1.53e-02 s53=1.53e-02 s54=1.53e-02 s60=1.53e-02 s61=1.53e-02 s62=1.53e-02 s63=1.53e-02 s64=1.53e-02 s65=1.53e-02 s66=1.53e-02 s67=1.53e-02 s68=1.53e-02 s70=1.53e-02 s71=1.53e-02 s72=1.53e-02 s73=1.53e-02  bit-identical across all shapes: False
  out_proj   k=4096   n=2688   M=256   s0=1.53e-02 s1=1.53e-02 s2=1.53e-02 s3=1.53e-02 s4=1.53e-02 s5=1.53e-02 s6=1.53e-02 s7=1.53e-02 s8=1.53e-02 s9=1.53e-02 s10=1.53e-02 s11=1.53e-02 s12=1.53e-02 s13=1.53e-02 s20=1.53e-02 s21=1.53e-02 s22=1.53e-02 s23=1.53e-02 s24=1.53e-02 s25=1.53e-02 s26=1.53e-02 s27=1.53e-02 s30=1.53e-02 s31=1.53e-02 s32=1.53e-02 s33=1.53e-02 s34=1.53e-02 s35=1.53e-02 s36=1.53e-02 s37=1.53e-02 s50=1.53e-02 s51=1.53e-02 s52=1.53e-02 s53=1.53e-02 s54=1.53e-02 s60=1.53e-02 s61=1.53e-02 s62=1.53e-02 s63=1.53e-02 s64=1.53e-02 s65=1.53e-02 s66=1.53e-02 s67=1.53e-02 s68=1.53e-02 s70=1.53e-02 s71=1.53e-02 s72=1.53e-02 s73=1.53e-02  bit-identical across all shapes: True
  out_proj   k=4096   n=2688   M=1000  s0=1.52e-02 s1=1.52e-02 s2=1.52e-02 s3=1.52e-02 s4=1.52e-02 s5=2.19e-01 s6=1.52e-02 s7=1.52e-02 s8=1.52e-02 s9=2.19e-01 s10=1.52e-02 s11=2.19e-01 s12=1.52e-02 s13=2.19e-01 s20=1.52e-02 s21=1.52e-02 s22=1.52e-02 s23=1.52e-02 s24=1.52e-02 s25=1.52e-02 s26=1.52e-02 s27=1.52e-02 s30=1.52e-02 s31=1.52e-02 s32=1.52e-02 s33=1.52e-02 s34=1.52e-02 s35=1.52e-02 s36=1.52e-02 s37=1.52e-02 s50=1.52e-02 s51=1.52e-02 s52=1.52e-02 s53=1.52e-02 s54=1.52e-02 s60=1.52e-02 s61=1.52e-02 s62=1.52e-02 s63=1.52e-02 s64=1.52e-02 s65=1.52e-02 s66=1.52e-02 s67=1.52e-02 s68=1.52e-02 s70=1.52e-02 s71=1.52e-02 s72=1.52e-02 s73=1.52e-02  bit-identical across all shapes: False
  out_proj   k=4096   n=2688   M=2048  s0=1.52e-02 s1=1.52e-02 s2=1.52e-02 s3=1.52e-02 s4=1.52e-02 s5=2.19e-01 s6=1.52e-02 s7=1.52e-02 s8=2.19e-01 s9=2.19e-01 s10=1.52e-02 s11=2.19e-01 s12=1.52e-02 s13=2.19e-01 s20=1.52e-02 s21=1.52e-02 s22=1.52e-02 s23=1.52e-02 s24=1.52e-02 s25=1.52e-02 s26=1.52e-02 s27=1.52e-02 s30=1.52e-02 s31=1.52e-02 s32=1.52e-02 s33=1.52e-02 s34=1.52e-02 s35=1.52e-02 s36=1.52e-02 s37=1.52e-02 s50=1.52e-02 s51=1.52e-02 s52=1.52e-02 s53=1.52e-02 s54=1.52e-02 s60=1.52e-02 s61=1.52e-02 s62=1.52e-02 s63=1.52e-02 s64=1.52e-02 s65=1.52e-02 s66=1.52e-02 s67=1.52e-02 s68=1.52e-02 s70=1.52e-02 s71=1.52e-02 s72=1.52e-02 s73=1.52e-02  bit-identical across all shapes: False
  out_proj   k=4096   n=2688   M=8192  s0=1.52e-02 s1=1.52e-02 s2=1.52e-02 s3=1.52e-02 s4=1.52e-02 s5=nan s6=1.52e-02 s7=1.52e-02 s8=1.52e-02 s9=nan s10=1.52e-02 s11=nan s12=1.52e-02 s13=nan s20=1.52e-02 s21=1.52e-02 s22=1.52e-02 s23=1.52e-02 s24=1.52e-02 s25=1.52e-02 s26=1.52e-02 s27=1.52e-02 s30=1.52e-02 s31=1.52e-02 s32=1.52e-02 s33=1.52e-02 s34=1.52e-02 s35=1.52e-02 s36=1.52e-02 s37=1.52e-02 s50=1.52e-02 s51=1.52e-02 s52=1.52e-02 s53=1.52e-02 s54=1.52e-02 s60=1.52e-02 s61=1.52e-02 s62=1.52e-02 s63=1.52e-02 s64=1.52e-02 s65=1.52e-02 s66=1.52e-02 s67=1.52e-02 s68=1.52e-02 s70=1.52e-02 s71=1.52e-02 s72=1.52e-02 s73=1.52e-02  bit-identical across all shapes: False
[sweep] compared 14 (linear, M) pair(s)
SHAPE SWEEP: FAIL

$ python3 sweep_shapes.py --layer 1 --linears shared_experts.up_proj shared_experts.down_proj
  shared_experts.up_proj k=2688   n=3712   M=1     s0=1.33e-02 s1=1.33e-02 s2=1.33e-02 s3=1.33e-02 s4=1.33e-02 s5=1.68e-01 s6=1.33e-02 s7=1.33e-02 s8=1.68e-01 s9=1.68e-01 s10=1.33e-02 s11=1.68e-01 s12=1.33e-02 s13=nan s20=1.33e-02 s21=1.33e-02 s22=1.33e-02 s23=1.33e-02 s24=1.33e-02 s25=1.33e-02 s26=1.33e-02 s27=1.33e-02 s30=1.33e-02 s31=1.33e-02 s32=1.33e-02 s33=1.33e-02 s34=1.33e-02 s35=1.33e-02 s36=1.33e-02 s37=1.33e-02 s50=1.33e-02 s51=1.33e-02 s52=1.33e-02 s53=1.33e-02 s54=1.33e-02 s60=1.33e-02 s61=1.33e-02 s62=1.33e-02 s63=1.33e-02 s64=1.33e-02 s65=1.33e-02 s66=1.33e-02 s67=1.33e-02 s68=1.33e-02 s70=1.33e-02 s71=1.33e-02 s72=1.33e-02 s73=1.33e-02  bit-identical across all shapes: False
  shared_experts.up_proj k=2688   n=3712   M=17    s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s5=nan s6=1.27e-02 s7=1.27e-02 s8=1.27e-02 s9=nan s10=1.27e-02 s11=nan s12=1.27e-02 s13=nan s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: False
  shared_experts.up_proj k=2688   n=3712   M=63    s0=1.28e-02 s1=1.28e-02 s2=1.28e-02 s3=1.28e-02 s4=1.28e-02 s5=nan s6=1.28e-02 s7=1.28e-02 s8=1.28e-02 s9=1.28e-02 s10=1.28e-02 s11=1.28e-02 s12=1.28e-02 s13=nan s20=1.28e-02 s21=1.28e-02 s22=1.28e-02 s23=1.28e-02 s24=1.28e-02 s25=1.28e-02 s26=1.28e-02 s27=1.28e-02 s30=1.28e-02 s31=1.28e-02 s32=1.28e-02 s33=1.28e-02 s34=1.28e-02 s35=1.28e-02 s36=1.28e-02 s37=1.28e-02 s50=1.28e-02 s51=1.28e-02 s52=1.28e-02 s53=1.28e-02 s54=1.28e-02 s60=1.28e-02 s61=1.28e-02 s62=1.28e-02 s63=1.28e-02 s64=1.28e-02 s65=1.28e-02 s66=1.28e-02 s67=1.28e-02 s68=1.28e-02 s70=1.28e-02 s71=1.28e-02 s72=1.28e-02 s73=1.28e-02  bit-identical across all shapes: False
  shared_experts.up_proj k=2688   n=3712   M=256   s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s5=nan s6=1.27e-02 s7=1.27e-02 s8=1.58e-01 s9=nan s10=1.27e-02 s11=nan s12=1.27e-02 s13=nan s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: False
  shared_experts.up_proj k=2688   n=3712   M=1000  s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s5=nan s6=1.27e-02 s7=1.27e-02 s8=1.59e-01 s9=nan s10=1.27e-02 s11=nan s12=1.27e-02 s13=nan s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: False
  shared_experts.up_proj k=2688   n=3712   M=2048  s0=1.28e-02 s1=1.28e-02 s2=1.28e-02 s3=1.28e-02 s4=1.28e-02 s5=1.28e-02 s6=1.28e-02 s7=1.28e-02 s8=1.28e-02 s9=1.28e-02 s10=1.28e-02 s11=1.28e-02 s12=1.28e-02 s13=1.28e-02 s20=1.28e-02 s21=1.28e-02 s22=1.28e-02 s23=1.28e-02 s24=1.28e-02 s25=1.28e-02 s26=1.28e-02 s27=1.28e-02 s30=1.28e-02 s31=1.28e-02 s32=1.28e-02 s33=1.28e-02 s34=1.28e-02 s35=1.28e-02 s36=1.28e-02 s37=1.28e-02 s50=1.28e-02 s51=1.28e-02 s52=1.28e-02 s53=1.28e-02 s54=1.28e-02 s60=1.28e-02 s61=1.28e-02 s62=1.28e-02 s63=1.28e-02 s64=1.28e-02 s65=1.28e-02 s66=1.28e-02 s67=1.28e-02 s68=1.28e-02 s70=1.28e-02 s71=1.28e-02 s72=1.28e-02 s73=1.28e-02  bit-identical across all shapes: True
  shared_experts.up_proj k=2688   n=3712   M=8192  s0=1.28e-02 s1=1.28e-02 s2=1.28e-02 s3=1.28e-02 s4=1.28e-02 s5=1.59e-01 s6=1.28e-02 s7=1.28e-02 s8=1.59e-01 s9=1.59e-01 s10=1.28e-02 s11=1.59e-01 s12=1.28e-02 s13=1.59e-01 s20=1.28e-02 s21=1.28e-02 s22=1.28e-02 s23=1.28e-02 s24=1.28e-02 s25=1.28e-02 s26=1.28e-02 s27=1.28e-02 s30=1.28e-02 s31=1.28e-02 s32=1.28e-02 s33=1.28e-02 s34=1.28e-02 s35=1.28e-02 s36=1.28e-02 s37=1.28e-02 s50=1.28e-02 s51=1.28e-02 s52=1.28e-02 s53=1.28e-02 s54=1.28e-02 s60=1.28e-02 s61=1.28e-02 s62=1.28e-02 s63=1.28e-02 s64=1.28e-02 s65=1.28e-02 s66=1.28e-02 s67=1.28e-02 s68=1.28e-02 s70=1.28e-02 s71=1.28e-02 s72=1.28e-02 s73=1.28e-02  bit-identical across all shapes: False
  shared_experts.down_proj k=3712   n=2688   M=1     s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s5=2.04e-01 s6=1.30e-02 s7=1.30e-02 s8=1.30e-02 s9=2.04e-01 s10=1.30e-02 s11=2.04e-01 s12=1.30e-02 s13=2.04e-01 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: False
  shared_experts.down_proj k=3712   n=2688   M=17    s0=1.25e-02 s1=1.25e-02 s2=1.25e-02 s3=1.25e-02 s4=1.25e-02 s5=2.14e-01 s6=1.25e-02 s7=1.25e-02 s8=1.25e-02 s9=nan s10=1.25e-02 s11=2.14e-01 s12=1.25e-02 s13=2.14e-01 s20=1.25e-02 s21=1.25e-02 s22=1.25e-02 s23=1.25e-02 s24=1.25e-02 s25=1.25e-02 s26=1.25e-02 s27=1.25e-02 s30=1.25e-02 s31=1.25e-02 s32=1.25e-02 s33=1.25e-02 s34=1.25e-02 s35=1.25e-02 s36=1.25e-02 s37=1.25e-02 s50=1.25e-02 s51=1.25e-02 s52=1.25e-02 s53=1.25e-02 s54=1.25e-02 s60=1.25e-02 s61=1.25e-02 s62=1.25e-02 s63=1.25e-02 s64=1.25e-02 s65=1.25e-02 s66=1.25e-02 s67=1.25e-02 s68=1.25e-02 s70=1.25e-02 s71=1.25e-02 s72=1.25e-02 s73=1.25e-02  bit-identical across all shapes: False
  shared_experts.down_proj k=3712   n=2688   M=63    s0=1.26e-02 s1=1.26e-02 s2=1.26e-02 s3=1.26e-02 s4=1.26e-02 s5=nan s6=1.26e-02 s7=1.26e-02 s8=1.26e-02 s9=2.18e-01 s10=1.26e-02 s11=nan s12=1.26e-02 s13=2.18e-01 s20=1.26e-02 s21=1.26e-02 s22=1.26e-02 s23=1.26e-02 s24=1.26e-02 s25=1.26e-02 s26=1.26e-02 s27=1.26e-02 s30=1.26e-02 s31=1.26e-02 s32=1.26e-02 s33=1.26e-02 s34=1.26e-02 s35=1.26e-02 s36=1.26e-02 s37=1.26e-02 s50=1.26e-02 s51=1.26e-02 s52=1.26e-02 s53=1.26e-02 s54=1.26e-02 s60=1.26e-02 s61=1.26e-02 s62=1.26e-02 s63=1.26e-02 s64=1.26e-02 s65=1.26e-02 s66=1.26e-02 s67=1.26e-02 s68=1.26e-02 s70=1.26e-02 s71=1.26e-02 s72=1.26e-02 s73=1.26e-02  bit-identical across all shapes: False
  shared_experts.down_proj k=3712   n=2688   M=256   s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s5=2.18e-01 s6=1.27e-02 s7=1.27e-02 s8=1.27e-02 s9=2.18e-01 s10=1.27e-02 s11=2.18e-01 s12=1.27e-02 s13=2.18e-01 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: False
  shared_experts.down_proj k=3712   n=2688   M=1000  s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s5=2.18e-01 s6=1.27e-02 s7=1.27e-02 s8=1.27e-02 s9=2.18e-01 s10=1.27e-02 s11=2.18e-01 s12=1.27e-02 s13=2.18e-01 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: False
  shared_experts.down_proj k=3712   n=2688   M=2048  s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s5=1.27e-02 s6=1.27e-02 s7=1.27e-02 s8=2.19e-01 s9=1.27e-02 s10=1.27e-02 s11=1.27e-02 s12=1.27e-02 s13=1.27e-02 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: False
  shared_experts.down_proj k=3712   n=2688   M=8192  s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s5=2.19e-01 s6=1.27e-02 s7=1.27e-02 s8=1.27e-02 s9=2.19e-01 s10=1.27e-02 s11=2.19e-01 s12=1.27e-02 s13=2.19e-01 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: False
[sweep] compared 14 (linear, M) pair(s)
SHAPE SWEEP: FAIL

$ python3 sweep_shapes.py --layer 5 --linears q_proj k_proj v_proj o_proj
  q_proj     k=2688   n=4096   M=1     s0=1.45e-02 s1=1.45e-02 s2=1.45e-02 s3=1.45e-02 s4=1.45e-02 s5=1.45e-02 s6=1.45e-02 s7=1.45e-02 s8=1.45e-02 s9=1.45e-02 s10=1.45e-02 s11=1.45e-02 s12=1.45e-02 s13=1.45e-02 s20=1.45e-02 s21=1.45e-02 s22=1.45e-02 s23=1.45e-02 s24=1.45e-02 s25=1.45e-02 s26=1.45e-02 s27=1.45e-02 s30=1.45e-02 s31=1.45e-02 s32=1.45e-02 s33=1.45e-02 s34=1.45e-02 s35=1.45e-02 s36=1.45e-02 s37=1.45e-02 s50=1.45e-02 s51=1.45e-02 s52=1.45e-02 s53=1.45e-02 s54=1.45e-02 s60=1.45e-02 s61=1.45e-02 s62=1.45e-02 s63=1.45e-02 s64=1.45e-02 s65=1.45e-02 s66=1.45e-02 s67=1.45e-02 s68=1.45e-02 s70=1.45e-02 s71=1.45e-02 s72=1.45e-02 s73=1.45e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=17    s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s5=1.29e-02 s6=1.29e-02 s7=1.29e-02 s8=1.29e-02 s9=1.29e-02 s10=1.29e-02 s11=1.29e-02 s12=1.29e-02 s13=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=63    s0=1.31e-02 s1=1.31e-02 s2=1.31e-02 s3=1.31e-02 s4=1.31e-02 s5=1.31e-02 s6=1.31e-02 s7=1.31e-02 s8=1.31e-02 s9=1.31e-02 s10=1.31e-02 s11=1.31e-02 s12=1.31e-02 s13=1.31e-02 s20=1.31e-02 s21=1.31e-02 s22=1.31e-02 s23=1.31e-02 s24=1.31e-02 s25=1.31e-02 s26=1.31e-02 s27=1.31e-02 s30=1.31e-02 s31=1.31e-02 s32=1.31e-02 s33=1.31e-02 s34=1.31e-02 s35=1.31e-02 s36=1.31e-02 s37=1.31e-02 s50=1.31e-02 s51=1.31e-02 s52=1.31e-02 s53=1.31e-02 s54=1.31e-02 s60=1.31e-02 s61=1.31e-02 s62=1.31e-02 s63=1.31e-02 s64=1.31e-02 s65=1.31e-02 s66=1.31e-02 s67=1.31e-02 s68=1.31e-02 s70=1.31e-02 s71=1.31e-02 s72=1.31e-02 s73=1.31e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=256   s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s5=1.29e-02 s6=1.29e-02 s7=1.29e-02 s8=1.29e-02 s9=1.29e-02 s10=1.29e-02 s11=1.29e-02 s12=1.29e-02 s13=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=1000  s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s5=1.29e-02 s6=1.29e-02 s7=1.29e-02 s8=1.29e-02 s9=1.29e-02 s10=1.29e-02 s11=1.29e-02 s12=1.29e-02 s13=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=2048  s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s5=1.29e-02 s6=1.29e-02 s7=1.29e-02 s8=1.29e-02 s9=1.29e-02 s10=1.29e-02 s11=1.29e-02 s12=1.29e-02 s13=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=8192  s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s5=1.29e-02 s6=1.29e-02 s7=1.29e-02 s8=1.29e-02 s9=1.29e-02 s10=1.29e-02 s11=1.29e-02 s12=1.29e-02 s13=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=1     s0=1.17e-02 s1=1.17e-02 s2=1.17e-02 s3=1.17e-02 s4=1.17e-02 s5=1.17e-02 s6=1.17e-02 s7=1.17e-02 s8=1.17e-02 s9=1.17e-02 s10=1.17e-02 s11=1.17e-02 s12=1.17e-02 s13=1.17e-02 s20=1.17e-02 s21=1.17e-02 s22=1.17e-02 s23=1.17e-02 s24=1.17e-02 s25=1.17e-02 s26=1.17e-02 s27=1.17e-02 s30=1.17e-02 s31=1.17e-02 s32=1.17e-02 s33=1.17e-02 s34=1.17e-02 s35=1.17e-02 s36=1.17e-02 s37=1.17e-02 s50=1.17e-02 s51=1.17e-02 s52=1.17e-02 s53=1.17e-02 s54=1.17e-02 s60=1.17e-02 s61=1.17e-02 s62=1.17e-02 s63=1.17e-02 s64=1.17e-02 s65=1.17e-02 s66=1.17e-02 s67=1.17e-02 s68=1.17e-02 s70=1.17e-02 s71=1.17e-02 s72=1.17e-02 s73=1.17e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=17    s0=1.23e-02 s1=1.23e-02 s2=1.23e-02 s3=1.23e-02 s4=1.23e-02 s5=1.23e-02 s6=1.23e-02 s7=1.23e-02 s8=1.23e-02 s9=1.23e-02 s10=1.23e-02 s11=1.23e-02 s12=1.23e-02 s13=1.23e-02 s20=1.23e-02 s21=1.23e-02 s22=1.23e-02 s23=1.23e-02 s24=1.23e-02 s25=1.23e-02 s26=1.23e-02 s27=1.23e-02 s30=1.23e-02 s31=1.23e-02 s32=1.23e-02 s33=1.23e-02 s34=1.23e-02 s35=1.23e-02 s36=1.23e-02 s37=1.23e-02 s50=1.23e-02 s51=1.23e-02 s52=1.23e-02 s53=1.23e-02 s54=1.23e-02 s60=1.23e-02 s61=1.23e-02 s62=1.23e-02 s63=1.23e-02 s64=1.23e-02 s65=1.23e-02 s66=1.23e-02 s67=1.23e-02 s68=1.23e-02 s70=1.23e-02 s71=1.23e-02 s72=1.23e-02 s73=1.23e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=63    s0=1.25e-02 s1=1.25e-02 s2=1.25e-02 s3=1.25e-02 s4=1.25e-02 s5=1.25e-02 s6=1.25e-02 s7=1.25e-02 s8=1.25e-02 s9=1.25e-02 s10=1.25e-02 s11=1.25e-02 s12=1.25e-02 s13=1.25e-02 s20=1.25e-02 s21=1.25e-02 s22=1.25e-02 s23=1.25e-02 s24=1.25e-02 s25=1.25e-02 s26=1.25e-02 s27=1.25e-02 s30=1.25e-02 s31=1.25e-02 s32=1.25e-02 s33=1.25e-02 s34=1.25e-02 s35=1.25e-02 s36=1.25e-02 s37=1.25e-02 s50=1.25e-02 s51=1.25e-02 s52=1.25e-02 s53=1.25e-02 s54=1.25e-02 s60=1.25e-02 s61=1.25e-02 s62=1.25e-02 s63=1.25e-02 s64=1.25e-02 s65=1.25e-02 s66=1.25e-02 s67=1.25e-02 s68=1.25e-02 s70=1.25e-02 s71=1.25e-02 s72=1.25e-02 s73=1.25e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=256   s0=1.26e-02 s1=1.26e-02 s2=1.26e-02 s3=1.26e-02 s4=1.26e-02 s5=1.26e-02 s6=1.26e-02 s7=1.26e-02 s8=1.26e-02 s9=1.26e-02 s10=1.26e-02 s11=1.26e-02 s12=1.26e-02 s13=1.26e-02 s20=1.26e-02 s21=1.26e-02 s22=1.26e-02 s23=1.26e-02 s24=1.26e-02 s25=1.26e-02 s26=1.26e-02 s27=1.26e-02 s30=1.26e-02 s31=1.26e-02 s32=1.26e-02 s33=1.26e-02 s34=1.26e-02 s35=1.26e-02 s36=1.26e-02 s37=1.26e-02 s50=1.26e-02 s51=1.26e-02 s52=1.26e-02 s53=1.26e-02 s54=1.26e-02 s60=1.26e-02 s61=1.26e-02 s62=1.26e-02 s63=1.26e-02 s64=1.26e-02 s65=1.26e-02 s66=1.26e-02 s67=1.26e-02 s68=1.26e-02 s70=1.26e-02 s71=1.26e-02 s72=1.26e-02 s73=1.26e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=1000  s0=1.26e-02 s1=1.26e-02 s2=1.26e-02 s3=1.26e-02 s4=1.26e-02 s5=1.26e-02 s6=1.26e-02 s7=1.26e-02 s8=1.26e-02 s9=1.26e-02 s10=1.26e-02 s11=1.26e-02 s12=1.26e-02 s13=1.26e-02 s20=1.26e-02 s21=1.26e-02 s22=1.26e-02 s23=1.26e-02 s24=1.26e-02 s25=1.26e-02 s26=1.26e-02 s27=1.26e-02 s30=1.26e-02 s31=1.26e-02 s32=1.26e-02 s33=1.26e-02 s34=1.26e-02 s35=1.26e-02 s36=1.26e-02 s37=1.26e-02 s50=1.26e-02 s51=1.26e-02 s52=1.26e-02 s53=1.26e-02 s54=1.26e-02 s60=1.26e-02 s61=1.26e-02 s62=1.26e-02 s63=1.26e-02 s64=1.26e-02 s65=1.26e-02 s66=1.26e-02 s67=1.26e-02 s68=1.26e-02 s70=1.26e-02 s71=1.26e-02 s72=1.26e-02 s73=1.26e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=2048  s0=1.26e-02 s1=1.26e-02 s2=1.26e-02 s3=1.26e-02 s4=1.26e-02 s5=1.26e-02 s6=1.26e-02 s7=1.26e-02 s8=1.26e-02 s9=1.26e-02 s10=1.26e-02 s11=1.26e-02 s12=1.26e-02 s13=1.26e-02 s20=1.26e-02 s21=1.26e-02 s22=1.26e-02 s23=1.26e-02 s24=1.26e-02 s25=1.26e-02 s26=1.26e-02 s27=1.26e-02 s30=1.26e-02 s31=1.26e-02 s32=1.26e-02 s33=1.26e-02 s34=1.26e-02 s35=1.26e-02 s36=1.26e-02 s37=1.26e-02 s50=1.26e-02 s51=1.26e-02 s52=1.26e-02 s53=1.26e-02 s54=1.26e-02 s60=1.26e-02 s61=1.26e-02 s62=1.26e-02 s63=1.26e-02 s64=1.26e-02 s65=1.26e-02 s66=1.26e-02 s67=1.26e-02 s68=1.26e-02 s70=1.26e-02 s71=1.26e-02 s72=1.26e-02 s73=1.26e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=8192  s0=1.26e-02 s1=1.26e-02 s2=1.26e-02 s3=1.26e-02 s4=1.26e-02 s5=1.26e-02 s6=1.26e-02 s7=1.26e-02 s8=1.26e-02 s9=1.26e-02 s10=1.26e-02 s11=1.26e-02 s12=1.26e-02 s13=1.26e-02 s20=1.26e-02 s21=1.26e-02 s22=1.26e-02 s23=1.26e-02 s24=1.26e-02 s25=1.26e-02 s26=1.26e-02 s27=1.26e-02 s30=1.26e-02 s31=1.26e-02 s32=1.26e-02 s33=1.26e-02 s34=1.26e-02 s35=1.26e-02 s36=1.26e-02 s37=1.26e-02 s50=1.26e-02 s51=1.26e-02 s52=1.26e-02 s53=1.26e-02 s54=1.26e-02 s60=1.26e-02 s61=1.26e-02 s62=1.26e-02 s63=1.26e-02 s64=1.26e-02 s65=1.26e-02 s66=1.26e-02 s67=1.26e-02 s68=1.26e-02 s70=1.26e-02 s71=1.26e-02 s72=1.26e-02 s73=1.26e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=1     s0=1.24e-02 s1=1.24e-02 s2=1.24e-02 s3=1.24e-02 s4=1.24e-02 s5=1.24e-02 s6=1.24e-02 s7=1.24e-02 s8=1.24e-02 s9=1.24e-02 s10=1.24e-02 s11=1.24e-02 s12=1.24e-02 s13=1.24e-02 s20=1.24e-02 s21=1.24e-02 s22=1.24e-02 s23=1.24e-02 s24=1.24e-02 s25=1.24e-02 s26=1.24e-02 s27=1.24e-02 s30=1.24e-02 s31=1.24e-02 s32=1.24e-02 s33=1.24e-02 s34=1.24e-02 s35=1.24e-02 s36=1.24e-02 s37=1.24e-02 s50=1.24e-02 s51=1.24e-02 s52=1.24e-02 s53=1.24e-02 s54=1.24e-02 s60=1.24e-02 s61=1.24e-02 s62=1.24e-02 s63=1.24e-02 s64=1.24e-02 s65=1.24e-02 s66=1.24e-02 s67=1.24e-02 s68=1.24e-02 s70=1.24e-02 s71=1.24e-02 s72=1.24e-02 s73=1.24e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=17    s0=1.33e-02 s1=1.33e-02 s2=1.33e-02 s3=1.33e-02 s4=1.33e-02 s5=1.33e-02 s6=1.33e-02 s7=1.33e-02 s8=1.33e-02 s9=1.33e-02 s10=1.33e-02 s11=1.33e-02 s12=1.33e-02 s13=1.33e-02 s20=1.33e-02 s21=1.33e-02 s22=1.33e-02 s23=1.33e-02 s24=1.33e-02 s25=1.33e-02 s26=1.33e-02 s27=1.33e-02 s30=1.33e-02 s31=1.33e-02 s32=1.33e-02 s33=1.33e-02 s34=1.33e-02 s35=1.33e-02 s36=1.33e-02 s37=1.33e-02 s50=1.33e-02 s51=1.33e-02 s52=1.33e-02 s53=1.33e-02 s54=1.33e-02 s60=1.33e-02 s61=1.33e-02 s62=1.33e-02 s63=1.33e-02 s64=1.33e-02 s65=1.33e-02 s66=1.33e-02 s67=1.33e-02 s68=1.33e-02 s70=1.33e-02 s71=1.33e-02 s72=1.33e-02 s73=1.33e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=63    s0=1.31e-02 s1=1.31e-02 s2=1.31e-02 s3=1.31e-02 s4=1.31e-02 s5=1.31e-02 s6=1.31e-02 s7=1.31e-02 s8=1.31e-02 s9=1.31e-02 s10=1.31e-02 s11=1.31e-02 s12=1.31e-02 s13=1.31e-02 s20=1.31e-02 s21=1.31e-02 s22=1.31e-02 s23=1.31e-02 s24=1.31e-02 s25=1.31e-02 s26=1.31e-02 s27=1.31e-02 s30=1.31e-02 s31=1.31e-02 s32=1.31e-02 s33=1.31e-02 s34=1.31e-02 s35=1.31e-02 s36=1.31e-02 s37=1.31e-02 s50=1.31e-02 s51=1.31e-02 s52=1.31e-02 s53=1.31e-02 s54=1.31e-02 s60=1.31e-02 s61=1.31e-02 s62=1.31e-02 s63=1.31e-02 s64=1.31e-02 s65=1.31e-02 s66=1.31e-02 s67=1.31e-02 s68=1.31e-02 s70=1.31e-02 s71=1.31e-02 s72=1.31e-02 s73=1.31e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=256   s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s5=1.29e-02 s6=1.29e-02 s7=1.29e-02 s8=1.29e-02 s9=1.29e-02 s10=1.29e-02 s11=1.29e-02 s12=1.29e-02 s13=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=1000  s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s5=1.29e-02 s6=1.29e-02 s7=1.29e-02 s8=1.29e-02 s9=1.29e-02 s10=1.29e-02 s11=1.29e-02 s12=1.29e-02 s13=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=2048  s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s5=1.30e-02 s6=1.30e-02 s7=1.30e-02 s8=1.30e-02 s9=1.30e-02 s10=1.30e-02 s11=1.30e-02 s12=1.30e-02 s13=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=8192  s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s5=1.30e-02 s6=1.30e-02 s7=1.30e-02 s8=1.30e-02 s9=1.30e-02 s10=1.30e-02 s11=1.30e-02 s12=1.30e-02 s13=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  o_proj     k=4096   n=2688   M=1     s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s5=2.36e-01 s6=1.29e-02 s7=1.29e-02 s8=1.29e-02 s9=2.36e-01 s10=1.29e-02 s11=2.36e-01 s12=1.29e-02 s13=2.36e-01 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: False
  o_proj     k=4096   n=2688   M=17    s0=1.35e-02 s1=1.35e-02 s2=1.35e-02 s3=1.35e-02 s4=1.35e-02 s5=1.49e-01 s6=1.35e-02 s7=1.35e-02 s8=1.49e-01 s9=1.49e-01 s10=1.35e-02 s11=1.49e-01 s12=1.35e-02 s13=1.49e-01 s20=1.35e-02 s21=1.35e-02 s22=1.35e-02 s23=1.35e-02 s24=1.35e-02 s25=1.35e-02 s26=1.35e-02 s27=1.35e-02 s30=1.35e-02 s31=1.35e-02 s32=1.35e-02 s33=1.35e-02 s34=1.35e-02 s35=1.35e-02 s36=1.35e-02 s37=1.35e-02 s50=1.35e-02 s51=1.35e-02 s52=1.35e-02 s53=1.35e-02 s54=1.35e-02 s60=1.35e-02 s61=1.35e-02 s62=1.35e-02 s63=1.35e-02 s64=1.35e-02 s65=1.35e-02 s66=1.35e-02 s67=1.35e-02 s68=1.35e-02 s70=1.35e-02 s71=1.35e-02 s72=1.35e-02 s73=1.35e-02  bit-identical across all shapes: False
  o_proj     k=4096   n=2688   M=63    s0=1.34e-02 s1=1.34e-02 s2=1.34e-02 s3=1.34e-02 s4=1.34e-02 s5=nan s6=1.34e-02 s7=1.34e-02 s8=1.52e-01 s9=1.88e-01 s10=1.34e-02 s11=nan s12=1.34e-02 s13=2.17e-01 s20=1.34e-02 s21=1.34e-02 s22=1.34e-02 s23=1.34e-02 s24=1.34e-02 s25=1.34e-02 s26=1.34e-02 s27=1.34e-02 s30=1.34e-02 s31=1.34e-02 s32=1.34e-02 s33=1.34e-02 s34=1.34e-02 s35=1.34e-02 s36=1.34e-02 s37=1.34e-02 s50=1.34e-02 s51=1.34e-02 s52=1.34e-02 s53=1.34e-02 s54=1.34e-02 s60=1.34e-02 s61=1.34e-02 s62=1.34e-02 s63=1.34e-02 s64=1.34e-02 s65=1.34e-02 s66=1.34e-02 s67=1.34e-02 s68=1.34e-02 s70=1.34e-02 s71=1.34e-02 s72=1.34e-02 s73=1.34e-02  bit-identical across all shapes: False
  o_proj     k=4096   n=2688   M=256   s0=1.33e-02 s1=1.33e-02 s2=1.33e-02 s3=1.33e-02 s4=1.33e-02 s5=2.19e-01 s6=1.33e-02 s7=1.33e-02 s8=1.33e-02 s9=2.19e-01 s10=1.33e-02 s11=2.19e-01 s12=1.33e-02 s13=2.19e-01 s20=1.33e-02 s21=1.33e-02 s22=1.33e-02 s23=1.33e-02 s24=1.33e-02 s25=1.33e-02 s26=1.33e-02 s27=1.33e-02 s30=1.33e-02 s31=1.33e-02 s32=1.33e-02 s33=1.33e-02 s34=1.33e-02 s35=1.33e-02 s36=1.33e-02 s37=1.33e-02 s50=1.33e-02 s51=1.33e-02 s52=1.33e-02 s53=1.33e-02 s54=1.33e-02 s60=1.33e-02 s61=1.33e-02 s62=1.33e-02 s63=1.33e-02 s64=1.33e-02 s65=1.33e-02 s66=1.33e-02 s67=1.33e-02 s68=1.33e-02 s70=1.33e-02 s71=1.33e-02 s72=1.33e-02 s73=1.33e-02  bit-identical across all shapes: False
  o_proj     k=4096   n=2688   M=1000  s0=1.34e-02 s1=1.34e-02 s2=1.34e-02 s3=1.34e-02 s4=1.34e-02 s5=2.18e-01 s6=1.34e-02 s7=1.34e-02 s8=1.34e-02 s9=2.18e-01 s10=1.34e-02 s11=2.18e-01 s12=1.34e-02 s13=2.18e-01 s20=1.34e-02 s21=1.34e-02 s22=1.34e-02 s23=1.34e-02 s24=1.34e-02 s25=1.34e-02 s26=1.34e-02 s27=1.34e-02 s30=1.34e-02 s31=1.34e-02 s32=1.34e-02 s33=1.34e-02 s34=1.34e-02 s35=1.34e-02 s36=1.34e-02 s37=1.34e-02 s50=1.34e-02 s51=1.34e-02 s52=1.34e-02 s53=1.34e-02 s54=1.34e-02 s60=1.34e-02 s61=1.34e-02 s62=1.34e-02 s63=1.34e-02 s64=1.34e-02 s65=1.34e-02 s66=1.34e-02 s67=1.34e-02 s68=1.34e-02 s70=1.34e-02 s71=1.34e-02 s72=1.34e-02 s73=1.34e-02  bit-identical across all shapes: False
  o_proj     k=4096   n=2688   M=2048  s0=1.34e-02 s1=1.34e-02 s2=1.34e-02 s3=1.34e-02 s4=1.34e-02 s5=1.34e-02 s6=1.34e-02 s7=1.34e-02 s8=2.18e-01 s9=1.34e-02 s10=1.34e-02 s11=1.34e-02 s12=1.34e-02 s13=1.34e-02 s20=1.34e-02 s21=1.34e-02 s22=1.34e-02 s23=1.34e-02 s24=1.34e-02 s25=1.34e-02 s26=1.34e-02 s27=1.34e-02 s30=1.34e-02 s31=1.34e-02 s32=1.34e-02 s33=1.34e-02 s34=1.34e-02 s35=1.34e-02 s36=1.34e-02 s37=1.34e-02 s50=1.34e-02 s51=1.34e-02 s52=1.34e-02 s53=1.34e-02 s54=1.34e-02 s60=1.34e-02 s61=1.34e-02 s62=1.34e-02 s63=1.34e-02 s64=1.34e-02 s65=1.34e-02 s66=1.34e-02 s67=1.34e-02 s68=1.34e-02 s70=1.34e-02 s71=1.34e-02 s72=1.34e-02 s73=1.34e-02  bit-identical across all shapes: False
  o_proj     k=4096   n=2688   M=8192  s0=1.34e-02 s1=1.34e-02 s2=1.34e-02 s3=1.34e-02 s4=1.34e-02 s5=1.34e-02 s6=1.34e-02 s7=1.34e-02 s8=1.34e-02 s9=1.34e-02 s10=1.34e-02 s11=1.34e-02 s12=1.34e-02 s13=1.34e-02 s20=1.34e-02 s21=1.34e-02 s22=1.34e-02 s23=1.34e-02 s24=1.34e-02 s25=1.34e-02 s26=1.34e-02 s27=1.34e-02 s30=1.34e-02 s31=1.34e-02 s32=1.34e-02 s33=1.34e-02 s34=1.34e-02 s35=1.34e-02 s36=1.34e-02 s37=1.34e-02 s50=1.34e-02 s51=1.34e-02 s52=1.34e-02 s53=1.34e-02 s54=1.34e-02 s60=1.34e-02 s61=1.34e-02 s62=1.34e-02 s63=1.34e-02 s64=1.34e-02 s65=1.34e-02 s66=1.34e-02 s67=1.34e-02 s68=1.34e-02 s70=1.34e-02 s71=1.34e-02 s72=1.34e-02 s73=1.34e-02  bit-identical across all shapes: True
[sweep] compared 28 (linear, M) pair(s)
SHAPE SWEEP: FAIL


=================================================================
THE SAME SWEEP RESTRICTED TO THE SHAPES WHOSE TILE_N DIVIDES N.
A 256-wide shape on a 128-but-not-256-aligned N is not a
disagreeing answer -- the grid truncated and the comparison is
against columns nothing wrote. With those excluded, every dense
geometry in this checkpoint is BIT-IDENTICAL across every legal
tile shape at every row count, which is the batch-invariance
contract the shape pin exists to protect.
=================================================================

$ python3 sweep_shapes.py --layer 0 --tile-n 64 128 --linears in_proj out_proj
  in_proj    k=2688   n=10368  M=1     s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s6=1.29e-02 s7=1.29e-02 s10=1.29e-02 s12=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  in_proj    k=2688   n=10368  M=17    s0=1.32e-02 s1=1.32e-02 s2=1.32e-02 s3=1.32e-02 s4=1.32e-02 s6=1.32e-02 s7=1.32e-02 s10=1.32e-02 s12=1.32e-02 s20=1.32e-02 s21=1.32e-02 s22=1.32e-02 s23=1.32e-02 s24=1.32e-02 s25=1.32e-02 s26=1.32e-02 s27=1.32e-02 s30=1.32e-02 s31=1.32e-02 s32=1.32e-02 s33=1.32e-02 s34=1.32e-02 s35=1.32e-02 s36=1.32e-02 s37=1.32e-02 s50=1.32e-02 s51=1.32e-02 s52=1.32e-02 s53=1.32e-02 s54=1.32e-02 s60=1.32e-02 s61=1.32e-02 s62=1.32e-02 s63=1.32e-02 s64=1.32e-02 s65=1.32e-02 s66=1.32e-02 s67=1.32e-02 s68=1.32e-02 s70=1.32e-02 s71=1.32e-02 s72=1.32e-02 s73=1.32e-02  bit-identical across all shapes: True
  in_proj    k=2688   n=10368  M=63    s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s6=1.29e-02 s7=1.29e-02 s10=1.29e-02 s12=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  in_proj    k=2688   n=10368  M=256   s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s6=1.30e-02 s7=1.30e-02 s10=1.30e-02 s12=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  in_proj    k=2688   n=10368  M=1000  s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s6=1.30e-02 s7=1.30e-02 s10=1.30e-02 s12=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  in_proj    k=2688   n=10368  M=2048  s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s6=1.30e-02 s7=1.30e-02 s10=1.30e-02 s12=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  in_proj    k=2688   n=10368  M=8192  s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s6=1.30e-02 s7=1.30e-02 s10=1.30e-02 s12=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  out_proj   k=4096   n=2688   M=1     s0=1.52e-02 s1=1.52e-02 s2=1.52e-02 s3=1.52e-02 s4=1.52e-02 s6=1.52e-02 s7=1.52e-02 s10=1.52e-02 s12=1.52e-02 s20=1.52e-02 s21=1.52e-02 s22=1.52e-02 s23=1.52e-02 s24=1.52e-02 s25=1.52e-02 s26=1.52e-02 s27=1.52e-02 s30=1.52e-02 s31=1.52e-02 s32=1.52e-02 s33=1.52e-02 s34=1.52e-02 s35=1.52e-02 s36=1.52e-02 s37=1.52e-02 s50=1.52e-02 s51=1.52e-02 s52=1.52e-02 s53=1.52e-02 s54=1.52e-02 s60=1.52e-02 s61=1.52e-02 s62=1.52e-02 s63=1.52e-02 s64=1.52e-02 s65=1.52e-02 s66=1.52e-02 s67=1.52e-02 s68=1.52e-02 s70=1.52e-02 s71=1.52e-02 s72=1.52e-02 s73=1.52e-02  bit-identical across all shapes: True
  out_proj   k=4096   n=2688   M=17    s0=1.53e-02 s1=1.53e-02 s2=1.53e-02 s3=1.53e-02 s4=1.53e-02 s6=1.53e-02 s7=1.53e-02 s10=1.53e-02 s12=1.53e-02 s20=1.53e-02 s21=1.53e-02 s22=1.53e-02 s23=1.53e-02 s24=1.53e-02 s25=1.53e-02 s26=1.53e-02 s27=1.53e-02 s30=1.53e-02 s31=1.53e-02 s32=1.53e-02 s33=1.53e-02 s34=1.53e-02 s35=1.53e-02 s36=1.53e-02 s37=1.53e-02 s50=1.53e-02 s51=1.53e-02 s52=1.53e-02 s53=1.53e-02 s54=1.53e-02 s60=1.53e-02 s61=1.53e-02 s62=1.53e-02 s63=1.53e-02 s64=1.53e-02 s65=1.53e-02 s66=1.53e-02 s67=1.53e-02 s68=1.53e-02 s70=1.53e-02 s71=1.53e-02 s72=1.53e-02 s73=1.53e-02  bit-identical across all shapes: True
  out_proj   k=4096   n=2688   M=63    s0=1.53e-02 s1=1.53e-02 s2=1.53e-02 s3=1.53e-02 s4=1.53e-02 s6=1.53e-02 s7=1.53e-02 s10=1.53e-02 s12=1.53e-02 s20=1.53e-02 s21=1.53e-02 s22=1.53e-02 s23=1.53e-02 s24=1.53e-02 s25=1.53e-02 s26=1.53e-02 s27=1.53e-02 s30=1.53e-02 s31=1.53e-02 s32=1.53e-02 s33=1.53e-02 s34=1.53e-02 s35=1.53e-02 s36=1.53e-02 s37=1.53e-02 s50=1.53e-02 s51=1.53e-02 s52=1.53e-02 s53=1.53e-02 s54=1.53e-02 s60=1.53e-02 s61=1.53e-02 s62=1.53e-02 s63=1.53e-02 s64=1.53e-02 s65=1.53e-02 s66=1.53e-02 s67=1.53e-02 s68=1.53e-02 s70=1.53e-02 s71=1.53e-02 s72=1.53e-02 s73=1.53e-02  bit-identical across all shapes: True
  out_proj   k=4096   n=2688   M=256   s0=1.53e-02 s1=1.53e-02 s2=1.53e-02 s3=1.53e-02 s4=1.53e-02 s6=1.53e-02 s7=1.53e-02 s10=1.53e-02 s12=1.53e-02 s20=1.53e-02 s21=1.53e-02 s22=1.53e-02 s23=1.53e-02 s24=1.53e-02 s25=1.53e-02 s26=1.53e-02 s27=1.53e-02 s30=1.53e-02 s31=1.53e-02 s32=1.53e-02 s33=1.53e-02 s34=1.53e-02 s35=1.53e-02 s36=1.53e-02 s37=1.53e-02 s50=1.53e-02 s51=1.53e-02 s52=1.53e-02 s53=1.53e-02 s54=1.53e-02 s60=1.53e-02 s61=1.53e-02 s62=1.53e-02 s63=1.53e-02 s64=1.53e-02 s65=1.53e-02 s66=1.53e-02 s67=1.53e-02 s68=1.53e-02 s70=1.53e-02 s71=1.53e-02 s72=1.53e-02 s73=1.53e-02  bit-identical across all shapes: True
  out_proj   k=4096   n=2688   M=1000  s0=1.52e-02 s1=1.52e-02 s2=1.52e-02 s3=1.52e-02 s4=1.52e-02 s6=1.52e-02 s7=1.52e-02 s10=1.52e-02 s12=1.52e-02 s20=1.52e-02 s21=1.52e-02 s22=1.52e-02 s23=1.52e-02 s24=1.52e-02 s25=1.52e-02 s26=1.52e-02 s27=1.52e-02 s30=1.52e-02 s31=1.52e-02 s32=1.52e-02 s33=1.52e-02 s34=1.52e-02 s35=1.52e-02 s36=1.52e-02 s37=1.52e-02 s50=1.52e-02 s51=1.52e-02 s52=1.52e-02 s53=1.52e-02 s54=1.52e-02 s60=1.52e-02 s61=1.52e-02 s62=1.52e-02 s63=1.52e-02 s64=1.52e-02 s65=1.52e-02 s66=1.52e-02 s67=1.52e-02 s68=1.52e-02 s70=1.52e-02 s71=1.52e-02 s72=1.52e-02 s73=1.52e-02  bit-identical across all shapes: True
  out_proj   k=4096   n=2688   M=2048  s0=1.52e-02 s1=1.52e-02 s2=1.52e-02 s3=1.52e-02 s4=1.52e-02 s6=1.52e-02 s7=1.52e-02 s10=1.52e-02 s12=1.52e-02 s20=1.52e-02 s21=1.52e-02 s22=1.52e-02 s23=1.52e-02 s24=1.52e-02 s25=1.52e-02 s26=1.52e-02 s27=1.52e-02 s30=1.52e-02 s31=1.52e-02 s32=1.52e-02 s33=1.52e-02 s34=1.52e-02 s35=1.52e-02 s36=1.52e-02 s37=1.52e-02 s50=1.52e-02 s51=1.52e-02 s52=1.52e-02 s53=1.52e-02 s54=1.52e-02 s60=1.52e-02 s61=1.52e-02 s62=1.52e-02 s63=1.52e-02 s64=1.52e-02 s65=1.52e-02 s66=1.52e-02 s67=1.52e-02 s68=1.52e-02 s70=1.52e-02 s71=1.52e-02 s72=1.52e-02 s73=1.52e-02  bit-identical across all shapes: True
  out_proj   k=4096   n=2688   M=8192  s0=1.52e-02 s1=1.52e-02 s2=1.52e-02 s3=1.52e-02 s4=1.52e-02 s6=1.52e-02 s7=1.52e-02 s10=1.52e-02 s12=1.52e-02 s20=1.52e-02 s21=1.52e-02 s22=1.52e-02 s23=1.52e-02 s24=1.52e-02 s25=1.52e-02 s26=1.52e-02 s27=1.52e-02 s30=1.52e-02 s31=1.52e-02 s32=1.52e-02 s33=1.52e-02 s34=1.52e-02 s35=1.52e-02 s36=1.52e-02 s37=1.52e-02 s50=1.52e-02 s51=1.52e-02 s52=1.52e-02 s53=1.52e-02 s54=1.52e-02 s60=1.52e-02 s61=1.52e-02 s62=1.52e-02 s63=1.52e-02 s64=1.52e-02 s65=1.52e-02 s66=1.52e-02 s67=1.52e-02 s68=1.52e-02 s70=1.52e-02 s71=1.52e-02 s72=1.52e-02 s73=1.52e-02  bit-identical across all shapes: True
[sweep] compared 14 (linear, M) pair(s) over 43 shape(s) with TILE_N in [64, 128]
SHAPE SWEEP: PASS

$ python3 sweep_shapes.py --layer 1 --tile-n 64 128 --linears shared_experts.up_proj shared_experts.down_proj
  shared_experts.up_proj k=2688   n=3712   M=1     s0=1.33e-02 s1=1.33e-02 s2=1.33e-02 s3=1.33e-02 s4=1.33e-02 s6=1.33e-02 s7=1.33e-02 s10=1.33e-02 s12=1.33e-02 s20=1.33e-02 s21=1.33e-02 s22=1.33e-02 s23=1.33e-02 s24=1.33e-02 s25=1.33e-02 s26=1.33e-02 s27=1.33e-02 s30=1.33e-02 s31=1.33e-02 s32=1.33e-02 s33=1.33e-02 s34=1.33e-02 s35=1.33e-02 s36=1.33e-02 s37=1.33e-02 s50=1.33e-02 s51=1.33e-02 s52=1.33e-02 s53=1.33e-02 s54=1.33e-02 s60=1.33e-02 s61=1.33e-02 s62=1.33e-02 s63=1.33e-02 s64=1.33e-02 s65=1.33e-02 s66=1.33e-02 s67=1.33e-02 s68=1.33e-02 s70=1.33e-02 s71=1.33e-02 s72=1.33e-02 s73=1.33e-02  bit-identical across all shapes: True
  shared_experts.up_proj k=2688   n=3712   M=17    s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s6=1.27e-02 s7=1.27e-02 s10=1.27e-02 s12=1.27e-02 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: True
  shared_experts.up_proj k=2688   n=3712   M=63    s0=1.28e-02 s1=1.28e-02 s2=1.28e-02 s3=1.28e-02 s4=1.28e-02 s6=1.28e-02 s7=1.28e-02 s10=1.28e-02 s12=1.28e-02 s20=1.28e-02 s21=1.28e-02 s22=1.28e-02 s23=1.28e-02 s24=1.28e-02 s25=1.28e-02 s26=1.28e-02 s27=1.28e-02 s30=1.28e-02 s31=1.28e-02 s32=1.28e-02 s33=1.28e-02 s34=1.28e-02 s35=1.28e-02 s36=1.28e-02 s37=1.28e-02 s50=1.28e-02 s51=1.28e-02 s52=1.28e-02 s53=1.28e-02 s54=1.28e-02 s60=1.28e-02 s61=1.28e-02 s62=1.28e-02 s63=1.28e-02 s64=1.28e-02 s65=1.28e-02 s66=1.28e-02 s67=1.28e-02 s68=1.28e-02 s70=1.28e-02 s71=1.28e-02 s72=1.28e-02 s73=1.28e-02  bit-identical across all shapes: True
  shared_experts.up_proj k=2688   n=3712   M=256   s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s6=1.27e-02 s7=1.27e-02 s10=1.27e-02 s12=1.27e-02 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: True
  shared_experts.up_proj k=2688   n=3712   M=1000  s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s6=1.27e-02 s7=1.27e-02 s10=1.27e-02 s12=1.27e-02 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: True
  shared_experts.up_proj k=2688   n=3712   M=2048  s0=1.28e-02 s1=1.28e-02 s2=1.28e-02 s3=1.28e-02 s4=1.28e-02 s6=1.28e-02 s7=1.28e-02 s10=1.28e-02 s12=1.28e-02 s20=1.28e-02 s21=1.28e-02 s22=1.28e-02 s23=1.28e-02 s24=1.28e-02 s25=1.28e-02 s26=1.28e-02 s27=1.28e-02 s30=1.28e-02 s31=1.28e-02 s32=1.28e-02 s33=1.28e-02 s34=1.28e-02 s35=1.28e-02 s36=1.28e-02 s37=1.28e-02 s50=1.28e-02 s51=1.28e-02 s52=1.28e-02 s53=1.28e-02 s54=1.28e-02 s60=1.28e-02 s61=1.28e-02 s62=1.28e-02 s63=1.28e-02 s64=1.28e-02 s65=1.28e-02 s66=1.28e-02 s67=1.28e-02 s68=1.28e-02 s70=1.28e-02 s71=1.28e-02 s72=1.28e-02 s73=1.28e-02  bit-identical across all shapes: True
  shared_experts.up_proj k=2688   n=3712   M=8192  s0=1.28e-02 s1=1.28e-02 s2=1.28e-02 s3=1.28e-02 s4=1.28e-02 s6=1.28e-02 s7=1.28e-02 s10=1.28e-02 s12=1.28e-02 s20=1.28e-02 s21=1.28e-02 s22=1.28e-02 s23=1.28e-02 s24=1.28e-02 s25=1.28e-02 s26=1.28e-02 s27=1.28e-02 s30=1.28e-02 s31=1.28e-02 s32=1.28e-02 s33=1.28e-02 s34=1.28e-02 s35=1.28e-02 s36=1.28e-02 s37=1.28e-02 s50=1.28e-02 s51=1.28e-02 s52=1.28e-02 s53=1.28e-02 s54=1.28e-02 s60=1.28e-02 s61=1.28e-02 s62=1.28e-02 s63=1.28e-02 s64=1.28e-02 s65=1.28e-02 s66=1.28e-02 s67=1.28e-02 s68=1.28e-02 s70=1.28e-02 s71=1.28e-02 s72=1.28e-02 s73=1.28e-02  bit-identical across all shapes: True
  shared_experts.down_proj k=3712   n=2688   M=1     s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s6=1.30e-02 s7=1.30e-02 s10=1.30e-02 s12=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  shared_experts.down_proj k=3712   n=2688   M=17    s0=1.25e-02 s1=1.25e-02 s2=1.25e-02 s3=1.25e-02 s4=1.25e-02 s6=1.25e-02 s7=1.25e-02 s10=1.25e-02 s12=1.25e-02 s20=1.25e-02 s21=1.25e-02 s22=1.25e-02 s23=1.25e-02 s24=1.25e-02 s25=1.25e-02 s26=1.25e-02 s27=1.25e-02 s30=1.25e-02 s31=1.25e-02 s32=1.25e-02 s33=1.25e-02 s34=1.25e-02 s35=1.25e-02 s36=1.25e-02 s37=1.25e-02 s50=1.25e-02 s51=1.25e-02 s52=1.25e-02 s53=1.25e-02 s54=1.25e-02 s60=1.25e-02 s61=1.25e-02 s62=1.25e-02 s63=1.25e-02 s64=1.25e-02 s65=1.25e-02 s66=1.25e-02 s67=1.25e-02 s68=1.25e-02 s70=1.25e-02 s71=1.25e-02 s72=1.25e-02 s73=1.25e-02  bit-identical across all shapes: True
  shared_experts.down_proj k=3712   n=2688   M=63    s0=1.26e-02 s1=1.26e-02 s2=1.26e-02 s3=1.26e-02 s4=1.26e-02 s6=1.26e-02 s7=1.26e-02 s10=1.26e-02 s12=1.26e-02 s20=1.26e-02 s21=1.26e-02 s22=1.26e-02 s23=1.26e-02 s24=1.26e-02 s25=1.26e-02 s26=1.26e-02 s27=1.26e-02 s30=1.26e-02 s31=1.26e-02 s32=1.26e-02 s33=1.26e-02 s34=1.26e-02 s35=1.26e-02 s36=1.26e-02 s37=1.26e-02 s50=1.26e-02 s51=1.26e-02 s52=1.26e-02 s53=1.26e-02 s54=1.26e-02 s60=1.26e-02 s61=1.26e-02 s62=1.26e-02 s63=1.26e-02 s64=1.26e-02 s65=1.26e-02 s66=1.26e-02 s67=1.26e-02 s68=1.26e-02 s70=1.26e-02 s71=1.26e-02 s72=1.26e-02 s73=1.26e-02  bit-identical across all shapes: True
  shared_experts.down_proj k=3712   n=2688   M=256   s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s6=1.27e-02 s7=1.27e-02 s10=1.27e-02 s12=1.27e-02 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: True
  shared_experts.down_proj k=3712   n=2688   M=1000  s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s6=1.27e-02 s7=1.27e-02 s10=1.27e-02 s12=1.27e-02 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: True
  shared_experts.down_proj k=3712   n=2688   M=2048  s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s6=1.27e-02 s7=1.27e-02 s10=1.27e-02 s12=1.27e-02 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: True
  shared_experts.down_proj k=3712   n=2688   M=8192  s0=1.27e-02 s1=1.27e-02 s2=1.27e-02 s3=1.27e-02 s4=1.27e-02 s6=1.27e-02 s7=1.27e-02 s10=1.27e-02 s12=1.27e-02 s20=1.27e-02 s21=1.27e-02 s22=1.27e-02 s23=1.27e-02 s24=1.27e-02 s25=1.27e-02 s26=1.27e-02 s27=1.27e-02 s30=1.27e-02 s31=1.27e-02 s32=1.27e-02 s33=1.27e-02 s34=1.27e-02 s35=1.27e-02 s36=1.27e-02 s37=1.27e-02 s50=1.27e-02 s51=1.27e-02 s52=1.27e-02 s53=1.27e-02 s54=1.27e-02 s60=1.27e-02 s61=1.27e-02 s62=1.27e-02 s63=1.27e-02 s64=1.27e-02 s65=1.27e-02 s66=1.27e-02 s67=1.27e-02 s68=1.27e-02 s70=1.27e-02 s71=1.27e-02 s72=1.27e-02 s73=1.27e-02  bit-identical across all shapes: True
[sweep] compared 14 (linear, M) pair(s) over 43 shape(s) with TILE_N in [64, 128]
SHAPE SWEEP: PASS

$ python3 sweep_shapes.py --layer 5 --tile-n 64 128 --linears q_proj k_proj v_proj o_proj
  q_proj     k=2688   n=4096   M=1     s0=1.45e-02 s1=1.45e-02 s2=1.45e-02 s3=1.45e-02 s4=1.45e-02 s6=1.45e-02 s7=1.45e-02 s10=1.45e-02 s12=1.45e-02 s20=1.45e-02 s21=1.45e-02 s22=1.45e-02 s23=1.45e-02 s24=1.45e-02 s25=1.45e-02 s26=1.45e-02 s27=1.45e-02 s30=1.45e-02 s31=1.45e-02 s32=1.45e-02 s33=1.45e-02 s34=1.45e-02 s35=1.45e-02 s36=1.45e-02 s37=1.45e-02 s50=1.45e-02 s51=1.45e-02 s52=1.45e-02 s53=1.45e-02 s54=1.45e-02 s60=1.45e-02 s61=1.45e-02 s62=1.45e-02 s63=1.45e-02 s64=1.45e-02 s65=1.45e-02 s66=1.45e-02 s67=1.45e-02 s68=1.45e-02 s70=1.45e-02 s71=1.45e-02 s72=1.45e-02 s73=1.45e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=17    s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s6=1.29e-02 s7=1.29e-02 s10=1.29e-02 s12=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=63    s0=1.31e-02 s1=1.31e-02 s2=1.31e-02 s3=1.31e-02 s4=1.31e-02 s6=1.31e-02 s7=1.31e-02 s10=1.31e-02 s12=1.31e-02 s20=1.31e-02 s21=1.31e-02 s22=1.31e-02 s23=1.31e-02 s24=1.31e-02 s25=1.31e-02 s26=1.31e-02 s27=1.31e-02 s30=1.31e-02 s31=1.31e-02 s32=1.31e-02 s33=1.31e-02 s34=1.31e-02 s35=1.31e-02 s36=1.31e-02 s37=1.31e-02 s50=1.31e-02 s51=1.31e-02 s52=1.31e-02 s53=1.31e-02 s54=1.31e-02 s60=1.31e-02 s61=1.31e-02 s62=1.31e-02 s63=1.31e-02 s64=1.31e-02 s65=1.31e-02 s66=1.31e-02 s67=1.31e-02 s68=1.31e-02 s70=1.31e-02 s71=1.31e-02 s72=1.31e-02 s73=1.31e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=256   s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s6=1.29e-02 s7=1.29e-02 s10=1.29e-02 s12=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=1000  s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s6=1.29e-02 s7=1.29e-02 s10=1.29e-02 s12=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=2048  s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s6=1.29e-02 s7=1.29e-02 s10=1.29e-02 s12=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  q_proj     k=2688   n=4096   M=8192  s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s6=1.29e-02 s7=1.29e-02 s10=1.29e-02 s12=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=1     s0=1.17e-02 s1=1.17e-02 s2=1.17e-02 s3=1.17e-02 s4=1.17e-02 s6=1.17e-02 s7=1.17e-02 s10=1.17e-02 s12=1.17e-02 s20=1.17e-02 s21=1.17e-02 s22=1.17e-02 s23=1.17e-02 s24=1.17e-02 s25=1.17e-02 s26=1.17e-02 s27=1.17e-02 s30=1.17e-02 s31=1.17e-02 s32=1.17e-02 s33=1.17e-02 s34=1.17e-02 s35=1.17e-02 s36=1.17e-02 s37=1.17e-02 s50=1.17e-02 s51=1.17e-02 s52=1.17e-02 s53=1.17e-02 s54=1.17e-02 s60=1.17e-02 s61=1.17e-02 s62=1.17e-02 s63=1.17e-02 s64=1.17e-02 s65=1.17e-02 s66=1.17e-02 s67=1.17e-02 s68=1.17e-02 s70=1.17e-02 s71=1.17e-02 s72=1.17e-02 s73=1.17e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=17    s0=1.23e-02 s1=1.23e-02 s2=1.23e-02 s3=1.23e-02 s4=1.23e-02 s6=1.23e-02 s7=1.23e-02 s10=1.23e-02 s12=1.23e-02 s20=1.23e-02 s21=1.23e-02 s22=1.23e-02 s23=1.23e-02 s24=1.23e-02 s25=1.23e-02 s26=1.23e-02 s27=1.23e-02 s30=1.23e-02 s31=1.23e-02 s32=1.23e-02 s33=1.23e-02 s34=1.23e-02 s35=1.23e-02 s36=1.23e-02 s37=1.23e-02 s50=1.23e-02 s51=1.23e-02 s52=1.23e-02 s53=1.23e-02 s54=1.23e-02 s60=1.23e-02 s61=1.23e-02 s62=1.23e-02 s63=1.23e-02 s64=1.23e-02 s65=1.23e-02 s66=1.23e-02 s67=1.23e-02 s68=1.23e-02 s70=1.23e-02 s71=1.23e-02 s72=1.23e-02 s73=1.23e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=63    s0=1.25e-02 s1=1.25e-02 s2=1.25e-02 s3=1.25e-02 s4=1.25e-02 s6=1.25e-02 s7=1.25e-02 s10=1.25e-02 s12=1.25e-02 s20=1.25e-02 s21=1.25e-02 s22=1.25e-02 s23=1.25e-02 s24=1.25e-02 s25=1.25e-02 s26=1.25e-02 s27=1.25e-02 s30=1.25e-02 s31=1.25e-02 s32=1.25e-02 s33=1.25e-02 s34=1.25e-02 s35=1.25e-02 s36=1.25e-02 s37=1.25e-02 s50=1.25e-02 s51=1.25e-02 s52=1.25e-02 s53=1.25e-02 s54=1.25e-02 s60=1.25e-02 s61=1.25e-02 s62=1.25e-02 s63=1.25e-02 s64=1.25e-02 s65=1.25e-02 s66=1.25e-02 s67=1.25e-02 s68=1.25e-02 s70=1.25e-02 s71=1.25e-02 s72=1.25e-02 s73=1.25e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=256   s0=1.26e-02 s1=1.26e-02 s2=1.26e-02 s3=1.26e-02 s4=1.26e-02 s6=1.26e-02 s7=1.26e-02 s10=1.26e-02 s12=1.26e-02 s20=1.26e-02 s21=1.26e-02 s22=1.26e-02 s23=1.26e-02 s24=1.26e-02 s25=1.26e-02 s26=1.26e-02 s27=1.26e-02 s30=1.26e-02 s31=1.26e-02 s32=1.26e-02 s33=1.26e-02 s34=1.26e-02 s35=1.26e-02 s36=1.26e-02 s37=1.26e-02 s50=1.26e-02 s51=1.26e-02 s52=1.26e-02 s53=1.26e-02 s54=1.26e-02 s60=1.26e-02 s61=1.26e-02 s62=1.26e-02 s63=1.26e-02 s64=1.26e-02 s65=1.26e-02 s66=1.26e-02 s67=1.26e-02 s68=1.26e-02 s70=1.26e-02 s71=1.26e-02 s72=1.26e-02 s73=1.26e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=1000  s0=1.26e-02 s1=1.26e-02 s2=1.26e-02 s3=1.26e-02 s4=1.26e-02 s6=1.26e-02 s7=1.26e-02 s10=1.26e-02 s12=1.26e-02 s20=1.26e-02 s21=1.26e-02 s22=1.26e-02 s23=1.26e-02 s24=1.26e-02 s25=1.26e-02 s26=1.26e-02 s27=1.26e-02 s30=1.26e-02 s31=1.26e-02 s32=1.26e-02 s33=1.26e-02 s34=1.26e-02 s35=1.26e-02 s36=1.26e-02 s37=1.26e-02 s50=1.26e-02 s51=1.26e-02 s52=1.26e-02 s53=1.26e-02 s54=1.26e-02 s60=1.26e-02 s61=1.26e-02 s62=1.26e-02 s63=1.26e-02 s64=1.26e-02 s65=1.26e-02 s66=1.26e-02 s67=1.26e-02 s68=1.26e-02 s70=1.26e-02 s71=1.26e-02 s72=1.26e-02 s73=1.26e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=2048  s0=1.26e-02 s1=1.26e-02 s2=1.26e-02 s3=1.26e-02 s4=1.26e-02 s6=1.26e-02 s7=1.26e-02 s10=1.26e-02 s12=1.26e-02 s20=1.26e-02 s21=1.26e-02 s22=1.26e-02 s23=1.26e-02 s24=1.26e-02 s25=1.26e-02 s26=1.26e-02 s27=1.26e-02 s30=1.26e-02 s31=1.26e-02 s32=1.26e-02 s33=1.26e-02 s34=1.26e-02 s35=1.26e-02 s36=1.26e-02 s37=1.26e-02 s50=1.26e-02 s51=1.26e-02 s52=1.26e-02 s53=1.26e-02 s54=1.26e-02 s60=1.26e-02 s61=1.26e-02 s62=1.26e-02 s63=1.26e-02 s64=1.26e-02 s65=1.26e-02 s66=1.26e-02 s67=1.26e-02 s68=1.26e-02 s70=1.26e-02 s71=1.26e-02 s72=1.26e-02 s73=1.26e-02  bit-identical across all shapes: True
  k_proj     k=2688   n=256    M=8192  s0=1.26e-02 s1=1.26e-02 s2=1.26e-02 s3=1.26e-02 s4=1.26e-02 s6=1.26e-02 s7=1.26e-02 s10=1.26e-02 s12=1.26e-02 s20=1.26e-02 s21=1.26e-02 s22=1.26e-02 s23=1.26e-02 s24=1.26e-02 s25=1.26e-02 s26=1.26e-02 s27=1.26e-02 s30=1.26e-02 s31=1.26e-02 s32=1.26e-02 s33=1.26e-02 s34=1.26e-02 s35=1.26e-02 s36=1.26e-02 s37=1.26e-02 s50=1.26e-02 s51=1.26e-02 s52=1.26e-02 s53=1.26e-02 s54=1.26e-02 s60=1.26e-02 s61=1.26e-02 s62=1.26e-02 s63=1.26e-02 s64=1.26e-02 s65=1.26e-02 s66=1.26e-02 s67=1.26e-02 s68=1.26e-02 s70=1.26e-02 s71=1.26e-02 s72=1.26e-02 s73=1.26e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=1     s0=1.24e-02 s1=1.24e-02 s2=1.24e-02 s3=1.24e-02 s4=1.24e-02 s6=1.24e-02 s7=1.24e-02 s10=1.24e-02 s12=1.24e-02 s20=1.24e-02 s21=1.24e-02 s22=1.24e-02 s23=1.24e-02 s24=1.24e-02 s25=1.24e-02 s26=1.24e-02 s27=1.24e-02 s30=1.24e-02 s31=1.24e-02 s32=1.24e-02 s33=1.24e-02 s34=1.24e-02 s35=1.24e-02 s36=1.24e-02 s37=1.24e-02 s50=1.24e-02 s51=1.24e-02 s52=1.24e-02 s53=1.24e-02 s54=1.24e-02 s60=1.24e-02 s61=1.24e-02 s62=1.24e-02 s63=1.24e-02 s64=1.24e-02 s65=1.24e-02 s66=1.24e-02 s67=1.24e-02 s68=1.24e-02 s70=1.24e-02 s71=1.24e-02 s72=1.24e-02 s73=1.24e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=17    s0=1.33e-02 s1=1.33e-02 s2=1.33e-02 s3=1.33e-02 s4=1.33e-02 s6=1.33e-02 s7=1.33e-02 s10=1.33e-02 s12=1.33e-02 s20=1.33e-02 s21=1.33e-02 s22=1.33e-02 s23=1.33e-02 s24=1.33e-02 s25=1.33e-02 s26=1.33e-02 s27=1.33e-02 s30=1.33e-02 s31=1.33e-02 s32=1.33e-02 s33=1.33e-02 s34=1.33e-02 s35=1.33e-02 s36=1.33e-02 s37=1.33e-02 s50=1.33e-02 s51=1.33e-02 s52=1.33e-02 s53=1.33e-02 s54=1.33e-02 s60=1.33e-02 s61=1.33e-02 s62=1.33e-02 s63=1.33e-02 s64=1.33e-02 s65=1.33e-02 s66=1.33e-02 s67=1.33e-02 s68=1.33e-02 s70=1.33e-02 s71=1.33e-02 s72=1.33e-02 s73=1.33e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=63    s0=1.31e-02 s1=1.31e-02 s2=1.31e-02 s3=1.31e-02 s4=1.31e-02 s6=1.31e-02 s7=1.31e-02 s10=1.31e-02 s12=1.31e-02 s20=1.31e-02 s21=1.31e-02 s22=1.31e-02 s23=1.31e-02 s24=1.31e-02 s25=1.31e-02 s26=1.31e-02 s27=1.31e-02 s30=1.31e-02 s31=1.31e-02 s32=1.31e-02 s33=1.31e-02 s34=1.31e-02 s35=1.31e-02 s36=1.31e-02 s37=1.31e-02 s50=1.31e-02 s51=1.31e-02 s52=1.31e-02 s53=1.31e-02 s54=1.31e-02 s60=1.31e-02 s61=1.31e-02 s62=1.31e-02 s63=1.31e-02 s64=1.31e-02 s65=1.31e-02 s66=1.31e-02 s67=1.31e-02 s68=1.31e-02 s70=1.31e-02 s71=1.31e-02 s72=1.31e-02 s73=1.31e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=256   s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s6=1.29e-02 s7=1.29e-02 s10=1.29e-02 s12=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=1000  s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s6=1.29e-02 s7=1.29e-02 s10=1.29e-02 s12=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=2048  s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s6=1.30e-02 s7=1.30e-02 s10=1.30e-02 s12=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  v_proj     k=2688   n=256    M=8192  s0=1.30e-02 s1=1.30e-02 s2=1.30e-02 s3=1.30e-02 s4=1.30e-02 s6=1.30e-02 s7=1.30e-02 s10=1.30e-02 s12=1.30e-02 s20=1.30e-02 s21=1.30e-02 s22=1.30e-02 s23=1.30e-02 s24=1.30e-02 s25=1.30e-02 s26=1.30e-02 s27=1.30e-02 s30=1.30e-02 s31=1.30e-02 s32=1.30e-02 s33=1.30e-02 s34=1.30e-02 s35=1.30e-02 s36=1.30e-02 s37=1.30e-02 s50=1.30e-02 s51=1.30e-02 s52=1.30e-02 s53=1.30e-02 s54=1.30e-02 s60=1.30e-02 s61=1.30e-02 s62=1.30e-02 s63=1.30e-02 s64=1.30e-02 s65=1.30e-02 s66=1.30e-02 s67=1.30e-02 s68=1.30e-02 s70=1.30e-02 s71=1.30e-02 s72=1.30e-02 s73=1.30e-02  bit-identical across all shapes: True
  o_proj     k=4096   n=2688   M=1     s0=1.29e-02 s1=1.29e-02 s2=1.29e-02 s3=1.29e-02 s4=1.29e-02 s6=1.29e-02 s7=1.29e-02 s10=1.29e-02 s12=1.29e-02 s20=1.29e-02 s21=1.29e-02 s22=1.29e-02 s23=1.29e-02 s24=1.29e-02 s25=1.29e-02 s26=1.29e-02 s27=1.29e-02 s30=1.29e-02 s31=1.29e-02 s32=1.29e-02 s33=1.29e-02 s34=1.29e-02 s35=1.29e-02 s36=1.29e-02 s37=1.29e-02 s50=1.29e-02 s51=1.29e-02 s52=1.29e-02 s53=1.29e-02 s54=1.29e-02 s60=1.29e-02 s61=1.29e-02 s62=1.29e-02 s63=1.29e-02 s64=1.29e-02 s65=1.29e-02 s66=1.29e-02 s67=1.29e-02 s68=1.29e-02 s70=1.29e-02 s71=1.29e-02 s72=1.29e-02 s73=1.29e-02  bit-identical across all shapes: True
  o_proj     k=4096   n=2688   M=17    s0=1.35e-02 s1=1.35e-02 s2=1.35e-02 s3=1.35e-02 s4=1.35e-02 s6=1.35e-02 s7=1.35e-02 s10=1.35e-02 s12=1.35e-02 s20=1.35e-02 s21=1.35e-02 s22=1.35e-02 s23=1.35e-02 s24=1.35e-02 s25=1.35e-02 s26=1.35e-02 s27=1.35e-02 s30=1.35e-02 s31=1.35e-02 s32=1.35e-02 s33=1.35e-02 s34=1.35e-02 s35=1.35e-02 s36=1.35e-02 s37=1.35e-02 s50=1.35e-02 s51=1.35e-02 s52=1.35e-02 s53=1.35e-02 s54=1.35e-02 s60=1.35e-02 s61=1.35e-02 s62=1.35e-02 s63=1.35e-02 s64=1.35e-02 s65=1.35e-02 s66=1.35e-02 s67=1.35e-02 s68=1.35e-02 s70=1.35e-02 s71=1.35e-02 s72=1.35e-02 s73=1.35e-02  bit-identical across all shapes: True
  o_proj     k=4096   n=2688   M=63    s0=1.34e-02 s1=1.34e-02 s2=1.34e-02 s3=1.34e-02 s4=1.34e-02 s6=1.34e-02 s7=1.34e-02 s10=1.34e-02 s12=1.34e-02 s20=1.34e-02 s21=1.34e-02 s22=1.34e-02 s23=1.34e-02 s24=1.34e-02 s25=1.34e-02 s26=1.34e-02 s27=1.34e-02 s30=1.34e-02 s31=1.34e-02 s32=1.34e-02 s33=1.34e-02 s34=1.34e-02 s35=1.34e-02 s36=1.34e-02 s37=1.34e-02 s50=1.34e-02 s51=1.34e-02 s52=1.34e-02 s53=1.34e-02 s54=1.34e-02 s60=1.34e-02 s61=1.34e-02 s62=1.34e-02 s63=1.34e-02 s64=1.34e-02 s65=1.34e-02 s66=1.34e-02 s67=1.34e-02 s68=1.34e-02 s70=1.34e-02 s71=1.34e-02 s72=1.34e-02 s73=1.34e-02  bit-identical across all shapes: True
  o_proj     k=4096   n=2688   M=256   s0=1.33e-02 s1=1.33e-02 s2=1.33e-02 s3=1.33e-02 s4=1.33e-02 s6=1.33e-02 s7=1.33e-02 s10=1.33e-02 s12=1.33e-02 s20=1.33e-02 s21=1.33e-02 s22=1.33e-02 s23=1.33e-02 s24=1.33e-02 s25=1.33e-02 s26=1.33e-02 s27=1.33e-02 s30=1.33e-02 s31=1.33e-02 s32=1.33e-02 s33=1.33e-02 s34=1.33e-02 s35=1.33e-02 s36=1.33e-02 s37=1.33e-02 s50=1.33e-02 s51=1.33e-02 s52=1.33e-02 s53=1.33e-02 s54=1.33e-02 s60=1.33e-02 s61=1.33e-02 s62=1.33e-02 s63=1.33e-02 s64=1.33e-02 s65=1.33e-02 s66=1.33e-02 s67=1.33e-02 s68=1.33e-02 s70=1.33e-02 s71=1.33e-02 s72=1.33e-02 s73=1.33e-02  bit-identical across all shapes: True
  o_proj     k=4096   n=2688   M=1000  s0=1.34e-02 s1=1.34e-02 s2=1.34e-02 s3=1.34e-02 s4=1.34e-02 s6=1.34e-02 s7=1.34e-02 s10=1.34e-02 s12=1.34e-02 s20=1.34e-02 s21=1.34e-02 s22=1.34e-02 s23=1.34e-02 s24=1.34e-02 s25=1.34e-02 s26=1.34e-02 s27=1.34e-02 s30=1.34e-02 s31=1.34e-02 s32=1.34e-02 s33=1.34e-02 s34=1.34e-02 s35=1.34e-02 s36=1.34e-02 s37=1.34e-02 s50=1.34e-02 s51=1.34e-02 s52=1.34e-02 s53=1.34e-02 s54=1.34e-02 s60=1.34e-02 s61=1.34e-02 s62=1.34e-02 s63=1.34e-02 s64=1.34e-02 s65=1.34e-02 s66=1.34e-02 s67=1.34e-02 s68=1.34e-02 s70=1.34e-02 s71=1.34e-02 s72=1.34e-02 s73=1.34e-02  bit-identical across all shapes: True
  o_proj     k=4096   n=2688   M=2048  s0=1.34e-02 s1=1.34e-02 s2=1.34e-02 s3=1.34e-02 s4=1.34e-02 s6=1.34e-02 s7=1.34e-02 s10=1.34e-02 s12=1.34e-02 s20=1.34e-02 s21=1.34e-02 s22=1.34e-02 s23=1.34e-02 s24=1.34e-02 s25=1.34e-02 s26=1.34e-02 s27=1.34e-02 s30=1.34e-02 s31=1.34e-02 s32=1.34e-02 s33=1.34e-02 s34=1.34e-02 s35=1.34e-02 s36=1.34e-02 s37=1.34e-02 s50=1.34e-02 s51=1.34e-02 s52=1.34e-02 s53=1.34e-02 s54=1.34e-02 s60=1.34e-02 s61=1.34e-02 s62=1.34e-02 s63=1.34e-02 s64=1.34e-02 s65=1.34e-02 s66=1.34e-02 s67=1.34e-02 s68=1.34e-02 s70=1.34e-02 s71=1.34e-02 s72=1.34e-02 s73=1.34e-02  bit-identical across all shapes: True
  o_proj     k=4096   n=2688   M=8192  s0=1.34e-02 s1=1.34e-02 s2=1.34e-02 s3=1.34e-02 s4=1.34e-02 s6=1.34e-02 s7=1.34e-02 s10=1.34e-02 s12=1.34e-02 s20=1.34e-02 s21=1.34e-02 s22=1.34e-02 s23=1.34e-02 s24=1.34e-02 s25=1.34e-02 s26=1.34e-02 s27=1.34e-02 s30=1.34e-02 s31=1.34e-02 s32=1.34e-02 s33=1.34e-02 s34=1.34e-02 s35=1.34e-02 s36=1.34e-02 s37=1.34e-02 s50=1.34e-02 s51=1.34e-02 s52=1.34e-02 s53=1.34e-02 s54=1.34e-02 s60=1.34e-02 s61=1.34e-02 s62=1.34e-02 s63=1.34e-02 s64=1.34e-02 s65=1.34e-02 s66=1.34e-02 s67=1.34e-02 s68=1.34e-02 s70=1.34e-02 s71=1.34e-02 s72=1.34e-02 s73=1.34e-02  bit-identical across all shapes: True
[sweep] compared 28 (linear, M) pair(s) over 43 shape(s) with TILE_N in [64, 128]
SHAPE SWEEP: PASS

