[tenancy before] (none)
[l2 scrub] mode='read' -- see the module docstring for the bias it carries

  shape  geometry                        thr  regs    smem blk/SM warps/SM   bank
  s24    128x128x128 2x4 st2 sw8         256   123   49152      2       16 64
  s27    128x128x128 1x4 st2 sw8         128   199   49152      2        8 128
  s80    128x128x128 2x4 st2 sw8 gs128   256   222   49152      1        8 64x2
  s110   128x128x128 1x4 st2 sw8         128   254   49152      2        8 128   <-- SPILLS
  s111   128x128x128 1x4 st2 sw8 gs128   128   255   50176      2        8 128x2

  out_proj k=6144 n=5120 M=2048   burst>= 50 ms, reps=9
    f110      446.28 us/call  inner=112    burst=   50.0 ms
    f111      484.95 us/call  inner=103    burst=   49.9 ms
    f27       432.98 us/call  inner=116    burst=   50.2 ms
    f80       621.49 us/call  inner=80     burst=   49.7 ms
    g110      300.45 us/call  inner=165    burst=   49.6 ms
    g111      320.58 us/call  inner=155    burst=   49.7 ms
    g27       287.55 us/call  inner=172    burst=   49.5 ms
    g80       454.79 us/call  inner=108    burst=   49.1 ms
    legA     1516.57 us/call  inner=33     burst=   50.0 ms
    legB      590.74 us/call  inner=87     burst=   51.4 ms
    per-arm ratios, paired within a rep:
      s27  per-token                                         gemm/s27 1.0000x  chain/s27 1.0000x
      s110 per-token                                         gemm/s27 1.0396x  chain/s27 1.0173x
      s111 gs128      bank vs s110 1.0614x [0.9856-1.1540]  gemm/s27 1.1028x  chain/s27 1.1210x
      s80  gs128                                             gemm/s27 1.6021x  chain/s27 1.4219x
      BEST grouped s111 vs BEST per-token s27: gemm 1.1028x [1.0076-1.1777]  chain 1.1210x [1.0332-1.1633]

  out_proj k=6144 n=5120 M=4096   burst>= 50 ms, reps=9
    f110      973.05 us/call  inner=52     burst=   50.6 ms
    f111     1023.31 us/call  inner=49     burst=   50.1 ms
    f27       940.90 us/call  inner=54     burst=   50.8 ms
    f80      1340.31 us/call  inner=39     burst=   52.3 ms
    g110      600.93 us/call  inner=87     burst=   52.3 ms
    g111      618.53 us/call  inner=82     burst=   50.7 ms
    g27       566.67 us/call  inner=91     burst=   51.6 ms
    g80       941.52 us/call  inner=55     burst=   51.8 ms
    legA     3130.05 us/call  inner=16     burst=   50.1 ms
    legB     1072.96 us/call  inner=48     burst=   51.5 ms
    per-arm ratios, paired within a rep:
      s27  per-token                                         gemm/s27 1.0000x  chain/s27 1.0000x
      s110 per-token                                         gemm/s27 1.0561x  chain/s27 1.0372x
      s111 gs128      bank vs s110 1.0418x [0.9365-1.1520]  gemm/s27 1.0959x  chain/s27 1.0844x
      s80  gs128                                             gemm/s27 1.6250x  chain/s27 1.4072x
      BEST grouped s111 vs BEST per-token s27: gemm 1.0959x [1.0544-1.2217]  chain 1.0844x [1.0542-1.2209]

  out_proj k=6144 n=5120 M=8192   burst>= 50 ms, reps=9
    f110     1998.69 us/call  inner=25     burst=   50.0 ms
    f111     2124.50 us/call  inner=24     burst=   51.0 ms
    f27      1992.94 us/call  inner=26     burst=   51.8 ms
    f80      2674.42 us/call  inner=19     burst=   50.8 ms
    g110     1163.51 us/call  inner=45     burst=   52.4 ms
    g111     1224.27 us/call  inner=42     burst=   51.4 ms
    g27      1107.90 us/call  inner=47     burst=   52.1 ms
    g80      1798.45 us/call  inner=28     burst=   50.4 ms
    legA     6123.41 us/call  inner=9      burst=   55.1 ms
    legB     2005.50 us/call  inner=26     burst=   52.1 ms
    per-arm ratios, paired within a rep:
      s27  per-token                                         gemm/s27 1.0000x  chain/s27 1.0000x
      s110 per-token                                         gemm/s27 1.0700x  chain/s27 1.0162x
      s111 gs128      bank vs s110 1.0362x [0.8551-1.1122]  gemm/s27 1.1042x  chain/s27 1.0890x
      s80  gs128                                             gemm/s27 1.6259x  chain/s27 1.3419x
      BEST grouped s111 vs BEST per-token s27: gemm 1.1042x [0.9522-1.1901]  chain 1.0890x [1.0469-1.1392]

  gate_proj k=5120 n=17408 M=2048   burst>= 50 ms, reps=9
    f110     1102.37 us/call  inner=47     burst=   51.8 ms
    f111     1131.32 us/call  inner=45     burst=   50.9 ms
    f27      1023.68 us/call  inner=51     burst=   52.2 ms
    f80      1561.95 us/call  inner=32     burst=   50.0 ms
    g110      855.10 us/call  inner=60     burst=   51.3 ms
    g111      908.07 us/call  inner=57     burst=   51.8 ms
    g27       818.59 us/call  inner=65     burst=   53.2 ms
    g80      1305.85 us/call  inner=38     burst=   49.6 ms
    legA     3568.35 us/call  inner=14     burst=   50.0 ms
    legB     1689.69 us/call  inner=31     burst=   52.4 ms
    per-arm ratios, paired within a rep:
      s27  per-token                                         gemm/s27 1.0000x  chain/s27 1.0000x
      s110 per-token                                         gemm/s27 1.0513x  chain/s27 1.0482x
      s111 gs128      bank vs s110 1.0488x [0.8805-1.1832]  gemm/s27 1.1045x  chain/s27 1.1002x
      s80  gs128                                             gemm/s27 1.6192x  chain/s27 1.5258x
      BEST grouped s111 vs BEST per-token s27: gemm 1.1045x [1.0651-1.2372]  chain 1.1002x [0.9512-1.1924]

  gate_proj k=5120 n=17408 M=4096   burst>= 50 ms, reps=9
    f110     2299.00 us/call  inner=23     burst=   52.9 ms
    f111     2307.95 us/call  inner=22     burst=   50.8 ms
    f27      2173.46 us/call  inner=23     burst=   50.0 ms
    f80      3169.51 us/call  inner=16     burst=   50.7 ms
    g110     1683.48 us/call  inner=31     burst=   52.2 ms
    g111     1783.66 us/call  inner=29     burst=   51.7 ms
    g27      1603.46 us/call  inner=33     burst=   52.9 ms
    g80      2605.77 us/call  inner=20     burst=   52.1 ms
    legA     7075.33 us/call  inner=8      burst=   56.6 ms
    legB     3063.31 us/call  inner=17     burst=   52.1 ms
    per-arm ratios, paired within a rep:
      s27  per-token                                         gemm/s27 1.0000x  chain/s27 1.0000x
      s110 per-token                                         gemm/s27 1.0524x  chain/s27 1.0644x
      s111 gs128      bank vs s110 1.0516x [0.8594-1.2449]  gemm/s27 1.1196x  chain/s27 1.0702x
      s80  gs128                                             gemm/s27 1.6396x  chain/s27 1.4505x
      BEST grouped s111 vs BEST per-token s27: gemm 1.1196x [1.0662-1.2604]  chain 1.0702x [0.9711-1.1410]

  gate_proj k=5120 n=17408 M=8192   burst>= 50 ms, reps=9
    f110     4600.15 us/call  inner=12     burst=   55.2 ms
    f111     4712.91 us/call  inner=11     burst=   51.8 ms
    f27      4296.87 us/call  inner=12     burst=   51.6 ms
    f80      6550.14 us/call  inner=8      burst=   52.4 ms
    g110     3427.33 us/call  inner=16     burst=   54.8 ms
    g111     3542.15 us/call  inner=15     burst=   53.1 ms
    g27      3229.70 us/call  inner=16     burst=   51.7 ms
    g80      5201.92 us/call  inner=10     burst=   52.0 ms
    legA    13270.91 us/call  inner=8      burst=  106.2 ms
    legB     5787.39 us/call  inner=8      burst=   46.3 ms
    per-arm ratios, paired within a rep:
      s27  per-token                                         gemm/s27 1.0000x  chain/s27 1.0000x
      s110 per-token                                         gemm/s27 1.0612x  chain/s27 1.0745x
      s111 gs128      bank vs s110 1.0428x [0.8399-1.1751]  gemm/s27 1.1146x  chain/s27 1.0968x
      s80  gs128                                             gemm/s27 1.6277x  chain/s27 1.5136x
      BEST grouped s111 vs BEST per-token s27: gemm 1.1146x [1.0596-1.1714]  chain 1.0968x [1.0473-1.2487]

  down_proj k=17408 n=5120 M=2048   burst>= 50 ms, reps=9
    f110     1328.90 us/call  inner=38     burst=   50.5 ms
    f111     1397.93 us/call  inner=36     burst=   50.3 ms
    f27      1286.28 us/call  inner=40     burst=   51.5 ms
    f80      1817.41 us/call  inner=29     burst=   52.7 ms
    g110      857.27 us/call  inner=61     burst=   52.3 ms
    g111      890.72 us/call  inner=57     burst=   50.8 ms
    g27       808.78 us/call  inner=64     burst=   51.8 ms
    g80      1270.04 us/call  inner=40     burst=   50.8 ms
    legA     3683.63 us/call  inner=14     burst=   51.6 ms
    legB     1526.51 us/call  inner=34     burst=   51.9 ms
    per-arm ratios, paired within a rep:
      s27  per-token                                         gemm/s27 1.0000x  chain/s27 1.0000x
      s110 per-token                                         gemm/s27 1.0503x  chain/s27 1.0302x
      s111 gs128      bank vs s110 1.0543x [1.0295-1.0939]  gemm/s27 1.1105x  chain/s27 1.0878x
      s80  gs128                                             gemm/s27 1.5680x  chain/s27 1.3863x
      BEST grouped s111 vs BEST per-token s27: gemm 1.1105x [1.0500-1.1546]  chain 1.0878x [1.0648-1.1185]

  down_proj k=17408 n=5120 M=4096   burst>= 50 ms, reps=9
    f110     2719.90 us/call  inner=19     burst=   51.7 ms
    f111     2876.25 us/call  inner=18     burst=   51.8 ms
    f27      2682.61 us/call  inner=19     burst=   51.0 ms
    f80      3713.37 us/call  inner=14     burst=   52.0 ms
    g110     1647.24 us/call  inner=31     burst=   51.1 ms
    g111     1743.12 us/call  inner=30     burst=   52.3 ms
    g27      1568.25 us/call  inner=32     burst=   50.2 ms
    g80      2527.78 us/call  inner=20     burst=   50.6 ms
    legA     7416.09 us/call  inner=8      burst=   59.3 ms
    legB     2741.89 us/call  inner=19     burst=   52.1 ms
    per-arm ratios, paired within a rep:
      s27  per-token                                         gemm/s27 1.0000x  chain/s27 1.0000x
      s110 per-token                                         gemm/s27 1.0546x  chain/s27 1.0311x
      s111 gs128      bank vs s110 1.0551x [1.0029-1.1865]  gemm/s27 1.1106x  chain/s27 1.0701x
      s80  gs128                                             gemm/s27 1.6135x  chain/s27 1.3679x
      BEST grouped s111 vs BEST per-token s27: gemm 1.1106x [1.0658-1.2401]  chain 1.0701x [1.0327-1.1112]

  down_proj k=17408 n=5120 M=8192   burst>= 50 ms, reps=9
    f110     5400.78 us/call  inner=10     burst=   54.0 ms
    f111     5788.83 us/call  inner=9      burst=   52.1 ms
    f27      5252.34 us/call  inner=10     burst=   52.5 ms
    f80      7417.22 us/call  inner=8      burst=   59.3 ms
    g110     3335.68 us/call  inner=16     burst=   53.4 ms
    g111     3468.83 us/call  inner=15     burst=   52.0 ms
    g27      3114.65 us/call  inner=17     burst=   52.9 ms
    g80      5230.89 us/call  inner=10     burst=   52.3 ms
    legA    14056.16 us/call  inner=8      burst=  112.4 ms
    legB     5199.87 us/call  inner=10     burst=   52.0 ms
    per-arm ratios, paired within a rep:
      s27  per-token                                         gemm/s27 1.0000x  chain/s27 1.0000x
      s110 per-token                                         gemm/s27 1.0574x  chain/s27 1.0281x
      s111 gs128      bank vs s110 1.0502x [1.0084-1.0862]  gemm/s27 1.1105x  chain/s27 1.0887x
      s80  gs128                                             gemm/s27 1.6290x  chain/s27 1.4122x
      BEST grouped s111 vs BEST per-token s27: gemm 1.1105x [1.0458-1.1617]  chain 1.0887x [1.0641-1.1440]

[sampler] 938 samples over 47.0 s of process life; 43.7 s of that was inside a timed block

[under load] sm clock median 2625 MHz (min 1875, max 2850)  power median 441 W  sw_power_cap 840/870 samples  hw_slowdown 0/870
[tenancy after ] 2700, 7166 MiB   (this process is pid 2700)
