# async-TP fused legs vs the collective they replace — bit-identity, 2026-09-07

Box 10.2.0.81, GPU0<->GPU1 (RTX A5500, NVLink NV4, sm_86), torch 2.12.1+cu130,
torchrun --nproc-per-node=2, M=2048 hidden=5120 bf16 — the 27B's per-rank TP2 shapes.

RS: fused_matmul_reduce_scatter vs chunk(all_reduce(mm(a,b)))
AG: fused_all_gather_matmul  vs mm(all_gather(a_shard), b)
'peer_contributed' rules out a leg that kept only the local partial, which would
compare equal to a reference computed the same wrong way.

RS K=  3072: bit_identical=True peer_contributed=True
RS K=  8704: bit_identical=True peer_contributed=True
RS K= 12288: bit_identical=True peer_contributed=True
RS K= 16384: bit_identical=True peer_contributed=True
RS K= 24576: bit_identical=False peer_contributed=True
AG N=  7168: bit_identical=True gather_exact=True
AG N= 17408: bit_identical=True gather_exact=True
VERDICT: MISMATCH

READ: every width the 27B issues (3072 o_proj/out_proj, 8704 down_proj) is EXACT.
K=24576 is NOT — the fused op chunks the contraction differently there, so the
partial sums are added in a different order and bf16 rounds differently. It is
also the width that regressed on time, but the two facts are independent and the
boot predicate refuses on either.
