flag census — measured 2026-09-11, arbi-whitebox-201 (2x RTX 4090, sm_89)
code: a723c2b1fdd794452078217f96a1fa30a536bfb7

Tier counts over all 328 RuntimeFlags fields:
  LIVE                 63
  INERT                 0
  DECLARED             28
  UNTESTABLE_HERE      17
  UNPROVEN            220

Evidence folded in:
  cpu effect probes   63 (56 must diverge, 7 controls must not) — tools/flag_census.py
  live differential   Qwen3.5-0.8B tp1, 33 runtime-scope params, one boot
  lane dumps          diag-on, piecewise-08b (tp1) + three TP2 arms below

THE CANONICAL CONFIG CANNOT SERVE A TOKEN ON THIS TREE
-----------------------------------------------------
Every contract with on=None is a claim about the canonical lane: Qwen3.6-27B-
AWQ-INT4, TP2, MTP5, tkv k4v4. Three arms were booted on both cards:

  canonical-no-precompile   booted 100.7s  smoke 19/19 HTTP 500
  canonical-no-devdrafts    booted  48.6s  smoke 19/19 HTTP 500
  canonical-tp2-no-mtp      booted  46.2s  smoke 0 errors, 22 serve counters

The fault is in the SPMD (TP>1) MTP verify path and it is NOT flag-gated:

  arbi_serve/spec_decode/mtp_verify_spmd.py:548  run_verify_step_spmd_worker
  -> arbi_serve/spec_decode/mtp_verify_plan.py:459  build_verify_plan
  -> arbi_serve/spec_decode/device_drafts.py:146   rows_on_device
  AttributeError: '_WorkerVerifyRequest' object has no attribute
                 'mtp_next_drafts_dev'. Did you mean: 'mtp_next_drafts'?

`_WorkerVerifyRequest` is the worker-rank shim whose docstring says it
'exposes exactly the subset those read off a mirror row'. That subset is
missing `mtp_next_drafts_dev`. mtp_device_drafts (default True) selects
WHICH line raises, not whether: plan line 459 reads the attribute via
rows_on_device when the flag is on, and line 461 reads it directly when the
flag is off. Booting with ARBI_MTP_DEVICE_DRAFTS=0 was measured and still
500s on 19/19. Dropping MTP is what makes TP2 serve, so the verify step is
the whole of it. TP1+MTP serves fine, which is why no TP1 lane caught it.

Consequence for the contract table: every phase='serve' MustFire owed by the
canonical lane has never been witnessed on the config it names, and cannot
be until this is fixed. That is the bulk of the 65 flags still reading
'declares counter X but no lane snapshot covers it'.

A SECOND BOOT-BLOCKER, THIS ONE TP-ONLY AND FLAG-GATED
-----------------------------------------------------
With precompile at its shipped default the TP2 boot dies outright:

  torch/_dynamo/guards.py:4447 serialize_guards -> raise_local_type_error
  TypeError: Type <class 'torch._C._distributed_c10d.ReduceOp.RedOpType'>
  for object RedOpType.SUM cannot be saved into torch.compile() package
  since it's defined in local scope.

caching_precompile defaults True and carries MustFire('dynamo_precompile_
store', phase='boot'). ReduceOp only enters the guard set when there is a
collective, so this is TP>1 only — a TP1 boot merely logs a precompile MISS
and carries on. All three TP2 arms above therefore ran ARBI_CACHING_
PRECOMPILE=0, and their lane names say so.

UNPROVEN breakdown (220):
   144  Inapplicable, never flipped
    65  MustFire declared, no lane snapshot covers it (see above)
     8  live-diff ran, UNREACHABLE_ON_THIS_BOOT
     3  live-diff ran, PATH_NOT_EXERCISED

Live differential (0.8B tp1), per verdict:
  UNREACHABLE_ON_THIS_BOOT    15
  LIVE                         7
  PATH_NOT_EXERCISED           5
  NEEDS_FRESH_BOOT             4
  NOT_FLIPPABLE                2

  null-control noise floor (counters that move between identical arms): activation_gate_rearm_requested, activation_gate_rearmed, forward_arena_over_accounted, tokenizer_prefix_reuse

  LIVE rows, each citing the flag's OWN declared counter:
    decode_input_pipeline          decode_input_pipeline:162->0
    decode_stream                  decode_stream_request:8->0
    gdn_chunk_indices_memo         gdn_chunk_indices_step_memo_hit:108->0
    inline_detok                   inline_detok_request:8->0
    persistent_input_batch         persistent_input_batch:162->0
    persistent_recurrent_index     persistent_recurrent_index_reuse:162->0
    recurrent_prefill_chunk        recurrent_prefill_chunk_cap:0->178

  Rows that decided nothing, and why:
    async_output_depth             NEEDS_FRESH_BOOT           contract phase 'boot' runs while the engine builds; a live overlay can
    cross_layer_fusion             NEEDS_FRESH_BOOT           contract phase 'boot' runs while the engine builds; a live overlay can
    decode_pad_cudagraph           PATH_NOT_EXERCISED         counter 'decode_pad_replay' is zero in both arms
    dflash_markov_semiar           UNREACHABLE_ON_THIS_BOOT   drafter='none', path needs one of ['dflash']
    dflash_selector_rerank         UNREACHABLE_ON_THIS_BOOT   drafter='none', path needs one of ['dflash']
    drafter_logit_temp             UNREACHABLE_ON_THIS_BOOT   drafter='none', path needs one of ['bundled_mtp', 'dflash']
    inductor_combo_kernels         UNREACHABLE_ON_THIS_BOOT   'inductor_combo_kernels' is consumed at the torch.compile config assem
    mtp_depth_gate_lag             UNREACHABLE_ON_THIS_BOOT   owning param mtp_depth_gate=False
    mtp_depth_gate_margin          UNREACHABLE_ON_THIS_BOOT   owning param mtp_depth_gate=False
    mtp_depth_gate_null            UNREACHABLE_ON_THIS_BOOT   owning param mtp_depth_gate=False
    mtp_device_drafts              UNREACHABLE_ON_THIS_BOOT   drafter='none', path needs one of ['bundled_mtp', 'dflash']
    mtp_fused_rejection            UNREACHABLE_ON_THIS_BOOT   drafter='none', path needs one of ['bundled_mtp', 'dflash', 'draft_mod
    mtp_spec_disable_batch         UNREACHABLE_ON_THIS_BOOT   drafter='none', path needs one of ['assistant', 'bundled_mtp', 'dflash
    prefill_replay                 UNREACHABLE_ON_THIS_BOOT   the flag is engaged and its declared counter 'prefill_graph_lookup' sh
    prefix_grouping                NEEDS_FRESH_BOOT           contract phase 'boot' runs while the engine builds; a live overlay can
    prefix_grouping_window         NEEDS_FRESH_BOOT           contract phase 'boot' runs while the engine builds; a live overlay can
    savepoint_decode_write         UNREACHABLE_ON_THIS_BOOT   the flag is engaged and its declared counter 'savepoint_decode_write' 
    sharded_verify                 UNREACHABLE_ON_THIS_BOOT   tp_size=1
    sharded_verify_n_cand          UNREACHABLE_ON_THIS_BOOT   tp_size=1
    split_mixed_decode_prefill     PATH_NOT_EXERCISED         no MustFire counter, so nothing witnesses the path running
    split_mixed_min_decode_rows    NOT_FLIPPABLE              no unambiguous opposite for 0; a numeric knob's 'other value' is a jud
    split_prefill_capture          UNREACHABLE_ON_THIS_BOOT   prefill_capture='eager', path needs one of ['full']
    tkv_per_row_bypass             PATH_NOT_EXERCISED         counter 'tkv_row_split' is zero in both arms
    watermark_enabled              PATH_NOT_EXERCISED         no MustFire counter, so nothing witnesses the path running
    watermark_h                    NOT_FLIPPABLE              no unambiguous opposite for 4; a numeric knob's 'other value' is a jud
    watermark_key                  PATH_NOT_EXERCISED         no MustFire counter, so nothing witnesses the path running

Flags proven INERT on a named config (live elsewhere, but doing nothing on
the config named — the dynamic-K shape):
  awq_no_a8                      sm_80, which has no a8 support — the arch declines the leg before the flag is read
  dflash_dynamic_k               a DFlash2 drafter whose checkpoint ships NO confidence head — the state every drafter in the tree is actuall
  gdn_conv_force_kernel          sm_89 (Ada, canonical): the arch fall-through already forces the kernel
  gdn_decode_kernel              any other decode geometry (the served 27B included): the shape overrides the request and both arms resolve t
  mtp_philox_verify              a CPU device — the Philox verify path needs a GPU, so the flag cannot bite
  serve_graph_pool_measure       a CPU host with no cuMem driver — there is nothing to measure against, so the flag cannot bite
  tokenizer_pool_workers         a boot whose CONFIG pins the worker count — the config shadows the env knob

Harness boundaries, not flag defects: savepoint_decode_write records 228
refusals of 'cache disabled for this request' because the probe sets
`cache: false` on every completion to stop an arm being served from the
prefix cache; prefill_replay records 'real_N above every captured prefill
rung' because the burst's lengths sit past what a 0.8B eager-prefill boot
captured. Also: `cache_enabled` is NOT a request field. CompletionRequest is
extra="allow", so that spelling — the one this session's own smoke rig has
been sending — is accepted in silence and the request runs WITH the prefix
cache. The served field is `cache`.

Gates at this commit: scripts/ci/cpu_suite.sh -> 20111 passed, 394 skipped,
0 failed; ruff 0.15.21 check + format --check clean over the whole repo.
