============================================================
PLAN
============================================================
  fixtures            : 9  (add-method, constants-in-module, find-usages, hidden-bug-in-body, large-file-search, multi-file-rename, re-read-after-edit, rename, run-tests-fix-bug)
  runs per side per fx: 2
  total claude calls  : 36
  per-call cost cap   : $0.50 USD (--max-budget-usd)
  worst-case spend    : $18.00 USD
  typical per-call    : <$0.30  (so usually $9.00-$12.60 total)

============================================================
FIXTURE: add-method
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=3 cost=$0.1273 turns=4 wall=13.9s  (multiply_then_add(2,3,1)=7; delegates to self.add)
    [2/2] vanilla... ✓ calls=3 cost=$0.1276 turns=4 wall=17.5s  (multiply_then_add(2,3,1)=7; delegates to self.add)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=3 cost=$0.1857 turns=4 wall=19.7s bl  (multiply_then_add(2,3,1)=7; delegates to self.add)
    [2/2] beaglelathe... ✓ calls=3 cost=$0.1833 turns=4 wall=19.9s bl  (multiply_then_add(2,3,1)=7; delegates to self.add)
============================================================
FIXTURE: constants-in-module
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=3 cost=$0.1252 turns=4 wall=14.7s  (answer.txt=7 (matches MAX_RETRIES from src/limits.py))
    [2/2] vanilla... ✓ calls=3 cost=$0.1252 turns=4 wall=12.0s  (answer.txt=7 (matches MAX_RETRIES from src/limits.py))
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=3 cost=$0.1866 turns=4 wall=13.1s bl  (answer.txt=7 (matches MAX_RETRIES from src/limits.py))
    [2/2] beaglelathe... ✓ calls=3 cost=$0.1871 turns=4 wall=12.5s bl  (answer.txt=7 (matches MAX_RETRIES from src/limits.py))
============================================================
FIXTURE: find-usages
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=20 cost=$0.5160 turns=21 wall=74.1s ⚠️  (6 callers identified correctly)
    [2/2] vanilla... ✓ calls=12 cost=$0.2427 turns=13 wall=34.5s  (6 callers identified correctly)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=7 cost=$0.2696 turns=8 wall=30.8s bl  (6 callers identified correctly)
    [2/2] beaglelathe... ✓ calls=7 cost=$0.2805 turns=8 wall=43.6s bl  (6 callers identified correctly)
============================================================
FIXTURE: hidden-bug-in-body
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=3 cost=$0.1303 turns=4 wall=15.2s  (all 5 truth-table cases pass)
    [2/2] vanilla... ✓ calls=3 cost=$0.1300 turns=4 wall=19.3s  (all 5 truth-table cases pass)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=3 cost=$0.1878 turns=4 wall=11.7s bl  (all 5 truth-table cases pass)
    [2/2] beaglelathe... ✓ calls=3 cost=$0.1879 turns=4 wall=13.2s bl  (all 5 truth-table cases pass)
============================================================
FIXTURE: large-file-search
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=4 cost=$0.1501 turns=5 wall=22.3s  (answer.txt=OrderProcessor (the class whose handle() returns 'authoritative'))
    [2/2] vanilla... ✓ calls=4 cost=$0.1513 turns=5 wall=19.0s  (answer.txt=OrderProcessor (the class whose handle() returns 'authoritative'))
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=4 cost=$0.2175 turns=5 wall=18.7s bl  (answer.txt=OrderProcessor (the class whose handle() returns 'authoritative'))
    [2/2] beaglelathe... ✓ calls=4 cost=$0.2137 turns=5 wall=19.0s bl  (answer.txt=OrderProcessor (the class whose handle() returns 'authoritative'))
============================================================
FIXTURE: multi-file-rename
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=27 cost=$0.5435 turns=28 wall=56.9s ⚠️  (renamed 12/12 files)
    [2/2] vanilla... ✗ calls=25 cost=$0.5661 turns=25 wall=62.6s ⚠️  (renamed=10/12; leftover handleAuth in: ['src/auth/login.ts', 'src/auth/session.ts']; handleLogin missing in: ['src/auth/login.ts', 'src/auth/session.ts'])
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=2 cost=$0.1709 turns=3 wall=9.4s bl  (renamed 12/12 files)
    [2/2] beaglelathe... ✓ calls=2 cost=$0.1705 turns=3 wall=7.8s bl  (renamed 12/12 files)
============================================================
FIXTURE: re-read-after-edit
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=5 cost=$0.1770 turns=6 wall=23.7s  (config edited; re-read confirmed DEBUG=True)
    [2/2] vanilla... ✓ calls=5 cost=$0.1713 turns=6 wall=42.7s  (config edited; re-read confirmed DEBUG=True)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=4 cost=$0.2058 turns=5 wall=15.6s bl  (config edited; re-read confirmed DEBUG=True)
    [2/2] beaglelathe... ✓ calls=4 cost=$0.2095 turns=5 wall=18.3s bl  (config edited; re-read confirmed DEBUG=True)
============================================================
FIXTURE: rename
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=9 cost=$0.3849 turns=10 wall=32.3s  (renamed 3/3 files)
    [2/2] vanilla... ✓ calls=8 cost=$0.1949 turns=9 wall=25.0s  (renamed 3/3 files)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=2 cost=$0.1617 turns=3 wall=7.5s bl  (renamed 3/3 files)
    [2/2] beaglelathe... ✓ calls=2 cost=$0.1695 turns=3 wall=11.8s bl  (renamed 3/3 files)
============================================================
FIXTURE: run-tests-fix-bug
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=6 cost=$0.1885 turns=7 wall=22.5s  (all 6 unittest tests pass; divide(10,2)=5)
    [2/2] vanilla... ✓ calls=10 cost=$0.2085 turns=11 wall=28.2s  (all 6 unittest tests pass; divide(10,2)=5)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=6 cost=$0.2354 turns=7 wall=31.8s bl  (all 6 unittest tests pass; divide(10,2)=5)
    [2/2] beaglelathe... ✓ calls=6 cost=$0.2364 turns=7 wall=19.4s bl  (all 6 unittest tests pass; divide(10,2)=5)

================================================================================================
PER-FIXTURE PASS RATE + STRUCTURAL DELTAS (means across passing runs only)
================================================================================================
  fixture                van pass  BL pass  BL-used   Δcalls   Δturns   Δoutput    Δcost
  --------------------------------------------------------------------------------------
  add-method               2/2      2/2      2/2       +0.0%    +0.0%    +19.7%   -44.8%
  constants-in-module      2/2      2/2      2/2       +0.0%    +0.0%     -3.8%   -49.2%
  find-usages              2/2      2/2      2/2      +56.2%   +52.9%    +38.4%   +27.5%
  hidden-bug-in-body       2/2      2/2      2/2       +0.0%    +0.0%    +14.3%   -44.3%
  large-file-search        2/2      2/2      2/2       +0.0%    +0.0%     -9.8%   -43.0%
  multi-file-rename        1/2      2/2      2/2      +92.6%   +89.3%    +92.6%   +68.6%
  re-read-after-edit       2/2      2/2      2/2      +20.0%   +16.7%    +40.6%   -19.2%
  rename                   2/2      2/2      2/2      +76.5%   +68.4%    +72.2%   +42.9%
  run-tests-fix-bug        2/2      2/2      2/2      +25.0%   +22.2%    +39.8%   -18.8%

================================================================================================
OVERALL (passing runs only, weighted equally per fixture)
================================================================================================
  pass rate     : vanilla 17/18  |  BL 18/18
  tool calls    : vanilla mean=7.5  |  BL mean=3.8
  turns         : vanilla mean=8.5  |  BL mean=4.8
  output tokens : vanilla mean=1,376  |  BL mean=704
  cost/run (actual)       : vanilla mean=$0.2173  |  BL mean=$0.2033
  cost/run (claude-sonnet-4-6 ): vanilla mean=$0.1248  |  BL mean=$0.1220
  cost/run (claude-opus-4-7   ): vanilla mean=$0.6241  |  BL mean=$0.6099
  wall time               : vanilla mean=27.9s  |  BL mean=18.0s
  BL tools used : 18/18 BL runs actually invoked a BL MCP tool (others fell back to vanilla)

  SAVINGS PER RUN (mean BL vs vanilla, positive = BL cheaper):
    actual                : $+0.0140 saved
    claude-sonnet-4-6     : $+0.0028 saved
    claude-opus-4-7       : $+0.0142 saved
  VERDICT: BL was cheaper than vanilla (mean cost lower across all priced models)

  raw event logs: /Users/thebeagle/dev/beagle-lathe-all/beagle-lathe/bench-logs/
