============================================================
PLAN
============================================================
  fixtures            : 9  (add-method, constants-in-module, find-usages, hidden-bug-in-body, large-file-search, multi-file-rename, re-read-after-edit, rename, run-tests-fix-bug)
  runs per side per fx: 2
  total claude calls  : 36
  per-call cost cap   : $0.50 USD (--max-budget-usd)
  worst-case spend    : $18.00 USD
  typical per-call    : <$0.30  (so usually $9.00-$12.60 total)
  experiment          : l2-trim-only
    BL side will run with --disallowedTools Read/Edit/Grep/Glob + BEAGLELATHE_TOOL_SET=minimal

============================================================
FIXTURE: add-method
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=3 cost=$0.2243 turns=4 wall=13.0s  (multiply_then_add(2,3,1)=7; delegates to self.add)
    [2/2] vanilla... ✓ calls=3 cost=$0.1247 turns=4 wall=22.1s  (multiply_then_add(2,3,1)=7; delegates to self.add)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=3 cost=$0.2798 turns=4 wall=20.5s bl  (multiply_then_add(2,3,1)=7; delegates to self.add)
    [2/2] beaglelathe... ✓ calls=3 cost=$0.1833 turns=4 wall=12.8s bl  (multiply_then_add(2,3,1)=7; delegates to self.add)
============================================================
FIXTURE: constants-in-module
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=3 cost=$0.1228 turns=4 wall=14.6s  (answer.txt=7 (matches MAX_RETRIES from src/limits.py))
    [2/2] vanilla... ✓ calls=3 cost=$0.1230 turns=4 wall=19.6s  (answer.txt=7 (matches MAX_RETRIES from src/limits.py))
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=3 cost=$0.1825 turns=4 wall=22.4s bl  (answer.txt=7 (matches MAX_RETRIES from src/limits.py))
    [2/2] beaglelathe... ✓ calls=3 cost=$0.1796 turns=4 wall=11.9s bl  (answer.txt=7 (matches MAX_RETRIES from src/limits.py))
============================================================
FIXTURE: find-usages
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=20 cost=$0.4986 turns=21 wall=67.6s  (6 callers identified correctly)
    [2/2] vanilla... ✓ calls=22 cost=$0.3768 turns=23 wall=49.0s  (6 callers identified correctly)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=5 cost=$0.2576 turns=6 wall=27.7s bl  (6 callers identified correctly)
    [2/2] beaglelathe... ✓ calls=6 cost=$0.2867 turns=7 wall=34.3s bl  (6 callers identified correctly)
============================================================
FIXTURE: hidden-bug-in-body
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=3 cost=$0.1247 turns=4 wall=17.2s  (all 5 truth-table cases pass)
    [2/2] vanilla... ✓ calls=3 cost=$0.1245 turns=4 wall=16.3s  (all 5 truth-table cases pass)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=3 cost=$0.1829 turns=4 wall=20.1s bl  (all 5 truth-table cases pass)
    [2/2] beaglelathe... ✓ calls=3 cost=$0.1801 turns=4 wall=14.7s bl  (all 5 truth-table cases pass)
============================================================
FIXTURE: large-file-search
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=4 cost=$0.1463 turns=5 wall=21.1s  (answer.txt=OrderProcessor (the class whose handle() returns 'authoritative'))
    [2/2] vanilla... ✓ calls=4 cost=$0.1435 turns=5 wall=17.1s  (answer.txt=OrderProcessor (the class whose handle() returns 'authoritative'))
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=4 cost=$0.2323 turns=5 wall=17.9s bl  (answer.txt=OrderProcessor (the class whose handle() returns 'authoritative'))
    [2/2] beaglelathe... ✓ calls=4 cost=$0.2330 turns=5 wall=17.3s bl  (answer.txt=OrderProcessor (the class whose handle() returns 'authoritative'))
============================================================
FIXTURE: multi-file-rename
============================================================
  -- vanilla --
    [1/2] vanilla... [claude exited 1]

✓ calls=27 cost=$0.6277 turns=28 wall=92.2s ⚠️  (renamed 12/12 files)
    [2/2] vanilla... [claude exited 1]

✗ calls=15 cost=$0.5031 turns=15 wall=50.9s ⚠️  (renamed=0/12; leftover handleAuth in: ['src/auth/login.ts', 'src/auth/session.ts', 'src/auth/token.ts', 'src/api/users.ts', 'src/api/posts.ts', 'src/api/comments.ts', 'src/middleware/guard.ts', 'src/middleware/logger.ts', 'src/admin/dashboard.ts', 'src/admin/settings.ts', 'src/admin/audit.ts', 'src/index.ts']; handleLogin missing in: ['src/auth/login.ts', 'src/auth/session.ts', 'src/auth/token.ts', 'src/api/users.ts', 'src/api/posts.ts', 'src/api/comments.ts', 'src/middleware/guard.ts', 'src/middleware/logger.ts', 'src/admin/dashboard.ts', 'src/admin/settings.ts', 'src/admin/audit.ts', 'src/index.ts'])
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=2 cost=$0.1659 turns=3 wall=12.5s bl  (renamed 12/12 files)
    [2/2] beaglelathe... ✓ calls=2 cost=$0.1661 turns=3 wall=8.3s bl  (renamed 12/12 files)
============================================================
FIXTURE: re-read-after-edit
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=5 cost=$0.1689 turns=6 wall=20.8s  (config edited; re-read confirmed DEBUG=True)
    [2/2] vanilla... ✓ calls=5 cost=$0.1682 turns=6 wall=21.2s  (config edited; re-read confirmed DEBUG=True)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=4 cost=$0.2012 turns=5 wall=28.1s bl  (config edited; re-read confirmed DEBUG=True)
    [2/2] beaglelathe... ✓ calls=4 cost=$0.1999 turns=5 wall=13.4s bl  (config edited; re-read confirmed DEBUG=True)
============================================================
FIXTURE: rename
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=9 cost=$0.3831 turns=10 wall=38.6s  (renamed 3/3 files)
    [2/2] vanilla... ✓ calls=9 cost=$0.3820 turns=10 wall=33.1s  (renamed 3/3 files)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=2 cost=$0.1626 turns=3 wall=7.7s bl  (renamed 3/3 files)
    [2/2] beaglelathe... ✓ calls=2 cost=$0.1665 turns=3 wall=11.2s bl  (renamed 3/3 files)
============================================================
FIXTURE: run-tests-fix-bug
============================================================
  -- vanilla --
    [1/2] vanilla... ✓ calls=5 cost=$0.1517 turns=6 wall=16.4s  (all 6 unittest tests pass; divide(10,2)=5)
    [2/2] vanilla... ✓ calls=5 cost=$0.1523 turns=6 wall=19.0s  (all 6 unittest tests pass; divide(10,2)=5)
  -- beaglelathe --
    [1/2] beaglelathe... ✓ calls=7 cost=$0.2399 turns=8 wall=22.2s bl  (all 6 unittest tests pass; divide(10,2)=5)
    [2/2] beaglelathe... ✓ calls=6 cost=$0.2204 turns=7 wall=20.8s bl  (all 6 unittest tests pass; divide(10,2)=5)

================================================================================================
PER-FIXTURE PASS RATE + STRUCTURAL DELTAS (means across passing runs only)
================================================================================================
  fixture                van pass  BL pass  BL-used   Δcalls   Δturns   Δoutput    Δcost
  --------------------------------------------------------------------------------------
  add-method               2/2      2/2      2/2       +0.0%    +0.0%    +14.9%   -32.7%
  constants-in-module      2/2      2/2      2/2       +0.0%    +0.0%    +11.9%   -47.4%
  find-usages              2/2      2/2      2/2      +73.8%   +70.5%    +55.9%   +37.8%
  hidden-bug-in-body       2/2      2/2      2/2       +0.0%    +0.0%    +14.2%   -45.6%
  large-file-search        2/2      2/2      2/2       +0.0%    +0.0%     +5.7%   -60.5%
  multi-file-rename        1/2      2/2      2/2      +92.6%   +89.3%    +92.9%   +73.6%
  re-read-after-edit       2/2      2/2      2/2      +20.0%   +16.7%    +45.4%   -19.0%
  rename                   2/2      2/2      2/2      +77.8%   +70.0%    +75.8%   +57.0%
  run-tests-fix-bug        2/2      2/2      2/2      -30.0%   -25.0%    -26.4%   -51.4%

================================================================================================
OVERALL (passing runs only, weighted equally per fixture)
================================================================================================
  pass rate     : vanilla 17/18  |  BL 18/18
  tool calls    : vanilla mean=7.8  |  BL mean=3.7
  turns         : vanilla mean=8.8  |  BL mean=4.7
  output tokens : vanilla mean=1,378  |  BL mean=670
  cost/run (actual)       : vanilla mean=$0.2378  |  BL mean=$0.2067
  cost/run (claude-sonnet-4-6 ): vanilla mean=$0.1381  |  BL mean=$0.1240
  cost/run (claude-opus-4-7   ): vanilla mean=$0.6904  |  BL mean=$0.6200
  wall time               : vanilla mean=29.4s  |  BL mean=18.0s
  BL tools used : 18/18 BL runs actually invoked a BL MCP tool (others fell back to vanilla)

  SAVINGS PER RUN (mean BL vs vanilla, positive = BL cheaper):
    actual                : $+0.0311 saved
    claude-sonnet-4-6     : $+0.0141 saved
    claude-opus-4-7       : $+0.0704 saved
  VERDICT: BL was cheaper than vanilla (mean cost lower across all priced models)

  raw event logs: /Users/thebeagle/dev/beagle-lathe-all/beagle-lathe/bench-logs/
