# Trusty-cage: improvements distilled from a 12-cage campaign

> **For the Claude Code session picking this up:** This document captures concrete, evidence-backed improvement requests for `trusty-cage` and the `cage-orchestrator` skill. It was written by an outer Claude that just finished orchestrating 12 cages end-to-end (a full phase-1 implementation of a monorepo project called Kanberoo). The observations here are from real dispatches, not speculation. Read section 2 ("Campaign context") first, then section 4 ("Enhancement backlog") for the concrete work.

---

## 1. What the user wants from this session

1. Work through the enhancement backlog in section 4. The user has already approved "address all of these, but not in this session" — meaning over time, not one marathon.
2. Suggest sequencing and tradeoffs before implementing (see section 5).
3. Always get user approval on a plan before implementing. Never auto-commit.
4. Static gates after every logical unit: `ruff format .`, `ruff check --fix .`, `pytest`.
5. For trusty-cage specifically: if adding new CLI flags or subcommands, update `--help` strings and any relevant README sections in the same PR. For the `cage-orchestrator` skill, edits live in `~/.claude/skills/cage-orchestrator/SKILL.md`.

---

## 2. Campaign context

**Project orchestrated:** Kanberoo — a kanban-style issue tracker (SQLite + FastAPI + Textual + Typer + MCP). Five-package `uv` workspace monorepo. Greenfield; built over one long outer-Claude session using 12 cage dispatches (cages A–L, though with a couple of spec-alignment PRs in between).

**Outer environment:** macOS, subscription auth, Docker + OrbStack running, `venv/bin/tc` v0.9.0. No `ANTHROPIC_API_KEY` — subscription mode throughout.

**Each cage dispatch followed roughly this pattern:**

```
git checkout -b feature/...
venv/bin/tc create <repo-url> --name <env> --auth-mode subscription --no-attach
venv/bin/tc launch <env> --test
# write /tmp/cage-prompt-<env>.txt
venv/bin/tc launch <env> --prompt-file /tmp/cage-prompt-<env>.txt --background
venv/bin/tc outbox <env> --poll --timeout 2400 --interval 30   # in outer bg task
# wait for task_complete...
venv/bin/tc export <env> --yes --output-dir .
# run host-side gates (ruff, mypy, pytest) and live smoke
git add -A && git commit -m "..." && git push && gh pr create
# user says "merged"
venv/bin/tc destroy <env> --yes
```

**What landed:** 371 tests, 187 formatted files, 150 typed source files across 5 packages, 15 merged PRs, zero lost work.

---

## 3. Observations

### What worked — do not regress these

- **`tc create → launch → poll → export` flow** was reliable across all 12 cages. `--no-attach` + `--background` was the right default for orchestrated use.
- **`tc export` preserved work even when cages died** mid-cleanup (cages D and F). `rsync --delete` respecting `.gitignore` was correct behavior — the host `venv/` survived because it was gitignored.
- **Subscription auth** with Keychain OAuth injection was seamless. Never had to do anything manual for creds.
- **`tc launch --test`** as a preflight was consistently useful; caught no failures during this campaign but trivially cheap insurance.
- **`tc logs <env>`** streaming was the right tool for diagnosing hangs.
- **The messaging protocol itself** (outbox/inbox with cursor files, lexicographic filenames) Just Worked. No issues.
- **Static-gates-only inner-prompt protocol** (used from cage G onward — ruff + mypy + pytest + commit, no live smoke inside the cage). Zero self-kills after adopting it. This is the single biggest stability improvement we found.

### What didn't work — these are the source material for section 4

| #   | Friction                                                                                                                                                                  | Frequency         | Severity |
| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | -------- |
| 1   | Inner Claude ran `pkill -f kanberoo-api` as smoke cleanup, matched its own process tree, died before sending `task_complete`                                              | 2/12 cages (D, F) | High     |
| 2   | When a cage died mid-cleanup, outer `tc outbox --poll` hung indefinitely with no indication that the inner was `<defunct>`                                                | 2/12 cages        | High     |
| 3   | Cage prompts ballooned to ~500 lines when they included smoke-test scaffolding. Harder to read, more surface for inner Claude to go off-script                            | ~3 cages          | Medium   |
| 4   | Cage F used system `python3` instead of `uv run python` for its smoke script; `websockets` wasn't installed. Trivial to spot externally but the cage didn't figure it out | 1/12 cages        | Medium   |
| 5   | When `tc outbox --poll` timed out, I had to manually check `pgrep claude`, `tc logs`, `.cage/outbox/`, `git status`. No single diagnostic command                         | every hang        | Medium   |
| 6   | No salvage path for "cage died with work intact but no `task_complete`" — had to kill the outer poll, inspect, `tc export` by hand                                        | 2/12 cages        | Medium   |
| 7   | Inconsistent cage env naming (`kanberoo-m04-rest`, `kanberoo-m06-ctl`, `kanberoo-m11-tui`, `kanberoo-m13-tui-detail`…) — I kept making new conventions                    | every cage        | Low      |
| 8   | `tc destroy` leaves the host clone at `~/.trusty-cage/envs/<env>/repo`. 12 of them accumulated across the campaign                                                        | every destroy     | Low      |
| 9   | PR merge detection was manual ("merged" typed 15 times by the user)                                                                                                       | every PR          | Low      |
| 10  | Repeated host-side curl smoke scripts in every task_complete body — nearly identical across cages                                                                         | 7 cages           | Low      |

---

## 4. Enhancement backlog

### A. Bake "no in-cage live smoke" into the `cage-orchestrator` default inner-prompt template

**Where:** `~/.claude/skills/cage-orchestrator/SKILL.md` (NOT a trusty-cage code change — this is a skill-file edit).

**Symptom:** Cages D and F died on post-smoke cleanup (`pkill -f <server>` matched their own `uv run <server>` process tree). Explicit "DO NOT use pkill" warnings in prompts were insufficient — inner Claude reached for familiar cleanup patterns under pressure.

**Root cause:** Running a backgrounded server inside the cage for an E2E smoke test creates a process the inner Claude must then clean up. Cleanup is where things break. The fundamental problem isn't the specific `pkill` incantation — it's that live-smoke-in-cage requires process lifecycle management, which is structurally fragile.

**Proposed fix:** The `cage-orchestrator` SKILL.md's Step 7 ("Launch Inner Claude") ships a default inner-prompt template. Update it so that:

1. The default acceptance gates listed for the inner are **static only**: format, lint, type-check, tests, commit.
2. Explicitly instruct inner Claude **not** to start backgrounded processes and **not** to use `pkill`/`pgrep` for cleanup.
3. Add a line: "End-to-end smoke tests against a live server are the outer orchestrator's job after `tc export`, not yours."
4. If the orchestrator _does_ need inner-run smoke (rare), it should be its own opt-in block with a checklist for process lifecycle.

**Acceptance criteria:**

- A cage dispatched with the default prompt never starts a backgrounded process.
- SKILL.md Step 7 lists static-gates-only as the default.
- An explicit "DO NOT use pkill/pgrep" bullet appears in the default inner-prompt template.

**Effort:** S (~30 min of editing one file).

**Evidence:** Cages D and F outcome logs; cages G–L ran zero backgrounded processes inside the cage and had zero self-kills.

---

### B. `tc outbox --poll` auto-diagnose on timeout

**Where:** `trusty-cage` repo — CLI command handler for `outbox --poll`.

**Symptom:** Outer poll hits its timeout with no actionable information. Operator has to manually run `docker exec -u trustycage isolated-dev-<env> pgrep -af claude`, notice `<defunct>`, check `~/.cage/outbox/` for the last message timestamp, inspect `tc logs`, etc.

**Proposed fix:** When `tc outbox --poll` reaches its timeout, before exiting, run a diagnostic sweep and print results:

1. Is the inner `claude` process alive, zombie (`<defunct>`), or absent?
2. When did the last outbox message arrive? What type was it?
3. What was the last line of the stream log?
4. Git status inside the cage: any modified/untracked files?
5. **Actionable suggestion:** "Inner Claude appears to have exited (zombie process). Work may be exportable via `tc export <env>`. Run `tc logs <env>` to see final inner-Claude output."

The diagnostic should run automatically on timeout; a `--no-diagnose` opt-out is fine but shouldn't be the default.

**Implementation sketch:**

- New helper module (maybe `trusty_cage/diagnostics.py`) that runs the sweep and returns a structured report.
- `outbox --poll` imports it and calls it on timeout.
- Same helper could be exposed as `tc diagnose <env>` for standalone use.

**Acceptance criteria:**

- On `tc outbox --poll` timeout, output includes: process state, last outbox message timestamp + type, git status summary, actionable suggestion.
- `tc diagnose <env>` runs the same sweep on demand.
- Tests cover: live process case, zombie case, no-process case, no-outbox-messages case.

**Effort:** M (~2–4 hours).

**Evidence:** Sessions around cages D (21:36 stream log timestamps) and F (00:07–00:23 hang). Both times I ran the same manual diagnostic sequence.

---

### C. `tc salvage <env>` command

**Where:** `trusty-cage` repo — new CLI subcommand.

**Symptom:** When a cage dies mid-cleanup with work intact but no `task_complete`, the normal workflow (`tc outbox --poll` → export → destroy) has no clean recovery path. Operator has to:

1. Kill the outer poll background task.
2. `tc export <env> --yes --output-dir .`
3. Inspect the git diff and the cage's `/home/trustycage/.cage/outbox/` to decide if the work is salvageable.
4. Commit as usual.

**Proposed fix:** `tc salvage <env>` that:

1. Runs the diagnostic sweep from enhancement B.
2. Prints the cage's last commit message from inside the container (if any).
3. Prints a summary of changed files (tracked + untracked).
4. Confirms with the operator, then runs `tc export` into the current directory.
5. Optionally runs `tc destroy` after.

**Flags:**

- `--no-confirm` for scripted use.
- `--keep-env` to skip destroy after export.
- `--output-dir` (passed through to export, defaults to `.`).

**Edge cases to handle:**

- Inner Claude alive but idle (sent `going_idle`). Salvage should work but warn that the session might still be resumable.
- No modified files — warn and let operator decide whether to proceed.
- Container stopped — error with remediation (probably needs `docker start`).

**Acceptance criteria:**

- `tc salvage <env>` produces a human-readable report and offers export without operator having to type the export command separately.
- `--no-confirm` flag enables scripted use.
- Tests cover: zombie inner, alive-idle inner, no-work-to-salvage, container-stopped.

**Effort:** M (~3–5 hours; reuses B's diagnostic helper).

**Depends on:** Enhancement B (reuses the diagnostic helper).

**Evidence:** Cages D, F — recovery required manual `TaskStop` on outer poll + `tc export` + manual inspection. Salvage would have been ~10 seconds instead of ~10 minutes.

---

### D. Cage-orchestrator: auto-poll `gh pr view` for merge detection

**Where:** `~/.claude/skills/cage-orchestrator/SKILL.md` — Step 10 (Review Changes with User) / Step 11 (Revision Decision).

**Symptom:** User had to type "merged" 15 times across the campaign. For long orchestration campaigns this is genuine friction; for one-off cages it's fine.

**Proposed fix:** After `gh pr create`, store the PR URL; when operator says "merged" (or when the skill re-enters a cycle), verify via `gh pr view <url> --json state,mergedAt`. Optionally poll passively if the operator explicitly enables long-campaign mode.

**Nuances:**

- Don't poll eagerly — it wastes API quota and the user's agency.
- Just _verify_ the merge claim with a cheap API call before proceeding. If they said "merged" but the PR isn't actually merged (race condition, wrong PR, whatever), catch it early.
- For autonomous/long-campaign flows, a `--auto-detect-merge` opt-in that polls every N minutes is reasonable.
- Add handling to bypass this behavior for non-github remotes, the gh cli will not work for these.

**Acceptance criteria:**

- SKILL.md documents a verification step: after user indicates merge, verify with `gh pr view --json state`.
- Optional autonomous-mode instructions describe the polling pattern and when to use it.

**Effort:** S (~30–60 min).

**Evidence:** 15 "merged" pings across the campaign; twice I'd have proceeded with a wrong assumption without verification.

---

### E. `tc destroy --purge-host-clone`

**Where:** `trusty-cage` repo — CLI handler for `destroy`.

**Symptom:** Every `tc destroy` output line ends with "Host clone preserved at `/Users/areese/.trusty-cage/envs/<env>/repo`". Across 12 destroys in this campaign that's ~12 stale clones accumulating in `~/.trusty-cage/envs/`.

**Proposed fix:** `--purge-host-clone` flag on `tc destroy` that also removes `~/.trusty-cage/envs/<env>/` entirely. Preserve current behavior by default (it's defensible — preserved clones have saved bacon before).

**Acceptance criteria:**

- `tc destroy --purge-host-clone <env>` removes the host clone.
- Default behavior unchanged (preserves clone, prints current "Host clone preserved at ..." message).
- Tests cover both paths.

**Effort:** S (~30 min).

**Evidence:** `ls /Users/areese/.trusty-cage/envs/` after this campaign shows 12 directories, one per cage, all preserved.

---

### F. Standardize cage env naming in `cage-orchestrator`

**Where:** `~/.claude/skills/cage-orchestrator/SKILL.md`.

**Symptom:** I made up env names inconsistently during the campaign: `kanberoo-m02-models`, `kanberoo-m03-auth`, `kanberoo-m04-rest`, `kanberoo-m05-epics-stories`, `kanberoo-m06-ctl` (shortened on the fly), `kanberoo-m08-ws`, `kanberoo-m09-cli`, `kanberoo-m11-tui`, `kanberoo-m13-tui-detail`, `kanberoo-m15-mcp`, `kanberoo-m16-export-gaps`, `kanberoo-m17-docs`. No real pattern.

**Proposed fix:** Recommend a naming convention in the skill, e.g. `<repo-basename>-<short-slug>` where slug is derived from the task description (lowercase, hyphenated, max N chars). Document as the default; operators can override with `--name`.

**Acceptance criteria:**

- SKILL.md documents a deterministic naming recipe.
- Orchestrator generates names via the recipe when one isn't provided.

**Effort:** S (~30 min).

**Evidence:** The 12 ad-hoc names in this campaign.

---

### G. Host-side smoke helper

**Where:** Either a `scripts/` directory in an orchestrator-adjacent repo, or a helper command in `trusty-cage` itself (`tc smoke <env-config>`).

**Symptom:** Nearly-identical curl-based host smoke scripts appeared in the task_complete body of cages D, E, F, G, H, I, and K. They all followed the pattern:

```bash
rm -rf /tmp/kb-smoke-<cage>
KANBEROO_CONFIG_DIR=/tmp/kb-smoke-<cage> uv run kb init
TOKEN=$(grep '^token' .../config.toml | cut -d'"' -f2)
# start server in background
SERVER_PID=$!
sleep 2
# curl sequence
# kill $SERVER_PID; wait; rm -rf
```

**Proposed fix:** A helper template or small CLI command that takes a YAML/TOML spec like:

```yaml
name: kanberoo-post-cage-smoke
setup:
  - run: kb init
  - set_env: TOKEN=$(grep '^token' ...)
  - start_server: kanberoo-api
    wait_for: http://127.0.0.1:$PORT/health
steps:
  - curl POST /api/v1/workspaces ...
  - curl GET /api/v1/workspaces ...
cleanup:
  - kill $SERVER_PID
```

Or simpler: just a documented pattern/template file that new cages copy from.

**This is the lowest-priority item.** Might be better as a doc than a tool.

**Effort:** S–M depending on ambition.

**Evidence:** See commit messages for PRs #6, #7, #9, #10, #11, #12, #14 — they all have a "Live E2E on host" section with near-duplicate scripts.

---

## 5. Recommended sequencing

**Tier 1 — do first, they unblock each other:**

1. **A (no-smoke-in-cage default)** — skill-file edit, no code. Highest leverage for preventing cage deaths. Do this first; it's cheap and it matters.
2. **B (outbox --poll auto-diagnose)** — prerequisite for C.
3. **C (tc salvage)** — builds on B.

**Tier 2 — nice-to-have, independent:**

4. **D (merge detection)** — skill file edit.
5. **E (destroy --purge-host-clone)** — tiny trusty-cage change.
6. **F (env naming)** — skill file edit.

**Tier 3 — defer or drop:**

7. **G (smoke helper)** — might be a doc instead of a tool.

**Suggested cadence:** One cycle per tier. Don't try to do tier 1 in one sitting if you're not sure about B/C's scope.

---

## 6. Testing the improvements without redoing the 12-cage campaign

**Cheapest validation target:** A tiny toy project (say: "add a `kanberoo-smoke-toy` feature that prints hello world") dispatched through a cage. This exercises the full lifecycle: create → launch → poll → export → commit → destroy.

**For enhancement A** (no-smoke-in-cage default): the test is negative — run a cage with the new default prompt and verify no backgrounded processes exist in the container at exit. Can check with `docker exec <container> ps -ef | grep -v grep | wc -l` before and after the inner work.

**For enhancement B** (outbox --poll auto-diagnose): simulate by manually running `kill -9` on the inner `claude` process, then let the poll time out. Assert the output includes the diagnostic report.

**For enhancement C** (salvage): kill the inner, run `tc salvage`, assert it produces a report and offers export.

**For enhancement D**: stub `gh pr view` responses.

**For enhancements E/F/G**: trivial — just exercise the flag / naming function / helper.

Don't redo the 12 cages. A handful of targeted toy dispatches is enough to validate.

---

## 7. Reference artifacts

### File paths

- Cage-orchestrator skill: `~/.claude/skills/cage-orchestrator/SKILL.md`
- Cage-iterate skill: `~/.claude/skills/cage-iterate/SKILL.md` (this is the skill that wrote this document)
- Local per-project memory (kanberoo's): `~/.claude/projects/-Users-areese-projects-personal-kanberoo/memory/`
- Trusty-cage host clones: `~/.trusty-cage/envs/<env>/repo/`
- Cage container message dirs: `/home/trustycage/.cage/{inbox,outbox,cursor}/` (inside each container)
- `tc`-in-venv: `<project>/venv/bin/tc` (this is how the orchestrator skill invokes it)

### Commands I used for manual diagnosis during the campaign

```bash
# Is the inner alive, zombie, or absent?
docker exec -u trustycage isolated-dev-<env> pgrep -af claude

# Last outbox messages (same view as tc outbox)
docker exec -u trustycage isolated-dev-<env> \
  sh -c 'ls -1 /home/trustycage/.cage/outbox/*.json 2>/dev/null | sort | tail -3 | while read f; do echo "=== $f ==="; cat "$f"; echo; done'

# Inside-cage git status
docker exec -u trustycage isolated-dev-<env> \
  sh -c 'cd /home/trustycage/project && git log --oneline -5 && git status --short'

# Container stream log tail
venv/bin/tc logs <env> 2>&1 | tail -40
```

These are the commands enhancement B should automate.

### Message envelope (for reference)

```json
{
  "id": "msg-YYYYMMDDTHHMMSSms-xxxx",
  "type": "task_complete|progress_update|error|info_request|going_idle|task_revision|info_response|ack",
  "timestamp": "ISO8601",
  "payload": {
    /* type-specific */
  },
  "version": 1
}
```

Filenames are `YYYY-MM-DDTHH-MM-SS.mmmZ.json` — lexicographic sort equals chronological order.

### The static-gates-only inner-prompt pattern (cage G onward)

The key section of the prompt that made cages G–L reliable:

> **No live-server / live-TUI / live-MCP smoke inside the cage.** Static gates only (ruff format + ruff check + mypy + pytest) + commit. Do NOT start backgrounded processes. Do NOT use pkill/pgrep.
>
> ACCEPTANCE (static gates only, in order):
>
> 1. `uv run ruff format --check .` → no changes
> 2. `uv run ruff check .` → all checks passed
> 3. `uv run mypy packages/` → no issues
> 4. `uv run pytest` → all tests pass

No cage that followed this died in cleanup. Every cage that didn't (D, F) had issues.

---

## 8. Anti-goals — things NOT to change

- **Don't change the messaging protocol.** It's simple, works, and every consumer is hard-coded against the current schema. Leave `outbox/`, `inbox/`, `cursor/` and the envelope alone.
- **Don't remove `tc logs` streaming.** It's the only real-time window into inner Claude. Preserve it.
- **Don't change default auth mode selection logic.** Subscription auth with Keychain injection worked perfectly; don't risk a regression.
- **Don't auto-commit during salvage.** Always let the operator review `git diff` before commit, even on recovery.
- **Don't auto-destroy environments.** Always confirm. Recovery scenarios depend on the env sticking around.

---

## 9. What to tell the user in this session

Do these things before writing any code:

1. **Acknowledge the backlog** and propose a tier-1 plan (enhancements A + B + C). Be specific about scope.
2. **Propose a validation approach** for each — section 6 has suggestions; iterate with the user.
3. **Ask what's in scope for this session.** The user said "all of these, but not in this session" in the parent conversation — they want a plan, they don't want you to try to do all of it at once. Confirm which tier fits this session's time budget.
4. **Get explicit approval on the plan before implementing.**
5. After each logical unit: format + lint + test. After each enhancement: ask before committing.

---

## Appendix: Summary stats from the source campaign

- 12 cages, 15 PRs (including 3 supporting: 1 docs-alignment, 1 milestone-1 scaffold, 1 mid-stream bug-fix branch).
- 371 tests total across 5 packages at the end.
- Two cage self-kills (D, F), both recovered without work loss.
- Zero lost work across the entire campaign.
- Zero auth failures.
- ~40 minutes of genuinely-manual recovery time across the two self-kills (would be ~1 minute with enhancements B+C).
- Prompt sizes: cages A–F were ~300–500 lines each (with smoke scaffolding). Cages G–L were ~200–300 lines each (static gates only).
