Metadata-Version: 2.4
Name: skill-conflict-detector
Version: 0.1.0
Summary: Detect and fix trigger-ambiguity conflicts in agent skill libraries: whole-library LLM analysis, actionable fix suggestions, safe apply with backup, and automatic re-verification.
Author: frkentno1-cloud
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/frkentno1-cloud/skill-conflict-detector
Project-URL: Repository, https://github.com/frkentno1-cloud/skill-conflict-detector
Project-URL: Documentation, https://github.com/frkentno1-cloud/skill-conflict-detector#readme
Project-URL: Bug Tracker, https://github.com/frkentno1-cloud/skill-conflict-detector/issues
Keywords: agent,skills,SKILL.md,LLM,routing,conflict-detection,claude-code,codex,cursor,gemini-cli,devtools,lint
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: typer>=0.12
Requires-Dist: httpx>=0.27
Requires-Dist: pyyaml>=6.0
Dynamic: license-file

# skill-conflict-detector

**Stop your agent from doing the wrong thing — before users notice.**

[中文版](README.zh-CN.md) · Apache-2.0 · PyPI release coming soon — `pip install git+https://github.com/frkentno1-cloud/skill-conflict-detector.git`

---

Your agent is only as reliable as its skill routing. When two skills both claim *"use me when the user asks for X"*, the router picks one — often the wrong one. Your agent quietly does the wrong thing, and nobody knows why until trust is gone.

**skill-conflict-detector finds those collisions automatically, proposes concrete fixes, applies them with your approval (with automatic backup), and re-tests until your library is clean.** One command. Minutes. A few cents.

## The problem, concretely

Every skill declares *when to trigger* in its `description`. With a handful of skills, overlaps are easy to spot. With 50+? 200+? They're inevitable — and hand-maintaining non-overlapping descriptions at that scale is impossible.

The failure is **silent**: no error, no crash, just a wrong tool doing the wrong job. It's the #1 hidden killer of agent reliability, and it compounds as your library grows.

## What this tool does

```
┌─────────┐   ┌──────────┐   ┌──────────┐   ┌────────┐   ┌────────┐
│  check  │ → │ suggest  │ → │  apply   │ → │ verify │ → │(loop)  │
│  detect │   │ propose  │   │(with     │   │ re-test│   │ until  │
│  clashes│   │  edits   │   │ backup)  │   │        │   │ clean  │
└─────────┘   └──────────┘   └──────────┘   └────────┘   └────────┘
```

1. **check** — one LLM call over your *entire* library (name + description of every skill). Whole-library view: it spots clusters, not just pairs. Output: JSON + HTML report.
2. **suggest** — for each clash, concrete edits: *which file, which field, change it to what*. It also classifies each conflict: **text-fixable** (rewrite descriptions to draw a boundary) vs **architectural** (the skills genuinely overlap — flagged for manual merge, never auto-applied).
3. **apply** — every edit is shown to you first (y/n). Originals are **backed up automatically**.
4. **verify** — re-runs the analysis. Clean? Done. Not clean? Loops (default max 3 rounds).

Everything is reversible: `rollback` restores any change from backup.

## Why you can trust the approach

- **Validated by published research.** [*A Single Rewrite Suffices: Empirical Lessons from Production Skill Description Optimization*](https://arxiv.org/abs/2606.30775) (arXiv 2606.30775) shows a **single LLM rewrite** of overlapping descriptions reaches ~79% F1 on routing accuracy — on par with hand-tuned descriptions (79.4%) — at **~32× lower effort** (3.8 min vs 120 min per skill). It also separates *text-fixable collisions* from *genuinely overlapping scopes* — exactly what our `fix_type` field encodes.
- **Precision over recall.** We'd rather miss a conflict than cry wolf: mere description similarity (daily report vs weekly report) is *not* a conflict. Real user requests must be genuinely ambiguous. In our own smoke test on an internal 37-skill library: **8/8 conflict clusters found, 0 false positives**.
- **Deterministic.** `temperature=0`, whole-library single-pass, structured JSON output with fabrication guards (invented skill IDs are dropped).

## Quick start (2 minutes)

```bash
# PyPI release coming soon — install from source for now:
pip install git+https://github.com/frkentno1-cloud/skill-conflict-detector.git
export LLM_API_KEY=sk-...          # any OpenAI-compatible or Anthropic key

# Point it at your skills directory and run the full loop:
skill-conflict-detector audit --path skills/
```

`audit` walks you through every proposed edit (y/n), applies the ones you approve, backs up the originals, and re-scans until clean — or until nothing further can be auto-applied (remaining conflicts need manual handling). That's the whole workflow in one command.

> Tired of the long name? `alias scd="skill-conflict-detector"`.

## Commands

| Command | What it does |
|---|---|
| `skill-conflict-detector check` | Detect conflicts → JSON + HTML report |
| `skill-conflict-detector suggest` | Generate fix suggestions (optionally fed misrouting evidence, below) |
| `skill-conflict-detector audit` | Full loop: detect → suggest → confirm → apply → re-test |
| `skill-conflict-detector apply <suggestions.json>` | Apply suggestions from a JSON file (interactive; add `--yes` for non-interactive CI) |
| `skill-conflict-detector rollback` | Restore the last change (`--session <ts>` / `--suggestion <id>` / `--all`) |
| `skill-conflict-detector backups` | List backup sessions |
| `skill-conflict-detector validate` | Hard checks, no LLM (duplicate names, description quality) |

## Configuration (`skill-conflict-detector.toml`)

```toml
[llm]
provider = "openai"            # openai / anthropic / deepseek / any OpenAI-compatible gateway
model = "gpt-4.1-mini"         # cheap models work; quality comes from the prompt
api_key_env = "LLM_API_KEY"    # key read from environment variable
base_url = ""                  # set for a self-hosted gateway

[scan]
paths = ["skills/"]
exclude = []

[report]
out_dir = "./skill-conflict-detector-reports"

[apply]
backup_dir = "./.skill-conflict-detector-backups"
confirm = "interactive"        # interactive / all
max_loop = 3                   # re-test loop cap
```

### Optional: feed it real misrouting evidence

Have logs of actual routing mistakes? Condition the fixes on them:

```bash
skill-conflict-detector suggest --path skills/ --evidence misrouting.json
```

`misrouting.json`: `[{"user_request": "...", "routed_skill_id": "a", "expected_skill_id": "b", "note": "..."}]`

## CI integration

Two different CIs are worth distinguishing:

- **This repo's CI** (`.github/workflows/skill-conflict-detector.yml`, shipped): installs the package and runs the unit-test suite on every push/PR.
- **Your skill library's CI** — keep conflicts out of `main` forever. Add a workflow to *your* skills repo:

```yaml
on:
  push:
    paths: ['skills/**', 'skill-conflict-detector.toml']
  pull_request:
    paths: ['skills/**', 'skill-conflict-detector.toml']
jobs:
  skillcheck:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.11' }
      - name: Install
        run: pip install -e git+https://github.com/frkentno1-cloud/skill-conflict-detector.git
      - name: Validate skills (no LLM)
        run: skill-conflict-detector validate --path skills/   # errors block the build
      - name: Conflict detection (LLM, optional)
        if: env.LLM_API_KEY != ''
        env: { LLM_API_KEY: ${{ secrets.LLM_API_KEY }} }
        run: skill-conflict-detector check --path skills/ --json
```

1. **validate** — hard checks, no LLM; duplicate skill names are errors and block the build (they make detection itself ambiguous).
2. **conflict detection** — runs when `secrets.LLM_API_KEY` is set, skipped otherwise.
3. **reports** — `check` writes JSON + HTML under `skill-conflict-detector-reports/`; upload them as artifacts.

> Run everything on your own repo first (`check` / `audit` locally) before wiring it into CI — the LLM needs a configured provider (`[llm]` in `skill-conflict-detector.toml`).

## FAQ

**Is this a syntax linter for SKILL.md?** No — structural linters exist. This targets *semantic* clashes: descriptions that would both match the same request. Syntax errors never broke a routing; ambiguity does.

**Which skill formats does it support?** Any library of Markdown files with YAML frontmatter (`name` + `description`) — the emerging open format adopted across Claude Code, Codex, Cursor, Gemini CLI, and many other agents. If your agent reads SKILL.md, this works.

**Does it need an LLM key?** Yes for conflict detection; `validate` works without one. Cost is trivial: a full analysis of ~200 skills is roughly **¥1–3** (~$0.15–0.45) on cheap models.

**Will it edit my skills without asking?** Never. Every edit is shown for approval, originals are backed up, and `rollback` restores them. `fix_type: scope` / `manual` items (merges, renames, platform-level issues) are never auto-applied, even if the model says otherwise. Skills with duplicate names in the library are never auto-edited (which copy would you be fixing?).

**Why does `check` refuse to run on my library?** Duplicate skill names are hard validation errors — detection with two skills sharing an id is meaningless. Run `skill-conflict-detector validate --path skills/` to see the offending files (`identical` = redundant copies; `version_conflict` = historical versions, pick one), fix them, and re-run.

**I'm a PM, not an engineer.** Then `audit` is your friend: one command, human-readable questions, human-readable HTML report. No code to write, nothing to configure beyond one API key.

## Design & development

- [DESIGN.md](DESIGN.md) — full design rationale (detection mechanism, closed loop, related work)
- Tests: `python tests/test_skeleton.py && python tests/test_rollback.py && python tests/test_robustness.py && python tests/test_validate_real.py` (54 cases; `test_validate_real.py` gracefully skips its real-library section when the internal `/skills` tree is absent)
- `testdata/` and `extract_skills.py` are intentionally gitignored (internal-library samples)

## License

Apache-2.0. See [LICENSE](LICENSE).
