Metadata-Version: 2.4
Name: regression-analyzer
Version: 0.1.0
Summary: Repo-agnostic CLI for mining regression patterns from git fix history and proposing agent rule updates.
License-Expression: Apache-2.0
Project-URL: Repository, https://github.com/scanadi/regression-analyzer
Project-URL: Issues, https://github.com/scanadi/regression-analyzer/issues
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: typer>=0.12
Requires-Dist: rich>=13
Requires-Dist: tomli-w>=1.0
Requires-Dist: pydantic>=2.5
Requires-Dist: anthropic>=0.40
Requires-Dist: sentence-transformers>=2.5
Requires-Dist: pydriller>=2.6
Requires-Dist: hdbscan>=0.8
Requires-Dist: scikit-learn>=1.4
Requires-Dist: tree-sitter>=0.21
Requires-Dist: tree-sitter-typescript>=0.21
Requires-Dist: tree-sitter-python>=0.21
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-cov>=5; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Dynamic: license-file

# regression-analyzer

Mines a git repository's fix history, clusters recurring regression patterns, and proposes agent-guidance rules (additions to `CLAUDE.md`, `AGENTS.md`, or `.cursor/rules/*`). The tool runs locally, redacts secrets before any provider call, and never edits source files — it only writes patches to `.regression-analyzer/reports/` for a human reviewer to apply.

## What it does

- Mines fix-like commits via `pydriller`, normalizes diffs, splits suspicious bundle commits, and extracts repo-generic subsystem plus semantic fix-type tags.
- Embeds redacted evidence locally (`sentence-transformers/all-MiniLM-L6-v2`) and clusters normalized vectors with HDBSCAN without building a quadratic pairwise-distance matrix.
- Calls Anthropic (Claude Opus 4.8 by default) once per cluster to produce a structured proposed rule + scores.
- Reuses unchanged embeddings, clustering results, and per-cluster analyses across runs.
- Compares clusters across runs and reports correlational `REDUCED` / `WORSENED` / `SHIFTED` pattern trends; it does not claim accepted rules caused the change.

## What it doesn't do

- No SaaS. Runs entirely on the developer's machine.
- No auto-apply. The CLI writes patches; the reviewer chooses to `git apply` or copy-paste.
- No cross-repo pooling. Each run is scoped to a single working directory.
- No AI attribution in generated commits or reports.
- No telemetry. No data is collected about runs.

## Install

Install from a local checkout:

```sh
git clone https://github.com/scanadi/regression-analyzer.git
cd regression-analyzer
pipx install -e .
```

## Quickstart

1. `cd <your-repo>`
2. `regression-analyzer init` — writes `.regression-analyzer.toml` and adds `.regression-analyzer/` to `.gitignore`.
3. Export env vars:
   ```sh
   export ANTHROPIC_API_KEY=sk-ant-api03-...
   # Optional — overrides [models] analysis_model (default claude-opus-4-8):
   export REGRESSION_ANALYZER_AI_MODEL=claude-opus-4-8
   ```
4. `regression-analyzer analyze --since 6m --propose-rules`
5. Inspect outputs under `.regression-analyzer/reports/`.

To reproduce the quickstart without pointing the analyzer at your own code, build
the deterministic 50-commit fixture repository:

```sh
uv run python scripts/build_sanitized_fixture_repo.py /tmp/regression-analyzer-fixture
regression-analyzer init --cwd /tmp/regression-analyzer-fixture
regression-analyzer analyze --cwd /tmp/regression-analyzer-fixture --since 1y --no-llm
```

The fixture and its ground-truth redaction corpus contain synthetic values only.
See [`examples/sanitized-fixture-repo/`](examples/sanitized-fixture-repo/).

## Required env vars

| Variable | When required | Purpose |
|---|---|---|
| `ANTHROPIC_API_KEY` | analysis stage (unless `--no-llm`) | Authenticates the Anthropic Messages API. |
| `REGRESSION_ANALYZER_AI_MODEL` | optional | Overrides `[models] analysis_model` (default `claude-opus-4-8`). Any Claude model string. |
| `REGRESSION_ANALYZER_AI_MAX_OUTPUT_TOKENS` | optional | Overrides the default 4096. |
| `REGRESSION_ANALYZER_AI_TIMEOUT_MS` | optional | Per-request timeout, default 180000. |

No env var is required for the embedding stage — `sentence-transformers/all-MiniLM-L6-v2` runs on CPU. With `--no-network`, model loading is cache-only (`local_files_only=true`); mining-only runs do not import Torch or load a model.

## Configuration

`regression-analyzer init` writes `.regression-analyzer.toml` at the repo root. Major sections:

| Section | Purpose |
|---|---|
| `[filter]` | Mining range, fix keywords, exclude paths, authors. |
| `[privacy]` | Redaction + network policy. See [`docs/config-privacy-keys.md`](docs/config-privacy-keys.md). |
| `[diff_handling]` | Hunk truncation, bundle-split thresholds. |
| `[clustering]` | HDBSCAN `min_cluster_size`, `min_samples`, `cluster_selection_method`. |
| `[models]` | Embedding + analysis provider, `analysis_model` (single source of truth, default `claude-opus-4-8`), token budget, `analysis_concurrency`. |
| `[guidance]` | Target guidance files, package globs, duplicate / conflict similarity thresholds. |
| `[scoring]` | Severity floor + weights, recency half-life, severe-singleton severities. |
| `[cost]` | Spend guardrails for the analysis stage: `max_cost_usd` (default 5.0), `max_clusters`, `require_confirmation`. |

CLI flags override config values for `--since`, `--until`, `--author`, `--min-cluster-size`, and `--allow-redacted-send`.

## Privacy defaults

| Key | Default | Effect |
|---|---|---|
| `redact_before_api` | `true` | Every provider-bound payload passes through the redaction gate first. |
| `fail_on_secret` | `true` | High-confidence secret findings abort the run before any provider call. |
| `cache_raw_diffs` | `false` | Raw diff text is not persisted to the local cache. |
| `allow_network` | `true` | Provider calls are permitted; setting `false` forces dry-run semantics. |
| `allowed_providers` | `["anthropic", "sentence_transformers", "claude_code"]` | Only these may be called. |

Full policy: [`docs/privacy-policy.md`](docs/privacy-policy.md). Outbound payload shape: [`docs/provider-payload-contract.md`](docs/provider-payload-contract.md). Per-provider disclosure: [`docs/provider-disclosure.md`](docs/provider-disclosure.md).

## Cost controls

The analysis stage is the only paid step (one Anthropic call per cluster + per severe singleton). It is guarded on three axes:

| Guardrail | Default | Effect |
|---|---|---|
| Pre-flight confirmation | on (TTY) | Prints cluster count + estimated cost and asks before spending. `--yes` skips it; auto-skipped in non-interactive shells (ceiling still applies). |
| `[cost] max_cost_usd` / `--max-cost` | `5.0` | Hard ceiling. The run stops before a call would push spend past it and writes a partial report (`Budget: stopped early` in the maintainer report). Worst-case overshoot is one cluster (cents). |
| `[cost] max_clusters` / `--max-clusters` | unset (all) | Analyze only the N most severe/largest clusters. |

A live dashboard (progress bar + running cost + severity-tagged findings feed)
renders in a TTY. Actual token usage and USD are recorded in the maintainer
report and the `analyze` summary.

## Outputs

All written under `.regression-analyzer/reports/`:

| File | When | Contents |
|---|---|---|
| `maintainer-report.md` | always | Headline summary: clusters, severe singletons, prompt-debt findings, and observed post-rule pattern trends (if `--compare-to`). |
| `prompt-debt-report.md` | analysis ran | Classified findings against existing guidance (`MISSING`, `VAGUE`, `DUPLICATE`, `CONFLICTING`, `MISPLACED`, `IGNORED`, `COVERED`). |
| `proposed-rules.patch` | `--propose-rules` | Pure unified diff bundle (no Markdown wrapping). Apply with `git apply --unidiff-zero proposed-rules.patch`. |
| `proposed-rules-summary.md` | `--propose-rules` | Human-readable index of each patch (target file, action, source cluster). |
| `calibration-queue.toml` | always | Empty review template the reviewer fills in. |
| `calibration-metrics.md` | after `review` subcommand | Gate results + per-cluster metrics. |
| `trends.md` | `--compare-to` set | Correlational pattern-trend classifications across runs; never presented as causal rule impact. |
| `agent-eval-fixtures/<rule_id>.json` | after review passes gates | Agent-evaluable fixture per accepted rule. |
| `accepted-rules-agent-context.md` | `export-agent-context` | Accepted lessons in an agent-readable, scope-aware format. |
| `pre-commit-hook` | `export-agent-context` | Optional hook template; never installed automatically. |
| `intelligence-benchmark.json` | `benchmark-intelligence` | Labeled semantic clustering metrics and gate result. |

The cache lives under `.regression-analyzer/cache/`: `redacted-items.json`, `embeddings.json`, `clusters.json`, `analysis.json`, `analysis-input-hashes.json`, `guidance-index.json`, `prompt-debt.json`, `accepted-rules.json`, plus calibration artifacts. Embedding reuse is keyed by evidence, model, dimensions, and composition version; analysis reuse is keyed by the full redacted cluster input, guidance excerpts, model, system prompt, and tool schema.

## CLI reference

| Command | Purpose |
|---|---|
| `regression-analyzer init` | Write `.regression-analyzer.toml` (use `--force` to overwrite). |
| `regression-analyzer analyze` | Run the full pipeline. Flags below. |
| `regression-analyzer preview-payload` | Run the local mining, redaction, embedding, and clustering stages, then print the redacted analysis payload for each cluster without calling an analysis provider. |
| `regression-analyzer review <path>` | Score a filled-in `calibration-queue.toml`, evaluate gates, persist accepted rules and agent-eval fixtures on pass. |
| `regression-analyzer trends` | Emit `trends.md` from cached snapshots (alternative to `analyze --compare-to`). |
| `regression-analyzer benchmark-intelligence` | Run the real local embedding + clustering path over 35 labeled cases and fail if ARI, V-measure, purity, coverage, or mixed-cluster gates regress. Cache-only by default; pass `--allow-network` to download a missing model. |
| `regression-analyzer export-agent-context` | Export accepted rules and an optional pre-commit hook template under `reports/`; never edits guidance or `.git/hooks`. |
| `regression-analyzer precheck [paths...]` | Print accepted rules applicable to planned paths. `--strict` exits non-zero for optional hook use. |
| `regression-analyzer inspect-cache` | Print cache file inventory and sizes. |
| `regression-analyzer purge-cache` | Delete `.regression-analyzer/cache/`. |
| `regression-analyzer --version` | Print version. |

`analyze` flags:

| Flag | Purpose |
|---|---|
| `--since <range>` | Lower bound for fix history. Accepts ISO dates (`2025-01-01`, `2025-11-01T00:00:00Z`) or relative ranges (`6m`, `30d`, `2w`, `1y`). Git refs (e.g. `v1.2.0`, `HEAD~10`) are NOT supported. |
| `--until <range>` | Upper bound for fix history. Same shapes as `--since`. |
| `--author <name>` | Restrict to authors (repeatable). |
| `--min-cluster-size <int>` | Override HDBSCAN `min_cluster_size`. |
| `--no-llm` | Skip the Anthropic analysis stage; emit cluster metadata only. |
| `--provider anthropic\|claude-code` | Override the configured analysis provider. `claude-code` uses the local `claude` CLI and does not require `ANTHROPIC_API_KEY`. |
| `--no-network` / `--dry-run` | Block outbound analysis calls and render the payloads that would be sent. |
| `--no-cache` | Skip cache reads and writes. |
| `--propose-rules` | Emit `proposed-rules.patch`. |
| `--allow-redacted-send` | Allow provider calls after redaction even if findings were redacted. |
| `--max-cost <usd>` | Ceiling on analysis spend (default `[cost] max_cost_usd`, 5.0). The run stops before a call would exceed it and writes a partial report. |
| `--max-clusters <int>` | Analyze only the N most severe/largest clusters (default: all). |
| `--yes` / `-y` | Skip the pre-flight cost confirmation prompt (for automation). |
| `--compare-to <run_id or last>` | Compare against a prior run; emits `trends.md`. |
| `--only-stage mining\|embedding\|clustering\|analysis\|reporting` | Stop after a named stage; later model-dependent stages remain unloaded. |

## Examples

First run, six months of history, with rule patches:

```sh
regression-analyzer analyze --since 6m --propose-rules
```

Local-only (skip Anthropic, embedding-only output):

```sh
regression-analyzer analyze --since 3m --no-llm
```

Cross-run comparison against the previous run:

```sh
regression-analyzer analyze --since 6m --compare-to last --propose-rules
```

## What it proposes vs. what it applies

The CLI ONLY writes patches under `.regression-analyzer/reports/`. It NEVER edits source guidance files (`CLAUDE.md`, `AGENTS.md`, `.cursor/rules/*`). After review, the reviewer chooses to `git apply --unidiff-zero .regression-analyzer/reports/proposed-rules.patch` (or copy individual sections from `proposed-rules-summary.md` into the target files).

## License

Licensed under the [Apache License 2.0](LICENSE).
