Metadata-Version: 2.4
Name: fixwitness
Version: 1.0.0
Summary: Prove that a regression test actually detects the bug your patch fixes.
Author: adondada
License: MIT
Project-URL: Homepage, https://github.com/adondada/fixwitness
Project-URL: Issues, https://github.com/adondada/fixwitness/issues
Keywords: testing,regression,git,tdd,ci
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Software Development :: Version Control :: Git
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

<p align="center">
  <img src="assets/fixwitness-demo.gif" alt="FixWitness proving that tests pass with the fix and fail when the implementation patch is removed" width="100%">
</p>

<h1 align="center">FixWitness</h1>

<p align="center">
  <strong>Prove that a regression test actually detects the bug your patch fixes.</strong>
</p>

<p align="center">
  <a href="https://github.com/adondada/fixwitness/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/adondada/fixwitness/actions/workflows/ci.yml/badge.svg"></a>
  <a href="https://github.com/adondada/fixwitness/releases"><img alt="Release" src="https://img.shields.io/github/v/release/adondada/fixwitness"></a>
  <a href="LICENSE"><img alt="MIT license" src="https://img.shields.io/badge/license-MIT-blue.svg"></a>
  <img alt="Python 3.11+" src="https://img.shields.io/badge/python-3.11%2B-3776AB.svg">
  <img alt="No runtime dependencies" src="https://img.shields.io/badge/runtime_dependencies-0-success.svg">
</p>

A test can raise coverage, pass CI, and still be unrelated to the bug fix. FixWitness checks the final diff in two isolated Git states:

| State | What FixWitness runs | Required result |
|---|---|---:|
| **Fixed** | New tests + implementation patch | Pass |
| **Bug restored** | New tests + old implementation | Fail |

Only that green-to-red transition proves the changed test notices the implementation fix disappearing.

## 30-second demo

Run a disposable example repository without installing FixWitness globally:

```bash
git clone https://github.com/adondada/fixwitness.git
cd fixwitness
./scripts/demo.sh
```

Expected result:

```text
[PASS] Green with the fix, red when the implementation patch is removed. Regression proved.
base: 085d83c304a7
head: 9b0898b53b3e
tests changed: 1
implementation files changed: 1
fixed: exit 0
bug restored: exit 1
```

The script creates a tiny buggy calculator repository, commits a fix with a regression test, and asks FixWitness to remove only the implementation fix. The new test then fails against the restored bug.

## Install

Install the current release directly from GitHub:

```bash
python -m pip install "git+https://github.com/adondada/fixwitness.git@v0.1.0"
```

Python 3.11+ and Git are required. FixWitness has no runtime Python dependencies.

For development:

```bash
git clone https://github.com/adondada/fixwitness.git
cd fixwitness
python -m pip install -e . pytest
pytest
```

## Use it

Compare the latest commit with its parent:

```bash
fixwitness prove --test "pytest -q"
```

Compare a feature branch with `main`:

```bash
fixwitness prove \
  --base origin/main \
  --head HEAD \
  --test "npm test"
```

Write reports for CI artifacts or pull-request comments:

```bash
fixwitness prove \
  --base origin/main \
  --test "pytest -q" \
  --json build/fixwitness.json \
  --markdown build/fixwitness.md
```

Use `--verbose` to print captured output. Use `--keep-worktree` when the red-state environment needs inspection.

## Why this is different

### Coverage cannot prove relevance

Coverage shows that a test executed code. It does not show that the test would fail if the implementation fix vanished.

### It is narrower than mutation testing

Mutation testing invents many synthetic code changes. FixWitness uses the actual implementation patch as the mutation, so it answers one focused question:

> Does the changed test detect the bug fixed by this diff?

### It recovers evidence lost by squashing

The cleanest regression-test workflow is test-red, fix-green. Squashed commits and coding agents often erase that history. FixWitness reconstructs the evidence from the final committed diff.

## Configure file detection

FixWitness reads optional settings from `pyproject.toml`:

```toml
[tool.fixwitness]
test_command = "pytest -q"
setup_command = "python -m pip install -e ."
test_globs = [
  "tests/**",
  "**/*_test.py",
  "**/*.spec.ts",
]
ignore_globs = [
  "docs/**",
  "**/*.md",
  ".github/**",
]
```

Then run:

```bash
fixwitness prove --base origin/main
```

## GitHub Action

```yaml
name: FixWitness

on:
  pull_request:

jobs:
  prove-regression:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - uses: adondada/fixwitness@v0.1.0
        with:
          base: ${{ github.event.pull_request.base.sha }}
          head: ${{ github.event.pull_request.head.sha }}
          test-command: pytest -q
          setup-command: python -m pip install -e . pytest

      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: fixwitness-report
          path: fixwitness-report.md
```

For repositories where not every pull request is a bug fix, run FixWitness only when a label such as `bug` is present, or expose it as a manually triggered workflow.

## Exit codes

| Code | Meaning |
|---:|---|
| `0` | Proved: fixed state passed and bug-restored state failed |
| `1` | Not proved: tests passed after removing the implementation patch |
| `2` | Fixed state failed |
| `3` | Configuration, Git, setup, or classification error |

## Limitations

- FixWitness currently works on committed revisions, not uncommitted changes.
- It assumes changed tests can run against the base implementation.
- Database migrations, generated files, lockfiles, and coupled configuration changes may require custom globs or a setup command.
- A red result proves the implementation diff matters to the selected test command. It does not prove the test captures every aspect of the bug.

## Contributing

Issues and focused pull requests are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md).

Useful first contributions include language-specific test globs, better monorepo support, and CI integrations.

## License

MIT. See [LICENSE](LICENSE).
