Metadata-Version: 2.4
Name: covopt
Version: 0.1.0a5
Summary: Budgeted local test selection
Author: Olle Lindgren
Requires-Python: >=3.10
Requires-Dist: diskcache>=5.6.3
Requires-Dist: numpy>=2.2.6
Requires-Dist: pygit2>=1.18.2
Provides-Extra: dev
Requires-Dist: coverage>=7.6; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.3; extra == 'dev'
Requires-Dist: ruff>=0.15.8; extra == 'dev'
Requires-Dist: ty>=0.0.26; extra == 'dev'
Requires-Dist: vulture; extra == 'dev'
Description-Content-Type: text/markdown

# covopt

`covopt` is a small command-line tool for **budgeted local test selection**.

It reads a diff from `stdin`, loads a precomputed model describing which tests are informative for which parts of the repository, and emits a subset of tests that maximizes expected signal under a runtime budget.

The intended workflow is:

```bash
git diff | covopt select -t 10 | xargs pytest
```

This is **not** a full test runner and does **not** integrate with `pytest` directly. Its only job is to:

1. parse changed code from a diff,
2. map those changes into an internal repository-space representation,
3. score candidate tests by expected value,
4. penalize redundant tests,
5. choose a subset that fits a wall-clock budget,
6. print selected test IDs to `stdout`, one per line.

## Motivation

Large test suites often contain many tests that are highly correlated. For local iteration, running all tests is too expensive, while running only “affected” tests can still waste time on near-duplicates.

`covopt` treats test selection as a **budgeted optimization problem**:

- **Return**: expected defect-detection signal in changed areas
- **Cost**: estimated runtime of a test
- **Risk**: residual untested change exposure
- **Correlation**: redundancy between tests

The goal is to select a small, diverse, high-value subset of tests for fast feedback, while leaving full validation to CI.

## Core idea

The repository is modeled as a feature space over code regions such as files, modules, classes, or functions.

Each test has:

- a sparse **signal vector** over that space,
- an estimated runtime cost,
- optional metadata such as historical failures or flakiness.

A diff is converted into a weighted **change vector** over the same space.

Selection then solves:

- maximize coverage of changed regions,
- prefer tests with high marginal gain per second,
- avoid picking multiple tests with nearly identical signal,
- stop when the budget is exhausted.

In practice this is implemented as a **budgeted greedy optimizer** over a diminishing-returns objective.

## Non-goals

For the first version, this project does **not** aim to provide:

- direct `pytest` plugin support,
- dynamic collection of coverage data,
- mutation testing,
- distributed execution,
- CI orchestration,
- perfect safety guarantees.

This is a **local feedback accelerator**, not a replacement for the full test suite.

## CLI

### Basic usage

```bash
git diff | covopt select -t 10
```

Prints selected test node IDs to `stdout`.

### Example

```bash
git diff HEAD~1 | covopt select -t 10 | xargs pytest
```

### Arguments

```text
usage: covopt select [-h] -t SECONDS [-n N] [--verbose]
```

#### Optional

- `-n N`
  Restrict optimization to the top `N` candidate tests after initial scoring.

- `--verbose`
  Emit diagnostics to `stderr`.

## Input / output contract

### Input

`stdin` must contain a unified diff, for example from:

```bash
git diff
git diff HEAD~1
git show <commit>
```

### Output

By default, the tool prints one test per line:

```text
tests/unit/foo/test_parser.py::test_basic_parse
tests/unit/bar/test_config.py::test_defaults
tests/integration/api/test_health.py::test_healthcheck
```

This makes it easy to pipe into `xargs pytest`.

With `--verbose`, the tool emits structured output including estimated total runtime and selection scores to stderr. Errors are always written to stderr.

## Model format

The selector consumes a model containing:

- repository feature definitions,
- per-test sparse signal vectors,
- pairwise or cluster-level redundancy information,
- test runtime estimates,
- optional weights and calibration parameters.

A minimal conceptual schema looks like this:

```json
{
  "features": [
    "pkg.module_a",
    "pkg.module_a.fn_x",
    "pkg.module_b",
    "pkg.module_b.ClassY.method_z"
  ],
  "tests": {
    "tests/unit/test_a.py::test_one": {
      "cost": 0.8,
      "signal": {
        "pkg.module_a": 0.9,
        "pkg.module_a.fn_x": 1.0
      }
    }
  }
}
```

The model is stored in a `.covopt` file that is generated when invoking the tool.

The exact on-disk format is an implementation detail and may evolve.

## Selection algorithm

At a high level:

1. Parse the diff from `stdin`
2. Map changed files / hunks / symbols to repository features
3. Build a weighted change vector
4. Score candidate tests by overlap with the change vector
5. Iteratively pick the test with the best **marginal gain / cost**
6. Apply diminishing returns to already-covered regions
7. Penalize redundant tests using similarity
8. Stop when the next test would exceed the budget

A typical utility function is:

```text
utility(S) = sum over features j of w_j * f(sum over tests i in S of A_ij)
```

Where:

- `w_j` is the change weight for feature `j`
- `A_ij` is the signal of test `i` on feature `j`
- `f(...)` is a saturating function so duplicate tests add less value

This makes the selector naturally prefer diverse tests over repeated variants of the same test shape.

## Project structure

```text
covopt/
  __main__.py         # CLI entrypoint
  cli.py              # argv parsing and I/O orchestration
  diff_parser.py      # unified diff parsing
  feature_space.py    # repo feature mapping
  scoring.py          # initial candidate scoring
  optimize.py         # budgeted greedy selection
  output.py           # stdout / json formatting
```

## Exit behavior

- `0`: successful selection
- non-zero: invalid arguments, malformed diff, or internal failure

Diagnostics should go to `stderr`. Selected tests should go to `stdout` only.

## Design principles

- **Fast startup**: suitable for local shell pipelines
- **Deterministic**: same diff => same selection
- **Composable**: works well with Unix pipes
- **Model-driven**: selection logic is decoupled from model construction
- **Conservative stdout**: only emit test IDs unless JSON mode is requested

## Example workflow

```bash
git diff | covopt select -t 10 | xargs pytest
```

Full CI still runs the complete suite:

```bash
pytest
```

## Future work

- richer repository feature extraction
- learned test-value calibration from historical failures
- improved redundancy modeling
- coverage/model builders
- optional `pytest` integration
- support for multiple optimization strategies

## Status

Early-stage experimental project focused on the core algorithm and CLI. The first milestone is a reliable selector that can consume a diff, apply a time budget, and emit a useful non-redundant subset of tests for local development.
