Metadata-Version: 2.4
Name: parity-ai
Version: 0.1.14
Summary: Automatically generate evals for every AI change
Author: Antoine Nguyen
License-Expression: MIT
Project-URL: Homepage, https://github.com/antoinenguyen27/parity
Project-URL: Repository, https://github.com/antoinenguyen27/parity
Project-URL: Issues, https://github.com/antoinenguyen27/parity/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: arize-phoenix-client==2.0.0
Requires-Dist: braintrust==0.9.0
Requires-Dist: claude-agent-sdk==0.1.48
Requires-Dist: click==8.3.1
Requires-Dist: httpx==0.28.1
Requires-Dist: langsmith==0.7.17
Requires-Dist: mcp==1.26.0
Requires-Dist: numpy==2.4.3
Requires-Dist: openai==2.28.0
Requires-Dist: pydantic==2.12.5
Requires-Dist: PyYAML==6.0.3
Requires-Dist: rich==14.3.3
Requires-Dist: tiktoken==0.12.0
Provides-Extra: dev
Requires-Dist: build==1.3.0; extra == "dev"
Requires-Dist: pytest==9.0.2; extra == "dev"
Requires-Dist: pytest-cov==7.0.0; extra == "dev"
Requires-Dist: respx==0.22.0; extra == "dev"
Dynamic: license-file

# Parity

[![PyPI](https://img.shields.io/pypi/v/parity-ai)](https://pypi.org/project/parity-ai/)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/downloads/)

Parity analyzes behavior-defining AI changes in pull requests, discovers the most relevant existing eval target, validates the real coverage gaps, and proposes native eval additions that fit the target suite.

Parity is not an eval runner. It does not create or mutate hosted evaluator infrastructure. It reuses the eval system you already have.

## What Parity Does

For each PR that changes prompts, instructions, guardrails, judges, validators, or similar behavior-defining assets, Parity:

1. Detects the behavioral change.
2. Resolves the best matching eval target and method.
3. Validates which gaps are actually uncovered.
4. Synthesizes native eval additions for that target.
5. Writes only `native_ready` evals after explicit approval.

## Support

| Path | Status | Notes |
|---|---|---|
| Promptfoo | Strong | Best fully native path. Assertions are row-local and writeback is straightforward. |
| LangSmith | Strong | Strong dataset discovery and writeback. Evaluator reuse is supported; evaluator mutation is out of scope. |
| Braintrust | Supported with limitations | Writeback works. Target discovery is weaker and evaluator recovery depends more on repo assets. |
| Arize Phoenix | Supported with limitations | Dataset read/write works. Evaluator discovery is weaker than Promptfoo and LangSmith. |
| Bootstrap mode | Built in | If no safe target is found, Parity proposes starter evals and abstains from unsafe writeback. |

More detail: [docs/platforms.md](docs/platforms.md)

## Public Commands

These are the commands most users need:

- `parity init`
- `parity doctor`
- `parity run-stage 1`
- `parity run-stage 2`
- `parity run-stage 3`
- `parity write-evals`

Parity also ships lower-level operational commands for GitHub comments, run lookup, embeddings, and similarity, but those are advanced surfaces rather than the main product path.

## Quick Start

```bash
pip install parity-ai
parity init
```

Then:

1. Fill in the generated `context/` files.
2. Add GitHub secrets: `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, and any platform keys you use.
3. Commit `parity.yaml`, `.github/workflows/parity.yml`, and `context/`.
4. Open a PR that changes AI behavior.
5. Add the fixed approval label `parity:approve` before merging if you want Parity to write approved evals back after merge.

## Docs

- [Configuration](docs/configuration.md)
- [Architecture](docs/spec.md)
- [Platform support](docs/platforms.md)
- [Example quickstart](examples/langgraph-agentic-rag/docs/quickstart.md)
- [Maintainer guide](docs/maintainers.md)

## License

[MIT](LICENSE)
