Metadata-Version: 2.4
Name: kneepoint
Version: 0.1.0
Summary: Find where your AI agent breaks: load, cost, and chaos testing for AI agents.
Project-URL: Homepage, https://kneepoint.dev
Project-URL: Repository, https://github.com/kneepoint-dev/kneepoint
Author: The Kneepoint Authors
License-Expression: MIT
License-File: LICENSE
Keywords: ai-agents,chaos-engineering,llm,load-testing,performance-testing
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Software Development :: Testing :: Traffic Generation
Requires-Python: >=3.11
Requires-Dist: httpx>=0.27
Requires-Dist: jinja2
Requires-Dist: kneed
Requires-Dist: plotly
Requires-Dist: pydantic>=2
Requires-Dist: pyyaml
Requires-Dist: typer>=0.12
Requires-Dist: uvicorn
Provides-Extra: dev
Requires-Dist: build; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-asyncio; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: twine; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material; extra == 'docs'
Description-Content-Type: text/markdown

# Kneepoint

> **Find where your AI agent breaks.** Load testing, cost-per-resolved-task, and chaos engineering for AI agents.

[![PyPI](https://img.shields.io/pypi/v/kneepoint)](https://pypi.org/project/kneepoint/)
[![CI](https://github.com/kneepoint-dev/kneepoint/actions/workflows/ci.yml/badge.svg)](https://github.com/kneepoint-dev/kneepoint/actions/workflows/ci.yml)
[![Python](https://img.shields.io/pypi/pyversions/kneepoint)](https://pypi.org/project/kneepoint/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

![kneepoint demo](https://raw.githubusercontent.com/kneepoint-dev/kneepoint/main/docs/assets/demo.gif)

Your agent passes its evals at concurrency 1. Kneepoint answers the three questions that decide whether it survives production:

1. **The Knee Point** — the concurrency where latency (and quality) stops being flat and starts to cliff. Every agent has one; almost nobody knows theirs.
2. **$ / Resolved Task** — spend divided by tasks *actually solved*, not tokens, not requests. Retries and failures make these numbers wildly different.
3. **Resilience Score** — how much resolution rate survives injected chaos: rate limits, server errors, tool timeouts, malformed tool JSON.

## Quickstart (zero API keys)

```bash
pip install kneepoint
kneepoint demo          # bundled agent + chaos + full report, ~90 seconds
```

The demo spins up a deliberately naive agent, injects faults into its LLM calls *and* its tool calls, and opens a report answering all three questions. Nothing leaves your machine; no keys, no spend.

![report](https://raw.githubusercontent.com/kneepoint-dev/kneepoint/main/docs/assets/report.png)

## Against your own agent

```bash
kneepoint init                            # starter kneepoint.yaml + prompt corpus
kneepoint run --scenario kneepoint.yaml   # ramp, judge, price, report
```

Any OpenAI-compatible endpoint works as a target. A scenario is one YAML file:

```yaml
target:
  url: http://127.0.0.1:8000/v1
  model: my-agent

workload:
  ramp: {from: 1, to: 50, step: 5, hold_seconds: 20}
  conversation:
    turns: {min: 1, max: 3}                       # multi-turn: context grows
    corpus: ./prompts/*.txt                       # real prompt distribution

resolution:
  check: {kind: contains, value: "[RESOLVED"}     # or a regex, or an LLM judge

cost:
  input_per_mtok: 3.00
  output_per_mtok: 15.00
  max_spend: 0.50                                 # hard cap: run stops if crossed

chaos:
  profile: standard                               # 429s, 503s, tool faults

slo:
  min_resolution_rate: 0.90                       # breach -> exit code 1 (CI gate)
```

Example scenarios: [support bot](examples/scenarios/support-bot/) · [RAG agent](examples/scenarios/rag-agent/) · [agent with MCP tools](examples/scenarios/mcp-tool-agent/)

## CI gate

Kneepoint exits non-zero when your agent regresses — wire it into CI like any other test:

```yaml
- run: pip install kneepoint
- run: kneepoint run --scenario kneepoint.yaml --out reports
- uses: actions/upload-artifact@v4
  if: always()
  with: {name: kneepoint-report, path: reports/}
```

Exit codes: `0` pass · `1` SLO breach · `2` usage error · `3` budget cap hit. [Full CI docs →](https://docs.kneepoint.dev/ci/)

## How it's different

| | Kneepoint | k6 / Locust | Eval platforms | Observability |
|---|---|---|---|---|
| Concurrency ramps + knee detection | ✅ | ✅ | — | — |
| Multi-turn sessions with growing context | ✅ | — | some, at n=1 | — |
| Task-resolution quality **under load** | ✅ | — | quality, no load | — |
| $/resolved task + retry waste | ✅ | — | — | after the fact |
| LLM + tool-layer chaos injection | ✅ | — | — | — |

Different layers, honestly: evals tell you the agent *can* do the task; observability tells you what happened in prod. Kneepoint tells you where it breaks *before* prod does. Use all three.

## Roadmap

- **The Kneepoint Index** — published knee/$/resilience benchmarks of popular agent frameworks
- Native MCP target adapter (fault-inject MCP tool calls first-class)
- More injectors: `stream_cut`, `tool_error`, `tool_stale_data`, `slow_tokens` — [good first issues](https://github.com/kneepoint-dev/kneepoint/issues?q=label%3A%22good+first+issue%22)
- N-run variance mode

## Contributing

`pip install -e ".[dev]"`, then `ruff check . && pytest -q` — the whole suite runs against a bundled mock agent, $0 and no keys. See [CONTRIBUTING.md](CONTRIBUTING.md); new fault injectors are the perfect first PR.

Built in public — follow along at [kneepoint.dev](https://kneepoint.dev). MIT licensed.
