Metadata-Version: 2.5
Name: agent-chaos-runner
Version: 0.5.0
Summary: Chaos engineering for autonomous AI agents
Project-URL: Homepage, https://github.com/Coroz2/agent-chaos
Project-URL: Documentation, https://github.com/Coroz2/agent-chaos#readme
Project-URL: Repository, https://github.com/Coroz2/agent-chaos
Project-URL: Issues, https://github.com/Coroz2/agent-chaos/issues
Project-URL: Changelog, https://github.com/Coroz2/agent-chaos/blob/main/CHANGELOG.md
Author: Carlos Orozco
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: ai-agents,chaos-engineering,resilience,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.12
Requires-Dist: httpx<1,>=0.27
Requires-Dist: pydantic<3,>=2.8
Requires-Dist: pyyaml<7,>=6.0
Requires-Dist: starlette<1,>=0.38
Requires-Dist: typer<1,>=0.12
Requires-Dist: uvicorn<1,>=0.30
Provides-Extra: dev
Requires-Dist: mypy<2,>=1.11; extra == 'dev'
Requires-Dist: pytest-asyncio<1,>=0.24; extra == 'dev'
Requires-Dist: pytest<9,>=8.3; extra == 'dev'
Requires-Dist: ruff<1,>=0.6; extra == 'dev'
Requires-Dist: twine<8,>=7; extra == 'dev'
Requires-Dist: types-pyyaml<7,>=6.0; extra == 'dev'
Description-Content-Type: text/markdown

# Agent Chaos

**Chaos engineering for autonomous AI agents.**

AI agents increasingly depend on unreliable model APIs, tools, databases, and HTTP services.
Agent Chaos intentionally disrupts those dependencies so developers can measure whether an
agent-like workload tolerates a fault, retries successfully, or fails.

Agent Chaos is an early open-source vertical slice. It is framework-independent and does not
require an OpenAI or Anthropic API key.

```mermaid
flowchart LR
    A["Agent workload"] --> C["Agent Chaos proxy"]
    C --> D["HTTP dependency"]
    C -. "inject fault" .-> C
```

## Install

The published Python distribution is named `agent-chaos-runner`; the product and installed
command remain Agent Chaos and `agentchaos`.

```bash
uv tool install agent-chaos-runner
agentchaos --version
```

Agent Chaos supports Python 3.12+ on macOS and Linux.

## Quick start

Clone the repository to run the deterministic local demo without API keys:

```bash
git clone https://github.com/Coroz2/agent-chaos.git
cd agent-chaos
uv sync --extra dev --locked
uv run agentchaos run examples/scenarios/api_503_recovery.yaml
```

The scenario starts its deterministic fake dependency automatically. A successful run ends with
`RECOVERED` and writes its artifacts under `.agentchaos/runs/<run-id>/`.

Other examples:

```bash
uv run agentchaos run examples/scenarios/no_fault.yaml
uv run agentchaos run examples/scenarios/api_latency_recovery.yaml
uv run agentchaos run examples/scenarios/api_429_recovery.yaml
uv run agentchaos run examples/scenarios/api_429_failure.yaml
uv run agentchaos run examples/scenarios/api_503_failure.yaml
uv run agentchaos run examples/scenarios/api_503_schedule_recovery.yaml
uv run agentchaos run examples/scenarios/api_503_schedule_incomplete.yaml
uv run agentchaos run examples/scenarios/api_503_probability_recovery.yaml
uv run agentchaos run examples/scenarios/api_503_probability_incomplete.yaml
uv run agentchaos run examples/scenarios/http_disconnect_recovery.yaml
uv run agentchaos run examples/scenarios/http_disconnect_failure.yaml
uv run agentchaos run examples/scenarios/http_malformed_json_recovery.yaml
uv run agentchaos run examples/scenarios/http_malformed_json_failure.yaml
```

The 429, 503, disconnect, malformed-JSON, incomplete-schedule, and incomplete-window examples
deliberately exit with status 1 because the experiment's required recovery or trigger completion is
not observed.

## Scenario

```yaml
schema_version: 1
name: api-503-recovery

dependency:
  type: http
  base_url: http://127.0.0.1:19103
  start:
    command: [python, fake_api.py, --port, "19103"]
    cwd: ..
    readiness:
      path: /health

workload:
  name: demo-agent
  command: [python, demo_agent.py]
  cwd: ..
  proxy_url_env: CUSTOMER_API_URL

fault:
  type: http_error
  target:
    method: GET
    path: /customer/*
  trigger:
    occurrence: 2
  status_code: 503

success:
  exit_code: 0
```

The optional managed dependency is intended for local tests. Omit `dependency.start` when the
upstream already exists. Agent Chaos always exposes the generated proxy URL as
`AGENTCHAOS_PROXY_URL`; `proxy_url_env` maps it into the variable an existing workload expects.

Agent Chaos also supports deterministic HTTP rate-limit injection:

```yaml
fault:
  type: http_rate_limit
  target:
    method: GET
    path: /customer/*
  trigger:
    occurrence: 2
  retry_after_seconds: 1
```

The selected request receives HTTP 429 with an integer `Retry-After` value and
`X-Agent-Chaos-Fault: http_rate_limit`; the upstream is not contacted for that request.

To test application-level JSON handling despite a successful HTTP transport status, configure the
fixed malformed-JSON fault:

```yaml
fault:
  type: http_malformed_json
  target:
    method: GET
    path: /customer/*
  trigger:
    occurrence: 2
```

On the selected occurrence, Agent Chaos returns status 200 with `Content-Type: application/json`,
`X-Agent-Chaos-Fault: http_malformed_json`, and one fixed invalid JSON body. It does not contact
the upstream or accept configurable response content. A matching retry that receives valid JSON
is classified as recovery; exiting without a successful matching retry is a failed experiment.

To test recovery from an abruptly terminated HTTP connection, configure the disconnect fault:

```yaml
fault:
  type: http_disconnect
  target:
    method: GET
    path: /customer/*
  trigger:
    occurrence: 2
```

Agent Chaos does not forward the selected request upstream and terminates the client connection
before a complete response arrives. The portable guarantee is a client-visible transport or HTTP
protocol failure; a literal TCP reset and a particular client-library exception are not
guaranteed. A matching retry that succeeds through the upstream is classified as recovery.

To inject the same configured fault more than once, use an occurrence schedule:

```yaml
fault:
  type: http_error
  target:
    method: GET
    path: /customer/*
  trigger:
    occurrences: [2, 4]
  status_code: 503
```

Schedule entries must be positive, unique, and strictly increasing. The scalar
`occurrence: 2` form remains supported as a one-entry schedule. Target-matching retries count
toward the schedule and can themselves be selected for injection. A schedule is complete only
when every configured occurrence injects; reaching only part of it fails with
`FAULT_SCHEDULE_INCOMPLETE` even if every observed failure recovered.

To select faults reproducibly from a finite matching-request window, configure a probability and
required seed:

```yaml
fault:
  type: http_error
  target:
    method: GET
    path: /customer/*
  trigger:
    probability: 0.5
    seed: 10
    window:
      start_occurrence: 1
      end_occurrence: 5
  status_code: 503
```

The bounds are inclusive occurrence ordinals. Probability supports up to six decimal places, the
seed is an unsigned 64-bit integer, and retries participate in counting. Selection uses a published
SHA-256 algorithm, so the same scenario and matching-request order select the same operations on
every supported platform. A window that starts but does not finish fails with
`TRIGGER_WINDOW_INCOMPLETE`, even when every observed failure recovered.

## Results

- `PASSED`: a baseline succeeds, or the workload tolerates injected latency without failure.
- `RECOVERED`: every fault-related failure has its own successful matching retry, and the workload
  succeeds.
- `FAILED`: execution fails, the fault never fires, a schedule is incomplete, or any required
  recovery is not observed.

An injected disconnect always records a failed operation. It produces `RECOVERED` only when a
matching retry succeeds; an expected workload exit without that retry produces `FAILED`.

Successful and recovered experiments exit 0. Experiment failures exit 1, invalid scenarios and
invalid saved reports exit 2, setup or unexpected internal failures exit 3, and interruptions exit
130. Inspecting a structurally valid saved report exits 0 even when its recorded experiment result
is `FAILED`.

Every valid run contains:

```text
.agentchaos/runs/<run-id>/
├── scenario.yaml
├── events.jsonl
├── stdout.log
├── stderr.log
├── dependency.stdout.log
├── dependency.stderr.log
└── report.json
```

`events.jsonl` remains a schema-1, sequence-ordered event stream. New runs write schema-3
`report.json` files with a generalized occurrence-schedule or seeded-probability trigger object plus
one evidence row per fault-related failed operation. Probability reports include the seed, window,
evaluation count, and selected occurrence ordinals. Each recovery row identifies its successful
retry and latency, or uses null values when recovery was not observed.

## Commands

```bash
uv run agentchaos --help
uv run agentchaos --version
uv run agentchaos version
uv run agentchaos validate examples/scenarios/api_503_recovery.yaml
uv run agentchaos run examples/scenarios/api_503_recovery.yaml
uv run agentchaos inspect .agentchaos/runs/<run-id>
uv run agentchaos inspect .agentchaos/runs/<run-id>/report.json
```

Use `--output-dir PATH` to place run directories somewhere other than `.agentchaos/runs`.

`inspect` accepts either a run directory or its `report.json` file and prints the same result
summary as `run`. It strictly validates schema-1, schema-2, and schema-3 saved reports without running a
workload, starting a dependency, contacting the network, reading other artifacts, migrating the
report, or modifying the run directory.

## Documentation

- [Project vision](docs/PROJECT-VISION.md): stable mission, principles, capability map, and
  boundaries.
- [Documentation index](docs/README.md): authority map for specifications, release guidance, and
  archived planning.
- [v0.5 specification](docs/specs/v0.5.md): complete contract for the current released behavior.
- [v0.4 specification](docs/specs/v0.4.md): immutable contract for the prior release.
- [v0.3 specification](docs/specs/v0.3.md): immutable contract for the prior release.
- [v0.2 specification](docs/specs/v0.2.md): immutable contract for the prior release.
- [v0.1 specification](docs/specs/v0.1.md): immutable contract for the initial release.

## Development

```bash
uv sync --extra dev
uv run pytest
uv run ruff check .
uv run ruff format --check .
uv run mypy
```

See [CONTRIBUTING.md](CONTRIBUTING.md) for the branch, pull request, verification, and release
workflow.

## Limitations

Agent Chaos supports one HTTP dependency and zero or one fault on macOS and Linux. It is a reverse
proxy, not transparent network interception: the workload must accept the proxy base URL through
its configuration. Request bodies and responses are buffered up to 10 MiB. Disconnect injection
does not provide packet-level reset controls or partial-response faults. Streaming, SSE,
WebSockets, CONNECT tunneling, TLS interception, multiple faults, wall-clock triggers, unbounded
probabilistic triggers, and model-specific grading are not implemented.

Retry classification uses a deterministic fingerprint of method, path, hashed query, and body.
It is useful black-box evidence, not proof of the workload's internal intent.

## Roadmap

After v0.5's probability-window work, the next logical steps are multi-fault campaigns,
then another dependency adapter such as MCP. These are broad, nonbinding directions; detailed
release scope begins only in an approved version specification.

Licensed under Apache-2.0.
