Metadata-Version: 2.4
Name: pywrkr
Version: 1.6.0
Summary: Python HTTP benchmarking tool with extended statistics, inspired by wrk and Apache ab
Author: pywrkr contributors
License: MIT
Project-URL: Homepage, https://github.com/kurok/pywrkr
Classifier: Development Status :: 5 - Production/Stable
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: System :: Benchmark
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp~=3.14
Provides-Extra: tui
Requires-Dist: rich>=13.0; extra == "tui"
Provides-Extra: otel
Requires-Dist: opentelemetry-api>=1.20; extra == "otel"
Requires-Dist: opentelemetry-sdk>=1.20; extra == "otel"
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.20; extra == "otel"
Provides-Extra: http2
Requires-Dist: httpx[http2]>=0.27; extra == "http2"
Provides-Extra: all
Requires-Dist: rich>=13.0; extra == "all"
Requires-Dist: opentelemetry-api>=1.20; extra == "all"
Requires-Dist: opentelemetry-sdk>=1.20; extra == "all"
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.20; extra == "all"
Requires-Dist: httpx[http2]>=0.27; extra == "all"
Requires-Dist: pytest>=7.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=9.0.3; extra == "dev"
Requires-Dist: pytest-xdist>=3.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: aiohttp>=3.14.0; extra == "dev"
Requires-Dist: httpx[http2]>=0.27; extra == "dev"
Requires-Dist: hypercorn>=0.17; extra == "dev"
Requires-Dist: pyyaml>=6.0; extra == "dev"
Provides-Extra: lint
Requires-Dist: mypy>=1.0; extra == "lint"
Requires-Dist: ruff>=0.1.0; extra == "lint"
Provides-Extra: security
Requires-Dist: bandit>=1.7; extra == "security"
Requires-Dist: pip-audit>=2.6; extra == "security"
Provides-Extra: pytest
Requires-Dist: pytest>=7.0; extra == "pytest"
Dynamic: license-file

# pywrkr

[![CI](https://github.com/kurok/pywrkr/actions/workflows/python-package.yml/badge.svg)](https://github.com/kurok/pywrkr/actions/workflows/python-package.yml)
[![CodeQL](https://github.com/kurok/pywrkr/actions/workflows/codeql.yml/badge.svg)](https://github.com/kurok/pywrkr/actions/workflows/codeql.yml)
[![Latest release](https://img.shields.io/github/v/release/kurok/pywrkr)](https://github.com/kurok/pywrkr/releases/latest)
[![Python versions](https://img.shields.io/pypi/pyversions/pywrkr)](https://pypi.org/project/pywrkr/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![codecov](https://codecov.io/gh/kurok/pywrkr/graph/badge.svg)](https://codecov.io/gh/kurok/pywrkr)

**Load-test any HTTP endpoint in one command — and get wrk/ab-grade numbers without the wrk/ab setup.** `pywrkr` is a pure-Python benchmarking CLI: point it at a URL and get latency percentiles (p50–p99.99), a throughput timeline, status/error breakdowns, and CI-ready SLO checks.

Five load modes (duration, fixed-count, virtual users, constant rate, traffic profiles), HAR-file import to turn a browser recording into a test, and OpenTelemetry / Prometheus export — no JVM, no YAML, no cluster.

## Demo

![Benchmarking an HTTP endpoint with pywrkr: a live requests/sec counter during the run, then a full report with latency percentiles, a status-code breakdown, and a throughput timeline](https://raw.githubusercontent.com/kurok/pywrkr/main/docs/assets/demo.gif)

<sub>Recorded with [asciinema](https://asciinema.org) + [agg](https://github.com/asciinema/agg) — regenerate with `docs/record-demo.sh`.</sub>

## Install

```bash
pip install pywrkr
```

## Minimal example

```bash
# 10 connections, 5-second benchmark
pywrkr https://example.com -c 10 -d 5
```

That's it. Add `--json results.json`, `-w report.html`, `--threshold "p95<300ms"`, or `-u 1000` for virtual users when you need more — see [Quick Start](#quick-start) below.

> **See also:** [awesome-http-benchmark](https://github.com/denji/awesome-http-benchmark) — a curated list of HTTP(S) load & benchmarking tools (wrk, ab, k6, vegeta, …) where pywrkr fits in.

## Features

- **HAR import** (`har-import`): convert browser-recorded HAR files into pywrkr scenarios or URL lists — dramatically cuts test authoring time
- **Scripted scenarios** (`--scenario`): multi-step flows with variable extraction and `${var}` correlation — log in, capture the token, and hit authenticated endpoints with it
- **Library API** (`pywrkr.run` / `pywrkr.arun`): pure Python, so load tests live inside pytest suites, notebooks, and CI scripts — typed `Result`, thresholds as verdicts instead of `exit()`
- **HTTP/2** (`--http2`): protocol-representative load against modern edges, via a pluggable client backend — the negotiated protocol is reported, never assumed
- **Per-user sessions:** each virtual user keeps its own cookie jar, so cookie-session logins work and N users look like N real clients rather than one anonymous loop
- **Data-driven testing** (`--data`): CSV/JSON feeders with `loop`/`sequential`/`random`/`unique` strategies plus built-in generators (`${uuid()}`, `${randint()}`, `${counter()}`, …) — 1000 users with 1000 distinct payloads, not 1000 copies of one request
- **Five benchmarking modes:**
  - **Duration mode** (`-d`): wrk-style, run for N seconds
  - **Request-count mode** (`-n`): ab-style, send exactly N requests
  - **User simulation mode** (`-u`): simulate virtual users with ramp-up and think time
  - **Rate limiting mode** (`--rate`): send requests at a controlled, constant rate (with optional ramp)
  - **Traffic profiles** (`--traffic-profile`): realistic traffic shaping — sine waves, spikes, step functions, business-hour curves, and CSV replay
  - **Autofind mode** (`--autofind`): automatically ramp load to find maximum sustainable capacity
- **Detailed latency statistics:** min/max/mean/median/stdev, percentiles (p50-p99.99), histogram, and ab-style "percentage served within" table
- **Throughput timeline:** requests/sec over time in ASCII bar chart
- **Multiple output formats:** terminal, CSV (`-e`), JSON (`--json`), HTML (`-w`)
- **HTTP features:** keep-alive toggle, Basic auth (`-A`), cookies (`-C`), custom headers (`-H`), POST body (`-b`/`-p`), content-length verification (`-l`)
- **Cache-busting** (`-R`): append a unique random query parameter to each request URL
- **Graceful shutdown:** handles SIGINT/SIGTERM cleanly
- **Live progress display** with requests/sec, error count, and active user count
- **SLO-aware thresholds** (`--threshold`): pass/fail criteria like `p95 < 300ms`, `error_rate < 1%` with non-zero exit code on breach — CI-ready
- **Regression detection** (`pywrkr compare`, `--baseline`): gate a PR on *relative* change — "fail if p95 got 10% worse than main" — with a markdown delta table for the PR comment
- **Native observability export:** OpenTelemetry (`--otel-endpoint`) and Prometheus remote write (`--prom-remote-write`), streamed live during the run with `--export-interval` — windowed percentiles, cumulative counters
- **Test metadata tags** (`--tag`): attach environment, build, region labels to metrics and JSON output

### HAR / Browser-Recording Import

Convert browser-recorded [HAR files](http://www.softwareishard.com/blog/har-12-spec/) (from Chrome DevTools, Firefox, Charles Proxy, Fiddler, etc.) into pywrkr scenarios or URL lists. Similar to k6's HAR converter and JMeter's HTTP(S) Test Script Recorder.

```bash
# Convert HAR to a pywrkr scenario (JSON):
pywrkr har-import recording.har -o scenario.json

# Then run the generated scenario (URLs come from the scenario):
pywrkr --scenario scenario.json -u 100 -d 60

# Or convert to a URL file for --url-file mode:
pywrkr har-import recording.har --format url-file -o urls.txt
pywrkr --url-file urls.txt -c 50 -d 30
```

**Recording a HAR file:**

1. Open Chrome DevTools (F12) → Network tab
2. Navigate through your application
3. Right-click the network log → "Save all as HAR with content"

**Filtering options:**

```bash
# Only include requests to specific domain(s):
pywrkr har-import recording.har --domain api.example.com -o scenario.json

# Include static assets (CSS, JS, images — excluded by default):
pywrkr har-import recording.har --include-static -o scenario.json

# Exclude analytics/tracking URLs:
pywrkr har-import recording.har --exclude '/analytics' --exclude '/tracking' -o scenario.json

# Only include specific URL patterns:
pywrkr har-import recording.har --include '/api/v2' -o scenario.json

# Preserve original request headers (default: only Content-Type):
pywrkr har-import recording.har --preserve-headers -o scenario.json

# Add status code assertions from recorded responses:
pywrkr har-import recording.har --assert-status -o scenario.json

# Adjust think time (inter-request delay derived from recording):
pywrkr har-import recording.har --think-time-multiplier 0.5 -o scenario.json   # 2x faster
pywrkr har-import recording.har --no-think-time -o scenario.json               # no delays
```

**HAR import options:**

| Flag | Description |
|------|-------------|
| `har_file` | Path to the HAR file (positional, required) |
| `-o` / `--output` | Output file path (default: print to stdout) |
| `--format` | Output format: `scenario` (default) or `url-file` |
| `--name` | Scenario name (default: derived from filename) |
| `--include-static` | Include static assets (CSS, JS, images, fonts) |
| `--domain` | Only include requests to this domain (repeatable) |
| `--exclude` | Exclude URLs matching regex pattern (repeatable) |
| `--include` | Only include URLs matching regex pattern (repeatable) |
| `--preserve-headers` | Keep original request headers |
| `--no-think-time` | Don't derive think times from recorded timing |
| `--think-time-multiplier` | Scale derived think times (default: 1.0) |
| `--assert-status` | Assert recorded 2xx/3xx status codes |

## Library usage

pywrkr is pure Python, so a load test can live *inside* a pytest suite, a notebook, or an
orchestration script — no subprocess, no JSON parsing:

```python
import pywrkr

result = pywrkr.run("https://api.example.com/health", connections=50, duration=30)

assert result.percentiles.p95 < 0.3
assert result.error_rate < 1.0
print(f"{result.requests_per_sec:,.0f} req/s over {result.duration:.1f}s")
```

Nothing is printed, no signal handlers are installed, and a breached threshold comes back as a
**verdict on the result** rather than an `exit()`:

```python
result = pywrkr.run(url, duration=30, thresholds=["p95 < 300ms", "error_rate < 1%"])
for verdict in result.thresholds:
    print(verdict.expression, "->", "pass" if verdict.passed else "FAIL", verdict.actual)
if not result.passed:
    raise SystemExit(result.exit_code)  # same code the CLI would use
```

**Async-native.** `arun()` never calls `asyncio.run`, so it is safe to await inside an existing
loop; `run()` raises a clear error if called from one.

```python
results = await asyncio.gather(
    pywrkr.arun(url, connections=5, duration=30),
    pywrkr.arun(url, connections=50, duration=30),
)
```

**Full control** via `Config`, which is the same object the CLI builds — anything the CLI can
express, the library can:

```python
config = pywrkr.Config(
    url="https://api.example.com",
    users=100,
    ramp_up=10,
    think_time=0.5,
    scenario=pywrkr.load_scenario("flow.yaml"),
)
result = await pywrkr.arun(config)
print(result.steps["checkout"]["p95"])
```

**Live progress** through `on_tick`, called about once a second (an exception from it is logged,
not fatal):

```python
pywrkr.run(url, duration=60, on_tick=lambda s: print(s.elapsed, s.requests_per_sec))
```

### API reference

| Name | What it is |
|------|-----------|
| `run(target, **opts) -> Result` | Blocking run. Raises `RuntimeError` inside a running loop |
| `arun(target, **opts) -> Result` | Async run, awaitable from an existing loop |
| `Config` | The run configuration (alias of `BenchmarkConfig`) — every CLI option is a field |
| `Result` | Typed results; see below |
| `Latency` / `Percentiles` / `ThresholdVerdict` / `LiveStats` | Result components |
| `load_scenario(path) -> Scenario` | Load a JSON/YAML scenario file |

`target` is a URL string plus keyword options, or a prepared `Config`. `thresholds` accepts
expression strings (`"p95 < 300ms"`) as well as parsed objects.

`Result` exposes `total_requests`, `total_errors`, `error_rate`, `requests_per_sec`, `duration`,
`total_bytes`, `latency`, `percentiles`, `status_codes`, `error_types`, `http_versions`,
`rps_timeline`, `steps`, `thresholds`, `passed`, `exit_code`, and the raw `stats`.

**`result.to_dict()` is exactly what `--json` writes** — same schema, same `schema_version` — so a
result can be fed straight to `pywrkr compare`, a dashboard, or a golden file. `to_json()`
serializes it identically.

`percentiles` is both attribute- and key-addressed, so tail percentiles that only exist for large
samples stay reachable: `result.percentiles.p95`, `result.percentiles["p99.9"]`.

### Stability

`pywrkr.__all__` is the supported surface and is what the versioning promise covers: breaking
changes to those names require a major release. Everything else is an implementation detail. A
few worker internals that leaked into the package namespace before this API existed
(`pywrkr.worker`, `pywrkr.make_url`, …) still import for one more minor release but emit a
`DeprecationWarning` pointing at `pywrkr.workers`.

The package ships a `py.typed` marker, so type checkers see the annotations.

Runnable examples: [`examples/library_usage.py`](examples/library_usage.py).

## Requirements

- Python 3.10+

```bash
pip install pywrkr
```

## Quick Start

```bash
# Basic 10-second benchmark with 10 connections
pywrkr http://localhost:8080/

# 30 seconds, 200 concurrent connections
pywrkr -c 200 -d 30 http://localhost:8080/api

# Send exactly 1000 requests with 50 connections (ab-style)
pywrkr -n 1000 -c 50 http://localhost:8080/

# Simulate 1500 users for 5 minutes with 30s ramp-up and 1s think time
pywrkr -u 1500 -d 300 --ramp-up 30 --think-time 1.0 http://localhost:8080/

# Cache-busting mode (bypass HTTP caches with random query param)
pywrkr -R -c 100 -d 10 http://localhost:8080/

# Constant rate: 500 requests/sec for 30 seconds
pywrkr --rate 500 -d 30 http://localhost:8080/

# Rate ramp: linearly increase from 100 to 1000 req/s over 60 seconds
pywrkr --rate 100 --rate-ramp 1000 -d 60 http://localhost:8080/

# Traffic profiles: sine wave oscillating up to 500 req/s
pywrkr --rate 500 -d 120 --traffic-profile sine http://localhost:8080/

# Traffic profiles: periodic spikes at 5x baseline
pywrkr --rate 200 -d 60 --traffic-profile "spike:interval=10,multiplier=5" http://localhost:8080/

# Traffic profiles: replay production traffic from CSV
pywrkr --rate 1000 -d 300 --traffic-profile "csv:traffic.csv" http://localhost:8080/

# Autofind: automatically find max sustainable load
pywrkr --autofind --max-error-rate 1 --max-p95 5.0 http://localhost:8080/

# SLO thresholds: exit code 2 if any threshold breached (CI-friendly)
pywrkr --threshold "p95 < 300ms" --threshold "error_rate < 1%" \
    -c 100 -d 30 http://localhost:8080/

# Export metrics to OpenTelemetry collector
pywrkr --otel-endpoint http://localhost:4318 \
    --tag environment=staging --tag build=v1.2.3 \
    -c 100 -d 30 http://localhost:8080/

# Push metrics to Prometheus Pushgateway
pywrkr --prom-remote-write http://pushgateway:9091 \
    --tag region=us-east-1 --tag service=api \
    -c 100 -d 30 http://localhost:8080/

# POST with auth, cookies, and JSON output
pywrkr -n 500 -c 20 -m POST -b '{"key":"val"}' \
    -H "Content-Type: application/json" \
    -A user:pass -C "session=abc123" \
    --json results.json http://localhost:8080/api
```

## Usage

```
usage: pywrkr [-h] [-c CONNECTIONS] [-d DURATION] [-n NUM_REQUESTS]
              [-t THREADS] [-m METHOD] [-H NAME:VALUE] [-b BODY]
              [-p POST_FILE] [-A user:pass] [-C COOKIE] [-k]
              [--no-keepalive] [-l] [-v VERBOSITY] [--timeout TIMEOUT]
              [--ssl-verify] [--ca-bundle FILE] [-R] [-e FILE] [-w]
              [--json FILE] [--html-report FILE] [--live]
              [--latency-breakdown] [--tag TAGS] [--otel-endpoint URL]
              [--prom-remote-write URL] [--threshold THRESHOLDS]
              [-u USERS] [--ramp-up RAMP_UP] [--think-time THINK_TIME]
              [--think-jitter THINK_JITTER] [--rate RATE]
              [--rate-ramp RATE_RAMP] [--traffic-profile PROFILE]
              [--scenario FILE] [--autofind]
              [--max-error-rate MAX_ERROR_RATE] [--max-p95 MAX_P95]
              [--step-duration STEP_DURATION] [--start-users START_USERS]
              [--max-users MAX_USERS] [--step-multiplier STEP_MULTIPLIER]
              [--url-file FILE] [--master] [--worker HOST:PORT]
              [--expect-workers N] [--bind ADDR] [--port PORT]
              [url]
```

### Options

| Flag | Long | Description |
|------|------|-------------|
| `url` | | Target URL to benchmark (required) |
| `-c` | `--connections` | Number of concurrent connections (default: 10) |
| `-d` | `--duration` | Test duration in seconds (default: 10) |
| `-n` | `--num-requests` | Total number of requests (ab-style, overrides `-d`) |
| `-t` | `--threads` | Number of worker groups (default: 4) |
| `-m` | `--method` | HTTP method: GET, POST, PUT, DELETE, etc. (default: GET) |
| `-H` | `--header` | Custom header, e.g. `-H "Content-Type: application/json"` (repeatable) |
| `-b` | `--body` | Request body string |
| `-p` | `--post-file` | File containing POST body data |
| `-A` | `--basic-auth` | Basic HTTP auth as `user:pass` |
| `-C` | `--cookie` | Cookie as `name=value` (repeatable) — always sent, in every mode |
| | `--no-session-cookies` | Ignore `Set-Cookie`. By default each virtual user keeps its own cookie jar |
| | `--http2` | Use the HTTP/2 backend (needs `pywrkr[http2]`). `-c` then bounds concurrent streams, not connections |
| | `--data` | Attach a CSV/JSON data set as `NAME=FILE` (repeatable), referenced as `${NAME.column}`. Requires `--scenario` |
| | `--data-strategy` | Row hand-out strategy as `NAME=STRATEGY` (repeatable): `loop`, `sequential`, `random`, `unique` |
| | `--baseline` | Compare against previous `--json` results (file or glob to average) and apply `--fail-on` |
| | `--save-baseline` | Write this run's results to a file for later `--baseline` comparison |
| | `--fail-on` | Regression rule on the baseline delta (repeatable), e.g. `"p95 > +10%"`. Exit code 3 when one fires |
| | `--strict-config` | Fail instead of warning when the baseline used a different load shape |
| | `--compare-format` | Baseline comparison format: `table` (default), `markdown`, `json` |
| `-k` | `--keepalive` | Enable keep-alive (default: on) |
| | `--no-keepalive` | Disable keep-alive |
| `-l` | `--verify-length` | Verify response Content-Length consistency |
| `-v` | `--verbosity` | 0=quiet, 2=warnings, 3=status codes, 4=full detail |
| | `--timeout` | Request timeout in seconds (default: 30) |
| `-e` | `--csv` | Write CSV percentile table to file |
| `-w` | `--html` | Print results as HTML table |
| | `--json` | Write JSON results to file |
| `-R` | `--random-param` | Append unique `_cb=<uuid>` query param per request (cache-buster) |
| | `--rate` | Target requests per second (constant rate mode) |
| | `--rate-ramp` | Linearly ramp rate from `--rate` to this value over the duration |
| | `--traffic-profile` | Traffic shaping profile: `sine`, `step`, `sawtooth`, `square`, `spike`, `business-hours`, or `csv:file.csv` |
| | `--html-report` | Generate interactive Gatling-style HTML report to file |
| | `--live` | Live TUI dashboard during benchmark (requires `pywrkr[tui]`) |
| | `--scenario` | Path to JSON/YAML scenario file for scripted multi-step requests (supports `extract` + `${var}` correlation) |
| | `--latency-breakdown` | Show detailed per-phase latency breakdown (DNS, TCP, TLS, TTFB, transfer) |
| | `--threshold` / `--th` | SLO threshold (repeatable), e.g. `--threshold "p95 < 300ms"`. Exit code 2 on breach |
| | `--tag` | Metadata tag as `key=value` (repeatable), e.g. `--tag environment=staging` |
| | `--otel-endpoint` | Export metrics to OpenTelemetry collector (OTLP/HTTP) |
| | `--prom-remote-write` | Push metrics to Prometheus Pushgateway endpoint |
| | `--export-interval` | Stream metric snapshots every N seconds instead of only at the end (needs an export endpoint) |
| | `--ssl-verify` / `PYWRKR_SSL_VERIFY` | Enable TLS certificate verification (default: disabled). Recommended when using `--basic-auth` or `--cookie` against `https://` targets |
| | `--ca-bundle PATH` / `PYWRKR_CA_BUNDLE` | Path to a custom CA certificate bundle (PEM format). Used when `--ssl-verify` is enabled and the target uses a private or corporate CA |

### User Simulation Options

| Flag | Long | Description |
|------|------|-------------|
| `-u` | `--users` | Number of virtual users (enables simulation mode) |
| | `--ramp-up` | Seconds to gradually start all users (default: 0) |
| | `--think-time` | Mean pause between requests per user in seconds (default: 1.0) |
| | `--think-jitter` | Think time jitter factor 0-1 (default: 0.5, i.e. +/-50%) |

## Output

### Terminal Output

```
======================================================================
  BENCHMARK RESULTS
======================================================================
  Mode:              300 virtual users, 120.0s
  Duration:          124.15s
  Virtual Users:     300
  Ramp-up:           10.00s
  Think Time:        1.00s (+/-50%)
  Avg Reqs/User:     50.8
  Keep-Alive:        yes
  Total Requests:    15,229
  Total Errors:      1
  Requests/sec:      122.66
  Transfer/sec:      119.34MB/s
  Total Transfer:    14.46GB

======================================================================
  LATENCY STATISTICS
======================================================================
    Min:          449.00ms
    Max:            4.85s
    Mean:           961.00ms
    Median:         870.00ms
    Stdev:          520.00ms

  Latency Percentiles:
    p50           870.00ms
    p75             1.10s
    p90             1.56s
    p95             2.98s
    p99             4.85s
```

### JSON Output

Use `--json results.json` to save structured results:

```json
{
  "duration_sec": 124.15,
  "connections": 300,
  "total_requests": 15229,
  "total_errors": 1,
  "requests_per_sec": 122.66,
  "transfer_per_sec_bytes": 125120000.0,
  "total_bytes": 15533200000,
  "latency": {
    "min": 0.449,
    "max": 4.85,
    "mean": 0.961,
    "median": 0.87,
    "stdev": 0.52
  },
  "percentiles": {
    "p50": 0.87,
    "p75": 1.1,
    "p90": 1.56,
    "p95": 2.98,
    "p99": 4.85
  }
}
```

## Benchmarking Modes

### Duration Mode (wrk-style)

Runs for a fixed duration with a pool of persistent connections:

```bash
pywrkr -c 100 -d 30 http://localhost:8080/
```

### Request-Count Mode (ab-style)

Sends exactly N requests, then stops:

```bash
pywrkr -n 10000 -c 50 http://localhost:8080/
```

### User Simulation Mode

Simulates realistic user behavior with configurable think time and gradual ramp-up:

```bash
pywrkr -u 500 -d 300 --ramp-up 30 --think-time 1.0 http://localhost:8080/
```

Each virtual user:
1. Sends a request
2. Waits for the response
3. Pauses for think time (with jitter)
4. Repeats until duration expires

The ramp-up period gradually introduces users to avoid a thundering herd at startup.

### Scripted Scenarios

A scenario file (JSON or YAML) describes a multi-step flow that every virtual user replays in a loop:

```bash
pywrkr --scenario examples/scenario-correlation.json -u 100 -d 60
```

Each step takes a `path`, plus optional `method`, `headers`, `body`, `think_time`, `name`,
`assert_status`, and `assert_body_contains`. The target host comes from the positional URL, or
from the scenario's own `base_url` when no URL is given.

#### Step assertions

`assert_status` alone lets a load test pass while the API returns well-formed garbage. Each step
can check what actually makes a response correct:

```yaml
steps:
  - name: get-user
    path: /users/42
    assert_status: 200
    assert_body_contains: "email"
    assert_body_regex: '"id":\s*42'
    assert_json:
      "$.id": 42          # must equal
      "$.email": "*"      # must exist, any value
    assert_header:
      X-Trace: "abc123"                    # exact match
      Content-Type: {regex: "^application/json"}   # or a regex
    assert_max_latency: 500ms
```

| Assertion | Checks |
|-----------|--------|
| `assert_status` | Exact status code |
| `assert_body_contains` | Substring is present in the body |
| `assert_body_regex` | Regex matches somewhere in the body |
| `assert_json` | JSONPath → expected value, or `"*"` for "must exist" |
| `assert_header` | Header equals a string, or matches `{regex: "..."}` |
| `assert_max_latency` | This request took no longer than `500ms` / `1.5s` / `250us` |

`assert_json` uses the same JSONPath subset as `extract`, so the two never disagree. Numbers
compare numerically (`42` matches `42.0`), but booleans stay distinct from numbers — `true` does
not satisfy an expected `1`.

Exact `assert_header` matches are exact: a server sending `application/json; charset=utf-8` will
*not* match `"application/json"`. Use the regex form for prefixes.

A failed assertion counts the request as **one** error however many rules broke, and each broken
rule gets its own key in the error distribution. Those keys name the *rule*, never the observed
value — otherwise a per-request latency or payload id would mint a fresh key every time and
overflow the breakdown. The observed value goes to the log at `-v 2`.

Bad regexes, unsupported JSONPaths, and nonsense durations are rejected when the scenario file
loads, naming the step.

#### Per-step reporting

Scenario runs report each step separately, because an aggregate p95 blends them: if `login` is
40ms and `checkout` is 2s, the headline number describes neither.

```
  PER-STEP BREAKDOWN
    Step          Count   Errors      Req/s         p50         p95         p99         Max
    get-user         35        0       17.3    509.00us    872.00us    894.00us    894.00us
    checkout         34       34       16.8     52.67ms     53.10ms     53.10ms     53.10ms
```

The same blocks appear under `step_stats` in `--json` (with `count`, `errors`, `requests_per_sec`,
`min`/`max`/`mean`/`median`/`stdev` and `p50`/`p95`/`p99`) and as a table in `--html-report`. In
distributed mode they are merged across workers like the global stats.

#### Variable extraction & correlation

Steps are not limited to replaying static requests: an `extract` block pulls values out of a
response, and later steps reference them as `${var}`. This is what makes authenticated and
stateful flows testable — login → capture token → call the API with it.

```yaml
name: Login and read profile
on_extract_failure: abort_iteration   # or: continue
on_template_error: abort_iteration    # or: keep_literal
steps:
  - name: login
    method: POST
    path: /auth/login
    body: '{"user": "demo", "pass": "demo"}'
    extract:
      token:
        json: "$.access_token"                 # JSONPath into the JSON body
      session_id:
        header: "X-Session-Id"                 # response header value
      csrf:
        regex: 'name="csrf" value="([^"]+)"'   # first capture group

  - name: get-profile
    path: /me
    headers:
      Authorization: "Bearer ${token}"
      X-Session: "${session_id}"
    assert_status: 200

  - name: submit-form
    method: POST
    path: /form
    body:
      csrf: "${csrf}"
      user_token: "${token}"
```

**Extraction sources** — a rule names exactly one of:

| Source | Expression | Notes |
|--------|-----------|-------|
| `json` | `$.a.b[0].c` | Dotted JSONPath subset: object keys, array indices (negative allowed), and `["quoted keys"]`. Wildcards, slices, filters, and recursive descent are not supported. `$` selects the whole document. |
| `header` | `X-Session-Id` | Response header, matched case-insensitively. |
| `regex` | `value="([^"]+)"` | Searched against the response body; capture group 1 is used. The pattern must have at least one group. |

Non-string JSON values keep their JSON spelling (`true`, not `True`); objects and arrays are
re-serialized compactly, so a whole sub-document can be carried between steps.

**Where `${var}` works:** the step `path`, header names and values, and the `body` — including
inside nested JSON object/array bodies. Values are inserted verbatim, so URL-encode anything that
needs it on the server side. `${...}` is the entire template language: no expressions, no logic.

**Variable scope:** each virtual user has its own variable set, cleared at the start of every
iteration. Users never see each other's tokens, and every iteration starts from the same known
state.

**Failure handling** — both options are scenario-level:

| Option | Values | Behavior |
|--------|--------|----------|
| `on_extract_failure` | `abort_iteration` (default), `continue` | An `extract` rule that produces no value skips the rest of the iteration, or is ignored and the flow continues. |
| `on_template_error` | `abort_iteration` (default), `keep_literal` | A `${var}` that is not bound aborts the iteration, or is sent to the server literally. |

Failures are visible in three places: the `Extract Failures` / `Template Errors` counters in the
terminal summary, the `extract_failures` / `template_errors` fields in JSON output, and the error
distribution as distinct `ExtractFailure: ...` / `TemplateError: ...` keys naming the variable and
the reason. Bad regexes, unsupported JSONPaths, and invalid option values are rejected when the
scenario file loads — not mid-run.

Those dedicated counters record every occurrence, but the headline `Total Errors` (and therefore
`error_rate` thresholds) charges an iteration at most once: a 401, the extraction that failed on
its body, and the `${var}` that could not resolve as a result are one broken flow, not three.

Working example: [`examples/scenario-correlation.json`](examples/scenario-correlation.json).

### HTTP/2

Most production edges (CDNs, ALBs, nginx, Envoy) serve HTTP/2, and their behaviour under load is
qualitatively different: h2 multiplexes streams over one connection instead of holding a
connection per in-flight request. `--http2` generates protocol-representative load against them.

```bash
pip install 'pywrkr[http2]'
pywrkr --http2 -c 100 -d 30 https://edge.example.com/
```

**`-c` means concurrent streams, not connections.** Under HTTP/1.1, `-c 100` opens 100 sockets.
Under HTTP/2 the client multiplexes, so `-c 100` bounds in-flight streams and the socket count is
far lower — worth remembering when comparing an h1 baseline against an h2 run.

**Protocol negotiation is reported, never assumed.** Over `https://`, ALPN decides; a server that
only offers HTTP/1.1 is used as such, counted separately, and warned about, so a run can't quietly
claim to be an HTTP/2 test:

```
  NEGOTIATED PROTOCOL
    HTTP/2:        9,321 (100.0%)
```

JSON output carries the same counts in `http_versions`. Over `http://` there is no ALPN handshake,
so HTTP/2 is used with prior knowledge (h2c) — the only way `--http2` can mean anything against a
cleartext target. A cleartext server that does not speak h2c will fail the requests rather than
silently downgrade.

**`--latency-breakdown` reports less on this backend.** The HTTP/2 client has no hooks for the DNS,
TCP and TLS phases, so those are **omitted** rather than reported as zero — a zero would read as an
impossibly fast connection phase. TTFB, transfer and total are still measured. Connection-reuse
counts are omitted for the same reason: under h2, "200 new connections" would be one connection
carrying 200 streams.

Everything else — virtual users, rate limiting, traffic profiles, scenarios with correlation and
feeders, thresholds, and baseline comparison — works identically on both backends. Distributed
workers must each have the extra installed; a worker without it refuses the run and says so rather
than contributing HTTP/1.1 load to an HTTP/2 result.

### Regression Testing in CI

Absolute gates (`--threshold "p95 < 300ms"`) rot: loose enough never to fire, or tight enough to
flake on infrastructure noise. What a PR gate usually wants is relative — *"fail if p95 got more
than 10% worse than the last known-good run"*:

```bash
# Record a baseline on main
pywrkr --save-baseline baseline.json -c 100 -d 30 https://api.example.com/

# Gate a PR against it, in one command
pywrkr --baseline baseline.json \
       --fail-on "p95 > +10%" --fail-on "rps < -5%" \
       -c 100 -d 30 https://api.example.com/
```

Or compare two existing `--json` files after the fact:

```bash
pywrkr compare baseline.json current.json --fail-on "p95 > +10%"
```

**`--fail-on` expressions** state the condition under which the gate *fails*, and always compare
the **delta**, never the raw value:

| Expression | Fails when |
|------------|-----------|
| `p95 > +10%` | p95 is more than 10% higher than the baseline |
| `rps < -5%` | throughput dropped by more than 5% |
| `p99 > +50ms` | p99 grew by more than 50ms in absolute terms |
| `error_rate > +0.5` | the error rate rose by more than 0.5 **percentage points** |
| `step:checkout.mean > +20ms` | that scenario step's mean latency grew by more than 20ms |

Metrics: `rps`, `error_rate`, `total_requests`, `total_errors`, `total_bytes`, `transfer_rate`,
`duration`, `min_latency`, `max_latency`, `avg_latency`, `median_latency`, `stdev_latency`, any
percentile (`p50`…`p99.99`), and `step:<name>.<field>`. A `%` suffix makes a rule relative;
anything else is an absolute delta in the metric's own unit (`ms`/`us`/`s` accepted for latency).

Note the asymmetry for `error_rate`: `+0.5` is half a percentage point, while `+10%` is 10%
*relative* to the baseline error rate.

**Exit codes:** `0` no regression · `2` an absolute `--threshold` was breached · `3` a `--fail-on`
rule fired · `1` usage or schema error. When both a threshold and a regression fire, `2` wins.

**A threshold on a metric the run did not produce fails.** If nothing was measured there is no p95,
and a gate that goes green because it found nothing to check is worse than no gate — the table
prints `not measured` and the run exits `2`. A genuine zero is still a zero: a run with requests and
no errors passes `error_rate < 1%`.

**Output formats:** `--format markdown` produces a table ready to paste into a PR comment;
`--format json` gives a machine-readable verdict. (On the main command the flag is
`--compare-format`.)

**Per-step thresholds.** For a scenario, the aggregate `p95` is a blend of every step — so a
checkout flow whose login is 1ms and whose payment call is 250ms sits comfortably under an aggregate
budget while the step that matters is five times over it, and *adding fast steps improves the
number*. Scope a threshold to one step instead:

```bash
pywrkr --scenario checkout.json -u 50 -d 60 \
       --threshold "step:login p95 < 200ms" \
       --threshold "step:payment p99 < 3s" \
       --threshold "step:payment error_rate < 0.5%" \
       --threshold "p95 < 1s"                        # the aggregate still works
```

```
  Expression                    Actual   Status
  step:login p95 < 50ms         1.07ms     PASS
  step:payment p95 < 50ms     253.91ms     FAIL
  p95 < 500ms                 253.91ms     PASS   <-- the blend hides it
```

Every metric the aggregate form accepts works per step. The step name runs from `step:` to the
metric, so a name containing spaces or colons — which the default `METHOD /path` naming produces —
needs no quoting.

Two failures are caught **before any load is applied**, because they need different fixes: a
`step:` threshold with no `--scenario` has no steps to measure, and a step name the scenario does not
define is a typo (the error lists the names it does define). A step that exists but never ran —
because an earlier step aborted the iteration — reports `not measured` and fails, like any other
unmeasurable metric.

Note the spelling difference from `--fail-on`, which is one token and so uses a dot:
`--fail-on "step:checkout.mean > +20ms"` against a baseline, `--threshold "step:checkout p95 < 800ms"`
for an absolute gate.

**Comparability.** Results carry a `schema_version` and a snapshot of the load shape (mode,
connections, users, duration, host). Comparing a 10-user run against a 1000-user baseline is
arithmetically fine and completely meaningless, so `compare` warns when they differ — and fails
with `--strict-config`.

**Riding out noise.** A single baseline run makes every later run look like a regression when the
baseline happened to be lucky. Point `--baseline` at a glob to average several:

```bash
pywrkr compare 'baselines/*.json' current.json --fail-on "p95 > +10%"
```

A recommended recipe: run 3–5 repetitions, discard the first as warm-up, and keep the rest as the
baseline set.

#### GitHub Action

```yaml
- uses: kurok/pywrkr@v1
  with:
    url: http://localhost:8080/
    args: -c 50 -d 30
    thresholds: |
      p95 < 500ms
      error_rate < 1%
    comment-pr: true
```

That runs the benchmark, gates the job on the thresholds, writes the table to the job summary, and
posts it on the PR — **editing its own previous comment instead of adding a new one**, so a branch
with twenty pushes has one report, not twenty.

Gate on a baseline instead of, or alongside, absolute thresholds:

```yaml
- uses: kurok/pywrkr@v1
  with:
    url: http://localhost:8080/
    args: -c 50 -d 30
    baseline: perf/baseline.json
    fail-on: |
      p95 > +10%
      rps < -5%
    save-baseline: perf/candidate.json
```

| Input | Default | Description |
|-------|---------|-------------|
| `url` | — | Target URL. Omit only if `args` supplies its own target. |
| `args` | `""` | Any other pywrkr flags, e.g. `-c 50 -d 30 --rate 200`. |
| `thresholds` | `""` | Absolute gates, one per line (`p95 < 500ms`). |
| `baseline` | `""` | Results file or glob to compare against. |
| `fail-on` | `""` | Regression rules, one per line (`p95 > +10%`). Requires `baseline`. |
| `save-baseline` | `""` | Also write this run's results here, to commit or cache. |
| `version` | `latest` | Version to install, or `local` for the checked-out tree. |
| `comment-pr` | `false` | Post/update the report on the PR. Needs `pull-requests: write`. |
| `soft-fail` | `false` | Report a breach without failing the step. |
| `html-report` | `""` | Also write a standalone HTML report here. |
| `results-file` | `pywrkr-results.json` | Where the JSON results go. |
| `summary-file` | `pywrkr-report.md` | Where the rendered markdown goes. |
| `job-summary` | `true` | Append the report to the job summary. |
| `title` | `pywrkr performance report` | Heading used in the report. |

Outputs: `passed`, `verdict` (`pass` / `threshold` / `regression` / `error`), `p50`, `p95`, `p99`,
`rps`, `error-rate`, `total-requests`, `results-file`, `summary-file`. Latencies are in seconds and
`error-rate` is a percentage; a metric the run did not produce comes back as an empty string, so a
downstream step can tell "no data" from "zero".

```yaml
- uses: kurok/pywrkr@v1
  id: perf
  with: { url: http://localhost:8080/, args: -c 50 -d 30, soft-fail: "true" }
- run: echo "p95 was ${{ steps.perf.outputs.p95 }}s at ${{ steps.perf.outputs.rps }} req/s"
```

Two behaviours worth knowing about:

- **A threshold on a metric the run never produced fails.** If the target was unreachable there is
  no p95, and a gate that goes green because it found nothing to check is worse than no gate.
- **The action depends on no other action.** It installs pywrkr into a private venv using the
  runner's own Python; there is no `uses:` inside it, so adopting it does not pull anything else
  into your supply chain. Every input reaches the shell through the environment rather than being
  interpolated into a script, so an input from an untrusted fork cannot inject commands.

If you would rather wire it up by hand, `pywrkr summary` is the same code the action calls:

```bash
pywrkr -c 50 -d 30 --json results.json http://localhost:8080/
pywrkr summary results.json \
       --threshold "p95 < 500ms" \
       --baseline perf/baseline.json --fail-on "p95 > +10%" \
       --output report.md --github-output "$GITHUB_OUTPUT"
```

It re-reads the results file rather than re-running anything, and exits `0` / `2` / `3` on the same
rules as the main command.

### WebSocket Benchmarking

Real-time features — chat, live dashboards, trading feeds, collaborative editing — ride on
WebSockets, and what matters about their load is invisible to a request/response benchmarker: a
connection storm, how many sockets a server holds open, how long a message takes to come back on a
socket that is already established. A `ws://` or `wss://` URL switches modes automatically:

```bash
# 500 sockets, each sending a message every second for 60s, measuring round-trip latency
pywrkr wss://ws.example.com/feed -c 500 -d 60 \
       --ws-message '{"op":"ping"}' --ws-message-interval 1 --ws-expect-reply

# Connection storm: open 1000 sockets over 30s and hold them, counting server pushes
pywrkr wss://ws.example.com/feed -c 1000 -d 300 --ramp-up 30
```

`-c` is concurrent sockets, `-d` is how long to hold them, and `--ramp-up` staggers the handshakes
so a connection storm has the shape you asked for instead of arriving all at once.

| Option | Default | Description |
|--------|---------|-------------|
| `--ws-message TEXT` | — | Payload to send; repeat to cycle several. Without any, the run connects and listens |
| `--ws-message-interval S` | `1.0` | Seconds between sends on one socket; `0` sends as fast as the socket allows |
| `--ws-expect-reply` | off | Wait for a reply to each message and report round-trip latency |
| `--ws-reply-timeout S` | `--timeout` | How long to wait for that reply |
| `--ws-subprotocol NAME` | — | `Sec-WebSocket-Protocol` to offer; repeatable |
| `--ws-ping-interval S` | off | Send a ping every S seconds to keep idle sockets alive |
| `--ws-max-message-size B` | 4 MiB | Reject frames larger than this |
| `--ws-close-timeout S` | `5.0` | How long to wait for the peer's close frame |
| `--ws-reconnect` | off | Reopen a socket the server closed instead of leaving the slot empty |
| `--ws-reconnect-delay S` | `1.0` | Pause before reconnecting |

**Which number is the latency?** Stated explicitly rather than left to be inferred, both in the
terminal output and as `websocket.latency_metric` in `--json`:

- With `--ws-expect-reply`, the run's latency — and therefore `--threshold "p95 < 100ms"` — is the
  **message round-trip time**.
- Without it, there is no reply to time, so the latency is the **handshake**.

Handshake and round-trip latency are *also* always reported separately, because a service that
connects instantly and answers slowly and one that does the reverse are different problems that a
single latency line cannot tell apart.

Likewise, `requests_per_sec` counts messages when there are messages to send and connections
otherwise; `websocket.primary_metric` says which.

**What is reported.** On top of the usual percentile/threshold/JSON/HTML machinery, `--json` gains
a `websocket` block:

```json
{
  "websocket": {
    "latency_metric": "rtt",
    "primary_metric": "messages",
    "connections": {"opened": 500, "failed": 0, "dropped": 3, "reconnects": 0,
                    "peak_concurrent": 500},
    "messages": {"sent": 29847, "received": 29844, "sent_per_sec": 497.45,
                 "received_per_sec": 497.40, "bytes_sent": 447705,
                 "bytes_received": 1790640, "reply_timeouts": 3,
                 "unexpected_replies": 0},
    "handshake": {"count": 500, "min": 0.0012, "max": 0.041, "mean": 0.0089,
                  "percentiles": {"p50": 0.0081, "p95": 0.0223, "p99": 0.0388}},
    "rtt":       {"count": 29844, "min": 0.0004, "max": 0.112, "mean": 0.0021,
                  "percentiles": {"p50": 0.0018, "p95": 0.0044, "p99": 0.0091}},
    "close": {"frames_sent": 500, "unacknowledged": 0, "codes": {"1000": 500}}
  }
}
```

`close.unacknowledged` counts sockets whose close frame the server never answered — a server that
does not read its sockets shows up here instead of as a silent zero.

**Clean shutdown.** Every socket is closed with a close frame, on normal completion and on
`Ctrl-C` alike, so a benchmark does not leave the server holding thousands of half-open connections
that poison whatever you measure next. That teardown is deliberately excluded from the reported
duration: waiting on an unresponsive peer is not load, and counting it would deflate every rate
derived from it.

`wss://` uses the same TLS settings as `https://` — `--ssl-verify` and `--ca-bundle` behave
identically.

#### Mixed HTTP + WebSocket scenarios

A `ws:` step in a scenario opens a socket on the *same session* as the HTTP steps around it, so it
inherits their cookies, and `${var}` correlation works across the protocol boundary:

```json
{
  "steps": [
    {"name": "login", "path": "/api/login", "method": "POST",
     "extract": {"token": {"json": "$.token"}}},
    {"name": "subscribe", "ws": "wss://ws.example.com/feed?auth=${token}",
     "send": "{\"op\":\"subscribe\",\"channel\":\"orders\"}",
     "expect_message_contains": "\"subscribed\"",
     "hold": "30s",
     "extract": {"sid": {"json": "$.sid"}}}
  ]
}
```

| Key | Description |
|-----|-------------|
| `ws` | The `ws://`/`wss://` URL. Absolute — `base_url` is not prepended. Templated. |
| `send` | Payload to send once the socket is open. Templated. |
| `expect_message_contains` | Wait for a message containing this substring. Scans every arriving message, not just the first, so a confirmation behind a welcome frame or a heartbeat still matches. Its text is what the step's `extract` rules run against. |
| `hold` | Keep the socket open afterwards (`"30s"`, `"250ms"`, or a bare number of seconds), counting what the server pushes. |

The step's latency is the whole thing — handshake, send, and the wait for the expected message —
because that is what a user of a "subscribe and get the first update" flow actually waits for.
`hold` afterwards is passive listening and is not counted in it. HTTP-only keys (`method`, `body`,
`assert_status`) are rejected on a `ws:` step rather than silently ignored.

See [`examples/scenario-websocket.json`](examples/scenario-websocket.json).

**Not supported yet:** distributed WebSocket mode (`--master` rejects a `ws://` target; a mixed
HTTP/WebSocket *scenario* does run distributed), Socket.IO/SockJS protocol layers, and gRPC/SSE.

### pytest Integration

Performance testing usually lives in its own silo, runs rarely, and rots. Being pure Python is
pywrkr's structural advantage over wrk/k6/Gatling, and this is what cashes it in: an SLO becomes a
test in the suite that already exists, failing a PR the way a unit test does.

```bash
pip install "pywrkr[pytest]"
```

The plugin registers itself; there is nothing to enable.

```python
import pytest


@pytest.mark.pywrkr(
    url="/health", connections=20, duration=10, thresholds=["p95 < 200ms", "error_rate < 1%"]
)
def test_health_meets_slo(pywrkr_result):
    assert 200 in pywrkr_result.status_codes


def test_search_stays_under_budget(pywrkr_bench):
    result = pywrkr_bench("/api/search?q=widget", connections=50, duration=15)
    assert result.percentiles.p95 < 0.5
    assert result.requests_per_sec > 100
```

A breached threshold **fails the test**, naming the metric, the bound and what was measured:

```
E   Failed: pywrkr threshold(s) breached for /health:
E     - p95 < 200ms (measured p95 = 412.30ms)
```

**Benchmarks skip by default.** They put real load on whatever the base URL points at, and a test
suite is run casually and often. Pass `--pywrkr-run` to execute them; without it they skip with a
reason saying so. Put the flag in your performance CI job, not in `addopts`.

| Option / setting | Where | Description |
|------------------|-------|-------------|
| `--pywrkr-run` | CLI | Actually run the benchmarks. Required. |
| `--pywrkr-json DIR` | CLI | Write each benchmark's JSON result into `DIR`, named after the test's node id. Feeds `pywrkr compare`. |
| `pywrkr_base_url` | ini | Prepended to a relative target, so tests name paths and the host stays an environment detail. An absolute URL in a test always wins. |
| `pywrkr_duration` | ini | Default duration in seconds. |
| `pywrkr_connections` | ini | Default connection count. |

```ini
[pytest]
pywrkr_base_url = http://localhost:8080
pywrkr_duration = 10
pywrkr_connections = 20
```

`pywrkr_bench(url, **options)` takes any [`Config`](#library-usage) field, so the load shape is not
limited to connections and duration — `users`, `ramp_up`, `think_time`, `rate`, `scenario`,
`method`/`body`/`headers` all work. It returns the same `Result` object as `pywrkr.run()`.

**Feeding the baseline workflow.** `--pywrkr-json` writes one schema-valid file per test, which is
exactly what `pywrkr compare` reads:

```bash
pytest -m pywrkr --pywrkr-run --pywrkr-json perf-results/
pywrkr compare 'baselines/*.json' perf-results/tests-test_perf.py-test_health.json \
       --fail-on "p95 > +10%"
```

**Terminal summary.** After the run, every benchmark that executed is tabulated:

```
============================= pywrkr benchmarks =============================
Test                       Requests      Req/s        p50        p95        p99   Errors  Verdict
-------------------------------------------------------------------------------------------------
test_health_meets_slo        98,412    9,841.2     1.94ms     3.11ms     5.02ms    0.00%  PASS
test_search_under_budget     12,004      800.3    58.10ms   412.30ms   890.00ms    0.02%  FAIL
                                breached: p95 < 200ms
```

**pytest-xdist is refused, deliberately.** `--pywrkr-run` together with `-n` is a usage error, not
a missing feature: benchmarks running in parallel contend for the same CPU, sockets and target, so
four workers each opening 50 connections put 200 on the host while each reports 50. Every number
produced would be wrong in a way nothing downstream could detect. Run benchmarks in their own
non-parallel invocation:

```bash
pytest -n auto -m "not pywrkr"                  # the fast suite, in parallel
pytest -p no:xdist -m pywrkr --pywrkr-run       # the benchmarks, alone
```

**Zero cost when unused.** pytest imports every registered plugin at startup, so the plugin is a
top-level `pytest_pywrkr` module rather than `pywrkr.pytest_plugin` — anything under `pywrkr.`
would pull in the package `__init__` and its whole public API on every pytest run of any project
that merely depends on pywrkr. Nothing but pytest is imported at module scope; the benchmark
runner is imported inside the fixture that needs it.

See [`examples/test_perf_example.py`](examples/test_perf_example.py).

### OpenAPI Import

`har-import` covers "I can click through the app in a browser". For an API-first service there is
often no browser flow to record, but an OpenAPI document already exists — FastAPI and most modern
frameworks publish one for free.

```bash
# From a local spec (JSON or YAML, OpenAPI 3.0/3.1)
pywrkr openapi-import openapi.json -o scenario.json

# Straight from a running FastAPI app
pywrkr openapi-import http://localhost:8000/openapi.json -o scenario.json

# Filter, and opt in to mutating methods
pywrkr openapi-import spec.yaml --include '/api/v2' --exclude '/admin' \
       --method GET --method POST --tag public \
       --base-url https://staging.example.com --assert-status -o scenario.json
```

| Option | Description |
|--------|-------------|
| `-o`, `--output` | Write here instead of stdout |
| `--format` | `scenario` (default) or `url-file` |
| `--name` | Scenario name (default: the spec's `info.title`) |
| `--method` | Method to include; repeatable. **Default: GET and HEAD only** |
| `--include` / `--exclude` | Path regexes; repeatable. Exclude wins |
| `--tag` | Only operations carrying this tag; repeatable |
| `--base-url` | Override the spec's `servers[]` entry |
| `--assert-status` | Add `assert_status` from each operation's documented success code |
| `--think-time` | Scenario-wide think time between steps |
| `--timeout`, `--ssl-verify`, `--ca-bundle` | For fetching a remote spec |

**It does not invent data.** That is the whole design. A spec says an endpoint takes a `user_id`;
it rarely says which user ids exist. Guessing produces a scenario that benchmarks a 404 handler,
which is worse than useless because it looks like it worked. So:

- Values come from the schema's `example`, then `default`, then the first `enum` value.
- A **required** parameter with none of those becomes `${placeholder}` and is listed at the end.
- An **optional** parameter with none of those is *omitted* — leaving it out is a request the spec
  explicitly allows, whereas `?q=string` is a different request that may take a different path.
- Request bodies are the opposite case: the shape has to match for the request to be accepted at
  all, so missing optional fields get type-appropriate stubs. A missing **required** field still
  becomes a placeholder. A `format: uuid` field becomes `${uuid()}`, not a fixed value — a
  constant uuid would make every virtual user collide.
- **Credentials are never invented.** `securitySchemes` produces a templated
  `Authorization: Bearer ${token}` or an apiKey header, plus printed guidance.

The report goes to **stderr**, so `pywrkr openapi-import spec.json > scenario.json` still yields
valid JSON:

```
Authentication (no credentials were invented):
  - `bearerAuth` is HTTP bearer: the Authorization header is templated as `${token}`.

Needs input -- 2 value(s) the spec did not supply. Each is a ${placeholder}; bind them
with --data or an earlier extract step:
  - listWidgets: query `cursor` -- required, and the spec gives no example/default/enum
  - getWidget: path `widgetId` -- required, and the spec gives no example/default/enum

Skipped:
  - 2 mutating operation(s) (POST/PUT/PATCH/DELETE). Only safe methods are generated by
    default; add --method POST to include them.
```

Bind the placeholders with a [data set](#data-driven-testing) or an earlier
[`extract`](#scripted-scenarios) step, and the scenario is ready to run.

**Safe methods only by default.** Generating a scenario that DELETEs its way through an API
because the spec documented the endpoint is not a helpful default; naming `--method DELETE` makes
it a conscious choice. The count of operations skipped this way is reported, not hidden.

**Limits.** Swagger 2.0 is rejected with a pointer to a converter rather than half-translated —
its parameter model differs enough that a partial translation would silently drop request bodies.
`$ref` resolution is single-document; bundle first (e.g. `redocly bundle`) for multi-file specs.
Which POST feeds which GET is not inferred — placeholders plus `extract` cover that by hand.

See [`examples/openapi-widget-api.json`](examples/openapi-widget-api.json) and the scenario it
generates, [`examples/openapi-import-scenario.json`](examples/openapi-import-scenario.json).

### Data-Driven Testing

Identical payloads systematically overstate cache performance and understate database and
session-store load. A scenario can declare named **data sets** so every virtual user works from
its own row:

```yaml
data:
  users:
    file: users.csv        # or users.json (a list of flat objects)
    strategy: unique       # loop | sequential | random | unique
steps:
  - name: login
    method: POST
    path: /auth/login
    body: '{"user": "${users.username}", "pass": "${users.password}"}'
```

```bash
pywrkr --scenario examples/scenario-data-driven.json -u 100 -d 60

# Or attach a data set from the CLI, without touching the scenario file:
pywrkr --scenario flow.json --data users=users.csv --data-strategy users=unique -u 100 -d 60
```

Each user draws one row per data set at the **start of every iteration** and references its
columns as `${dataset.column}`, anywhere templating works — path, headers, and body.

**Strategies** — the cursor is shared by all users in a run, so `unique` really is unique rather
than unique-per-user:

| Strategy | Behavior |
|----------|----------|
| `loop` (default) | Rows handed out round-robin, wrapping around forever |
| `sequential` | Like `loop`, but users stop when the rows run out |
| `random` | A uniformly random row per iteration, with replacement |
| `unique` | Each row used at most once for the whole run; users stop when spent |

`unique` is checked **before the run starts**: if there are fewer rows than the load needs — one
per virtual user, and one per iteration when `-n` fixes the request count — pywrkr refuses to
start rather than quietly running short. In distributed mode the master hands each worker a
disjoint slice of the rows, so `unique` and `sequential` stay globally unique across nodes.

**File format.** CSV needs a header row, which supplies the field names; values are strings. JSON
must be an array of flat objects; scalars keep their JSON spelling (`true`, not `True`). Relative
`file:` paths resolve against the scenario file's own directory, so a scenario and its data travel
together. Rows are read into memory once at startup — fine for the hundreds-of-thousands range,
but streaming very large files is deliberately not supported.

### Built-in Template Functions

Available anywhere `${...}` works, with no data file needed:

| Function | Expands to |
|----------|-----------|
| `${uuid()}` | A random UUID4 |
| `${randint(1,100)}` | A random integer in the inclusive range |
| `${randstr(12)}` | A random alphanumeric string of that length |
| `${counter()}` / `${counter(name)}` | A run-wide counter starting at 1; named counters are independent |
| `${now()}` / `${now(unix)}` | ISO 8601 UTC timestamp / epoch seconds |

```json
{ "reference": "order-${counter(orders)}", "id": "${uuid()}", "placed_at": "${now()}" }
```

Counters are shared across virtual users, so `counter()` is strictly monotonic for the run rather
than restarting per user. Unknown functions and nonsense arguments (`${randint(9,1)}`) are
rejected when the scenario file loads, naming the step — not once per request mid-run.

There is deliberately no expression language: no arithmetic, no conditionals, no nesting.

Working example: [`examples/scenario-data-driven.json`](examples/scenario-data-driven.json) with
[`examples/users.csv`](examples/users.csv).

### Sessions & Cookies

In user-simulation and scenario modes every virtual user gets **its own cookie jar**, so
`Set-Cookie` is stored and replayed for that user across steps and iterations. Cookie-session
logins — the most common form of web auth — work without any correlation setup:

```bash
# The server sets a session cookie on /login; each user carries its own from then on
pywrkr --scenario examples/scenario-cookie-session.json -u 100 -d 60
```

This also means N virtual users look like N distinct clients to the target, which matters for
anything keyed on identity: session-store load, sticky-session balancing, per-user rate limits,
and cache hit rates.

| Setting | Where | Effect |
|---------|-------|--------|
| default | — | one cookie jar per virtual user, kept for the whole run |
| `session: fresh_per_iteration` | scenario file | empty the jar at the start of each iteration, so every pass is a brand-new visitor |
| `--no-session-cookies` | CLI | ignore `Set-Cookie` entirely; only the static `-C` cookies are sent |

**Static `-C` cookies** are sent on every request in every mode. They travel in the request's
`Cookie` header rather than the jar, so they survive `session: fresh_per_iteration` and are
unaffected by `--no-session-cookies`. A server-set cookie of the same name is sent alongside them.

**IP-address targets:** cookie jars normally refuse to store cookies for a bare IP host, which
would silently disable sessions against the `http://127.0.0.1:8080` targets load tests usually
point at. pywrkr detects an IP literal in the target URL and opens the jar (`unsafe`) for it, so
loopback and internal-IP targets behave like named hosts.

**Plain connection mode** (`-c`/`-d`, no `-u`) is unchanged: there is no per-user identity to
isolate, so it keeps the client library's default jar. `--no-session-cookies` still applies.

**Distributed mode:** jars live per virtual user inside each worker process. There is no shared
session state between worker nodes, so a session started on one node is never continued on another.

### Skipping the Response Body

`--no-read-body` releases the connection instead of reading the response body, for runs where
nothing looks at it.

```bash
pywrkr --no-read-body -c 100 -d 30 http://localhost:8080/large.json
```

**Read this before using it.** The flag is opt-in because it is not the straightforward win it
sounds like. Measured against a local server, three variants of the send path differing only in body
handling — `read` (the default), `release` (this flag), and a control that waits for the body without
building an object:

| payload | `--no-read-body` vs default | control: wait, but don't build the object |
|---------|-----------------------------|--------------------------------------------|
| 0.1 KiB | rps **+4.6%**, p95 −4.2% | rps **−0.9%** |
| 123 KiB | rps **+5.3%**, p95 −5.2% | rps **−5.0%** |
| 1.2 MiB | rps **−15.5%**, p95 +18.5% | — |

Three things follow, and they are the whole story:

1. **Avoiding the allocation saves nothing.** That is what the control isolates: skip building the
   bytes object but still wait for the body, and it is *slower* at every size. `await resp.read()` is
   already an efficient bulk read.
2. **The gain is the run no longer timing the response.** `release()` returns before the body has
   arrived, so the measured latency stops including receipt. That is why p95 "improves" on a 0.1 KiB
   body, where there is nothing to copy — a gain that does not scale with payload size is not a
   saving on payload handling.
3. **On large payloads it loses outright** — 15% slower at 1.2 MiB, because the un-awaited drain
   contends with the next request.

The bytes cross the wire either way: an HTTP/1.1 keep-alive connection has to be drained before it
can carry the next response, so there was never bandwidth to save.

If you want it anyway, what it does is honest about itself:

- `total_bytes` and `transfer_per_sec_bytes` count only what was read, and `--json` records
  `config.read_body: false` so a zero is never ambiguous.
- `pywrkr compare` warns when one run read bodies and the other did not, instead of reporting the
  transfer-rate collapse as a regression.
- **Anything that inspects the body reads it regardless**: a step with an `extract` rule or a body
  assertion (`assert_body_contains`, `assert_body_regex`, `assert_json`), plus `--verify-length` and
  `-v 3`. The decision is per step, so a scenario can mix both kinds. The flag cannot silently break
  a flow that depends on response content.
- `--http2` (the httpx backend) is unaffected: its non-streaming send has already read the body by
  the time it returns, so there is nothing to skip.

### Cache-Busting Mode

Append `-R` to any mode to bypass HTTP caches by adding a unique query parameter to each request:

```bash
pywrkr -R -u 300 -d 120 https://example.com/
# Each request hits: https://example.com/?_cb=<unique-uuid>
```

This is useful for testing origin server performance without CDN/proxy cache interference.

### Rate Limiting Mode

Instead of sending requests as fast as possible, `--rate` sends them at a controlled, constant rate. This is critical for SLA testing and finding exact server breaking points.

```bash
# Constant 500 req/s for 30 seconds
pywrkr --rate 500 -d 30 http://localhost:8080/

# Rate with request count: 50 req/s, stop after 200 requests
pywrkr --rate 50 -n 200 http://localhost:8080/

# Rate limiting with multiple connections (rate is global, shared across all workers)
pywrkr --rate 100 -c 10 -d 60 http://localhost:8080/

# Combine with user simulation (applies when think_time is 0)
pywrkr --rate 200 -u 50 -d 120 --think-time 0 http://localhost:8080/
```

**Rate Ramp** (`--rate-ramp`): Linearly increase the rate over the test duration. This is useful for finding the exact breaking point automatically:

```bash
# Start at 100 req/s, linearly increase to 1000 req/s over 60 seconds
pywrkr --rate 100 --rate-ramp 1000 -d 60 http://localhost:8080/
```

At `--rate 500`, the tool sends one request every 2ms. If the server cannot keep up (latency exceeds the interval), requests queue up -- this is expected and useful for identifying saturation points.

**Comparison with default "max throughput" mode:**

| Mode | Use Case |
|------|----------|
| Default (no `--rate`) | Find maximum throughput; stress test |
| `--rate N` | SLA validation; controlled load; latency-under-load testing |
| `--rate N --rate-ramp M` | Find breaking point; gradual load increase |
| `--rate N --traffic-profile P` | Realistic traffic patterns (sine, spikes, CSV replay) |

Results include "Target RPS" vs "Actual RPS" and "Rate Limit Waits" count (how many times the limiter had to slow down a worker).

### Traffic Profiles

Shape your test traffic to match real-world patterns using `--traffic-profile`. Requires `--rate` (base/peak rate) and `-d` (duration).

```bash
# Sine wave: smooth oscillation up to 1000 req/s, 3 cycles
pywrkr --rate 1000 -d 120 --traffic-profile "sine:cycles=3,min=0.2" http://localhost:8080/

# Step function: jump between discrete load levels
pywrkr --rate 1000 -d 90 --traffic-profile "step:levels=100,500,1000" http://localhost:8080/

# Spike: baseline at 20% with 5x bursts every 10 seconds
pywrkr --rate 200 -d 60 --traffic-profile "spike:interval=10,multiplier=5" http://localhost:8080/

# Business hours: 24h daily pattern compressed into test duration
pywrkr --rate 2000 -d 300 --traffic-profile business-hours http://localhost:8080/

# CSV replay: replay real production traffic from a file
pywrkr --rate 1000 -d 300 --traffic-profile "csv:traffic.csv" http://localhost:8080/
```

**Built-in profiles:**

| Profile | Pattern | Use case |
|---------|---------|----------|
| `sine` | Smooth wave | Gradual load changes, auto-scaling tests |
| `step` | Discrete jumps | Testing specific load tiers |
| `sawtooth` | Repeated ramps | Repeated warm-up behavior |
| `square` | On/off toggle | Sudden load change recovery |
| `spike` | Periodic bursts | Flash sale / viral event simulation |
| `business-hours` | Day/night curve | Realistic daily traffic patterns |
| `csv:file` | Custom curve | Replaying real production traffic |

**CSV format:** Two columns — `time_sec,rate` (absolute RPS) or `time_sec,multiplier` (factor applied to `--rate`). Values are linearly interpolated between points.

### Latency Breakdown

Use `--latency-breakdown` to see where each request spends its time. This breaks down latency into individual phases using aiohttp's tracing infrastructure:

```bash
# Show latency breakdown for each phase
pywrkr --latency-breakdown -n 1000 -c 50 https://example.com/

# Combine with JSON output
pywrkr --latency-breakdown --json results.json -d 30 https://example.com/
```

Output includes averages with min/max/p50/p95 for each phase:

```
======================================================================
  LATENCY BREAKDOWN (averages)
======================================================================
    DNS Lookup:          2.15ms  (min=1.20ms, max=5.30ms, p50=2.00ms, p95=4.10ms)
    TCP Connect:        12.34ms  (min=10.00ms, max=18.50ms, p50=12.00ms, p95=16.20ms)
    TLS Handshake:      45.67ms  (min=40.00ms, max=55.00ms, p50=45.00ms, p95=52.00ms)
    TTFB:               89.12ms  (min=60.00ms, max=150.00ms, p50=85.00ms, p95=130.00ms)
    Transfer:           34.56ms  (min=20.00ms, max=80.00ms, p50=30.00ms, p95=65.00ms)
    Total:             183.84ms  (min=131.20ms, max=308.80ms, p50=174.00ms, p95=267.30ms)

    New Connections:    50
    Reused Connections: 950
```

**Phases:**
- **DNS Lookup** -- Time to resolve the hostname via DNS
- **TCP Connect** -- Time to establish the TCP connection
- **TLS Handshake** -- Time for TLS negotiation (HTTPS only)
- **TTFB** -- Time to first byte, from sending the request to receiving the first response byte
- **Transfer** -- Time to read the full response body

**Connection reuse:** When keep-alive is enabled (the default), most requests reuse existing connections. For reused connections, DNS/Connect/TLS phases will be zero. The breakdown reports how many connections were new vs. reused.

When `--json` is used, the breakdown data is included in the JSON output under the `latency_breakdown` key.

### Auto-Ramping / Step Load (Autofind)

Automatically increase load until the server's capacity is found. The `--autofind` flag starts with a small number of users, runs short tests at increasing load levels, and uses binary search to pinpoint the maximum sustainable load.

```bash
# Find max capacity with default thresholds (1% error rate, 5s p95)
pywrkr --autofind https://example.com/

# Custom thresholds: 0.5% error rate, 2s p95, 15s steps
pywrkr --autofind --max-error-rate 0.5 --max-p95 2.0 \
    --step-duration 15 https://example.com/

# Start from 50 users, up to 5000, multiply by 1.5x each step
pywrkr --autofind --start-users 50 --max-users 5000 \
    --step-multiplier 1.5 https://example.com/

# Save detailed results to JSON
pywrkr --autofind --json autofind_results.json https://example.com/

# With cache-busting and custom think time
pywrkr --autofind -R --think-time 0.5 https://example.com/
```

**How it works:**

1. Start with `--start-users` (default: 10) virtual users
2. Run a short test (`--step-duration`, default: 30s) at that load
3. Check if error rate exceeds `--max-error-rate` or p95 latency exceeds `--max-p95`
4. If OK, multiply users by `--step-multiplier` (default: 2x) and repeat
5. If thresholds exceeded, binary search between the last good and first bad user count
6. Report the maximum sustainable load with a summary table

**Example output:**

```
============================================================
  AUTOFIND RESULTS
============================================================
  Maximum sustainable load: 280 users

  Step Results:
  Users |      RPS |     p50 |     p95 |     p99 | Errors | Status
     10 |      9.8 |   120ms |   180ms |   200ms |   0.0% | OK
     20 |     19.5 |   125ms |   190ms |   220ms |   0.0% | OK
     40 |     38.2 |   130ms |   250ms |   300ms |   0.0% | OK
     80 |     75.1 |   180ms |   400ms |   600ms |   0.0% | OK
    160 |    140.2 |   350ms |    1.2s |    2.1s |   0.0% | OK
    320 |    135.5 |    2.1s |    8.5s |   15.2s |   5.2% | FAIL
    240 |    138.1 |   800ms |    3.2s |    5.1s |   0.8% | OK
    280 |    136.8 |    1.1s |    4.8s |    7.2s |   0.9% | OK
    300 |    135.2 |    1.5s |    5.5s |    9.1s |   1.2% | FAIL
============================================================
```

**Autofind options:**

| Flag | Description |
|------|-------------|
| `--autofind` | Enable auto-ramping mode |
| `--max-error-rate` | Stop when error rate exceeds this percent (default: 1.0) |
| `--max-p95` | Stop when p95 latency exceeds this in seconds (default: 5.0) |
| `--step-duration` | Duration of each step test in seconds (default: 30) |
| `--start-users` | Starting number of users (default: 10) |
| `--max-users` | Maximum users to try (default: 10000) |
| `--step-multiplier` | Multiply users by this each step (default: 2.0) |

### SLO-Aware Thresholds

Define pass/fail criteria for your benchmarks. If any threshold is breached, pywrkr exits with code 2 — making it usable in CI/CD pipelines.

```bash
# Single threshold
pywrkr --threshold "p95 < 300ms" -c 100 -d 30 http://localhost:8080/

# Multiple thresholds
pywrkr \
    --th "p95 < 300ms" \
    --th "p99 < 1s" \
    --th "error_rate < 1%" \
    --th "rps > 100" \
    -c 100 -d 30 http://localhost:8080/
```

**Supported metrics:**
- `p50`, `p75`, `p90`, `p95`, `p99` — latency percentiles
- `avg_latency`, `max_latency`, `min_latency` — latency aggregates
- `error_rate` — error percentage (e.g., `error_rate < 1%` or `error_rate < 1`)
- `rps` — requests per second

**Operators:** `<`, `>`, `<=`, `>=`

**Time units:** `ms` (milliseconds), `s` (seconds), `us` (microseconds). Default is seconds if no unit.

**Example output:**
```
======================================================================
  SLO THRESHOLDS
======================================================================
    p95 < 300ms         Actual: 245.00ms       PASS
    p99 < 1s            Actual: 820.00ms       PASS
    error_rate < 1%     Actual: 0.00%          PASS
    rps > 100           Actual: 523.45         PASS

  Result: ALL THRESHOLDS PASSED
```

**CI usage:**
```bash
pywrkr --th "p95 < 500ms" --th "error_rate < 0.1%" \
    -c 50 -d 60 http://api.staging/health || echo "Performance regression detected!"
```

### Observability Export

Export benchmark metrics directly to your observability stack.

#### OpenTelemetry

```bash
pip install pywrkr[otel]
pywrkr --otel-endpoint http://localhost:4318 \
    --tag environment=staging --tag build=$(git rev-parse --short HEAD) \
    -c 100 -d 30 http://localhost:8080/
```

Exports gauges and counters: `pywrkr.requests.total`, `pywrkr.errors.total`, `pywrkr.requests_per_sec`, `pywrkr.latency.p50/p95/p99/mean/max`, `pywrkr.transfer_bytes_per_sec`, `pywrkr.duration_sec`.

#### Prometheus Remote Write (Pushgateway)

```bash
pywrkr --prom-remote-write http://pushgateway:9091 \
    --tag region=us-east-1 --tag service=api \
    -c 100 -d 30 http://localhost:8080/
```

Uses stdlib `urllib` — no extra dependencies. Pushes metrics in Prometheus text format to `{endpoint}/metrics/job/pywrkr`.

#### Live streaming during the run (`--export-interval`)

By default metrics are pushed **once, at the end**. For the runs where observability matters most
— a 30-minute soak, an autofind ramp, a traffic profile — that leaves you blind until it's over,
and a killed run exports nothing at all. `--export-interval` streams snapshots as the run happens:

```bash
pywrkr --otel-endpoint http://collector:4318 --export-interval 10 \
    --tag test=soak-v2 --rate 500 -d 1800 https://api.example.com/
```

**Counters stay cumulative** (`pywrkr_requests_total`, `pywrkr_errors_total`), so Prometheus
`rate()` and OTel deltas work normally. **Percentiles are windowed** — computed over the last
interval only, so a spike 25 minutes ago is not still dragging your current p95 around. The
run-cumulative percentiles are still what the end-of-run export carries.

Each streamed snapshot is labelled `export="interval"`, and the one emitted at shutdown
`export="final"`, so a dashboard can separate live points from the closing state. **A run
interrupted with Ctrl-C still emits that final snapshot**, so an aborted soak leaves its last
state in the TSDB rather than a cliff.

**A slow collector never slows the run.** Sampling and sending are separate tasks joined by a
bounded queue: if the endpoint is unreachable, snapshots are dropped rather than backing up into
the request path, and the count is reported at the end —

```
  Streaming export: 0 snapshot(s) exported, 3 never delivered (collector unresponsive)
```

— never a silent success. Measured cost against an unreachable collector: ~1% of throughput.

Without `--export-interval` nothing changes: one export at the end, exactly as before.

**Grafana walkthrough.** Point pywrkr at a collector that writes to your Prometheus, then graph:

| Panel | Query |
|-------|-------|
| Achieved throughput | `rate(pywrkr_requests_total[1m])` |
| Error rate | `rate(pywrkr_errors_total[1m]) / rate(pywrkr_requests_total[1m])` |
| Windowed p95 | `pywrkr_latency_p95_ms{export="interval"}` |
| Target vs achieved | `pywrkr_requests_per_sec` against your `--rate` |

Put those beside your service's own dashboards and a latency spike lines up with the deploy, GC
pause, or scaling event that caused it — while the test is still running. With `--autofind`, every
snapshot carries a `step_users` label, so each step of the ramp is a separable series instead of
one smeared line.

#### Test Metadata Tags

Tags are attached to all exported metrics and included in JSON output:

```bash
pywrkr --tag environment=production --tag build=v2.1.0 \
    --tag region=eu-west-1 --tag test_name=api_stress \
    --json results.json -c 100 -d 30 http://localhost:8080/
```

#### Distributed streaming

`--export-interval` works on a distributed run too. The master merges what the workers report and
exports the cluster-wide view on its own interval, so a multi-node soak is visible live rather than
only after it finishes:

```bash
# On the master
pywrkr --master --expect-workers 4 --export-interval 10 \
       --otel-endpoint http://collector:4318/v1/metrics \
       -u 4000 -d 1800 https://api.example.com/
```

Nothing extra is needed on the workers: the master asks for progress in the config it already
sends, and only when it has somewhere to put the data. A run without an export endpoint puts no
extra traffic on the wire, and an older master simply never asks.

Master-side exports carry `role="master"` and a `workers_reporting` count so they are separable
from any per-worker exports, and `export="final"` marks the closing snapshot — which is what leaves
a run killed mid-flight with its last state rather than nothing.

Two things worth knowing about the numbers:

- **Counters are the sum of each worker's latest report**, so they stay monotonic however the
  workers' intervals interleave. A counter that goes backwards makes `rate()` report negative
  throughput.
- **Percentiles come from the workers' pooled samples**, not from averaging their percentiles —
  which is not a meaningful operation. Each worker sends a capped, evenly-strided sample of its
  interval so throughput cannot turn one window into an unbounded payload.

A worker that stops reporting keeps its completed requests in the cumulative totals but drops out
of the current window, with a warning naming it. Its traffic really happened; it just cannot speak
for the interval it missed.

### Multi-URL Mode

Test multiple endpoints in a single benchmark run using a URL file:

```bash
# Create a URL file (one URL per line)
cat urls.txt
http://localhost:8080/api/users
http://localhost:8080/api/products
http://localhost:8080/api/orders

# Run benchmark against all URLs
pywrkr --url-file urls.txt -c 50 -d 30
```

| Flag | Description |
|------|-------------|
| `--url-file` | Path to file containing URLs to test (one per line) |

Requests are distributed across all URLs. Results include per-URL breakdowns alongside aggregate statistics.

### Distributed Mode

Scale benchmarks across multiple machines by running one master and multiple workers:

```bash
# On the master node: coordinate 3 workers
pywrkr http://target:8080/ --master --expect-workers 3 -c 300 -d 60

# On each worker node: connect back to the master
pywrkr --worker master-host:9220
```

| Flag | Description |
|------|-------------|
| `--master` | Run as distributed master (coordinates workers) |
| `--worker HOST:PORT` | Run as distributed worker, connecting to master at HOST:PORT |
| `--expect-workers` | Number of workers the master should wait for before starting |
| `--bind` | Master bind address (default: `0.0.0.0`) |
| `--port` | Master listen port (default: `9220`) |

The master splits the workload evenly across workers, collects results, and produces a single aggregated report.

### TLS / SSL Verification

By default, SSL certificate verification is **disabled** to allow benchmarking dev/staging servers with self-signed certs. Enable it for production targets and supply a custom CA bundle when needed:

```bash
# Enable standard TLS verification
pywrkr https://example.com --ssl-verify -c 50 -d 30

# Benchmark an internal HTTPS service with a corporate CA
pywrkr https://internal.corp.example.com/api/health \
  --ssl-verify \
  --ca-bundle /etc/ssl/certs/corporate-ca.pem \
  --duration 60
```

The same options are available via environment variables: `PYWRKR_SSL_VERIFY=true` and `PYWRKR_CA_BUNDLE=/path/to/ca.pem`.

## Installation

```bash
# Basic (aiohttp only)
pip install pywrkr

# With live TUI dashboard
pip install pywrkr[tui]

# With OpenTelemetry export
pip install pywrkr[otel]

# Everything
pip install pywrkr[all]
```

## Development Setup

```bash
# Install in editable mode with dev + lint dependencies
pip install -e ".[dev,lint]"
```

## Testing

```bash
# Run all tests
python -m pytest tests/ -v

# Run a specific test file
python -m pytest tests/test_pywrkr.py -v
python -m pytest tests/test_har_import.py -v

# Run a specific test class
python -m pytest tests/test_pywrkr.py::TestMakeUrl -v

# Run tests sequentially (useful for debugging)
python -m pytest tests/ -v -n 0
```

The test suite includes unit and integration tests covering:
- Formatting helpers, percentiles, histogram, timeline, CSV/JSON/HTML output
- Integration tests with a real aiohttp test server (duration mode, request-count mode, POST, auth, cookies, content-length verification, keepalive, cache-buster)
- User simulation integration tests (think time, ramp-up, jitter, error handling, output formats)
- Autofind integration tests (healthy server, error endpoint, threshold enforcement, binary search, JSON output, summary table)
- HAR import tests (parsing, filtering, scenario generation)
- Reporting module tests (formatting, percentile computation, threshold evaluation, CSV/JSON output)
- Multi-URL mode tests (URL file loading, entry parsing)
- Distributed mode tests (config/stats serialization, merge operations, TCP protocol)
- Worker utility tests (URL construction, headers, stats merging, breakdown aggregation)

## Contributing

Contributions are welcome! Please read the [Contributing Guide](CONTRIBUTING.md) for details on how to get started, report bugs, suggest features, and submit pull requests.

This project follows the [Contributor Covenant Code of Conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code.

## License

MIT
