Metadata-Version: 2.4
Name: agentqa-sdk
Version: 0.1.0
Summary: Regression testing and monitoring for AI agents — record every run, test every change.
Project-URL: Homepage, https://agentqa-phi.vercel.app
Author: AgentQA
License-Expression: MIT
Keywords: agents,ai,llm,observability,qa,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.9
Requires-Dist: httpx>=0.24
Provides-Extra: dev
Requires-Dist: pytest-httpx>=0.30; extra == 'dev'
Requires-Dist: pytest>=7; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Description-Content-Type: text/markdown

# agentqa-sdk

Regression testing and monitoring for AI agents — record every run, test
every change, and catch failures before they break real workflows.

## Install

```bash
pip install agentqa-sdk
```

(The package installs as `agentqa-sdk`; the import name is `agentqa`.)

## Configure

```bash
export AGENTQA_API_KEY=aq_live_...        # from your project's Settings page
export AGENTQA_API_URL=https://your-agentqa-deployment
```

## Record a run

```python
import agentqa

with agentqa.track_run(
    agent_name="sales_research_agent",
    task_input="Research this lead and update the CRM",
) as run:
    run.log_tool_call("crm.lookup", {"company": "Acme"}, {"id": 42})
    run.log_llm_call("claude-sonnet-5", tokens_in=800, tokens_out=200, cost_usd=0.01)
    run.log_screenshot("page.png")          # browser agents
    run.set_output("Lead updated.")
    run.set_cost(0.014)
```

Every step, tool call, screenshot, cost, and error appears in your AgentQA
dashboard seconds later. If your agent raises, the run is recorded as
`errored` with the traceback — and the exception is re-raised untouched.

The SDK never blocks or crashes your agent: steps are sent from a
background thread with retries, and network failures degrade to dropped
telemetry, not broken automation.

## Run a test suite

```python
import agentqa

def my_agent(task_input: str) -> str:
    ...  # your agent, any framework
    return "final answer"

summary = agentqa.run_suite(my_agent, suite_id="...", label="prompt-v4")
print(summary)  # {'suite_run_id': ..., 'cases': 20, 'executed': 20, 'crashed': 0}
```

Each test case executes locally in your environment — AgentQA never runs
your code or holds your credentials. Verdicts and the failure diagnosis
appear on the suite run's page as results stream in.

## Redact sensitive data

```python
with agentqa.track_run(..., redact=[r"[\w.+-]+@[\w-]+\.[\w.]+"]) as run:
    ...
```

Redaction runs client-side, before anything leaves your environment. Pass
regex patterns or a callable `(str) -> str`.