Metadata-Version: 2.4
Name: failscope
Version: 0.1.2
Summary: AI-Powered Root Cause Analysis for pytest — Dual-Agent (Analyzer→Critic) pipeline that automatically triages test failures
Author-email: Yani <yani@example.com>
License: MIT
Project-URL: Homepage, https://github.com/Yaniv2809/failscope
Project-URL: Repository, https://github.com/Yaniv2809/failscope
Project-URL: Issues, https://github.com/Yaniv2809/failscope/issues
Keywords: pytest,testing,ai,rca,root-cause-analysis,flaky-tests,ci-cd,quality-assurance,automation
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pytest>=7.0
Requires-Dist: httpx>=0.24
Provides-Extra: dev
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Dynamic: license-file

# 🔬 Failscope

**AI-Powered Root Cause Analysis for pytest**

Failscope is a lightweight pytest plugin that automatically analyzes test failures using a Dual-Agent (Analyzer → Critic) AI pipeline. It eliminates CI noise, deduplicates failures, and gives you actionable fix suggestions — not raw logs.

## Features

- **Dual-Agent RCA** — Analyzer generates diagnosis, Critic validates against evidence (prevents hallucinations)
- **Smart Log Truncation** — Preserves setup context (first 10%) + error zone (last 90%)
- **Error Fingerprinting** — Clusters similar failures, LLM analyzes only unique root causes
- **A-F Stability Scoring** — Track test reliability over time with flakiness detection
- **Zero Config** — Just `pip install failscope` and add `--failscope` flag
- **Multi-Provider** — Supports Groq (free), OpenAI, and Anthropic (BYOK)
- **Offline Mode** — Rule-based analysis when no API key is available

## Quick Start

```bash
pip install failscope
```

### With AI (recommended)

```bash
export GROQ_API_KEY=your-key-here  # free at console.groq.com
pytest --failscope
```

### Without AI (offline mode)

```bash
pytest --failscope --fs-offline
```

### With stability report

```bash
pytest --failscope --fs-report
```

## How It Works

```
Test Failure
    │
    ▼
┌─────────────────────┐
│  Log Preprocessor   │  Strip noise, smart truncate,
│                     │  extract error signals
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│ Error Fingerprinting│  Deduplicate: only unique
│                     │  root causes reach the LLM
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│  Analyzer (temp 0.4)│  Creative analysis:
│  [Actor Agent]      │  root cause + fix suggestion
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│  Critic (temp 0.0)  │  Validate every claim against
│  [Evaluator Agent]  │  raw evidence. Override
│                     │  hallucinations.
└────────┬────────────┘
         │
         ▼
   Developer-facing
   RCA Report (JSON)
```

## CLI Options

| Flag | Description |
|------|-------------|
| `--failscope` | Enable Failscope analysis |
| `--fs-offline` | Use rule-based analysis (no API key needed) |
| `--fs-report` | Generate stability report after run |
| `--fs-provider` | LLM provider: `groq`, `openai`, or `anthropic` |
| `--fs-output` | Output directory (default: `.failscope/`) |

## Output

Failscope generates two reports in `.failscope/`:

**`rca_report.json`** — Root cause analysis for each unique failure:
```json
{
  "root_cause": "API endpoint /login returns 401 due to expired test token",
  "category": "assertion_failure",
  "severity": "high",
  "fix_suggestion": "Refresh auth token in conftest.py fixture before each test",
  "confidence": 0.87,
  "was_critic_override": false,
  "affected_tests": ["test_auth.py::test_login", "test_auth.py::test_profile"],
  "occurrence_count": 2
}
```

**`stability_report.json`** — A-F grading per test:
```json
{
  "test_name": "test_checkout.py::test_payment_flow",
  "grade": "C",
  "pass_rate": "72.0%",
  "flakiness_score": 58,
  "verdict": "Flaky",
  "trend": "degrading"
}
```

## LLM Providers

| Provider | Model | Env Variable | Cost |
|----------|-------|-------------|------|
| **Groq** (default) | Llama 3.1 70B | `GROQ_API_KEY` | Free tier available |
| OpenAI | GPT-4o | `OPENAI_API_KEY` | Pay per token |
| Anthropic | Claude 3.5 Sonnet | `ANTHROPIC_API_KEY` | Pay per token |

## License

MIT
