Metadata-Version: 2.4
Name: whyfail
Version: 3.0.0
Summary: Evidence-based diagnostics for Python failures. Explains why an exception happened using runtime evidence — fully local, deterministic, offline, and protected by SecretShield redaction.
Home-page: https://github.com/Sam3360/whyfail
Author: Samarth Chugh (Sam3360)
License-Expression: MIT
Project-URL: Homepage, https://github.com/Sam3360/whyfail
Project-URL: Repository, https://github.com/Sam3360/whyfail
Project-URL: Documentation, https://github.com/Sam3360/whyfail#readme
Keywords: debugging,exceptions,diagnostics,error-analysis,developer-tools
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Debuggers
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: secretshield<1.0,>=0.4.1
Provides-Extra: test
Requires-Dist: pytest>=7.0; extra == "test"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Dynamic: home-page
Dynamic: license-file

# whyfail

**Evidence-based diagnostics for Python failures.**

Python exceptions tell you *what* failed:

```
KeyError: 'user'
```

`whyfail` tells you *why it most likely happened* — by analyzing the actual
failure: the traceback, the inspected runtime values, and the source around
the failing line. No AI, no cloud, no network. It reads the failed program
like a careful debugger would, records only what it can observe, and says
**"I don't have enough evidence"** instead of guessing when it cannot tell.

Runtime values can contain credentials, so whyfail v2 routes every value that
enters a diagnostic through **SecretShield** — pattern- and entropy-based
secret detection — before anything is rendered. Useful crash diagnostics,
without turning them into credential leaks.

```text
KeyError: 'user'
================

What failed
-----------
response["user"] raised KeyError.

Execution path
--------------
  <module> (app.py:1)
    ↓
  fetch_user() (app.py:10)
    ↓
  load() (app.py:4)   ← raised here

Value history
-------------
  - `response` was assigned the return value of `fetch_user()` at line 3
    (`fetch_user()` returns the expression `payload` at line 10 — a static
    observation, not a runtime claim).

Broken assumption
-----------------
  - the code assumes the mapping accessed by `response` contains the key
    'user'
    Runtime evidence against it: the available keys are: 'account', 'status'

Likely cause
------------
The mapping does not contain the key 'user'.

Runtime evidence
----------------
  - the subscripted value is a dict.
  - value: {'account': {'id': 7}, 'status': 'active'}
  - available keys (2): 'account', 'status'
  - the failing subscript targeted key 'user'.

Expected vs actual
------------------
  Expected:
    the mapping accessed by `response` contains the key 'user'
  Actual:
    the available keys are: 'account', 'status'

Failure location
----------------
  app/users.py:4 in load()

  2 | def load():
  3 |     response = {"account": {"id": 7}, "status": "active"}
  4 |     return response["user"]
    |            ^^^^^^^^^^^^^^^^
What to investigate
-------------------
  • Verify what `response` really contains when this code runs — its
    producer does not include the key 'user'.
  • If the key can legitimately be absent, guard the access (e.g. an `in`
    check or `.get(...)` default) so the missing key is handled explicitly.

Confidence: high
```

That is not generated text. Every statement in it was *observed*: the value's
type, its keys, the requested key, the exact source span, the frames that
called the failing code, and the assignments that produced the failing value.
The narrative is assembled from that evidence — nothing is invented to make
the story read better.

---

## Why it exists

Most error messages describe the immediate operation ("key not found"), not
the reason it happened. Humans debug by looking at the runtime values around
the crash — and so does `whyfail`, automatically, at the moment of failure
while the values still exist.

`whyfail` is built around one rule, above all others:

> **It does not guess what happened. It analyzes what the failed program can
> actually tell us.**

A mediocre tool says *"your API probably changed"*. `whyfail` says:

```text
The mapping does not contain the key 'user'.

I cannot determine why the key is missing from the available runtime evidence.
```

A conservative diagnosis is better than a confident but incorrect one.

## Features

- **One engine, three interfaces** — the CLI, the Python API, and the pytest
  plugin all share the same diagnostic engine.
- **Deep, but bounded, evidence** — exception type/message, full chains,
  source context with AST analysis, safe inspection of runtime locals,
  function arguments, mapping keys, sequence lengths, dataclass fields and
  public attributes.
- **v3 root-cause model** — each diagnosis separates the *immediate failure*
  (the operation that raised) from the *likely root cause*, states the
  assumption the code acted on, and pairs **Expected vs Actual** using only
  observed evidence.
- **Value provenance** — where the failing value came from: assignment,
  parameter, call-site argument, import — traced through source and the
  live frames, bounded to a handful of hops and never quoting raw values.
- **Call-chain analysis** — a compact outermost-first execution path to the
  failure, with the raise site marked, propagated frames distinguished, and
  recursive runs folded into an observed repeat count.
- **Execution story** — an ordered, numbered narrative of how the program
  reached the failure.
- **Machine-readable output** — `whyfail run --format json python app.py`
  emits the structured diagnostic as JSON for editors, CI, and scripts.
- **Per-exception diagnostics** for `KeyError`, `IndexError`, `TypeError`,
  `AttributeError`, `NameError`/`UnboundLocalError`, `ZeroDivisionError`,
  `ValueError`, `ImportError`/`ModuleNotFoundError`, and `AssertionError`.
- **Exception chains** — `raise X from Y` and implicit context are followed
  to the underlying failure that matters.
- **Honest source highlighting** — the caret is derived from real AST +
  bytecode column information; when the failing expression cannot be pinned
  down, no misleading caret is drawn.
- **Confidence levels** — `high`, `medium`, `low`, and explicit
  *insufficient evidence* statements. Speculation is labelled as speculation
  ("Possible explanations"), never as fact.
- **SecretShield-powered redaction (v2)** — every runtime value that enters a
  diagnostic passes through SecretShield's pattern + entropy detection before
  it reaches any renderer. SecretShield is installed automatically as a
  dependency of whyfail — you never install or configure it separately.
- **Redaction by default** — local values whose names suggest secrets
  (password, token, api_key, authorization, private_key, cookies, ...) and
  values that look like credentials (`sk-…`, `ghp_…`, `BEGIN ... PRIVATE
  KEY`, JWTs, `Bearer …`) are never printed; high-entropy tokens with
  innocuous names are caught by SecretShield.
- **Fully local, deterministic, offline** — no telemetry, no network, ever.

---

## Installation

```bash
pip install whyfail
```

Requires Python 3.10+. Installing `whyfail` also installs **SecretShield**
automatically — it is a declared dependency, so you never run
`pip install secretshield` yourself.

## Releases

### whyfail 3.0 — runtime root-cause analysis

```text
whyfail 3.0
───────────

• Structured root-cause model: immediate failure vs. likely root cause
• Expected vs. actual, generated from runtime evidence
• Broken-assumption detection with the contradicting observation
• Value provenance (assignment / parameter / call-site history)
• Compact call-chain analysis with recursion folding
• Execution-story narrative
• Evidence-based "what to investigate" suggestions per failure
• JSON diagnostics for tooling (--format json)
• Bounded analysis: size/depth/frame budgets with explicit notes
• All v2 diagnostics, CLI, API, pytest integration and redaction preserved
```

### whyfail 2.0 — SecretShield protection

```text
whyfail 2.0
───────────

• SecretShield-powered runtime protection
• Automatic sensitive-value redaction
• Secure nested runtime inspection
• Safer diagnostic evidence
• Existing v1 diagnostics preserved
• CLI preserved
• Python API preserved
• pytest integration preserved

(Requires Python 3.10+ — SecretShield itself requires 3.10+.)
```

## How to read a diagnosis: observed, inferred, possible, suggested

Every section in a whyfail diagnostic belongs to exactly one of four
categories, and the layout tells you which is which:

- **Observed** — statements that were inspected directly: the exception
  message, the runtime type of a value, a mapping's key list, a sequence
  length, the source snippet, the frame records. *What failed, Runtime
  evidence, Expected/Actual rows, Value history lines, and the Execution
  path are observed* (value-history lines that come from source are labelled
  "static observation, not a runtime claim").
- **Inferred** — the engine's evidence-backed answer. The *Likely cause*
  and *Broken assumption* sections are inferences, and every one carries a
  confidence (`high` / `medium` / `low`).
- **Possible** — genuinely plausible explanations the evidence cannot
  prove, under *Possible explanations* (for example "this may be a typo"
  only when a similarly named key/attribute actually exists).
- **Suggested** — advice for what to investigate or change, under *What to
  investigate*. Suggestions are tied to the specific failure; there is no
  generic boilerplate advice.

When the evidence cannot support even an inference, whyfail says
**"Insufficient runtime evidence…"** at `low` confidence instead of
inventing a cause. Never reward sounding confident — reward being correct.

## Quick start

### 1. The Python API

```python
import whyfail

try:
    run_request()
except Exception as exc:          # note: the except block is where frames live
    diagnostic = whyfail.explain(exc)
    print(whyfail.format_diagnostic(diagnostic))
    # or machine-readable:
    diagnostic.to_dict()
    diagnostic.to_json()
```

`whyfail.explain(exception)` returns a structured
[`Diagnostic`](src/whyfail/models.py) — exception chain, failure location,
observed facts, immediate-failure statement, expected vs actual rows,
broken assumptions, provenance, execution story, evidence-based suggestions,
confidence, redaction summary, and the call chain. Rendering is separate, so
JSON/editor integrations need no engine changes.

### 2. The CLI

Run any Python program (or pytest) and get a diagnosis of the unhandled
failure, *after* the program's own output:

```bash
whyfail run python app.py
whyfail run python -m mypackage
whyfail run pytest
whyfail run --format json python app.py   # machine-readable diagnostics
```

```bash
$ whyfail run python app.py
Traceback (most recent call last):      # ← unchanged original output
  ...
KeyError: 'user'

====================================
whyfail diagnosis
====================================
KeyError: 'user'
What failed
-----------
response["user"] raised KeyError.
...
Confidence: high
```

The child process runs as normally as possible: stdout, stderr, arguments,
environment, and exit code are preserved. `whyfail` only observes; it never
patches the running program. A failing program exits with its own exit code
— a successful diagnosis never turns a failure into a success.

```bash
whyfail --help
whyfail --version
```

### Machine-readable output (`--format json`)

```bash
whyfail run --format json python app.py
```

prints one JSON document to stdout — an array of
`{"context", "diagnostic"}` entries (one per diagnosed failure, e.g. one per
failed pytest test):

```json
[
  {
    "context": null,
    "diagnostic": {
      "whyfail": 1,
      "schema": 2,
      "exception_type": "KeyError",
      "diagnosis": {
        "analyzer": "KeyError",
        "failure": "data[\"user\"] raised KeyError.",
        "expected": ["..."],
        "actual": ["..."],
        "broken_assumptions": [{"assumption": "...", "runtime": "..."}],
        "suggestions": ["..."],
        "provenance": [{"variable": "...", "origin": "...", ...}],
        "story": ["..."],
        "facts": [...],
        "cause": {"text": "...", "confidence": "high", ...},
        "notes": [...]
      },
      "call_chain": [{"function": "...", "role": "call", ...}],
      "location": { ... },
      "chain": [ ... ],
      "redacted_count": 0
    }
  }
]
```

Notes for tooling:

- The `--format` option belongs to whyfail and must come *before* the
  command: `whyfail run --format json python app.py` (the spec-style
  `... --format json` after the program path is passed to the child).
- In JSON mode the **only** thing written to stdout is the JSON document.
  The child program's own stdout/stderr is captured and not forwarded, so
  `jq`/parsers always see pure JSON. Use plain `whyfail run` (text mode)
  when you want the program's output inline.
- Exit status still mirrors the child: `0` = the program succeeded,
  non-zero = it failed. A successful run prints `[]`.
- The document is the exact structured model the engine produces
  (`Diagnostic.to_dict()`), already redacted and bounded — the CLI adds no
  extra fields.
- The Python API offers the same data in-process via
  `whyfail.explain(exc).to_dict()` / `.to_json()`.

### 3. The pytest plugin

No test rewrites needed:

```bash
pytest --whyfail
```

When a test fails, its diagnosis is printed alongside the normal failure
output, with the failing test as context:

```text
test_api.py::test_returns_user [call failed]
KeyError: 'user'
```

Failure analysis happens at the moment the exception is raised — inside
pytest's own reporting hooks — so test frames and locals are still alive when
the engine inspects them.

## Supported diagnostics

| Exception | What whyfail shows (from evidence) |
| --- | --- |
| `KeyError` | requested key, subject type, its keys, similar-key typo suggestions (only when a similar key actually exists) |
| `IndexError` | attempted index, sequence length, valid index range |
| `TypeError` | unsupported operands (from the message types + inspected operands), calling non-callables, subscripting non-subscriptables, argument-count mismatches, iterating non-iterables, `None` operands |
| `AttributeError` | object type, requested attribute, attributes that *do* exist, naming-mistake suggestions backed by a similar real attribute |
| `NameError` / `UnboundLocalError` | the missing name, scope, what is bound in that scope, bindings earlier/later in the source |
| `ZeroDivisionError` | the division expression and the runtime divisor when resolvable |
| `ValueError` | failed `int()`/`float()` conversions with the actual literal and runtime argument |
| `ImportError` / `ModuleNotFoundError` | missing module vs. missing symbol, importable parent prefixes, local shadowing files, circular-import wording |
| `AssertionError` | the asserted condition and the operand values that made it false; deliberate `raise AssertionError` is reported as such |
| anything else | an honest generic diagnosis — facts about the location, and an explicit *insufficient evidence* statement |

## Example output

```text
IndexError: list index out of range
===================================

What failed
-----------
items[10] raised IndexError.

Value history
-------------
  - `items` was assigned a literal List value at line 2.

Broken assumption
-----------------
  - the code assumes `items` contains at least 11 element(s)
    Runtime evidence against it: observed length: 3

Likely cause
------------
Index 10 is out of range: the sequence has 3 element(s), so valid indexes are 0..2.

Expected vs actual
------------------
  Expected:
    index 10 falls inside the bounds of `items`
  Actual:
    the sequence has 3 element(s); with index 10, valid indexes would be 0..2

Failure location
----------------
  app/main.py:6 in main()

  4 | def main():
  5 |     items = ["a", "b", "c"]
  6 |     return items[10]
    |            ^^^^^^^^^
What to investigate
-------------------
  • Check how many elements `items` really holds when this code runs (it had
    3 at failure time).
  • If the length is not guaranteed, guard the access with a `len()` check
    or iterate instead of indexing blindly.

Confidence: high
```

With insufficient evidence, the output says so instead:

```text
Likely cause
------------
Insufficient runtime evidence to determine the root cause from the available
local evidence.
```

At low confidence the narrative sections that would need runtime data are
simply absent — the tool never fills them with guesses.

### Why did this happen? — three questions

Every v3 diagnosis attempts to answer, in order:

```text
What failed?          items[10] raised IndexError.
Why did it fail?      index 10 is outside a 3-element sequence.
Why was that          `items` was built with only 3 elements at line 2;
possible?             nothing re-populated it before the access.
```

The third level (the *why was that possible* layer) comes from provenance
and call-chain evidence. When the evidence cannot support it, that layer is
omitted rather than invented.

## Architecture

```
Runtime failure
       │
       ▼
Failure Capture          capture.py   — exception chains + frames (live)
       │
       ▼
Evidence Collection      runtime.py   — bounded, guarded value inspection
       │                  redact.py    — conservative secret redaction
       ▼
Context Analysis         source.py    — source reading, AST ops, carets
       │
       ▼
Diagnosis Engine         engine.py + per-exception analyzers
       │                  ├─ analyzers (keyerror, indexerror, ...) fill
       │                  │    expected/actual/broken assumption/suggestions
       │                  ├─ provenance.py — bounded value history
       │                  ├─ callchain.py  — execution path + recursion folding
       │                  └─ story assembly
       ▼
Security Boundary        shield.py    — SecretShield pass over every string
       ▼
Diagnostic Model         models.py    — structured data, never raw values
       │
       ├── CLI renderer      renderer.py / cli.py (text + --format json)
       ├── Python API        api.py
       └── Pytest renderer   pytest_plugin.py
```

The pipeline runs *inside the failing process* — an `except` block, an
installed `sys.excepthook` (CLI child), or pytest's reporting hooks — so the
exception's frames and locals are alive when inspected. Rendered diagnostics
are plain text built from the structured model.

### Offline capture (`whyfail run`)

For `whyfail run python app.py`, the CLI launches the program with an
observing `sys.excepthook`. On an unhandled failure the *child itself* runs
the engine while frames are alive, redacts, and writes the structured result
to a temporary JSON sidecar that only the parent CLI reads and renders.
Normal behavior — output, traceback, exit code — is untouched.

For `whyfail run pytest`, the CLI launches pytest with the whyfail plugin
loaded; the plugin writes the same sidecar protocol and the CLI renders it.
Running `pytest --whyfail` directly renders inline instead.

## Runtime Privacy

whyfail analyzes local runtime information (traceback frames, local
variables, function arguments, inspected objects) to explain failures.

Because runtime variables can contain credentials — passwords, API keys,
tokens, cookies, authorization headers — whyfail integrates **SecretShield**
to detect and redact sensitive values *before* diagnostic information is
rendered. The sanitization boundary sits inside the engine: every string that
enters the structured `Diagnostic` passes through SecretShield first, so the
CLI, the Python API, and the pytest plugin all receive the same already-safe
evidence — there is no per-renderer redaction to forget.

- **SecretShield is installed automatically as part of whyfail.** It is a
  declared dependency; users never install or configure it separately, and
  security is on by default with no opt-in flag.
- **No runtime data is sent to a remote service.** Detection is fully local;
  whyfail and SecretShield perform no network I/O.
- **Structural evidence is preserved.** Redaction hides sensitive *values*,
  not the shape of the data: `response is a dict`, its key list, sequence
  lengths, and types are still reported while values are protected.
- **Fail-closed.** If SecretShield is unavailable or errors, whyfail falls
  back to its own conservative rules (sensitive variable names and
  unmistakable credential shapes) and notes the fallback in the diagnostic —
  it never emits raw values just because the primary layer failed.
- **Redaction cannot guarantee detection of every possible secret.** Both
  layers are heuristic; a novel secret format with an innocuous variable
  name may evade detection. Treat diagnostics the way you treat tracebacks.
- **Source lines are displayed like tracebacks.** Context lines that contain
  credential literals are masked; the failing source line is shown verbatim,
  exactly as Python's own tracebacks show it.

## Privacy and local-only guarantees

- **No network.** Ever. The engine performs no I/O beyond reading local
  files (source) and the local import path. There is no telemetry, no
  external service, no analytics.
- **No raw values leave the process.** Values are summarized and redacted
  before being recorded; the sidecar contains only rendered text.
- **whyfail never alters process behavior.** Importing whyfail does not wrap
  or filter your program's stdout/stderr: SecretShield's stream guardians,
  which it installs on import, are disabled again by whyfail so the traced
  program's output stays byte-for-byte identical. Protection happens on the
  diagnostic data itself. (If you want SecretShield's stream-level
  protection for your own prints, `import secretshield` directly.)

## Limitations

- `whyfail` provides **evidence-based likely explanations, not guaranteed
  root-cause analysis**. The cause section states what the evidence supports,
  at a stated confidence; speculation appears only under "Possible
  explanations".
- **Provenance is static and best-effort.** The value history comes from
  reading assignments, parameters and call sites in the source plus the live
  frame records; it is deliberately bounded (a handful of hops) and is not a
  full dataflow or debugger trace. When it cannot establish an origin it
  says nothing rather than guessing.
- **Only failing programs are analyzed.** v3 analyzes live runtime failures
  (API, CLI, pytest). A static "explain this source file" mode and
  automatic code-fixing remain future work — the model already carries
  structured fix suggestions.
- **Traceback frames are capped (500)** and the call chain is folded and
  summarized past 24 entries; a folded recursion shows an observed repeat
  count, and truncation is stated in the diagnostic notes.
- Analysis happens at the moment the exception is alive. If a program
  overrides `sys.excepthook`, `whyfail run` cannot capture that failure.
- The interpreter must be able to import `whyfail` (the CLI runs children
  with the interpreter it is installed into).
- A pathological object whose `__repr__` loops forever without raising
  cannot be interrupted from inside the same process (standard for any
  diagnostic tool); all raising/misbehaving cases are fully guarded.
- Source files are read from disk at analysis time, so source edits made
  between a crash and the analysis can make code context stale.
- Secret redaction is heuristic and not perfect (see *Runtime Privacy*).
  whyfail hides sensitive values that SecretShield's patterns and entropy
  detection — plus whyfail's own name rules — recognize; it cannot
  guarantee that an unrecognized secret format never appears.
- Python 3.10+ is required (whyfail 2 depends on SecretShield, which
  requires 3.10+).

## Development

```bash
git clone https://github.com/Sam3360/whyfail
cd whyfail
pip install -e ".[dev]"
python -m pytest          # full suite incl. golden + limits tests
```

### Project layout

```
pyproject.toml           packaging / metadata / entry points
src/whyfail/             the package (engine, analyzers, renderers, CLI, plugin)
tests/                   unit + integration + false-positive tests
  golden/                realistic failing programs, asserted structurally
benchmarks/              overhead comparison script (dev only)
.github/workflows/ci.yml CI (Linux, Python 3.10–3.14, build check)
```

### Testing

The suite covers every supported exception type, runtime inspection, source
extraction and highlighting, exception chaining, the v3 narrative layers
(expected/actual, broken assumptions, provenance, call chains, recursion
folding, JSON output), **golden programs** asserting structured fields,
resource limits (huge mappings, deep nesting, cyclic values, giant
tracebacks), confidence, redaction, hostile/recursive/huge values, the CLI
(exit codes, output preservation, `-m`, `--format json` and pytest
invocations), the pytest plugin, and — importantly — **false-positive
tests** proving the engine refuses to invent causes when the evidence does
not support them.

### Performance

The engine only runs when a diagnostic is requested, so normal program
performance is unaffected. Each analysis runs under hard budgets — frame
capture (≤500), call-chain entries (≤24, middle summarized), provenance hops
(≤5), rendered value size/depth/width, per-snippet line length, and total
traceback walking — so a pathological input yields a short diagnosis with an
explicit note, never a hang or an unbounded dump. `python
benchmarks/bench_diagnose.py` compares plain execution against a
diagnose-then-continue loop for a rough overhead feel.

### Releasing

```bash
python -m build
twine upload dist/*     # after review
```

## License

MIT — see [LICENSE](LICENSE).
