Metadata-Version: 2.4
Name: decisive
Version: 0.0.1
Summary: A causal-read and cheap-experimentation toolkit for teams without experimentation infrastructure.
Author: Aman Singh
License: MIT
Project-URL: Homepage, https://github.com/Aman12x/decisive
Project-URL: Repository, https://github.com/Aman12x/decisive
Project-URL: Issues, https://github.com/Aman12x/decisive/issues
Project-URL: Roadmap, https://github.com/Aman12x/decisive/blob/main/ROADMAP.md
Keywords: causal-inference,difference-in-differences,synthetic-control,interrupted-time-series,quasi-experiment,ab-testing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Mathematics
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.23
Requires-Dist: pandas>=1.5
Provides-Extra: plot
Requires-Dist: matplotlib>=3.5; extra == "plot"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Dynamic: license-file

# decisive

**A causal-read toolkit that returns a verdict, not a number — and tells you when *not* to trust it.**

Most causal-inference libraries hand you a coefficient and a p-value and leave the hard part —
*should I believe this, and what should I do about it?* — to you. `decisive` is built around that
hard part. You describe the data you have; it **routes** to the right quasi-experimental design,
**checks the identifying assumptions loudly**, and returns a **trust level and a recommendation** —
or refuses, and points you back to a design that could answer the question.

The core rule: **statistical significance never overrides a design failure.**
A confounded estimate with `p < 0.001` still comes back `unreliable`.

---

## Why it exists

The whole Python causal ecosystem is sharp *estimation engines* (`statsmodels`, `linearmodels`,
`EconML`, `DoWhy`, …) that assume you have already chosen a design, satisfied identification, and can
read a coefficient table. Three things are missing across all of them, and `decisive` is built to be
the layer that has them:

1. **Routing** — you describe the data; it picks the method.
2. **Loud refusal** — a failed identifying assumption *blocks* the causal claim, it doesn't get a footnote.
3. **A verdict** — effect, uncertainty, robustness, trust, and a ship / don't-ship recommendation gated
   on trust and on *your cost of being wrong* — never on a p-value alone.

Plus a **ground-truth teaching layer** (`datasets/`) — parameterizable data-generating processes where
*you set the true effect* — so you can watch a method recover the truth when its assumptions hold and
produce confident nonsense when they don't.

---

## Install

```bash
pip install decisive             # runtime deps: just numpy + pandas. No scipy.
pip install "decisive[plot]"     # + matplotlib, for result.plot() / decision.plot()
```

---

## 60-second tour: the teaching loop

Generate data with a **known** effect, then watch the verdict track — or refuse — it.

```python
from decisive.datasets import make_did
from decisive import analyze

# same true effect (+3.0) both times; the second injects a pre-existing trend
clean  = make_did(true_effect=3.0, parallel_trends=True,  seed=11)
broken = make_did(true_effect=3.0, parallel_trends=False, seed=11)

analyze(clean,  treatment="treated", outcome="y", time="period", unit="unit", post="post").trust
# 'strong'      → the effect is trustworthy; ships.

analyze(broken, treatment="treated", outcome="y", time="period", unit="unit", post="post").trust
# 'unreliable'  → the estimate is biased to ~+5.7 with p≈0.000, and decisive REFUSES it.
```

The broken read *looks* maximally significant. It is refused because the treated group was already
diverging from control **before** treatment — significance cannot launder a broken design.

---

## The three moments

`decisive` shows up at the three points where expensive mistakes happen, not just at analysis.

### Moment A — before the ship: make a holdout cheap and costed

```python
from decisive.plan import plan_holdout

print(plan_holdout(daily_traffic=20_000, baseline_rate=0.04, mde=0.05).summary())
# Hold out 5% (~79,315 units) and run ~80 day(s): you can detect a lift of 5.0%
# or larger (baseline 4.0%). duration solved from your minimum detectable effect.
```

And the argument *for* holding out — the salvage penalty if you don't:

```python
from decisive.plan.design_forecast import forecast_no_holdout

print(forecast_no_holdout(daily_traffic=20_000, baseline_rate=0.04, mde=0.05).summary())
# With a holdout: detect a 5.0% lift with randomized, assumption-free inference ...
# Without one: your best read is observational ... capped at 'moderate' trust ...
# The penalty: a confounder ... could bias the estimate by ~2.5% — about 0.5x the
# lift you're trying to detect, often enough to flip ship/kill.
```

### Moment B — after the ship: read what happened honestly

```python
from decisive import analyze

result = analyze(data, treatment="region_treated", outcome="revenue",
                 time="week", unit="region", post="post", min_effect=0.01)
print(result.summary())
```

```
Effect: +3.08 (95% CI: +2.71 to +3.44)  ·  design: difference-in-differences
Trust: STRONG
  ✓ parallel pre-trends hold (p=0.80)
  ✓ 18 control units available
  ✓ 24 clusters — cluster-robust inference is sound
  Robustness: ~17.5 standard errors of systematic bias would be needed to erase this.
  Placebo test: permutation p=0.000.
  HonestDiD (robust): survives a post-period trend violation up to ~7.9× the largest pre-trend.

→ Ship: the effect clears your threshold with room to spare.
```

When the design can't support the claim, the verdict says so and routes you back to Moment A:

```
Trust: UNRELIABLE
  ✗ pre-trends diverge before treatment (interaction p=0.00) — difference-in-differences is not identified
→ This design can't answer the question. See plan/holdout.py for the holdout that would.
```

### Moment C — the decision handoff: ship on the cost of being wrong

A verdict is not a decision. `decide()` gates ship / don't-ship on a **dollar cost model** — the
market white space no incumbent occupies — while still respecting the trust layer.

```python
decision = result.decide(value_per_unit=2.0, scale=50_000, ship_cost=40_000, unit_label="user")
print(decision.summary())
```

```
Business decision: SHIP
  Breakeven effect: +0.4  (covers the $40,000 ship cost across 50,000 users).
  Effect +3.08  ·  expected value of shipping: $267,669.
  P(net-positive): 100.0%  ·  expected downside if you ship: $0.

→ Ship: the effect covers its cost with room to spare (trust: STRONG).
```

The trust gate survives the dollar layer: an `unreliable` read returns `inconclusive` no matter how
large the expected value. Significance can't launder a broken design, and neither can a big number.

---

## What `analyze()` supports

You describe the columns; the router picks the design from the data's shape — and refuses to
mis-dispatch to a method it can't stand behind.

| Data shape | Design chosen | Estimator | Identifying check (can **block**) |
|---|---|---|---|
| Several treated units, **common** treatment timing, with controls | **Difference-in-differences** | 2×2 OLS interaction | parallel pre-trends |
| Several treated units, **varying** adoption times, with controls | **Staggered adoption** (Callaway–Sant'Anna) | group-time ATT(g,t) | pre-period ATT(g,t) |
| **One** treated unit + a donor pool | **Synthetic control** | simplex-weighted donors | pre-period fit quality |
| A single interrupted series (no control group) | **Interrupted time series** | segmented regression | pre-trend curvature |
| No time dimension | *(refused, with guidance)* | — | — |

Every method returns the same `ExperimentResult`, so the trust/verdict/plot/decision surface is
identical regardless of which design ran.

### The methods, briefly

- **Difference-in-differences** — cluster-robust standard errors by unit with a Student-t(G−1)
  critical value (the Bertrand–Duflo–Mullainathan fix + the small-cluster correction), a **wild
  cluster bootstrap** CI when clusters are few, and a permutation/placebo cross-check. The pre-trends
  test is cluster-robust and *power-aware*: `p ≥ 0.05` is not a pass if the test was too weak to rule
  out a trend large enough to fabricate the effect.
- **Staggered adoption** — Callaway–Sant'Anna group-time ATT(g,t) with clean (not-yet-treated)
  comparisons, cluster-bootstrap inference, and a pre-period-ATT parallel-trends block. Recovers the
  true ATT under dynamic effects where naive two-way fixed effects is biased (the Goodman-Bacon
  "forbidden comparison").
- **Synthetic control** — a donor counterfactual via simplex-constrained weights (projected gradient,
  no scipy). Inference is placebo-in-space permutation (there is no analytic SE); the identifying
  block fires when the donor pool can't reconstruct the treated unit's pre-period. Caps its own claim
  honestly.
- **Interrupted time series** — segmented regression with Prais-Winsten AR(1) GLS inference (ITS
  residuals are serially correlated), a blocking pre-trend-curvature check, and placebo-in-time
  robustness. Capped at `moderate` trust by construction — a single series has no control group.

---

## The trust model (the differentiator)

`trust` is **derived from the assumption checks, never asserted** (`result.py`):

- Any check with `severity="block"` that fails → `trust = "unreliable"`, the effect is reported as
  descriptive, and the recommendation becomes `inconclusive_run_a_real_test` (routing back to `plan/`).
  **This holds regardless of the p-value.**
- Each failing `severity="warn"` check erodes `strong` → `moderate`.
- A method that declines a causal claim (e.g. clustering) reports `descriptive_only`.

The recommendation is then gated on trust **and** your `min_effect` (or, via `decide()`, your dollar
breakeven) — never on significance alone.

---

## Robustness & sensitivity

Every `ExperimentResult` carries an interpretable robustness statement, not just a p-value:

- **Confounder-to-overturn** — how many standard errors of systematic bias it would take to erase the effect.
- **Placebo tests** — permutation (DiD, SC) or placebo-in-time (ITS) p-values.
- **HonestDiD** (Rambachan–Roth relative magnitudes) — a **breakdown value**: how many times larger than
  the largest pre-period violation the post-period parallel-trends violation could be before the effect's
  CI includes zero. Folded into the DiD verdict, and available standalone with an event-study +
  sensitivity-curve plot:

  ```python
  from decisive import honest_did_sensitivity
  hr = honest_did_sensitivity(data, treatment="treated", outcome="y",
                              time="period", unit="unit", post="post")
  print(hr.summary())   # "survives a violation up to ~7.9× the largest pre-trend"
  hr.plot()
  ```

  > Note: this is a deliberately *conservative* identified-set + sampling-error bound, not the exact
  > Rambachan–Roth fixed-length CI. Labeled as such everywhere.

---

## Diagnostic plots

Every result renders a **verdict-colored** diagnostic panel (green = strong/ship, amber = moderate,
red = unreliable/route-back), so the reasoning is visible, not asserted:

```python
result.plot()          # DiD: trajectories + placebo null + effect/CI
                       # SC: treated-vs-synthetic + gap-spaghetti + effect
                       # ITS: series + counterfactual + placebo-in-time + effect
                       # staggered: event study + overall ATT
decision.plot()        # effect distribution vs breakeven, loss zone shaded
```

---

## Ground-truth datasets (for learning and validation)

The analog of `seaborn.load_dataset` for causal inference — each generator carries `true_effect` and a
knob that *breaks* its identifying assumption, so you can watch a method succeed and fail on data whose
truth you set. Four route to a live verdict today; the other two ship runnable numpy recovery snippets.

```python
from decisive.datasets import (
    make_did,                 # 2×2 difference-in-differences        (routes to a verdict)
    make_staggered,           # staggered adoption                   (routes to a verdict)
    make_synthetic_control,   # one treated unit + a donor pool      (routes to a verdict)
    make_its,                 # single interrupted series            (routes to a verdict)
    make_confounded,          # cross-sectional selection bias       (teaching data + fixture)
    make_rdd,                 # sharp regression discontinuity       (teaching data + fixture)
)
```

---

## Programmatic use

```python
result.to_dict()   # JSON-safe: effect, ci, trust, recommendation, assumptions, robustness, ...
```

---

## The gallery

Ten copy-runnable lessons live in [`notebooks/`](https://github.com/Aman12x/decisive/tree/main/notebooks),
each setting a known truth and watching the verdict track or refuse it. Every cell runs in CI
(`tests/test_gallery.py`), so the gallery can't rot. Start with
**[01 — the teaching loop](https://github.com/Aman12x/decisive/blob/main/notebooks/01_the_teaching_loop.ipynb)**. Highlights:
cluster-robust SEs, honest pre-trends, the DGP catalog, DiD-vs-ITS routing, synthetic control,
HonestDiD, staggered adoption (Goodman-Bacon), and the dollar decision.

---

## Scope & honest limitations

This release is **observational causal reads + the decision layer**. Be aware of the edges:

- **Experiments are not built yet.** The variance-reduction / sequential-testing half of the vision
  (CUPED, mSPRT, stratification — Tier-2) is stubbed. `decisive` today is for reading data you already
  have, plus planning and deciding around it.
- **No cross-sectional estimator.** Data with no time dimension is refused with guidance (add a
  pre/post time dimension to unlock a design); selection-on-observables (matching / IPW) is roadmap.
- **`make_confounded` / `make_rdd`** generate ground-truth data but have no wired estimator yet.
- **ITS** single-series coverage is ~0.88–0.93 and degrades under strong serial correlation (disclosed
  via the autocorrelation warning); **synthetic control** placebo inference is coarse with few donors.

See [`ROADMAP.md`](https://github.com/Aman12x/decisive/blob/main/ROADMAP.md) for what's next and
[`docs/v0.1-readiness.md`](https://github.com/Aman12x/decisive/blob/main/docs/v0.1-readiness.md)
for the release status.

---

## Design principles & firewalls

Two structural firewalls are enforced in CI, not by convention:

1. The causal router **cannot import** descriptive clustering — clustering is for targeting, and can
   never feed a verdict.
2. Heterogeneous-treatment-effect analysis **hard-depends** on multiplicity correction — no subgroup
   effect is ever returned without it.

See [`CLAUDE.md`](https://github.com/Aman12x/decisive/blob/main/CLAUDE.md) for the full design
rationale, the verdict rules, and the build order — read it before adding methods.

---

## Develop

```bash
git clone https://github.com/Aman12x/decisive && cd decisive
pip install -e ".[dev,plot]"        # dev = pytest, plot = matplotlib
pytest                              # the test suite is Monte-Carlo-gated
python notebooks/build_gallery.py   # re-execute the gallery (needs nbformat/nbconvert/ipykernel)
```

Every method ships with its ground-truth DGP and a Monte-Carlo test (unbiasedness, coverage, verdict
behavior). No estimator lands without proof it recovers the truth and its intervals cover.
