Metadata-Version: 2.4
Name: replaysafe
Version: 0.2.0
Summary: Deterministic static analysis for data-pipeline recoverability
Project-URL: Homepage, https://github.com/loxux/ReplaySafe
Project-URL: Repository, https://github.com/loxux/ReplaySafe
Project-URL: Issues, https://github.com/loxux/ReplaySafe/issues
Project-URL: Changelog, https://github.com/loxux/ReplaySafe/blob/main/CHANGELOG.md
Author: ReplaySafe contributors
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: airflow,data-engineering,dbt,sql,static-analysis
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.12
Requires-Dist: click<9,>=8.1
Requires-Dist: pyyaml<7,>=6.0
Requires-Dist: sqlglot<31,>=26
Provides-Extra: dev
Requires-Dist: mypy<2,>=1.15; extra == 'dev'
Requires-Dist: pytest<9,>=8.3; extra == 'dev'
Requires-Dist: ruff<1,>=0.11; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0.12; extra == 'dev'
Description-Content-Type: text/markdown

# ReplaySafe

[![CI](https://github.com/loxux/ReplaySafe/actions/workflows/ci.yml/badge.svg)](https://github.com/loxux/ReplaySafe/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/replaysafe.svg)](https://pypi.org/project/replaysafe/)
[![Python](https://img.shields.io/pypi/pyversions/replaysafe.svg)](https://pypi.org/project/replaysafe/)

ReplaySafe is a deterministic static analyzer for data pipelines that cannot safely retry, replay, backfill, partially fail, or run concurrently. The v0.1 scanner focuses on SQL plus statically extractable Python/Airflow and dbt-style SQL. It runs locally and never imports or executes scanned repository code.

## Five-minute quick start

ReplaySafe requires Python 3.12 or newer.

```bash
python -m pip install replaysafe
replaysafe scan dags/
```

Scan output explains the exact evidence, failure sequence, likely consequence, remediation, and deterministic confidence. A finding at or above `fail_on` exits with status 1.

```bash
replaysafe scan . --dialect postgres --format json
replaysafe scan . --format sarif > replaysafe.sarif
replaysafe explain RS003
replaysafe init
```

## What v0.1 checks

| Rule | Default | Proven pattern |
| --- | --- | --- |
| RS001 Wall-clock dependency | High | Wall-clock function in `WHERE`, `JOIN`, `HAVING`, or `QUALIFY` |
| RS002 Retry-unsafe blind append | High | `INSERT` append without a visible replay guard |
| RS003 Target-derived unsafe watermark | Critical | `> (SELECT MAX(...) FROM target)` where the task writes target |
| RS004 Non-atomic destructive replacement | Critical | Same-task `DELETE` then `INSERT` outside an explicit transaction |
| RS006 Unstable pagination | High | `LIMIT/OFFSET` without `ORDER BY` |
| RS008 Non-unique watermark | High | Bounded scalar time cursor without a secondary key |
| RS014 External side effect | Critical, disabled | Retried Airflow task with obvious HTTP POST and no idempotency key |
| RS017 Non-deterministic deduplication | High | Proven `ROW_NUMBER`/`RANK = 1` survivor selection without ordering |

ReplaySafe prefers no finding over a speculative blocking result. See [rule boundaries](docs/rules.md) and the [precision report](docs/precision-report.md).

## Supported inputs

| Input | v0.1 support |
| --- | --- |
| Standalone SQL | Multi-statement parsing and recovery semantics |
| PostgreSQL, Snowflake, BigQuery, StarRocks | SQLGlot parsing with explicit dialect selection |
| Python | Literal/f-string/local-variable SQL passed to known execution methods |
| Airflow | Static `@dag`, `@task`, common SQL operators, retries, task IDs, logical-time Jinja |
| dbt | Automatic manifest discovery, compiled SQL analysis, relations, dependencies, materializations, incremental strategies, and unique keys |

Dynamic Python assembly, arbitrary symbolic execution, runtime warehouse metadata, broad lineage catalogs, Spark plans, and generic data-quality checks remain intentionally outside the current scope.

## dbt projects

Scan a dbt project from its root:

```bash
replaysafe scan path/to/dbt-project
```

When `target/manifest.json` exists, ReplaySafe discovers it automatically and reads it without invoking dbt. Current `compiled_code` and legacy `compiled_sql` fields are preferred over unexpanded model templates. ReplaySafe maps findings back to `original_file_path` and carries model identity, physical relation, `ref`/`source` dependencies, materialization, incremental strategy, and `unique_key` into the semantic model.

Incremental models produce an explicit destination-write semantic: append without a key remains visible to RS002, `merge`/`delete+insert` with a key is treated as an upsert, and `insert_overwrite`/`microbatch` is treated as overwrite. Generated `target`, `logs`, and `dbt_packages` trees are excluded from ordinary source discovery to avoid duplicate findings.

For a nonstandard artifact location, pass it explicitly:

```bash
replaysafe scan . --dbt-manifest build/dbt/manifest.json
```

Without a manifest, dbt-style SQL is still scanned with non-executing Jinja placeholder handling, but macro-expanded relations and adapter materialization behavior are unavailable.

## Configuration and suppressions

Generate a documented starting point with `replaysafe init`. A fuller example is:

```yaml
version: 1
dialect: snowflake
fail_on: high
exclude:
  - "tests/**"
rules:
  RS014:
    enabled: false
assets:
  analytics.audit_log:
    append_only: true
    duplicate_tolerant: true
suppressions:
  - rule: RS002
    file: dags/audit.py
    reason: "Warehouse DDL enforces an event ID outside this repository."
    expires: "2027-01-01"
```

Inline exceptions apply to the following nearby statement and should explain the external guarantee:

```sql
-- replaysafe: ignore RS002 reason="event_id is unique in warehouse DDL"
INSERT INTO analytics.audit_log SELECT * FROM staged_events;
```

Use `--ci` (or set `CI`) to require suppression reasons. Unused config suppressions produce diagnostics instead of silently going stale.

## Output and exit codes

- `text` groups findings by file and presents the full failure path.
- `json` uses deterministic schema version `1.0.0`.
- `sarif` emits SARIF 2.1.0 with rule help and stable fingerprints.
- Exit 0: scan completed below the failure threshold.
- Exit 1: at least one finding reached `fail_on`.
- Exit 2: invalid invocation, configuration, or scan path.

The Python API is the core interface:

```python
from pathlib import Path
from replaysafe.analysis import scan_repository

result = scan_repository(Path("dags"))
```

## GitHub Actions

The repository includes a [SARIF workflow](.github/workflows/replaysafe.yml). It uploads results even when the configured finding threshold fails, then preserves the failed check. For stable install time, cache pip using `actions/setup-python` and pin the ReplaySafe release in production workflows.

## Security model

Default scans are local, offline, and parser-only. ReplaySafe does not import DAGs, call `eval`/`exec`, invoke dbt, connect to a warehouse, follow directory symlinks, or upload source. File size is bounded and likely credentials are redacted from evidence. See [SECURITY.md](SECURITY.md) for the threat boundary and reporting process.

## Development

```bash
python -m pip install -e ".[dev]"
pytest
ruff check .
ruff format --check .
mypy replaysafe
```

Contributions should add positive, safe-negative, and ambiguous fixtures before changing rule behavior. See [CONTRIBUTING.md](CONTRIBUTING.md).

The included 1,000-file smoke benchmark and current indicative result are documented in [docs/performance.md](docs/performance.md).

Licensed under Apache-2.0.
