Metadata-Version: 2.5
Name: correr
Version: 0.1.0
Summary: Time-series alignment and lag/correlation analysis
License: MIT
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: scipy>=1.10
Provides-Extra: all
Requires-Dist: matplotlib>=3.7; extra == 'all'
Requires-Dist: statsmodels>=0.14; extra == 'all'
Provides-Extra: dev
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: stats
Requires-Dist: statsmodels>=0.14; extra == 'stats'
Provides-Extra: viz
Requires-Dist: matplotlib>=3.7; extra == 'viz'
Description-Content-Type: text/markdown

# correr

Time-series alignment and lag/correlation analysis for Python.

`correr` answers one question well: **given two time series on different
clocks, at what time offset are they most correlated, how strong is that
correlation, and is it statistically real once you account for the fact that
time series are autocorrelated?**

It is not a general data-preprocessing library -- pandas and scikit-learn
already own that ground. It's a focused tool for the alignment + lag +
significance problem that normally takes real boilerplate and statistical
care to get right, plus a set of complementary tools (rolling lag tracking,
Granger causality, distributed-lag shape, DTW, a stationarity guardrail, a
CLI, and one-call HTML reports) for the ways that basic approach can mislead
you or fall short.

## Why

Say you have solar panel output sampled every 5 minutes and a sunspot index
reported daily. You suspect panel output responds to solar activity with
some delay. Answering that with pandas/scipy alone means:

1. Resampling both series onto a common grid by hand.
2. Shifting one series across a range of lags and computing a correlation at
   each shift.
3. Reporting a p-value for the best lag -- except a naive Pearson p-value on
   autocorrelated data is badly biased toward "significant," picking the
   best of many lags tested compounds that further, and two merely
   *trending* series will look correlated almost by construction regardless
   of any of the above.

`correr` does all three, with the statistical corrections included by
default instead of left as a footgun.

```python
import correr

match = correr.find_best_lag(signups, ad_spend, max_lag=10, freq="1D")
# LagMatch(lag=3, sub_lag=2.91, correlation=0.88, p_value=0.0003, ci=(0.80, 0.91))
```

## Install

```bash
pip install correr
```

That gets you the core package (`align`, `cross_correlate`, `find_best_lag`,
`compare_many`, `rolling_lag_correlation`, `lag_kernel`, `dtw_align`, the
CLI). For plotting and the stats-backed tools, install with extras:

```bash
pip install correr[viz,stats]   # or: pip install correr[all]
```

For local development instead (editable install, running the test suite):

```bash
git clone https://github.com/OliverVillson/correr.git
cd correr
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev,viz,stats]"
```

`viz` (matplotlib) is needed for `plot_lag_correlogram`/`plot_aligned`/
`write_report`; `stats` (statsmodels) for `check_stationarity` and
`granger_causality`.

## Quickstart

```python
import correr

# a_aligned, b_aligned share one DatetimeIndex, gaps interpolated
a_aligned, b_aligned = correr.align(series_a, series_b, freq="1D")

# correlation at every lag from -14 to +14 periods
result = correr.cross_correlate(a_aligned, b_aligned, max_lag=14)

# align + scan + pick the best lag + attach significance, in one call
match = correr.find_best_lag(series_a, series_b, max_lag=14, freq="1D")
print(match.lag, match.correlation, match.p_value, match.ci)

# same, against several candidate predictors at once, ranked by |correlation|
ranked = correr.compare_many(
    target=series_a,
    candidates={"b": series_b, "c": series_c},
    max_lag=14,
)

correr.plot_lag_correlogram(result)
correr.plot_aligned(a_aligned, b_aligned, lag=match.lag)
```

## Demo

[`demo.py`](demo.py) runs the core pipeline on two generated-but-realistic
datasets in [`datasets/`](datasets): daily online ad spend
([`ad_spend.csv`](datasets/ad_spend.csv)) and daily new website signups
([`website_signups.csv`](datasets/website_signups.csv)), the latter missing
a handful of days from a simulated analytics outage. The data was built with
a known ground truth -- signups respond to ad spend mostly 3 days later --
so the demo's output can be checked against something real.

```bash
python demo.py
```

```
ad_spend:  180 days  (2026-01-01 to 2026-06-29)
signups:   174 days  (2026-01-01 to 2026-06-29)

signups is missing 6 day(s): ['2026-03-03', '2026-03-26', '2026-04-05', '2026-04-13', '2026-05-10', '2026-06-23']
aligned onto a common 180-day grid (gaps linearly interpolated)

best lag: signups follow ad spend by 3 day(s)  (sub-lag estimate: 2.91)
correlation:  r = 0.88
p-value:      0.0003237  (autocorrelation-adjusted, Bonferroni-corrected for the lag scan)
95% CI:       [0.80, 0.91]

=> conclusion: a real, significant relationship between ad spend and signups at a 3-day lag.
```

`correr` recovers the injected 3-day lag exactly, from noisy daily data with
missing days, without being told it.

**Correlogram** -- correlation at every tested lag, best lag highlighted:

![lag correlogram](datasets/demo_correlogram.png)

**Aligned overlay** -- signups against ad spend shifted by the recovered lag
(the campaign spikes in April and June line up once shifted):

![aligned overlay](datasets/demo_aligned_overlay.png)

A second example, [`examples/solar_vs_sun_activity.py`](examples/solar_vs_sun_activity.py),
runs the same pipeline against a genuinely related series *and* an
unrelated one side by side, to show `compare_many` correctly rejecting the
unrelated series (p=1.0) rather than reporting a spurious lag.

### Advanced demo: the rest of the toolkit

[`demo_advanced.py`](demo_advanced.py) runs the same ad-spend/signups data
through everything beyond the core pipeline:

```bash
python demo_advanced.py
```

```
1. stationarity check
signups    ADF p=0.523  likely_nonstationary=True
ad_spend   ADF p=0.320  likely_nonstationary=True

correlation at lag=3: raw=0.88, after detrending=0.86
-> still strongly correlated after removing the shared trend, so this isn't just
   two things trending upward together -- there's a real lagged relationship.

2. rolling lag-correlation (40-day windows)
...
lag=3 in 67% of windows -- the relationship is stable, not a one-time coincidence.

3. Granger causality (does ad_spend help predict signups?)
        f_stat       p_value
lag
1    26.366516  7.430614e-07
2    51.125048  3.585592e-18
3    98.869825  4.458976e-37
4    74.351029  4.438010e-36
5    60.230558  2.250832e-35
6    49.753016  3.103037e-34

4. distributed-lag kernel (response shape across nearby lags)
...
 2    0.0857
 3    0.2084
 4    0.0920
...
-> weight concentrated around lags 2-4, matching how the demo data was built
   (a decaying response centered on day 3), not a single spike at one lag.

5. DTW alignment (for drift/stretch, not a fixed lag)
same waveform, stretched from 40 to 55 points.
DTW distance: 3.927
naive same-length correlation (ignoring the stretch): r=-0.12

6. HTML report
wrote datasets/demo_report.html
```

Worth noting: the rolling lag-correlation output shows the relationship is
*noisy* in the first ~3.5 months (low ad spend, low signal-to-noise) and
locks onto lag=3 consistently from late April on, once spend -- and the
campaign spikes in it -- picked up. `correr` surfaces that instead of
hiding it behind one overall number.

## How it works

```
raw series A, B (different freq/timestamps)
        |
        v
   check_stationarity()   flag a shared trend before it fakes a correlation
        |                  (detrend() to fix it)
        v
   align()            resample + interpolate onto a common grid
        |
        v
   cross_correlate()  shift B across a range of lags, score each with a
        |             correlation metric (pearson/spearman/kendall)
        v
   find_best_lag()    pick the strongest lag (+ a parabolic sub-lag
        |             estimate), then attach:
        |               - an autocorrelation-adjusted p-value
        |                 (effective-sample-size correction, Pyper &
        |                 Peterman 1998)
        |               - Bonferroni correction for the number of lags
        |                 scanned (picking the best of many candidates
        |                 and testing it as if it were the only one
        |                 inflates false positives)
        |               - a moving-block bootstrap confidence interval
        v
   compare_many()     run the above for one target against many candidate
                       series, ranked by |correlation|, with a further
                       Benjamini-Hochberg FDR correction across candidates
```

Alongside that pipeline: `rolling_lag_correlation` (does the lag/strength
hold steady over time?), `granger_causality` (does the candidate actually
improve prediction, not just correlate?), `lag_kernel` (the whole
distributed-lag response shape, not just its peak), `dtw_align` (alignment
for series that stretch/drift rather than shift by a constant offset), and
`write_report` (a self-contained HTML writeup of a `compare_many` run).

Full design rationale and module-by-module responsibilities are in
[ARCHITECTURE.md](ARCHITECTURE.md).

## Command line

Every core operation is also available without writing Python:

```bash
correr find-best-lag datasets/website_signups.csv datasets/ad_spend.csv --max-lag 10
correr compare datasets/website_signups.csv datasets/ad_spend.csv --max-lag 10
correr align datasets/website_signups.csv datasets/ad_spend.csv --out aligned.csv
correr report datasets/website_signups.csv datasets/ad_spend.csv --max-lag 10 --out report.html
```

CSVs need a date column (`--date-col`, default `date`) and a value column
(`--a-col`/`--b-col`/`--target-col`; inferred automatically if the file has
exactly one non-date column, as the demo CSVs do).

## API reference

| Function | Signature | Returns |
|---|---|---|
| `align` | `align(a, b, freq=None, method="linear", how="outer")` | `(aligned_a, aligned_b)` on a shared `DatetimeIndex` |
| `detect_gaps` | `detect_gaps(s, freq)` | `DatetimeIndex` of missing expected timestamps |
| `cross_correlate` | `cross_correlate(a, b, max_lag, method="pearson")` | `LagResult(lags, correlations, method)` |
| `find_best_lag` | `find_best_lag(a, b, max_lag, freq=None, method="pearson", warn_nonstationary=True)` | `LagMatch(lag, sub_lag, correlation, p_value, ci)` |
| `compare_many` | `compare_many(target, candidates, max_lag, freq=None, method="pearson", warn_nonstationary=True)` | `DataFrame` ranked by `\|correlation\|`, with `p_value_fdr` |
| `rolling_lag_correlation` | `rolling_lag_correlation(a, b, window, max_lag, freq=None, step=None, method="pearson")` | `DataFrame` indexed by window end: `[lag, correlation, n]` |
| `granger_causality` | `granger_causality(a, b, max_lag, freq=None)` | `DataFrame` indexed by lag: `[f_stat, p_value]` (`correr[stats]`) |
| `lag_kernel` | `lag_kernel(a, b, max_lag, freq=None, alpha=1.0)` | `Series` of ridge-regression weight per lag |
| `dtw_align` | `dtw_align(a, b, window=None, normalize=True)` | `DTWResult(distance, path)` |
| `check_stationarity` | `check_stationarity(s, alpha=0.05)` | `StationarityReport(adf_statistic, adf_p_value, likely_nonstationary)` (`correr[stats]`) |
| `detrend` | `detrend(s, how="linear")` | `Series` with the trend removed |
| `benjamini_hochberg` | `benjamini_hochberg(p_values)` | FDR-adjusted p-values (`ndarray`) |
| `write_report` | `write_report(target, candidates, max_lag, freq=None, path="correr_report.html", warn_nonstationary=True)` | path written; self-contained HTML (`correr[viz]`) |
| `plot_lag_correlogram` | `plot_lag_correlogram(result, ax=None)` | `Axes` (`correr[viz]`) |
| `plot_aligned` | `plot_aligned(a, b, lag=0, ax=None)` | `(ax, ax2)` twin axes (`correr[viz]`) |

`a`/`b`/`s` are `pandas.Series` with a sorted, deduplicated `DatetimeIndex`.
`method` is one of `"pearson"`, `"spearman"`, `"kendall"`. Lag is expressed
in periods of the aligned frequency (an `int`; `sub_lag` is the
parabolic-interpolated `float` estimate), not `pd.Timedelta`.

## Development

```bash
pip install -e ".[dev,viz,stats]"
python -m pytest -q
python demo.py
python demo_advanced.py
python datasets/generate_datasets.py   # regenerate the demo CSVs (fixed seed)
```

## License

MIT.
