Metadata-Version: 2.4
Name: edgepoint
Version: 5.0.0
Summary: Find the point in a numeric column above which a binary outcome becomes meaningfully and reliably better.
Author: Henry
License: MIT
Keywords: threshold,decision-threshold,binary-classification,feature-engineering,data-analysis,analytics,statistics,edge,threshold-detection
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.21
Requires-Dist: pandas>=1.5
Provides-Extra: test
Requires-Dist: pytest>=7.0; extra == "test"
Dynamic: license-file
Dynamic: requires-python

# edgepoint

Built for data analysts who aren't data scientists, people who know a little Python but don't want to build or babysit a modeling toolbox, and who need a quick, accurate read to make a decision now.

Finds the point in a numeric column where a binary outcome (a `hit`/`0`-`1` column) starts performing meaningfully better - above it, below it, or inside a range, whichever direction the data actually supports - plus the best combination of columns for that. Splits your data into train/test, searches train, replays the exact result on test, so what you get back is checked against data it never saw, not just fit once and trusted.

**Where this applies**: any numeric metric + binary outcome pair:

- Marketing: what engagement score is where churn drops off
- Credit: what score is where default rate becomes acceptable
- Healthcare: what biomarker level is where an outcome rate jumps
- Product: what usage count is where upgrade-to-paid spikes
- Trading / betting: what signal strength is where a pick's hit rate gets reliable (the original itch this scratched)

**Who's this for**: anyone who wants a plain "here's exactly where the line is, and proof it held up on data it hadn't seen" instead of a black-box model, and doesn't want to babysit a modeling toolbox to get there.

**Two things changed in this version.** It's no longer one-directional; it used to only check "does it get better above this point," now it checks every direction (above, below, or a range) and picks whichever one actually holds. And it's no longer a plain function (`edgepoint.search(df, ...)`), it's wrapped in a class called `Engine`, which also saves/loads results to disk for you. If you're used to the old style, read on, both are different now.

---

## Install

```
pip install edgepoint
```

```python
from edgepoint import Engine
```

---

## Quick start

```python
from edgepoint import Engine

engine = Engine(dir_name="my_dataset")   # dir_name = where results get saved

result = engine.train(df, outcome_col="hit")

result.edgepoints   # per-column thresholds (DataFrame)
result.combos       # best combo(s) of thresholds (dict or list of dicts)
```

Everything below is a method on `Engine`. Create one `Engine` per dataset (or reuse one and pass `dir_name=` per call if you're juggling several).

---

## `outcome_col`: telling edgepoint what "good" means

Before `train()` scans a single edge, it needs one thing from you that matters more than any parameter: `outcome_col`. This is you, the analyst, labeling every row as a win or a loss, a pass or a fail. `edgepoint` doesn't decide what "success" looks like in your data, it has no idea what a good sale, a good patient outcome, or a good bet even is. You decide that, once, by pointing at a column, and every number the library produces after that is downstream of that one judgment.

Concretely: `outcome_col` must be a column of `1`/`0`, `True`/`False`, or a mix of both, one value per row, `1` (or `True`) meaning "this row counts as a success," `0` (or `False`) meaning "this row doesn't." No other values are allowed: `edgepoint` raises a `ValueError` before doing any work if it finds anything else in that column, rather than silently guessing which values you meant.

---

## Creating an Engine

```python
engine = Engine(
    dir_name=None,        # dataset folder name; None = don't save to disk
    range_bins=15,        # number of candidate edges checked per column
    min_coverage=20,      # min % of rows a threshold must cover, 5-100
    gap_weight=65,        # 0-100, how much weight gap gets vs coverage
    shrinkage_k=30,       # small-sample discount strength
    show_progress=True,   # print progress as it runs
    date_match="closest", # how read_file/read_combo fall back if a date is missing
    max_lookback_n=30,    # how many days to look back/forward for date_match
    retain_saves_n=None,  # cap on saved dates kept per dataset; None = keep all
)
```

Every one of these is checked when you set it, pass the wrong type (like a bool where a number's expected) and you'll get a clear error immediately, not a confusing crash later.

### Overfitting: gap_weight and min_coverage are a pair

They're not two separate knobs. The defaults (`gap_weight=65`, `min_coverage=20`) are chosen to get good results while still guarding against overfitting: `min_coverage=20` is loose enough to let thinner, more specific slices of data qualify, more chances at a genuine edge, while `gap_weight=65` stops an impressive-looking gap on one of those thin slices from winning purely on gap size, coverage still carries real weight (35%) in the score. If you push `min_coverage` even lower to chase very specific slices, pull `gap_weight` down further with it, `50`-`60` is a reasonable range. Raising `min_coverage` back up (toward `33`+) lets you push `gap_weight` higher again, since a higher coverage floor is already doing more of the overfitting-guarding on its own.

---

## `update()`: change settings without rebuilding

```python
engine.update(min_coverage=25, show_progress=False)
```

Same checks as the constructor. Only touches the settings listed above, nothing else.

---

## `train()`: run the search, optionally save it

```python
result = engine.train(
    df,
    outcome_col="hit",
    top_combo_n=3,        # how many top combos to keep
    dir_name=None,         # overrides the Engine's dir_name for this call
    date=None,             # defaults to today; used as the save's date stamp
    show_progress=None,    # overrides the Engine's default for this call
    verbose=False,         # passed through to the underlying search
    overwrite=False,       # False = reuse an existing save for that date instead of rerunning
    direction="all",       # "all"/"left"/"right"/"range", or a dict of per-column overrides
    anchors=None,          # column(s) that must appear in every combo - see below
    top_col=None,          # cap on free-pool columns considered for combos; None = auto-scaled
    gen_combos=True,       # False = skip combo generation, edgepoints only
)
```

Returns a `TrainResult` with `.edgepoints` and `.combos` (also unpacks as `edgepoints, combos = engine.train(df)` if you prefer that) when `gen_combos=True` (the default). When `gen_combos=False`, this returns `edgepoints_df` directly instead, a single value, not a `TrainResult`, not a tuple, since there's no combo data in this mode to wrap.

If `dir_name` resolves to a real name, this also saves the results to disk, dated with today (or whatever `date=` you gave). Calling `train()` again for the same date does nothing but read back the old save, unless you pass `overwrite=True`. With `gen_combos=False`, only the edgepoints get saved (no `combos.json` is written, not even an empty one).

### `direction`, `anchors`, and `top_col`: your main overfitting guards

These three are the newest additions, and they're the most important levers you have against overfitting once you go past single-column edgepoints into combo search. The core risk with any combo search is the multiple-comparisons problem: the more combos you let it test, the more likely one of them looks great purely by chance, not because it's a real pattern. `min_coverage`/`gap_weight`/`shrinkage_k` all guard against overfitting *within* a single combo's stats; `direction`, `anchors`, and `top_col` guard against overfitting by shrinking the *search space itself* using what you already know about your data, so there are fewer chances for a lucky-looking combo to slip through in the first place.

#### `direction`: which side of the threshold to search

Controls which direction(s) get checked per column: `"left"` (does it get better below this point), `"right"` (does it get better above this point), `"range"` (does it get better inside a band), or `"all"` (the default, checks every direction and keeps whichever one actually holds).

```python
# You already know engagement should look better ABOVE some point -
# only test "right", instead of also trying "left" and "range" on it
result = engine.train(df, outcome_col="hit", direction="right")

# Per-column overrides: everything else still defaults to "all"
result = engine.train(
    df,
    outcome_col="hit",
    direction={
        "engagement_score": "right",     # only "does it get better above X"
        "days_since_signup": "left",     # only "does it get better below X"
        "risk_band": "range",            # only "does it get better inside a band"
    },
)
```

Why this protects against overfitting: every extra direction tested on a column is another independent chance for a threshold to look good purely from noise. If you already know engagement only ever goes one way, telling `direction` that instead of leaving it on `"all"` cuts that column's chances of a false-positive edge by up to two-thirds.

#### `anchors`: forcing specific columns into every combo

By default, combo generation is a free search across every eligible column. `anchors` lets you require that one or more specific columns show up in every combo it builds, useful when you already know a column has to be part of the story and just want to see what pairs well with it.

```python
# Require "engagement_score" in every combo generated
result = engine.train(df, outcome_col="hit", anchors="engagement_score")

# At least one of these two must be present in each combo (each is its own requirement)
result = engine.train(df, outcome_col="hit", anchors=["engagement_score", "tenure"])

# Bond two columns together - they always appear as a pair, never split apart
result = engine.train(df, outcome_col="hit", anchors=[["engagement_score", "tenure"]])

# Mix standalone and bonded: engagement_score alone, OR the bonded pair (tenure, plan_type)
result = engine.train(
    df,
    outcome_col="hit",
    anchors=["engagement_score", ["tenure", "plan_type"]],
)
```

- A bare column name requires that one column in every combo.
- A list of column names means at least one of them must be present, each is its own standalone requirement.
- A nested list/tuple inside that list bonds those columns together.

Up to 4 anchor requirements total (a bonded pair still only counts as one). Column names are matched case-insensitively against your data.

Why this protects against overfitting: instead of a free search testing every possible combination (thousands of chances for a lucky-looking combo), `anchors` narrows the search to only combos built around a column you already have a real reason to trust. Fewer combos tested means fewer chances for coincidence to win.

#### `top_col`: capping how wide the combo search goes

Combo generation can get large fast as more columns are eligible. `top_col` caps how many non-anchor ("free pool") columns are even considered, keeping only the best-scoring ones up to that count.

```python
# Only let the 10 best-scoring free-pool columns compete for combo slots
result = engine.train(df, outcome_col="hit", top_col=10)

# Auto-scale instead (the default) - tighter with fewer rows, looser with more
result = engine.train(df, outcome_col="hit", top_col=None)
```

Why this protects against overfitting: same logic as `anchors`, a smaller candidate pool means far fewer possible combos overall (the count grows combinatorially with pool size), so there's less room for a combo to win by chance rather than genuine signal, this matters most exactly when you're tempted to widen the search on a small dataset, which is the classic overfitting setup.

#### All three together

```python
result = engine.train(
    df,
    outcome_col="hit",
    direction={"engagement_score": "right"},   # only test the direction you expect
    anchors="engagement_score",                # every combo must include it
    top_col=10,                                # only the 10 best free-pool columns compete
)
```

### `gen_combos`: skip combo generation entirely

Set `gen_combos=False` when you only want the per-column edgepoints and don't need combo search at all, this skips it entirely rather than running it and discarding the result, so it's meaningfully faster on a wide dataset. `train()` then returns `edgepoints_df` directly instead of a `TrainResult`, and nothing gets written to `combos.json` even if you're saving to disk.

---

## `read_file()`: read back a saved run's thresholds

```python
df = engine.read_file(
    dir_name=None,
    date=None,            # defaults to today
    show_progress=None,
    max_lookback_n=None,
    date_match=None,      # "exact", "backward_first", "forward_only", "closest", "newest"
)
```

Returns a DataFrame (empty if nothing was found). If the exact date isn't saved, it falls back using `date_match`, e.g. `"closest"` checks nearby dates on both sides and takes the nearest one.

---

## `read_combo()`: read back a saved run's best combo(s)

```python
combos = engine.read_combo(
    dir_name=None,
    date=None,
    show_progress=None,
    max_lookback_n=None,
    date_match=None,
)
```

Same date-fallback behavior as `read_file()`. Returns a dict (single combo) or list of dicts, or `{}` if nothing found.

---

## `predict()`: check one row of data against a saved combo

```python
result = engine.predict(
    row,                  # dict of {column: value}, or a single-row DataFrame
    dir_name=None,
    date=None,
    check_n=3,             # how many saved top combos to check
    strict_n=2,            # how many of those must pass for an overall pass
    pick_metric="hit_rate",  # "hit_rate" or "coverage" - what to pick the winner by
    pick_on="train",         # "train" or "test" - which side of that metric to use
    show_progress=None,
    max_lookback_n=None,
)
```

Returns `{"status": bool, "combos": dict}`.

- If enough combos passed (`status=True`): `combos` is the single winning combo, among the ones that passed, whichever has the lowest `pick_metric` on `pick_on`. Ties break on the other metric (also lowest wins).
- If not enough passed (`status=False`): `combos` is `{}`, empty.

---

## `delete_combos()`: remove a saved date

```python
engine.delete_combos(date, dir_name=None, show_progress=None)
```

`date` is required (no "today" default here, you have to name what you're deleting). Deletes both the saved results file and combo file for that exact date.

---

## `list_dir()`: list every saved date for a dataset

```python
engine.list_dir(dir_name=None, show_progress=None)
```

Returns a list of `{"name": ..., "days_ago": ...}`, oldest first.

---

## Parameter reference

A quick lookup for every parameter across every method. Anything not listed here behaves exactly as its plain-English name suggests.

### `Engine(...)` / `update(...)`

| Param | Default | What it does |
|---|---|---|
| `dir_name` | `None` | Dataset namespace. `None` means nothing saves to disk unless you pass `dir_name=` on a specific call. |
| `range_bins` | `15` | Number of candidate edge points checked per column. Higher = finer-grained search, slower. |
| `min_coverage` | `20` | Minimum % of rows a threshold must cover to qualify, `5`-`100`. |
| `gap_weight` | `65` | `0`-`100`, how much the scoring favors gap size vs coverage. See the overfitting note below. |
| `shrinkage_k` | `30` | How hard small samples get discounted, higher = more skeptical of thin slices. |
| `show_progress` | `True` | Print progress/results as it runs. |
| `date_match` | `"closest"` | How `read_file`/`read_combo` (and anything that calls them, like `predict`) fall back when the exact date isn't saved. Five modes, see below. |
| `max_lookback_n` | `30` | How many days to search outward (back and/or forward) when `date_match` needs to fall back. |
| `retain_saves_n` | `None` | Cap on saved dates kept per dataset. `None` = keep everything forever; a number = oldest saves beyond that count get auto-deleted the next time `train()` writes a new one. |

`update()` takes the same params (any subset) and applies the same validation, it never touches anything outside this list.

### `date_match`: the five fallback modes

Used whenever a requested date isn't saved exactly. In every mode, an exact match on the date you asked for always wins immediately, these five only kick in when there's nothing there.

| Mode | Behavior |
|---|---|
| `"exact"` | No fallback at all. If the exact date isn't saved, that's a miss. |
| `"backward_first"` | Walks backward day by day up to `max_lookback_n`. Only if the entire backward sweep finds nothing does it then walk forward. |
| `"forward_only"` | Walks forward day by day up to `max_lookback_n`. Never falls back to backward. |
| `"closest"` (default) | Interleaves outward by distance, 1 day forward, 1 day back, 2 forward, 2 back, and so on. On a tie at the same distance, forward wins. |
| `"newest"` | Scans the whole window in both directions and returns whichever saved date is the single most recent one found, not necessarily the closest to what you asked for. |

### `train(df, ...)`

| Param | Default | What it does |
|---|---|---|
| `outcome_col` | `"hit"` | Your win/loss column. See the dedicated section above. |
| `top_combo_n` | `3` | How many top-ranked combos to keep. |
| `dir_name` | `None` | Overrides the Engine's `dir_name` for this call only. |
| `date` | `None` | Defaults to today. Used as the save's date stamp and what `overwrite` checks against. |
| `show_progress` | `None` | Overrides the Engine's default for this call only. |
| `verbose` | `False` | Passed straight through to the underlying search for extra detail. |
| `overwrite` | `False` | `False` = if a save already exists for this `dir_name`+`date`, skip recomputing and read the old one back. `True` = always recompute. |
| `direction` | `"all"` | Which direction(s) (`"left"`/`"right"`/`"range"`) to check per column. `"all"` checks every direction and keeps whichever one holds. A single string applies that direction to every column. A dict (e.g. `{"engagement_score": "right"}`) overrides just the columns named, everything else still defaults to `"all"`. See the dedicated section above. |
| `anchors` | `None` | Column(s) that must appear in every combo generated. A bare column name, a list of column names (each its own standalone requirement, at least one must be present), or a list containing nested 1-2 column lists/tuples to bond together (always appear as a pair, never split). Up to 4 anchor units total. See the dedicated section above. |
| `top_col` | `None` | Caps how many non-anchor ("free pool") columns are considered for combo-building, ranked by score. `None` = auto-scaled from row count and pool size. See the dedicated section above. |
| `gen_combos` | `True` | `True` = generate/validate combos as usual, `train()` returns a `TrainResult`. `False` = skip combo generation entirely, `train()` returns `edgepoints_df` directly (no `TrainResult`, no `.combos`, nothing written to `combos.json`). |

### `read_file(...)` / `read_combo(...)`

| Param | Default | What it does |
|---|---|---|
| `dir_name` | `None` | Falls back to the Engine's stored `dir_name`. |
| `date` | `None` | Defaults to today. |
| `show_progress` | `None` | Falls back to the Engine's default. |
| `max_lookback_n` | `None` | Falls back to the Engine's default. |
| `date_match` | `None` | Falls back to the Engine's default. |

### `predict(row, ...)`

| Param | Default | What it does |
|---|---|---|
| `row` | required | `dict` of `{column: value}`, or a single-row DataFrame. |
| `check_n` | `3` | How many of the saved top combos to check `row` against. Fewer than `check_n` saved = automatic reject. |
| `strict_n` | `2` | How many of those `check_n` combos must pass for an overall pass. Must be `<= check_n`. |
| `pick_metric` | `"hit_rate"` | `"hit_rate"` or `"coverage"`, which stat picks the winner among combos that passed (lowest wins; the other metric breaks ties, also lowest). |
| `pick_on` | `"train"` | `"train"` or `"test"`, which split of `pick_metric` to use. |
| `show_progress`, `max_lookback_n`, `date_match` | `None` | Same fallback behavior as `read_file`/`read_combo`, since `predict` reads a saved combo under the hood. |

### `delete_combos(date, ...)`

`date` is the only required parameter anywhere in this library, no "today" default, since deleting needs an explicit target. Exact match only, no fallback walk.

### `list_dir(...)`

Just `dir_name` and `show_progress`, both falling back to the Engine's stored defaults.

---

## Correlation, not causation

`edgepoint` finds where an outcome rate changes along a metric, it doesn't tell you *why*. A threshold that looks great on train and holds up on test is still just an association, not proof that crossing that point *causes* the better outcome. There could be a third factor driving both. Treat what it returns as "here's where the pattern sits and how well it held up," not "here's a lever you can pull."

That ties into the same habit as the coverage/gap_weight guidance above: the result is a snapshot worth trusting *as a snapshot*, not a mechanism you've proven. It's also not permanent, re-run it as new data comes in, since a threshold that held today can shift as more rows accumulate.

---

## Notes

- `dir_name` is just a folder name under a fixed root, it's not a file path, and results/combos live in their own subfolders under it.
- Everything above validates its own inputs. Pass a bad type or an out-of-range value and you'll get a clear error naming the exact param and what's expected.
- `retain_saves_n` is the cleanup knob: set it once and older saves beyond that count get deleted automatically the next time `train()` writes a new one.

---

No coefficients to decode, no black box, just "here's the line, and here's the proof it held." Point it at a metric and a `hit` column and let it do the counting for you.

## License

Copyright (c) 2026 osas2henry@gmail.com. All rights reserved.
