Metadata-Version: 2.4
Name: stapel-alerts
Version: 0.2.3
Summary: An alert store for a Stapel fleet that has no Sentry
License: MIT
Project-URL: Homepage, https://github.com/usestapel/stapel-alerts
Project-URL: Repository, https://github.com/usestapel/stapel-alerts
Project-URL: Documentation, https://github.com/usestapel/stapel-alerts#readme
Project-URL: Changelog, https://github.com/usestapel/stapel-alerts/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/usestapel/stapel-alerts/issues
Keywords: django,stapel,alerts,errors,observability,sentry
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Django
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: System :: Monitoring
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: stapel-core<1.0,>=0.68.1
Provides-Extra: sentry
Requires-Dist: sentry-sdk>=2.0; extra == "sentry"
Provides-Extra: all
Requires-Dist: sentry-sdk>=2.0; extra == "all"
Dynamic: license-file

<!-- Generated by stapel-readme from docs/readme.md + docs/*.json. Do not edit this file; edit docs/readme.md and re-run `make readme`. -->

# stapel-alerts

[![CI](https://img.shields.io/github/actions/workflow/status/usestapel/stapel-alerts/ci.yml?branch=main&logo=github&label=CI)](https://github.com/usestapel/stapel-alerts/actions/workflows/ci.yml?query=branch%3Amain)
[![coverage](https://img.shields.io/codecov/c/github/usestapel/stapel-alerts?branch=main&logo=codecov&label=coverage)](https://app.codecov.io/gh/usestapel/stapel-alerts)
[![status](https://img.shields.io/badge/status-unreleased-orange)](https://github.com/usestapel/stapel-alerts)
[![license](https://img.shields.io/github/license/usestapel/stapel-alerts)](https://github.com/usestapel/stapel-alerts/blob/main/LICENSE)
[![llms.txt](https://img.shields.io/badge/llms.txt-blue)](https://github.com/usestapel/stapel-alerts/blob/main/docs/llms.txt)

> An alert store for a fleet that has no Sentry, and the same capture() call whether one is connected or not. One row per BUG (fingerprint, status, count, first/last seen, the release that claimed the fix, whether it came back) rather than a search over log text — because the question a fix wave is reconciled against is 'which bugs are open, which are fixed, and which regressed', and that needs a status column. Every input is wired by installing the app: a WARNING+ log handler, the DRF 5xx hook, Celery task failures, core's DLQ parks, and the comm handler exceptions that are RETURNED rather than raised and which no except clause anywhere would otherwise see. Monoliths write in process; microservices POST with a per-service key, buffered and retried, and fall through to Telegram when the owner is unreachable — never over the bus, because the bus is the thing whose failures this records. A Prometheus blind-spot watchdog files the monitoring stack's own gaps as issues, on the principle that 'no alerts firing' is the observable state of both a healthy fleet and a blind one.

Part of the [Stapel framework](https://github.com/usestapel) — composable Django apps that deploy as a monolith or as microservices without changing module code.

## Install

Not published on PyPI yet. Install from source:

```bash
pip install git+https://github.com/usestapel/stapel-alerts
```

## At a glance

| Fact | Value |
|---|---|
| Version | `0.2.3` |
| Python | `>=3.11` (3.11, 3.12, 3.13, 3.14) |
| HTTP operations | 5 |
| Config axes | 6 |
| Usage surface | 18 |
| Extension points | 7 |
| Error codes | 49 |
| Fleet dependencies | [`celery`](https://github.com/usestapel/celery) (optional) · [`sentry-sdk`](https://github.com/usestapel/sentry-sdk) (optional) · [`stapel-core`](https://github.com/usestapel/stapel-core) · [`stapel-notifications`](https://github.com/usestapel/stapel-notifications) (optional) |

## Documentation

**Errors:** [English](https://github.com/usestapel/stapel-alerts/blob/main/docs/errors.en.md) · [Español](https://github.com/usestapel/stapel-alerts/blob/main/docs/errors.es.md) · [Русский](https://github.com/usestapel/stapel-alerts/blob/main/docs/errors.ru.md) · [OpenAPI](https://github.com/usestapel/stapel-alerts/blob/main/docs/schema.json) · [capabilities.json](https://github.com/usestapel/stapel-alerts/blob/main/docs/capabilities.json) · [llms.txt (for agents)](https://github.com/usestapel/stapel-alerts/blob/main/docs/llms.txt)

## Why it exists

A day spent reading a production incident out of `docker logs` is the reason
this exists. The signals were all there — a foreign key violation repeating on
three accounts, twenty-two events parked in a dead-letter queue, a JWT refusal
that turned out to be correct behaviour with a misleading log line — and every
one of them had to be found by a human grepping containers, because nothing
collected them.

This library collects them. Same interface with Sentry or without it:

```python
from stapel_alerts import capture

capture(exc)
capture("STT provider exhausted", level="error", context={"provider": "x"})
```

## Two-line mount

**A monolith** — this process owns the store:

```python
INSTALLED_APPS = [..., "stapel_alerts"]
urlpatterns = [path("alerts/", include("stapel_alerts.urls"))]
```

That is all. `STAPEL_ALERTS["SERVICE"]` names the service in the tracker, and a
system check tells you at boot if you forgot it.

**A microservice** — this process reports to the owner:

```python
INSTALLED_APPS = [..., "stapel_alerts"]          # no urlconf mount
STAPEL_ALERTS = {
    "SERVICE": "svc-billing",
    "OWNER_URL": "https://api.example.com",
    "SERVICE_KEY": "<from `manage.py alerts_service svc-billing` on the owner>",
    "FALLBACK": {"TELEGRAM_BOT_TOKEN": "...", "TELEGRAM_CHAT_ID": "..."},
}
```

Nothing else changes. Every input below is wired by `AppConfig.ready()`.

## What it captures, without you instrumenting anything

| input | what it catches |
|---|---|
| `AlertsLogHandler` | every log record at WARNING+ , rate-limited per fingerprint |
| `alerts_exception_handler` | anything the fleet exception handler answers with a 5xx |
| Celery `task_failure` | a task that failed after its retries |
| `bus_event_parked` | a DLQ park or a task-ledger `unprocessable` — work dropped |
| `deliver_to_subscribers` | a comm Action handler that raised |
| `capture(...)` | whatever a library decides is worth saying |
| the watchdog | a blind spot in the monitoring itself — see below |

## Two models, because they answer two questions

**`Issue`** is the tracker: one row per bug, with `status`, `count`,
`first_seen`/`last_seen`, and the version that fixed it. It is
machine-addressable on purpose — an agent lists it, fixes the code, and closes
the issue over the API.

**`ErrorEvent`** is one occurrence: the full trace, the context (redacted
through core's redaction seam), the request path, the trace id.

## Grouping is by similarity, not by exact match

Two tracebacks that differ only in a uuid, an id, a timestamp, a memory
address or a checkout path are **one issue**. The normaliser strips those, the
fingerprint is taken over the normalised top frame plus the escaping
exception's message, and a near-duplicate scoring ≥ 0.9 over normalised frames
joins the existing issue.

A **short** trace (fewer than four frames) groups only on an exact match after
normalisation. Two four-line traces hit 0.9 by coincidence, and a wrong merge
is worse than a duplicate issue: it hides a live bug behind one somebody has
already marked fixed.

The rule is tested on real production traces, including the case that must
*not* group — a different constraint on the same table, from the same frames.

## Statuses, and where they come from

```
new ──▶ fixed ──▶ regressed
 │        ▲           │
 └── muted┘◀──────────┘
```

`fixed` is set by an API call or by CI on a commit referencing
`alerts:<issue-id>`, with the version and sha that claim the fix.
**`regressed` is never set by a caller** — the store sets it when a fixed issue
receives a new event. A caller who could assert it could also decline to.

## API

```
GET    /alerts/api/v1/issues              ?status= &level= &service= &since= &open= &offset= &limit=
GET    /alerts/api/v1/issues/{id}         + the last 20 events
PATCH  /alerts/api/v1/issues/{id}         {status, note, muted_until}
POST   /alerts/api/v1/issues/{id}/fix     {version, sha}
POST   /alerts/api/v1/report              {events: [...]}   X-Service-Key
```

The list is a page, `{count, offset, limit, results}` (`IssuePage` in the
schema). `limit` is 1..200, default 50; an out-of-range value is clamped and
the envelope echoes the limit that was applied. A `PATCH` carrying only
`muted_until` is a mute (`null` = no deadline); leaving `muted` by any route
clears the deadline.

Staff session for the tracker, service key for `/report`, and **not the other
way round**: a reporter's key lives in every container in the fleet, so the
blast radius of one leaking must not include everything the store has ever
recorded. JSON only, stable uuid ids, ETag on the reads.

## When the owner is unreachable

Buffer, retry with backoff, and after `FALLBACK_AFTER_MINUTES` send a digest
to the `NOTIFY` seam — Telegram by default, through stapel-notifications'
channel if that is configured, otherwise a direct Bot API call. A dead alert
store must be loud, not silent. The buffer is bounded and drops oldest first:
a reporter that ran out of memory holding alerts about an outage would be a
second outage.

## The monitoring watchdog

A Prometheus blind spot is itself an issue in this tracker, because "nothing
is firing" is the observable state of a healthy fleet *and* of a monitoring
stack that has stopped looking. Something outside the stack has to ask.

```python
STAPEL_ALERTS = {
    "MONITORING": {
        "PROMETHEUS_URL": "http://prometheus:9090",
        "TARGETS": ["svc-billing", "svc-api"],     # up == 0, and no `up` at all
        "METRICS": ["alerts_open_total"],          # absent()
        "HEARTBEAT_ALERT": "Watchdog",             # a dead-man's switch
        "ALERTMANAGER_URL": "http://alertmanager:9093",
    },
}

# and the schedule, shipped by the library:
from stapel_alerts.beat import get_alerts_beat_schedule
CELERY_BEAT_SCHEDULE = {**get_alerts_beat_schedule(), ...}   # every 5 minutes
```

No Celery? `manage.py alerts_watch_monitoring` is the same pass, for cron.
`--dry-run` prints what it would say without filing or notifying anything.

| check | what it means |
|---|---|
| `exporter_down` | a scrape target reporting `up == 0` |
| `scrape_missing` | a target with **no** `up` series — dropped from the config or renamed, so every alert over its metrics silently stopped firing |
| `metric_absent` | `absent(metric)` returned a series: the metric stopped existing |
| `heartbeat_missing` | the always-firing alert stopped firing — the whole chain is broken |
| `alertmanager_silenced` | an active silence, keyed by its matchers so re-silencing the same alert raises one count instead of opening a new issue |
| `alertmanager_unreachable` / `prometheus_unreachable` | the stack itself did not answer |

Each finding becomes a `kind="monitoring"` issue **and** a message on the
`NOTIFY` seam, on every run that has one. That is deliberate: the notification
thresholds (new / regressed / spike) would be silent on a blind spot's ninth
consecutive run, which is not less urgent than its first. A check that stops
failing closes its issue with `fixed_in` = `recovered <ts>`; one that comes
back regresses it. A check that is no longer configured is **not** closed —
nothing recovered, nobody looked.

## Two facts on the comm bus

```python
from stapel_core.comm import on_action

@on_action("alerts.issue.opened")
def page_somebody(event): ...

@on_action("alerts.issue.regressed")
def reopen_the_ticket(event): ...
```

Schemas are in `schemas/emits/`, registered by core's autoloader and validated
on every emit. Only these two — an event per *occurrence* would put a failing
loop's whole traffic on the bus and make every subscriber re-derive the
grouping this module already did.

`alerts.issue.regressed` carries the release that claimed the fix next to the
release that is running, which is the pair that separates "the fix is wrong"
from "the fix is not deployed".

The emit happens **after** the issue has committed, in a transaction of its
own — not in the ingest's atomic block, where the rest of the fleet emits.
`emit` marks its transaction rollback-only when it fails, so emitting inside
the ingest would mean a broken outbox *deletes* the alert. The bus is the
thing whose failures this store records; it has to record them on the day the
bus is what is broken.

## Sentry

Set `SENTRY_DSN` (or `STAPEL_ALERTS["SENTRY_DSN"]`) and install the extra
(`pip install 'stapel-alerts[sentry]'`): each event is forwarded and its
Sentry id stored back on the row. Storing locally is never conditional on it.

## Retention and GDPR

`manage.py alerts_sweep` drops events past their level's retention and closed
issues with nothing left. An **open** issue is never swept, however old:
deleting it would turn "unresolved" into "never happened". `erase_subject`
(wired into the GDPR provider registry) drops the `user_id` link and scrubs
context that repeats it, and keeps the failure — an alert store's rows are the
record of a system failing, not a record about a person.

## Metrics

`alerts_new_total{level,service}` (counter) and `alerts_open_total{level,service}`
(gauge, declared at zero for every known pair, because a series that has never
existed cannot be alerted on).

## License

MIT — see [LICENSE](https://github.com/usestapel/stapel-alerts/blob/main/LICENSE).

---

<sub>This page is assembled by `stapel-readme` from `docs/readme.md` plus the contract artifacts in `docs/`. Edit the prose in `docs/readme.md`; the badges, facts and links above and below it are generated — do not hand-edit `README.md`.</sub>
