Metadata-Version: 2.4
Name: llm-driftscope
Version: 0.1.0
Summary: Local-first semantic drift detection for LLM outputs.
Project-URL: Homepage, https://github.com/Sreechandh22/Driftscope
Project-URL: Repository, https://github.com/Sreechandh22/Driftscope
Project-URL: Issues, https://github.com/Sreechandh22/Driftscope/issues
Author: Sreechandh Devireddy
License: MIT
Keywords: drift,embeddings,llm,monitoring,observability
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Requires-Dist: click>=8.1
Requires-Dist: numpy>=1.23
Provides-Extra: changepoints
Requires-Dist: ruptures>=1.1; extra == 'changepoints'
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Provides-Extra: embeddings
Requires-Dist: sentence-transformers>=2.7; extra == 'embeddings'
Provides-Extra: full
Requires-Dist: fastapi>=0.110; extra == 'full'
Requires-Dist: hdbscan>=0.8; extra == 'full'
Requires-Dist: ruptures>=1.1; extra == 'full'
Requires-Dist: sentence-transformers>=2.7; extra == 'full'
Requires-Dist: uvicorn>=0.27; extra == 'full'
Provides-Extra: server
Requires-Dist: fastapi>=0.110; extra == 'server'
Requires-Dist: uvicorn>=0.27; extra == 'server'
Provides-Extra: topics
Requires-Dist: hdbscan>=0.8; extra == 'topics'
Description-Content-Type: text/markdown

# DriftScope

DriftScope is a local-first Python library for detecting semantic drift in LLM outputs. It wraps your model calls, stores prompts and responses in SQLite, embeds outputs locally, calibrates a baseline from your own traffic, and reports when later outputs move outside the learned normal range.

No hosted dashboard. No external data pipeline. No cloud account required.

## Install

```bash
pip install llm-driftscope
```

For sentence-transformer embeddings:

```bash
pip install "llm-driftscope[embeddings]"
```

## Quick Start

```python
from driftscope import monitor

@monitor(session="support-bot")
def call_llm(prompt: str) -> str:
    return my_llm_client.chat(prompt)

answer = call_llm("How do I reset my password?")
```

By default, events are stored at `.driftscope/driftscope.sqlite3`.

Generate a local report:

```bash
driftscope report --session support-bot
```

Try the included drift simulation:

```bash
python examples/simulate_drift.py
```

Sample output:

```text
DriftScope report

Session: demo
Samples: 9
Status: drift
Latest score: 1.0015
Latest distance: 1.0202
Threshold: 0.2229
Topic: warming up; topic drift needs two windows
Changepoint: warming up; changepoint detection needs more samples
Drift events: 3
```

## Why DriftScope Exists

Infrastructure metrics can tell you that a service is healthy. They cannot tell you that an LLM has quietly started answering in a different semantic region than it used to. Hosted observability platforms can help, but many developers need a small embeddable library that works inside an existing codebase and keeps data local.

DriftScope focuses on that gap.

## Methodology

Fixed distance thresholds are brittle. A customer support bot should usually be consistent. A creative writing assistant naturally has wider semantic variation. DriftScope avoids hardcoded assumptions by calibrating against each application's own warmup data.

During warmup, DriftScope stores output embeddings and estimates the empirical distribution of distances from the baseline centroid. Once enough samples exist, it sets the alert threshold from a high percentile of that null distribution. New outputs are compared against the centroid, smoothed with EWMA, and flagged when the smoothed drift score crosses the calibrated threshold.

The v1 detector reports:

- raw semantic distance from the baseline
- EWMA-smoothed drift score
- calibrated threshold
- drift status
- warmup progress
- topic distribution drift
- changepoint detection over semantic distance

## API

```python
from driftscope import DriftConfig, DriftScope

config = DriftConfig(
    warmup_samples=50,
    threshold_percentile=99,
    ewma_alpha=0.3,
)

scope = DriftScope(session="my-app", config=config)
result = scope.record(prompt="hello", output="world")

if result.drifted:
    print(result.score, result.threshold)
```

Decorator form:

```python
from driftscope import monitor

@monitor(session="my-app")
def generate(prompt: str) -> str:
    return "model output"
```

## CLI

```bash
driftscope report
driftscope report --session my-app
driftscope report --db .driftscope/driftscope.sqlite3
driftscope report --json
driftscope sessions
driftscope export --format csv --out events.csv
```

Optional sidecar API:

```bash
pip install "llm-driftscope[server]"
driftscope serve
```

The sidecar exposes `GET /health`, `POST /events`, and `GET /report`.

## Optional Extras

```bash
pip install "llm-driftscope[topics]"
pip install "llm-driftscope[changepoints]"
pip install "llm-driftscope[server]"
pip install "llm-driftscope[full]"
```

## Development

```bash
pip install -e ".[dev]"
python -m pytest
python -m build
```
