Metadata-Version: 2.5
Name: corpus-canary
Version: 1.0.0
Summary: Open-source RAG corpus health checker. No API key needed. Runs locally.
Project-URL: Homepage, https://github.com/oragen-ai/corpus-canary
Project-URL: Repository, https://github.com/oragen-ai/corpus-canary
Project-URL: Issues, https://github.com/oragen-ai/corpus-canary/issues
Author: oragen
License: Apache-2.0
License-File: LICENSE
Keywords: embeddings,evaluation,hallucination-detection,llm,rag,retrieval-augmented-generation,vector-database
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.20
Requires-Dist: numpy>=1.24
Requires-Dist: pydantic>=2.5
Requires-Dist: rich>=13.0
Requires-Dist: scikit-learn>=1.3
Requires-Dist: sentence-transformers>=2.5
Requires-Dist: transformers<5.0,>=4.36
Requires-Dist: typer>=0.9
Provides-Extra: all
Requires-Dist: anthropic>=0.18; extra == 'all'
Requires-Dist: ollama>=0.4; extra == 'all'
Requires-Dist: openai>=1.10; extra == 'all'
Requires-Dist: pgvector>=0.2; extra == 'all'
Requires-Dist: pinecone>=5.0; extra == 'all'
Requires-Dist: plotly>=5.18; extra == 'all'
Requires-Dist: psycopg[binary]>=3.1; extra == 'all'
Requires-Dist: qdrant-client>=1.7; extra == 'all'
Requires-Dist: streamlit>=1.30; extra == 'all'
Requires-Dist: torch>=2.1; extra == 'all'
Provides-Extra: dev
Requires-Dist: bandit>=1.9; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: gpu
Requires-Dist: torch>=2.1; extra == 'gpu'
Provides-Extra: ollama
Requires-Dist: ollama>=0.4; extra == 'ollama'
Provides-Extra: pgvector
Requires-Dist: pgvector>=0.2; extra == 'pgvector'
Requires-Dist: psycopg[binary]>=3.1; extra == 'pgvector'
Provides-Extra: pinecone
Requires-Dist: pinecone>=5.0; extra == 'pinecone'
Provides-Extra: premium
Requires-Dist: anthropic>=0.18; extra == 'premium'
Requires-Dist: openai>=1.10; extra == 'premium'
Provides-Extra: qdrant
Requires-Dist: qdrant-client>=1.7; extra == 'qdrant'
Provides-Extra: release
Requires-Dist: build>=1.2; extra == 'release'
Requires-Dist: twine>=5.0; extra == 'release'
Provides-Extra: tracing
Provides-Extra: viz
Requires-Dist: plotly>=5.18; extra == 'viz'
Requires-Dist: streamlit>=1.30; extra == 'viz'
Description-Content-Type: text/markdown

# corpus-canary

**Open-source RAG corpus health checker. No API key needed. Runs locally.**

```bash
pip install corpus-canary
```

RAG corpora rot quietly. Documents get rewritten, chunks stop getting retrieved, new questions show up that nothing in the corpus answers, and answers start drifting from what's actually in the index — usually nobody notices until users complain. corpus-canary calls this **Knowledge Decay**, and audits it by connecting **read-only** to your vector store. It never deletes, upserts, or reindexes anything — it only observes and reports.

It detects four independent ways a corpus decays:

- **Obsolescence** — chunks that are dead weight (never retrieved) or missing the metadata needed to even check
- **Coverage Gaps** — topics your users ask about that nothing in the corpus answers well
- **Contradictions** — chunk pairs that semantically disagree with each other
- **Faithfulness** — RAG answers that aren't actually grounded in the chunks retrieved for them

Every detector runs locally using open-source, Apache 2.0 Hugging Face models — `pip install corpus-canary` and you can run a full scan without ever creating an account or pasting in a secret. CPU always works with zero prerequisites; a GPU is auto-detected and used if present (see [GPU acceleration](#gpu-acceleration) below).

## Install

```bash
pip install "corpus-canary[qdrant]"    # or [pinecone], [pgvector]
```

Each vector store integration is an optional extra, so you only pull in the client library you actually need. See the connector docs: [Pinecone](docs/connectors/pinecone.mdx), [Qdrant](docs/connectors/qdrant.mdx), [pgvector](docs/connectors/pgvector.mdx).

## Docker

Prefer a container over `pip install`? The official image is an offline/air-gapped alternative — no Python environment to manage, and the `fat` variant works with **zero network access from the moment you build it**.

| Variant | Size | Models |
| --- | --- | --- |
| `fat` | ~1.6GB | Pre-baked (HHEM-2.1, MiniLM, DeBERTa-NLI) — true offline |
| `slim` | ~430MB | Downloaded on first run — mount a volume to persist them |

```bash
# fat: fully offline, no network access at all
docker build --target runtime-fat -t corpus-canary:fat .
docker run --rm --network none corpus-canary:fat corpus-canary scan --help

# real scan against a vector store reachable from the host -- mount a
# *directory* (not a single file), writable by the container's uid 1000,
# for both the input log and the output report:
mkdir -p ./data && chmod 777 ./data && cp query_log.jsonl ./data/
docker run --rm --network host -v "$(pwd)/data:/data" \
  corpus-canary:fat corpus-canary scan --connector qdrant --url http://localhost:6333 \
  --query-log /data/query_log.jsonl --output /data/report.md
```

```bash
# slim: smaller image, models download into the mounted volumes on first run.
# Two volumes are needed: the app's own model cache, and HF_HOME (where the
# HHEM model's dynamic module code caches separately from its weights).
docker build --target runtime-slim -t corpus-canary:slim .
docker run --rm --network host \
  -v corpus-canary-models:/home/canary/.cache/corpus-canary \
  -v corpus-canary-hf:/home/canary/.cache/huggingface \
  -v "$(pwd)/data:/data" \
  corpus-canary:slim corpus-canary scan --connector qdrant --url http://localhost:6333 \
  --query-log /data/query_log.jsonl --output /data/report.md
```

A few things worth knowing: these images ship **CPU-only** torch to stay under the size budget, so GPU acceleration (see below) isn't available inside Docker — use `pip install "corpus-canary[gpu]"` for that. The image isn't published to a registry yet — build it locally from this repo with the commands above. And `--network host` (used above to reach a vector store on the host) is Linux-only — on Docker Desktop (Mac/Windows) use a bridge network with a service alias instead (e.g. `docker network create cc-net`, run your vector store on it with `--network cc-net`, then run corpus-canary on the same network and address it by container name instead of `localhost`).

## Run your first scan

```bash
corpus-canary scan --connector qdrant --url http://localhost:6333 --collection docs
```

No query log, no trace log, no API keys at all. This runs `ContradictionDetector` (always on) and `ObsolescenceDetector` in its `missing_metadata`-only mode, and writes a Markdown report with the overall **Decay Score** — a single 0-100 number summarizing corpus health.

Add `--query-log`/`--trace-log` to unlock the other two detectors:

| Install level | What works | Decay Score components |
| --- | --- | --- |
| Vector store only | Contradiction + Obsolescence (`missing_metadata` only) | 2/4 dimensions, partial signal |
| + `query_log` | Adds Coverage Gap | 3/4 dimensions |
| + `trace_log` | Adds Faithfulness + Obsolescence (`dead_weight`) | 4/4 dimensions, complete |

Don't have a `trace_log` yet? corpus-canary **includes a tracing helper to instrument your pipeline in 3 lines**:

```python
from corpus_canary.tracing import RAGTracer

tracer = RAGTracer(output_file="trace_log.jsonl")
tracer.emit(query=q, retrieved_chunks=[...], answer=a)
```

No network calls, no monkey-patching — see [docs/tracing.mdx](docs/tracing.mdx) for both forms (`emit()` and the `span()` context manager).

## Explore a report visually

corpus-canary **includes a local viewer to explore your RAG report** — a Decay Score gauge plus one tab per detector, reading a `report.json` file you already generated:

```bash
pip install "corpus-canary[viz]"
corpus-canary view report.json
```

![viewer demo](docs/assets/viewer-demo.gif)

*(Demo GIF not yet recorded — see [docs/RECORDING.md](docs/RECORDING.md) for how to produce it.)*

## GPU acceleration

The embedding/NLI/HHEM models used by `ContradictionDetector`, `CoverageGapDetector`, and `FaithfulnessDetector` auto-detect a GPU (CUDA, then Apple MPS) and use it if present — CPU always works with zero prerequisites, so nothing extra is required to get started:

```bash
corpus-canary scan --connector qdrant --device auto   # default: CUDA -> MPS -> CPU
corpus-canary scan --connector qdrant --device cuda    # force CUDA (falls back to CPU with a warning if unavailable, never errors)
corpus-canary scan --connector qdrant --device cpu     # force CPU
```

`torch` is already a transitive dependency (via `sentence-transformers`/`transformers`); `pip install "corpus-canary[gpu]"` just pins a modern `torch` version explicitly as an opt-in signal — no CUDA dependency is ever added to the default install.

## Full documentation

The [docs/](docs/quickstart.mdx) directory has the complete reference: a [quickstart](docs/quickstart.mdx), one page per [detector](docs/detectors/) and [connector](docs/connectors/), [tracing](docs/tracing.mdx), the [viewer](docs/viewer.mdx), an [FAQ](docs/faq.mdx), and [model licenses](docs/licenses.mdx).

## License

Apache 2.0 — see [LICENSE](LICENSE). Every model corpus-canary downloads is Apache 2.0 too; see [docs/licenses.mdx](docs/licenses.mdx).
