Metadata-Version: 2.4
Name: chunk-bench
Version: 1.0.0
Summary: Benchmark RAG chunking strategies on your own documents. Compare fixed, sliding, paragraph, recursive, and semantic chunking with real retrieval metrics.
Author: Linda Oraegbunam
License: MIT
Project-URL: Homepage, https://github.com/obielin/chunk-bench
Keywords: rag,chunking,benchmark,retrieval,nlp,llm
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: black>=24.0.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"
Dynamic: license-file

# chunk-bench

**Benchmark RAG chunking strategies on your own documents. Compare fixed, sliding, paragraph, and recursive chunking with real retrieval metrics.**

[![Tests](https://img.shields.io/badge/Tests-48%20passing-brightgreen?style=flat-square)](tests/)
[![Dependencies](https://img.shields.io/badge/Dependencies-zero-brightgreen?style=flat-square)](pyproject.toml)
[![Python](https://img.shields.io/badge/Python-3.10%2B-blue?style=flat-square)](pyproject.toml)
[![License](https://img.shields.io/badge/License-MIT-green?style=flat-square)](LICENSE)
[![LinkedIn](https://img.shields.io/badge/-Linda_Oraegbunam-blue?logo=linkedin&style=flat-square)](https://www.linkedin.com/in/linda-oraegbunam/)

---

## Why chunk-bench?

Chunking strategy is one of the highest-impact decisions in a RAG pipeline — 2026 benchmarks show up to a 9% recall gap between best and worst strategy on the same corpus. But most teams pick a strategy once and never measure whether it's actually working for their documents.

`chunk-bench` lets you run all four strategies against your own text and queries in one call, and get back recall, precision, and MRR scores — no embeddings required, no external services.

---

## Install

```bash
pip install chunk-bench
```

---

## Quick start

```python
from chunk_bench import ChunkBench

bench = ChunkBench()

report = bench.run(
    text=your_document_text,
    queries=["What is retrieval augmented generation?",
             "How does chunking affect retrieval quality?",
             "What is the difference between fixed and semantic chunking?"],
)

print(report.summary_table())
```

Output:
```
Strategy           Chunks  Avg Tokens   Recall  Precision     MRR      F1
────────────────────────────────────────────────────────────────────────
recursive               18         94    0.857      0.733   0.833   0.790
paragraph               12        142    0.810      0.700   0.810   0.752
fixed                   24         71    0.762      0.667   0.762   0.711
sliding                 31         55    0.714      0.633   0.714   0.671
────────────────────────────────────────────────────────────────────────

  Best recall:    recursive (0.857)
  Best F1:        recursive (0.790)
  Best MRR:       recursive (0.833)
```

---

## Providing your own relevance terms

```python
report = bench.run(
    text=your_document,
    queries=["What is GDPR?", "What are data subject rights?"],
    relevant_terms=[
        ["GDPR", "General Data Protection", "regulation"],
        ["subject", "rights", "access", "erasure", "portability"],
    ],
    top_k=5,
)
```

---

## Use specific strategies

```python
report = bench.run(
    text=text,
    queries=queries,
    strategies=["fixed", "recursive"],  # skip sliding and paragraph
    chunk_size=512,
    overlap=50,
)
```

---

## Chunk any text directly

```python
from chunk_bench import chunk

chunks = chunk(text, strategy="recursive", chunk_size=512, overlap=50)
for c in chunks:
    print(f"[{c.index}] ~{c.token_count} tokens: {c.text[:60]}")
```

---

## CLI

```bash
chunk-bench document.txt --queries "What is X?" "How does Y work?" --strategies fixed recursive --json
```

---

## Strategies

| Strategy | Description | Best for |
|---|---|---|
| `fixed` | Split at regular character intervals with overlap | Simple, uniform documents |
| `sliding` | Overlapping windows stepping forward | When context preservation matters most |
| `paragraph` | Split on double newlines, merge small paragraphs | Structured documents with clear sections |
| `recursive` | Try paragraph → sentence → word boundaries | Most document types — good default |

---

**Linda Oraegbunam** | [LinkedIn](https://www.linkedin.com/in/linda-oraegbunam/) | [Twitter](https://twitter.com/Obie_Linda) | [GitHub](https://github.com/obielin)
