Metadata-Version: 2.5
Name: promptbudget
Version: 0.1.0
Summary: Deterministic context-window budgeting for LLM prompts, with a full audit trail of every token decision.
Project-URL: Homepage, https://github.com/mathewOracle/promptbudget
Project-URL: Repository, https://github.com/mathewOracle/promptbudget
Project-URL: Issues, https://github.com/mathewOracle/promptbudget/issues
Author-email: Mathew Kadambatt <49642721+mathewOracle@users.noreply.github.com>
License: MIT
License-File: LICENSE
Keywords: ai,budget,context-window,llm,prompt,prompt-engineering,rag,tokenizer,tokens
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.9
Provides-Extra: dev
Requires-Dist: hypothesis>=6.0; extra == 'dev'
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: tiktoken
Requires-Dist: tiktoken>=0.5; extra == 'tiktoken'
Description-Content-Type: text/markdown

# promptbudget

**Deterministic context-window budgeting for LLM prompts — with a full audit trail of every token decision.**

[![CI](https://github.com/mathewOracle/promptbudget/actions/workflows/ci.yml/badge.svg)](https://github.com/mathewOracle/promptbudget/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/promptbudget)](https://pypi.org/project/promptbudget/)
[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](https://pypi.org/project/promptbudget/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
[![Typed](https://img.shields.io/badge/typing-strict-blue)](https://peps.python.org/pep-0561/)

Your system prompt, tool schemas, chat history and retrieved chunks are all competing
for the same context window. Most codebases resolve that fight with a `while
total > limit: messages.pop(0)` loop buried in a helper — which silently evicts the
one document the answer depended on, and leaves you no way to find out.

`promptbudget` makes that allocation an explicit, testable, auditable decision.

```python
from promptbudget import Section, Item, pack

result = pack(
    [
        Section("system", SYSTEM_PROMPT, priority=100, strategy="all_or_nothing"),
        Section("tools", tool_schemas, priority=90, min_tokens=2_000),
        Section("history", messages, priority=50, strategy="drop_oldest"),
        Section("retrieval", chunks, priority=40, strategy="lowest_score_first"),
        Section("scratchpad", notes, priority=10, max_tokens=1_000),
    ],
    budget=128_000,
    reserved_output=4_000,
)

send(result.text)

for d in result.dropped():
    log.info("%s[%d] %s: %s", d.section, d.index, d.action.value, d.reason)
```

```
retrieval[7] dropped: lowest relevance score, evicted first to fit the section allocation
scratchpad[0] truncated: cut to the tokens remaining in the section allocation
```

## Why this exists

| The usual approach | What goes wrong |
|---|---|
| `messages[-10:]` | Breaks the moment one message is huge |
| `while tokens > limit: pop(0)` | No priorities, no guarantees, drops pinned context |
| Framework-native trimming | Handles chat history only — not tools, RAG chunks and history *together* |
| Retry on `context_length_exceeded` | Pays for a failed call, and still doesn't decide *what* to shed |
| All of the above | Cannot answer "why is that chunk missing from my prompt?" |

Existing tools each cover a slice: LangChain's `trim_messages` trims message lists,
LLMLingua compresses text, `tiktoken` counts tokens. None of them arbitrate a fixed
budget across several sections with declared guarantees and tell you what they did.

## Install

```bash
pip install promptbudget                 # zero dependencies
pip install "promptbudget[tiktoken]"     # exact OpenAI-family token counts
```

Python 3.9+. The core has **no runtime dependencies**.

## Core ideas

### Sections compete; priority decides who wins

Sections are served highest priority first. Ties break on name, so identical inputs
always produce an identical prompt — the same reason you pin dependency versions.

Rendering follows *your* list order, not priority order. Priority decides who gets
tokens; you decide what the prompt reads like.

### Guarantees are kept or the call fails

```python
Section("tools", tool_schemas, priority=90, min_tokens=2_000)
```

`min_tokens` is reserved before any lower-priority section is served — a greedy
high-priority section cannot eat tokens promised downstream. If the minimums
collectively don't fit, you get `BudgetTooSmallError` rather than a prompt that
quietly violates one:

```
BudgetTooSmallError: Guaranteed minimums require 6000 tokens but only 4000 are
available (budget=8000, reserved_output=4000). Lower a min_tokens value, raise
the budget, or reduce reserved_output.
```

`max_tokens` is the mirror image: a cap, so a runaway history can't crowd out retrieval
even when the window is empty.

### Reserve room for the reply

```python
pack(sections, budget=128_000, reserved_output=4_000)
```

Forgetting the model's own output is the most common way to get a context-length
error from a prompt that "obviously fits". Reserved tokens are never offered to a section.

### Strategies decide what gets shed

| Strategy | Behaviour | Use for |
|---|---|---|
| `"truncate"` *(default)* | Cuts the tail of the overflowing item | Prose, scratchpads |
| `"drop_oldest"` | Evicts from the front | Chat history |
| `"drop_newest"` | Evicts from the back | Append-only logs |
| `"lowest_score_first"` | Evicts least-relevant items | RAG chunks |
| `"all_or_nothing"` | Delivers intact or raises | System prompts, tool schemas |

Pin anything that must survive:

```python
Item("The user's actual question", pinned=True)
```

Pinned items are never dropped or cut. If a pinned item genuinely cannot fit, you get
`SectionOverflowError` — a loud failure beats a mangled prompt.

Bring your own by subclassing `Strategy`:

```python
from promptbudget import Strategy, StrategyResult, register_strategy


class DeduplicateThenTruncate(Strategy):
    name = "dedupe"

    def apply(
        self, section, items, allocated, tokenizer, separator
    ): ...  # return StrategyResult(items, decisions)


register_strategy("dedupe", DeduplicateThenTruncate())
```

### Any tokenizer, or none

```python
from promptbudget import TiktokenTokenizer, pack

pack(sections, budget=128_000, tokenizer=TiktokenTokenizer("gpt-4o"))
pack(sections, budget=8_000, tokenizer=lambda text: len(text) // 4)  # any str -> int
pack(sections, budget=8_000)  # built-in heuristic
```

The allocator never imports a tokenizer directly — it talks to the `Tokenizer`
interface, so HuggingFace, a provider SDK, or your own estimator all drop straight in.

### The result explains itself

```python
result.text  # the fitted prompt
result.used_tokens  # what it actually costs
result.remaining_tokens  # headroom left
result.complete  # True when nothing was lost
result.dropped()  # every item that lost content, and why
result.section("rag")  # per-section accounting
result.to_json(indent=2)  # structured logging, ready to ship
```

Because packing is a pure function returning a plain value, you can assert on it:

```python
def test_the_user_question_always_survives():
    result = pack(sections, budget=4_000)
    assert "user_question" not in {d.section for d in result.dropped()}
```

## Guarantees

Verified by property-based tests across hundreds of generated inputs
([`tests/test_properties.py`](tests/test_properties.py)):

- The output **never** exceeds the budget.
- Reserved output tokens are **never** consumed.
- `max_tokens` caps are **never** breached.
- Identical inputs produce **byte-identical** prompts.
- Every item receives **exactly one** decision — nothing vanishes unrecorded.
- `complete` never lies about dropped content.

## Development

```bash
git clone https://github.com/mathewOracle/promptbudget && cd promptbudget
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytest && ruff check . && mypy
```

Contributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). New strategies are
especially welcome, and shouldn't require touching the allocator at all.

See [CHANGELOG.md](CHANGELOG.md) for release history and
[RELEASING.md](RELEASING.md) for how versions get published.

## License

MIT — see [LICENSE](LICENSE).
