Metadata-Version: 2.4
Name: token-overflow
Version: 0.1.0
Summary: Prevent LLM context window overflow with token counting, smart truncation, and budget management
Author-email: Zach <zacharie@astera.org>
License: MIT
Project-URL: Homepage, https://github.com/zachbg/token-overflow
Keywords: llm,tokens,context-window,overflow,truncation,openai,tiktoken
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# token-overflow

Prevent LLM context window overflow. Count tokens, enforce budgets, and smart-truncate prompts before they silently fail.

## The Problem

LLM APIs silently truncate or error when you exceed context windows. You don't know until you get garbage output or a 400 error. `token-overflow` catches this **before** the API call.

## Install

```bash
pip install token-overflow
```

## Quick Start

```python
from token_overflow import TokenGuard, fit

# Simple: fit text into a budget
short = fit("very long document...", max_tokens=1000)

# Guard a full prompt
guard = TokenGuard(model="gpt-4")  # auto-detects 8192 limit

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": huge_document},
]

# Check before calling API
if guard.would_overflow(messages):
    messages = guard.truncate(messages, strategy="tail")

# Or one-liner: safe messages guaranteed to fit
safe = guard.safe(messages)
```

## Token Counting

```python
from token_overflow import count_tokens, count_messages

# Count tokens in text
n = count_tokens("Hello world", model="gpt-4")  # 2

# Count tokens in chat messages
n = count_messages([
    {"role": "system", "content": "You are helpful."},
    {"role": "user", "content": "Hi"},
], model="gpt-4")
```

## Truncation Strategies

```python
guard = TokenGuard(model="gpt-4")

# Keep the tail (most recent), trim from head
guard.truncate(messages, strategy="tail")

# Keep the head (system prompt), trim from tail
guard.truncate(messages, strategy="head")

# Keep head and tail, trim middle
guard.truncate(messages, strategy="middle")

# Custom reserve: keep 500 tokens for response
guard.truncate(messages, strategy="tail", reserve=500)
```

## Model Limits

```python
from token_overflow import get_limit

get_limit("gpt-4")           # 8192
get_limit("gpt-4-turbo")     # 128000
get_limit("gpt-3.5-turbo")   # 16385
get_limit("claude-3-opus")   # 200000
get_limit("claude-3-sonnet") # 200000
```

## Features

- **Zero config**: Knows context limits for 30+ models
- **Multiple strategies**: head, tail, middle truncation
- **Message-aware**: Understands chat message format, preserves system prompts
- **Reserve tokens**: Leave room for the response
- **Fast counting**: Uses tiktoken when available, falls back to approximation
- **No dependencies required**: tiktoken optional for exact counts

## License

MIT
