Metadata-Version: 2.4
Name: patternbreak
Version: 0.2.0
Summary: Ratio-band sampling: force lateral thinking in LLMs by sampling from competitive-but-not-top token bands
Author: Ned Geeslin
License-Expression: MIT
Project-URL: Homepage, https://github.com/Ngeeslin/patternbreak
Project-URL: Source, https://github.com/Ngeeslin/patternbreak
Keywords: llm,sampling,lateral-thinking,logits,inference
Classifier: Development Status :: 3 - Alpha
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Provides-Extra: hf
Requires-Dist: transformers>=4.35.0; extra == "hf"
Provides-Extra: llamacpp
Requires-Dist: llama-cpp-python>=0.2.0; extra == "llamacpp"
Provides-Extra: vllm
Requires-Dist: vllm>=0.3.0; extra == "vllm"
Dynamic: license-file

# patternbreak

Force lateral thinking in LLMs by sampling from the model's "competitive but not top" token band during reasoning.

## How it works

At each decoding step, patternbreak looks at the probability distribution over tokens. When the model has several tokens that are *close-ish* in probability — a "shallow drop-off" — it suppresses both the obvious top choices and the long tail, forcing the model to pick from the middle: tokens it considered plausible but wouldn't normally choose.

This is controlled by two knobs:

- **`ratio_top`** (default 0.3) — upper bound. Tokens above this fraction of the top token's probability are suppressed.
- **`ratio_bottom`** (default 0.05) — lower bound. Tokens below this are suppressed.

With defaults, the model samples from tokens that are 5-30% as probable as its top choice. No warmup, no calibration, no corpus-specific tuning — it adapts naturally per decoding step.

By default, interventions stop after `</think>` is generated (for reasoning models like DeepSeek-R1), leaving the final answer untouched. This is configurable — set `think_end=None` to intervene on all tokens, or use a custom tag like `think_end="</reasoning>"` for other models.

## Install

```bash
pip install patternbreak              # core only (numpy)
pip install patternbreak[hf]          # + HuggingFace transformers
pip install patternbreak[llamacpp]    # + llama-cpp-python
pip install patternbreak[vllm]        # + vLLM
```

## Usage

### HuggingFace Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from patternbreak.adapters.hf import HFProcessor

model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-7B")
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-7B")

pb = HFProcessor(tokenizer)  # one line

output = model.generate(
    **tokenizer("Your prompt", return_tensors="pt"),
    logits_processor=[pb],
    max_new_tokens=512,
)
```

### llama-cpp-python

```python
from llama_cpp import Llama, LogitsProcessorList
from patternbreak.adapters.llamacpp import LlamaCppProcessor

model = Llama("model.gguf", n_gpu_layers=-1)
pb = LlamaCppProcessor(model)  # one line

output = model(
    "Your prompt",
    logits_processor=LogitsProcessorList([pb]),
    max_tokens=512,
)
```

### vLLM

```python
from vllm import LLM, SamplingParams
from patternbreak.adapters.vllm import VLLMProcessor

llm = LLM(model="deepseek-ai/DeepSeek-R1-Distill-Qwen-7B")
pb = VLLMProcessor(llm.get_tokenizer())  # one line

output = llm.generate("Your prompt", SamplingParams(logits_processors=[pb]))
```

## Configuration

```python
from patternbreak import ProcessorConfig

config = ProcessorConfig(
    ratio_top=0.3,        # suppress tokens above 30% of top prob
    ratio_bottom=0.05,    # suppress tokens below 5% of top prob
    min_candidates=3,     # need at least 3 tokens in band to trigger
    temperature=1.0,      # temperature applied to surviving tokens
    think_end="</think>", # stop after this tag (None = intervene on all tokens)
    debug=False,          # set True to log interventions to stdout
)

pb = HFProcessor(tokenizer, config=config)
```

### Preset ideas

| Style | ratio_top | ratio_bottom | min_candidates | Effect |
|-------|-----------|--------------|----------------|--------|
| Default | 0.3 | 0.05 | 3 | Good balance of divergence and coherence |
| Conservative | 0.2 | 0.08 | 4 | Fewer, more targeted interventions |
| Aggressive | 0.4 | 0.03 | 2 | More frequent, deeper interventions |
| Narrow | 0.2 | 0.1 | 3 | Tight band, very selective |
| Wide | 0.5 | 0.02 | 3 | Broad band, more chaotic |

## How the knobs map to intuition

- **`ratio_top=0.3`** means "suppress anything the model considers more than 30% as likely as its top choice" — this removes the obvious answers
- **`ratio_bottom=0.05`** means "suppress anything below 5% of top" — this removes noise
- What's left is the *competitive fringe*: tokens the model thought about but wouldn't normally pick
- **`min_candidates=3`** means "only intervene when there are at least 3 tokens in this band" — if the band is empty or too small, the model generates normally

## Core API

If you're building a custom integration:

```python
from patternbreak import PatternBreaker, ProcessorConfig
import numpy as np

breaker = PatternBreaker(ProcessorConfig(debug=False))

# Each decoding step:
breaker.update_think_state(decoded_text_so_far)
modified_logits = breaker.process(logits_np, tail_text, decode_fn)

# Check stats after generation:
print(f"{breaker.intervention_count} interventions over {breaker.token_count} tokens")
```

The core `PatternBreaker` class has zero framework dependencies — just numpy. Adapters are thin wrappers that handle tensor conversion.
