Metadata-Version: 2.5
Name: interp-engine
Version: 1.3.0
Summary: A fast, standardized interpretability engine that supports most modern models and architectures. Powers Neuronpedia.
Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
Project-URL: Repository, https://github.com/decoderesearch/interp-engine
Project-URL: Issues, https://github.com/decoderesearch/interp-engine/issues
License-Expression: Apache-2.0
License-File: LICENSE
Requires-Python: <3.14,>=3.11
Requires-Dist: einops
Requires-Dist: numpy>=1.24
Requires-Dist: torch>=1.10
Requires-Dist: transformers>=4.57.1
Provides-Extra: awq
Requires-Dist: accelerate>=1.0; extra == 'awq'
Requires-Dist: gptqmodel>=5.0; extra == 'awq'
Provides-Extra: dev
Requires-Dist: pyright<1.2,>=1.1.411; extra == 'dev'
Requires-Dist: pytest<9,>=8.3.1; extra == 'dev'
Requires-Dist: pyyaml>=6; extra == 'dev'
Requires-Dist: ruff<0.17,>=0.16.2; extra == 'dev'
Provides-Extra: parity
Requires-Dist: transformer-lens>=3.0; extra == 'parity'
Provides-Extra: quant
Requires-Dist: accelerate>=1.0; extra == 'quant'
Requires-Dist: kernels<0.17.0,>=0.15.2; extra == 'quant'
Provides-Extra: vllm
Requires-Dist: vllm>=0.27.1; (sys_platform == 'linux') and extra == 'vllm'
Description-Content-Type: text/markdown

# interp-engine

<p align="center">
  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ielogo.png" alt="interp-engine logo, a magnifying glass where the handle is a rocket" width="160">
</p>
<p align="center">
  🔗 <a href="https://interp-engine.org"><strong>interp-engine.org</strong></a>
</p>
<p align="center">
  <a href="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml"><img src="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml/badge.svg?branch=main" alt="CI status"></a>
  <a href="https://pypi.org/project/interp-engine/"><img src="https://img.shields.io/pypi/v/interp-engine.svg" alt="PyPI version"></a>
  <a href="LICENSE"><img src="https://img.shields.io/pypi/l/interp-engine.svg" alt="Apache-2.0 license"></a>
  <a href="https://join.slack.com/t/opensourcemechanistic/shared_invite/zt-3z9o0hxjl-MDX9pbATO2qESOazNDLpdQ"><img src="https://img.shields.io/badge/Slack-Open%20Source%20Mechanistic%20Interpretability-4A154B?logo=slack&logoColor=white" alt="Join the Slack"></a>
</p>


`interp-engine` is an interpretability engine that is fast (>40x tok/s vs HF eager), standardized (34 'points'/addresses across architectures), and easy to use and debug. It powers all of [Neuronpedia](https://neuronpedia.org)'s inference and is checked for accuracy against HF Transformers and other engines.

<p align="center">
  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ie-perf.png" alt="Tokens per second while capturing and generating: eager against IE-vLLM and IE-vLLM-static on qwen3.8-27b and deepseek-v4-flash-0731, 8 requests in flight" width="100%">
</p>
<p align="center">
  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/interp-engine-demo.gif" alt="interp-engine demo gif" width="100%">
</p>

This repo contains:

1. [`validator/`](validator/), which compares/validates it against TransformerLens, and nnsight/nnterp on real architectures.
2. [`visualizer-web/`](visualizer-web/), a "cheat sheet" hosted at [interp-engine.org](https://interp-engine.org) of each 'point' (eg `resid_post.16`), standardized across architectures.

## Installation

```bash
pip install 'interp-engine[vllm]'  # preferred install: includes vLLM support (CUDA required)
pip install interp-engine          # eager backend only
```

## Simple Usage

```python
from interp_engine import Address, load_model, run_with_cache

# VLLM (default): low VRAM, medium speed, every point, chosen per request
model = load_model("Qwen/Qwen3-8B")

# VLLM-STATIC: high VRAM, high speed, only the points you declare (default resid_post)
# model = load_model("Qwen/Qwen3-8B", backend="vllm-static")

# VLLM-GENERATE: fastest, generation only -- no capture, no steering
# model = load_model("Qwen/Qwen3-8B", backend="vllm-generate")

# EAGER: low VRAM, low speed
# model = load_model("Qwen/Qwen3-8B", backend="eager")

point = Address("resid_post", 10)      # or string: "resid_post.10"
cache = run_with_cache(model, model.to_tokens("Hello, world"), [point])
cache[point]  # [batch, pos, ...]
```

### AI Agents

Add "use interp-engine" to your prompt and let your agent figure it out - everything is fully documented in this repo and open source.

## Supported Points ("Addresses")

`interp-engine` supports 34 standardized points ("Addresses") across architectures: every one of them on the eager backend, 28 of them on vLLM. Check [interp-engine.org](https://interp-engine.org) for the "cheat sheet", or [SUPPORTED_POINTS.md](docs/SUPPORTED_POINTS.md) for a markdown version with the per-backend detail.

## Performance / Speed

vLLM gives `interp-engine` high throughput via concurrency, and `backend="vllm-static"` adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.

<!-- THROUGHPUT:START -->

<!-- Generated by `python -m benchmarks.report_bench`. Do not edit: rerun the sweep. -->

Measured on NVIDIA B200, bf16, 512-token prompt, 128 new tokens.

One stream (tok/s):

| model                    | eager | vLLM       | vLLM + static taps |
| ------------------------ | ----- | ---------- | ------------------ |
| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**     |
| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**    |
| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**     |
| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**      |
| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**      |

8 concurrent requests (aggregate tok/s):

| model                    | eager | vLLM        | vLLM + static taps |
| ------------------------ | ----- | ----------- | ------------------ |
| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**    |
| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**    |
| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**    |
| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**      |
| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**     |

<!-- THROUGHPUT:END -->

`backend="vllm-static"` is opt-in, and serves only the tap set it declared — `static_points="auto"` by default, or a list you name. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; [benchmarks/results-latest.md](benchmarks/results-latest.md) has every figure at full precision, including capture, steering and lens latencies; [benchmarks/README.md](benchmarks/README.md) has how the tables above are rounded.

## Correctness

We verify correctness in two main ways:

1. A test suite that checks results across several models - what each check is designed to catch is in [INTERNALS.md](docs/INTERNALS.md#correctness).
2. A full `validator` comparison engine that checks most hook points across 50+ models, at early, middle and late layers - fully reproducible, with detailed results saved in the git repo at [`validator/`](validator/).

## Why use an Interpretability Engine, instead of just having my AI code whatever it needs on the fly?

Software is easy to make in 2026, here's why an engine makes sense:

1. **Speed**: Get performance without sacrificing correctness.
2. **Standardization + Verification**: Eliminate ambiguity when referring to points, plus a full test suite included.
3. **Faster Dev / Fewer Tokens Used**: You could spend ten million tokens and have your AI write, test, and make production-ready an interpretability engine. Or you could just `pip install interp-engine[vllm]`.

## Development / Contributing

Activate the shared git hooks once per clone — they format staged Python, rebuild the generated files, and run CI's static checks before a push. Details in [CONTRIBUTING.md](CONTRIBUTING.md).

```bash
make hooks    # or: git config core.hooksPath .githooks
```

## Contact

Bugs and feature requests belong in [issues](https://github.com/decoderesearch/interp-engine/issues). For anything else: [johnny@neuronpedia.org](mailto:johnny@neuronpedia.org).

## License

Apache 2.0
