Metadata-Version: 2.4
Name: amperor-strata
Version: 0.1.0
Summary: Run Mixture-of-Experts models larger than your RAM on Apple Silicon by paging experts from SSD.
Author-email: Amperor <licensing@amperor.ai>
License-Expression: BUSL-1.1
Project-URL: Homepage, https://strata.amperor.ai
Project-URL: Documentation, https://strata.amperor.ai/docs
Project-URL: Repository, https://github.com/amperor-org/amperor-strata
Keywords: mlx,moe,mixture-of-experts,llm,inference,apple-silicon,local-ai,paging,on-device
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: MacOS
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Provides-Extra: mlx
Requires-Dist: mlx-lm>=0.31; extra == "mlx"
Dynamic: license-file

# Strata

**Run Mixture-of-Experts models larger than your RAM** — on Apple Silicon.

Strata keeps only a budget of experts resident and streams the rest from SSD, so
a machine that could never hold the full model can still load and generate with
it.

```bash
pip install 'amperor-strata[mlx]'
```

## Use it in your app (3 lines)

```python
import strata

# GLM-4.5-Air is ~60 GB on disk — load it under a 16 GB budget
model = strata.load("mlx-community/GLM-4.5-Air-4bit", target_ram_gb=16)
print(model.generate("Explain how a CPU pipeline works.", max_tokens=256))
```

Measured on a 128 GB M5 Max, that ~60 GB model loads and generates at a **~11.5
GB peak**. `model` accepts a Hugging Face repo id or a local MLX directory;
`model.stream(...)` yields tokens, and `generate` also takes OpenAI-style
`messages`.

## Or serve it (OpenAI-compatible)

```bash
strata serve --config strata-deployment.yaml   # http://127.0.0.1:8399/v1
```

Point any OpenAI SDK at `base_url` — no code changes.

## Scope

Apple Silicon / MLX, sparse-MoE models (GLM-4.5-Air, Qwen3-MoE, Mixtral,
DeepSeek-V2, OLMoE, …), single-node. Throughput is device-dependent.

- **Docs:** https://strata.amperor.ai/docs
- **How numbers are measured:** https://strata.amperor.ai/methods

## License

Source-available under the **Business Source License 1.1** — free for
non-production use (evaluation, development, research, personal); production or
commercial use requires a license until the Change Date (2030-08-04), when each
release converts to Apache 2.0. **Not** an OSI-approved open-source license.
Commercial licensing: licensing@amperor.ai.
