Metadata-Version: 2.4
Name: zeta-mlx-inference
Version: 0.3.2
Summary: MLX inference engine and API for Zeta MLX
Keywords: mlx,llm,apple-silicon,inference,openai,vllm
Author: ZetaLab
Author-email: zeta9044@gmail.com
Requires-Python: >=3.10,<3.13
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: fastapi (>=0.115,<0.116)
Requires-Dist: mlx (>=0.30,<0.31)
Requires-Dist: mlx-lm (>=0.28,<0.29)
Requires-Dist: uvicorn[standard] (>=0.32,<0.33)
Requires-Dist: zeta-mlx-core (>=0.3.2,<0.4.0)
Project-URL: Homepage, https://github.com/zeta9044/zeta-mlx
Project-URL: Repository, https://github.com/zeta9044/zeta-mlx
Description-Content-Type: text/markdown

# zeta-mlx-inference

MLX-based LLM inference engine with OpenAI/vLLM compatible API.

## Installation

```bash
pip install zeta-mlx-inference
```

## Features

- **MLX Optimized**: Native Apple Silicon performance
- **OpenAI Compatible**: `/v1/chat/completions` endpoint
- **vLLM Compatible**: `/tokenize`, `/detokenize` endpoints
- **Streaming**: Real-time token streaming support
- **Multi-model**: Support for Qwen, Llama, and more

## Usage

```python
from zeta_mlx.inference import InferenceEngine

# Create engine
engine = InferenceEngine("mlx-community/Qwen3-8B-4bit")

# Generate response
result = engine.generate(messages, params)
```

## API Endpoints

- `POST /v1/chat/completions` - Chat completions
- `POST /tokenize` - Tokenize text
- `POST /detokenize` - Detokenize tokens
- `GET /health` - Health check

## Links

- [GitHub](https://github.com/zeta9044/zeta-mlx)
- [Documentation](https://github.com/zeta9044/zeta-mlx#readme)

