Metadata-Version: 2.4
Name: mlx-vlm-batch-outlines
Version: 0.1.0
Summary: Qwen-focused MLX vision-language chat library with batched multimodal chat.
Author: Avishek Biswas
License-Expression: MIT
Keywords: mlx,vlm,qwen,structured-output,constrained-decoding,vision-language
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mlx>=0.31.1
Requires-Dist: huggingface_hub>=0.32.0
Requires-Dist: llguidance
Requires-Dist: numpy>=1.26.0
Requires-Dist: opencv-python>=4.10.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: requests>=2.31.0
Requires-Dist: torchvision>=0.26.0
Requires-Dist: tqdm>=4.66.0
Requires-Dist: transformers>=5.4.0
Provides-Extra: dev
Requires-Dist: pytest>=9.0.0; extra == "dev"
Dynamic: license-file

## mlx-vlm-batch-outlines

`mlx-vlm-batch-outlines` is a small Qwen-focused MLX vision-language package for:

- multimodal chat with images
- batched multimodal chat
- constrained decoding with `llguidance`
- structured outputs from Pydantic, regex, CFG, or JSON Schema
- simple video inference helpers

It is intentionally narrow. It is built for local MLX/Qwen workflows and does not try to support every `mlx-vlm` backend or the full Outlines API surface.

## Attribution

This project heavily reuses and adapts ideas and code paths from:

- [`mlx-vlm`](https://github.com/Blaizzy/mlx-vlm)
- [`outlines`](https://github.com/dottxt-ai/outlines)

In particular:

- the MLX/Qwen multimodal runtime and model code are derived from `mlx-vlm`
- the constrained decoding architecture and structured-output interface are derived from `outlines`

## Scope

This package currently targets Qwen vision models exposed through MLX, such as:

- `mlx-community/Qwen3.5-4B-MLX-4bit`
- `mlx-community/Qwen2-VL-2B-Instruct-4bit`

The public API is:

- `load(...)`
- `chat(...)`
- `chat_stream(...)`
- `batch_chat(...)`
- `video_chat(...)`
- `video_chunk_process(...)`

Structured output helpers:

- `Regex`
- `CFG`
- `JsonSchema`
- `regex(...)`
- `cfg(...)`
- `json_schema(...)`

## Install

```bash
uv sync
```

Or with plain pip:

```bash
pip install -e .
```

## Quick Start

```python
from PIL import Image
from mlx_vlm_batch_outlines import chat, load

model, processor = load("mlx-community/Qwen3.5-4B-MLX-4bit")
image = Image.open("cat.jpeg")

result = chat(
    model,
    processor,
    [
        {"role": "system", "content": "Answer concisely."},
        {"role": "user", "content": ["Describe this image.", image]},
    ],
    max_tokens=80,
)

print(result.text)
```

## Batched Image Chat

```python
from PIL import Image
from mlx_vlm_batch_outlines import batch_chat, load

model, processor = load("mlx-community/Qwen3.5-4B-MLX-4bit")
cat = Image.open("cat.jpeg")
dog = Image.open("dog.jpeg")

results = batch_chat(
    model,
    processor,
    [
        [{"role": "user", "content": ["Describe this image.", cat]}],
        [{"role": "user", "content": ["Describe this image.", dog]}],
    ],
    max_tokens=80,
)

for text in results.texts:
    print(text)
```

## Structured Outputs

### Pydantic

```python
from PIL import Image
from pydantic import BaseModel
from mlx_vlm_batch_outlines import chat, load


class VisualSummary(BaseModel):
    primary_subject: list[str]
    subject_count: int
    setting: str
    short_description: str


model, processor = load("mlx-community/Qwen3.5-4B-MLX-4bit")
image = Image.open("cat.jpeg")

result = chat(
    model,
    processor,
    [
        {
            "role": "user",
            "content": ["Describe this image as JSON using the requested schema.", image],
        }
    ],
    output_type=VisualSummary,
    max_tokens=140,
)

print(result.model_dump())
```

### CFG

```python
from PIL import Image
from mlx_vlm_batch_outlines import CFG, chat, load

model, processor = load("mlx-community/Qwen3.5-4B-MLX-4bit")
image = Image.open("cat.jpeg")

result = chat(
    model,
    processor,
    [
        {
            "role": "user",
            "content": [
                "What animal is most prominent in this image? Choose either cat or dog.",
                image,
            ],
        }
    ],
    output_type=CFG('start: "cat" | "dog"'),
    max_tokens=12,
)

print(result.text)
```

### Batch + Pydantic

```python
from PIL import Image
from pydantic import BaseModel
from mlx_vlm_batch_outlines import batch_chat, load


class VisualSummary(BaseModel):
    primary_subject: list[str]
    subject_count: int
    setting: str
    short_description: str


model, processor = load("mlx-community/Qwen3.5-4B-MLX-4bit")
cat = Image.open("cat.jpeg")
dog = Image.open("dog.jpeg")

results = batch_chat(
    model,
    processor,
    [
        [{"role": "user", "content": ["Describe this image.", cat]}],
        [{"role": "user", "content": ["Describe this image.", dog]}],
    ],
    output_type=VisualSummary,
    max_tokens=180,
)

for item in results:
    print(item.model_dump())
```

## Video Chat

`video_chat(...)` is the thin direct video path. It samples frames across the whole video and runs one multimodal generation call.

```python
from mlx_vlm_batch_outlines import load, video_chat

model, processor = load("mlx-community/Qwen2-VL-2B-Instruct-4bit")

result = video_chat(
    model,
    processor,
    video="path/to/video.mp4",
    prompt="Describe this video.",
    fps=1.0,
    max_pixels=(224, 224),
    max_tokens=100,
)

print(result.text)
```

## Chunked Video Processing

`video_chunk_process(...)` treats a single video as many independent image-chat chunks.

For each chunk it:

1. slices the video by time
2. samples frames inside that chunk
3. converts those frames into a normal multi-image chat
4. runs chunk chats through `batch_chat(...)` in mini-batches

This is useful when you want independent chunk summaries instead of one global video answer.

```python
from pydantic import BaseModel
from mlx_vlm_batch_outlines import load, video_chunk_process


class ChunkSummary(BaseModel):
    actions: list[str]
    scene: str


model, processor = load("mlx-community/Qwen3.5-4B-MLX-4bit")

results = video_chunk_process(
    model,
    processor,
    video="path/to/video.mp4",
    prompt="Describe what is happening in this chunk.",
    chunk_length_seconds=5.0,
    batch_size=4,
    fps=1.0,
    max_frames_per_chunk=8,
    output_type=ChunkSummary,
    max_tokens=120,
)

for item in results:
    print(item["chunk_index"], item["start_sec"], item["end_sec"], item["output"])
```

Each returned item looks like:

```python
{
    "chunk_index": 0,
    "start_sec": 0.0,
    "end_sec": 5.0,
    "output": ...,
}
```

If `output_type` is structured, `output` is the parsed structured object. Otherwise it is the raw text string for that chunk.

## Notes

- Default image resize is currently `224x224`.
- Smaller image sizes can improve batching throughput significantly because image token count drops quickly with resolution.
- Homogeneous batches usually perform better than mixed multimodal shapes.
- `batch_chat_stream(...)` is not implemented.
- This package is Qwen-focused and not intended as a generic VLM abstraction layer.

## Benchmark Notes

These are local measurements from the development machine, not a formal benchmark suite.

- 4B Qwen, homogeneous `3 x 1-image` batch, `768x768`:
  - sequential: about `19.25s`
  - batch: about `15.63s`
  - speedup: about `1.23x`

- 4B Qwen, homogeneous `3 x 1-image` batch, `384x384`, short CFG output:
  - sequential: about `6.25s`
  - batch: about `2.94s`
  - speedup: about `2.13x`

The practical takeaway is simple:

- batching helps more when image sizes are smaller
- batching helps more when the workload is homogeneous
- image resolution matters a lot because image token count grows quickly with width and height

## Local Verification

There is a simple verifier script in:

- [`verify_mlx_vlm_batch_outlines.py`](./verify_mlx_vlm_batch_outlines.py)

Run it with:

```bash
uv run python verify_mlx_vlm_batch_outlines.py
```
