Metadata-Version: 2.5
Name: unified-llm-provider
Version: 0.1.0
Summary: A unified async interface over multiple LLM providers (OpenAI, OpenRouter, Ollama).
Author: edesgan
License-Expression: MIT
License-File: LICENSE
Keywords: ai,async,llm,ollama,openai,openrouter,streaming
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: AsyncIO
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: httpx>=0.28.1
Requires-Dist: ollama>=0.5.1
Requires-Dist: openai>=3.11.0
Requires-Dist: pydantic>=2.0
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: mypy>=1.11; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Description-Content-Type: text/markdown

# unified-llm-provider

A small, dependency-light Python library that puts **one async interface in front of multiple LLM providers**. Build a single `LLMClient`, pick the provider in configuration, and stream or fetch chat completions through a normalized event model — no provider-specific branching in your application code.

Supported today: **OpenAI**, **OpenRouter** (OpenAI-compatible), and a local **Ollama** server.

## Features

- **Provider-agnostic API** — one `LLMClient` for OpenAI, OpenRouter, and Ollama. Switching providers is a config change, not a rewrite.
- **Normalized streaming events** — text and reasoning are surfaced as typed events (`MessageStartEvent`, `ReasoningDeltaEvent`, `TextDeltaEvent`, `MessageEndEvent`, `StreamEndEvent`, ...) so consumers never parse raw SSE or NDJSON.
- **Reasoning / thinking support** — chain-of-thought deltas are emitted as dedicated reasoning events, from `reasoning_content` (OpenAI, OpenRouter) and `thinking` (Ollama).
- **Per-request overrides** — model and sampling parameters can be set at call time and override the client defaults (including explicit `0` values).
- **Multimodal messages** — text plus images, where images may be remote URLs, `data:` URLs, or local file paths (auto-encoded to base64 with the correct MIME type).
- **Model discovery** — list the models exposed by the configured provider.
- **Predictable endpoints** — base URLs are normalized, so trailing `/chat/completions`, `/messages`, or `/api/chat` suffixes are stripped for you.
- **Minimal runtime** — `httpx` for chat calls, `pydantic` for config validation, and the `openai` / `ollama` SDKs only where they earn their keep.

## Requirements

- Python **>= 3.12**
- Dependencies (from `pyproject.toml`): `httpx>=0.28.1`, `pydantic>=2.0`, `openai>=3.11.0`, `ollama>=0.5.1`
- For Ollama: a local server (`ollama serve`) with the model you intend to use already pulled
- For OpenAI / OpenRouter: an API key

## Installation

```bash
pip install unified-llm-provider
```

For development, clone the repository and install with the dev tooling:

```bash
git clone <your-repo-url> unified-llm-provider
cd unified-llm-provider
uv sync --extra dev
```

## Quick start

### Basic chat (non-streaming)

```python
import asyncio
import os

from unified_llm_provider import LLMClient, LLMClientConfig

async def main():
    client = LLMClient(LLMClientConfig(
        provider_id="openai",
        api_key=os.environ["OPENAI_API_KEY"],   # required for openai / openrouter
        model_name="gpt-4o-mini",               # optional: can be given per request instead
    ))

    answer = await client.get_response("Explain async generators in two sentences.")
    print(answer)

asyncio.run(main())
```

`get_response()` is a convenience wrapper around `chat()` that returns the assistant text. Reach for `chat()` directly when you want the raw provider payload as a dictionary.

### Streaming with events

```python
import asyncio
import os

from unified_llm_provider import (
    ChatOptions,
    LLMClient,
    LLMClientConfig,
    MessageEndEvent,
    MessageStartEvent,
    ReasoningDeltaEvent,
    ReasoningEndEvent,
    ReasoningStartEvent,
    StreamEndEvent,
    TextDeltaEvent,
)

async def main():
    client = LLMClient(LLMClientConfig(
        provider_id="openai",
        api_key=os.environ["OPENAI_API_KEY"],
    ))

    async for event in client.chat_stream(
        "Write a haiku about streaming tokens.",
        ChatOptions(model="gpt-4o-mini"),
    ):
        match event:
            case MessageStartEvent():
                print("[stream opened]")
            case ReasoningDeltaEvent():
                print(event.content, end="", flush=True)
            case TextDeltaEvent():
                print(event.content, end="", flush=True)
            case MessageEndEvent():
                print(f"\n[done: {event.reason}]")
            case StreamEndEvent():
                print("[stream closed]")

asyncio.run(main())
```

Every event also carries a `type` attribute (`EventType.TEXT_DELTA`, `EventType.REASONING_DELTA`, ...), so `match event.type:` works just as well if you prefer dispatching on the enum.

### Per-request model and options

The model may live on the config, on the request, or both. Resolution is `options.model or config.model_name`, and a `ValueError` is raised at call time if neither is set. Any `ChatOptions` value overrides the defaults — explicit `0` included, since the library checks `is not None` rather than truthiness.

```python
options = ChatOptions(
    model="gpt-4o",                 # overrides config.model_name
    temperature=0.0,                # 0 is respected
    max_tokens=256,
    top_p=0.9,
    seed=42,
    stop=["\n\n"],
    response_format={"type": "json_object"},
)

payload = await client.chat("Return the answer as JSON.", options)
```

### Multi-turn and multimodal messages

A prompt is either a plain string (wrapped into a single `user` message) or a list of `Message` objects:

```python
from unified_llm_provider import Message

messages = [
    Message(role="system", content="You are terse."),
    Message(role="user", content="What is 2 + 2?"),
    Message(role="assistant", content="4"),
    Message(role="user", content="And multiplied by 10?"),
]

reply = await client.get_response(messages)
```

Content can be a list of parts for vision models. An `image_url` value may be an `http(s)://` URL, a `data:` URL, or a **local file path** (which is read and converted to a base64 data URL for OpenAI/OpenRouter, or to a bare base64 list for Ollama):

```python
messages = [Message(
    role="user",
    content=[
        {"type": "text", "text": "What is in this image?"},
        {"type": "image_url", "image_url": {"url": "./diagram.png"}},
    ],
)]
```

### Using Ollama

No API key is needed, and the endpoint defaults to `http://localhost:11434`:

```python
from unified_llm_provider import ChatOptions, LLMClient, LLMClientConfig

client = LLMClient(LLMClientConfig(
    provider_id="ollama",
    # endpoint="http://localhost:11434",  # optional, this is the default
    model_name="llama3.2",
))

print(await client.get_response("Why is the sky blue?"))

# Or override the model per request (e.g. a thinking model that emits `thinking` deltas)
async for event in client.chat_stream("Solve 17 * 23.", ChatOptions(model="deepseek-r1:8b")):
    print(event)
```

> [!NOTE]
> `get_response()` normalizes OpenAI-style payloads (`choices[0].message.content`). Ollama's non-streaming reply is shaped as `{"message": {"content": ...}}`, so with Ollama read the raw result from `await client.chat(...)` and use `payload["message"]["content"]`.

### Listing models

```python
models = await client.get_model()
print(models)   # ["gpt-4o-mini", "gpt-4o", ...] / ["llama3.2", ...]
```

Model listing goes through the official SDKs (`AsyncOpenAI.models.list()` for OpenAI/OpenRouter, `AsyncClient.list()` for Ollama). All chat and streaming traffic goes through `httpx`.

## Configuration reference

### `LLMClientConfig`

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `provider_id` | `str` | `"openai"` | One of `openai`, `ollama`, `openrouter` (lowercase). Any other value raises `ValueError`. |
| `endpoint` | `str` | provider-specific | Base URL. Must start with `http://` or `https://`. Defaults to the provider's public endpoint. |
| `api_key` | `str` | `None` | **Required** for `openai` and `openrouter`. Not needed for `ollama`. |
| `model_name` | `str` | `None` | Optional default model. Can be supplied per request via `ChatOptions(model=...)` instead. |

Config is a pydantic model with `extra="forbid"`, so unknown fields are rejected rather than silently ignored.

### `ChatOptions`

Per-request overrides. Every field defaults to `None`, meaning "fall back to the client default".

| Field | Type | Default | Applied as |
| --- | --- | --- | --- |
| `model` | `str` | `None` | Resolved against `config.model_name`; `ValueError` if both are missing. |
| `temperature` | `float` | `None` | Default `0.7`; `temperature` (`num_predict`-style options for Ollama). |
| `max_tokens` | `int` | `None` | Default `8192`; sent as `max_tokens` (OpenAI-compatible) / `num_predict` (Ollama). |
| `top_p` | `float` | `None` | `top_p` |
| `frequency_penalty` | `float` | `None` | `frequency_penalty` |
| `presence_penalty` | `float` | `None` | `presence_penalty` |
| `stop` | `str \| list[str]` | `None` | `stop` |
| `n` | `int` | `None` | `n` (OpenAI-compatible only) |
| `seed` | `int` | `None` | `seed` |
| `response_format` | `dict` | `None` | `response_format` (OpenAI-compatible only) |
| `**extra_kwargs` | `Any` | — | Accepted and stored on `options.extra_kwargs` (also reachable as attributes), but **not** serialized into the request payload today. |

Shared defaults live in `constants.py` as `DefaultModelSettings`: `temperature=0.7`, `max_tokens=8192`.

## Supported providers

| `provider_id` | Default endpoint | `api_key` | Chat path | Notes |
| --- | --- | --- | --- | --- |
| `openai` | `https://api.openai.com/v1` | Required | `/chat/completions` | Payload is OpenAI-compatible. Reasoning via `reasoning_content`. |
| `openrouter` | `https://openrouter.ai/api/v1` | Required | `/chat/completions` | OpenAI-compatible; model ids look like `openai/gpt-4o-mini`. |
| `ollama` | `http://localhost:11434` | Not used | `/api/chat` | Native Ollama payload (`options` block, bare-base64 images). Thinking via `thinking`. |

Endpoints are normalized before use: trailing slashes are trimmed, a trailing `/chat/completions`, `/messages`, or `/api/chat` is stripped back to the base URL, and for Ollama a trailing `/api` or `/v1` is removed. Passing `https://api.openai.com/v1/chat/completions` is therefore a valid way to configure the client.

## Event reference

| Event | `EventType` | Payload | Emitted when |
| --- | --- | --- | --- |
| `MessageStartEvent` | `MESSAGE_START` | — | Once, as the stream opens. |
| `ReasoningStartEvent` | `REASONING_START` | — | First reasoning/thinking delta arrives. |
| `ReasoningDeltaEvent` | `REASONING_DELTA` | `content: str` | Each reasoning/thinking chunk. |
| `ReasoningEndEvent` | `REASONING_END` | — | Reasoning finishes, or the answer text begins, or the stream ends. |
| `TextDeltaEvent` | `TEXT_DELTA` | `content: str` | Each answer text chunk. |
| `MessageEndEvent` | `MESSAGE_END` | `reason: str` | Message complete (`reason="stop"`, or `"length"` when truncated). |
| `StreamEndEvent` | `STREAM_END` | — | Once, after the message ends. |

Typical order for a reasoning-capable model:

```text
MESSAGE_START → [REASONING_START → REASONING_DELTA … → REASONING_END] → TEXT_DELTA … → MESSAGE_END → STREAM_END
```

Reasoning events are optional — providers that never emit reasoning content simply go straight to `TEXT_DELTA`.

## Project structure

```text
unified-llm-provider/
├── src/
│   └── unified_llm_provider/
│       ├── __init__.py        # public API re-exports
│       ├── client.py          # LLMClient: public entry point (chat, chat_stream, get_response, get_model)
│       ├── base.py            # BaseClient ABC: shared httpx chat/stream logic, message conversion, model resolution
│       ├── config.py          # LLMClientConfig, Message, ChatOptions
│       ├── constants.py       # ProviderType enum, DefaultModelSettings
│       ├── events.py          # EventType enum + normalized event dataclasses
│       ├── py.typed           # PEP 561 typing marker
│       └── providers/
│           ├── openai.py      # OpenAIClient
│           ├── openrouter.py  # OpenRouterClient
│           └── ollama.py      # OllamaClient (native /api/chat)
├── tests/
│   ├── conftest.py            # httpx stub + fixtures (offline)
│   ├── test_*.py              # unit tests
│   └── integration/           # opt-in live tests (marked `integration`)
├── pyproject.toml
├── LICENSE
└── README.md
```

## Running the tests

The default suite is fully offline — it stubs `httpx`, so no credentials or network are needed:

```bash
uv run pytest
```

Live tests against real providers live in `tests/integration/` and are excluded by default (they carry the `integration` marker). They need credentials or a running server, and self-skip when those are absent:

```bash
# OpenAI
OPENAI_API_KEY="sk-..." uv run pytest -m integration tests/integration/test_live_openai.py

# OpenRouter
OPENROUTER_API_KEY="sk-or-..." uv run pytest -m integration tests/integration/test_live_openrouter.py

# Ollama — requires `ollama serve` and the model already pulled
uv run pytest -m integration tests/integration/test_live_ollama.py
```

## Notes

- Provider selection is case-sensitive; use the exact lowercase ids listed above.
- The provider-specific extras are intentionally thin: `OpenAIClient` and `OpenRouterClient` add only model listing and auth/base-URL plumbing, while `OllamaClient` overrides the payload builder and chat path.
- Released under the MIT License — see `LICENSE`.
