Metadata-Version: 2.4
Name: aiyallm
Version: 0.1.2
Summary: A small, provider-agnostic gateway for OpenAI/Anthropic-style language and vision models.
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: openai>=1.66
Requires-Dist: anthropic>=0.49
Requires-Dist: httpx>=0.27
Provides-Extra: test
Requires-Dist: pytest>=8.3; extra == "test"
Requires-Dist: pytest-asyncio>=0.24; extra == "test"

# Aiyallm

Aiyallm is a lightweight Python gateway for OpenAI/Anthropic-style language and vision models.

> Aiyallm does not ship a built-in model catalog and never guesses model names. Provider endpoints, model lists, and pricing remain caller-owned. API keys can be passed directly, read from a specific environment variable, or inferred from common vendor variables such as `DEEPSEEK_API_KEY` and `DASHSCOPE_API_KEY`.

## Table of Contents

- [Features](#features)
- [Install](#install)
- [Quick Start](#quick-start)
- [Provider Configuration](#provider-configuration)
- [Routing](#routing)
- [Vision and Capability Filtering](#vision-and-capability-filtering)
- [Usage Ledger](#usage-ledger)
- [Cost Calculation](#cost-calculation)
- [Streaming and Async](#streaming-and-async)
- [Extending Providers](#extending-providers)

## Features

- OpenAI and OpenAI-compatible providers
- Anthropic and Anthropic-compatible providers
- Chinese OpenAI-compatible providers such as DeepSeek, Qwen/DashScope, Moonshot/Kimi, Zhipu GLM, and SiliconFlow
- Explicit model selection, `provider/model` references, and provider route lists
- Routing policies: `fallback`, `first`, `round_robin`, `random`, and `least_used`
- Vision and multimodal message passthrough with capability filtering
- Sync and async chat and streaming
- Detailed token accounting with cache hit/miss fields
- Optional SQLite or JSONL usage persistence
- Optional cost calculation from an application-owned price book

## Install

```bash
pip install -e .
```

Runtime dependencies are:

- `openai`
- `anthropic`
- `httpx`

## Quick Start

There are no default providers. Pass them explicitly.

```python
from aiyallm import Aiyallm

client = Aiyallm(
    providers=[
        {
            "name": "deepseek",
            "account": "billing-deepseek",
            "type": "openai_compatible",
            "base_url": "https://api.deepseek.com",
            "api_key": "your-deepseek-key",
            "models": ["deepseek-chat", "deepseek-reasoner"],
        }
    ]
)

response = client.chat(
    "用一句话解释什么是路由。",
    model="deepseek/deepseek-chat",
)

print(response.text)
print(response.usage)
```

> `models` is optional when the provider allows unknown models. Configured providers default to `allow_unknown_models=True`, but the provider itself must still be declared.

## Provider Configuration

Each provider accepts these common fields.

| Field | Meaning |
| --- | --- |
| `name` | Unique provider name used by `provider/model` routes |
| `account` | Optional billing or account name recorded in usage |
| `type` | `openai`, `openai_compatible`, `anthropic`, or `anthropic_compatible` |
| `base_url` | Endpoint for compatible services |
| `api_key` | Explicit API key |
| `api_key_env` | Optional environment variable name, or list of names, to read the key from |
| `api_key_prompt` | When `True`, prompt in the terminal if no key is found; defaults to `False` |
| `models` | Optional model names or `{"id", "capabilities", ...}` mappings |
| `allow_unknown_models` | Allow model IDs not listed in `models`; defaults to `True` |
| `timeout` | SDK timeout |

### OpenAI-Compatible Examples

```python
providers = [
    {
        "name": "deepseek",
        "type": "openai_compatible",
        "base_url": "https://api.deepseek.com",
        # `api_key_env` is optional; DEEPSEEK_API_KEY is inferred from `name`.
    },
    {
        "name": "qwen",
        "type": "openai_compatible",
        "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
        # DASHSCOPE_API_KEY or QWEN_API_KEY is inferred from `name`.
    },
    {
        "name": "moonshot",
        "type": "openai_compatible",
        "base_url": "https://api.moonshot.cn/v1",
        # MOONSHOT_API_KEY or KIMI_API_KEY is inferred from `name`.
    },
    {
        "name": "glm",
        "type": "openai_compatible",
        "base_url": "https://open.bigmodel.cn/api/paas/v4",
        # ZHIPU_API_KEY / ZHIPUAI_API_KEY / GLM_API_KEY is inferred from `name`.
    },
]
```

Use the `base_url` from each provider's current documentation. Aiyallm does not hard-code any endpoint or model name.

## Routing

Use `provider/model` to target a provider.

```python
response = client.chat("Hello", model="qwen/qwen-plus")
```

Use a route list for fallback.

```python
response = client.chat(
    "Hello",
    model=["deepseek/deepseek-chat", "qwen/qwen-plus"],
    route_policy="fallback",
)
```

Available policies:

- `fallback` - try candidates in order
- `first` - use only the first candidate
- `round_robin` - rotate through candidates
- `random` - choose a random candidate
- `least_used` - choose the candidate with the lowest recorded token usage

The default policy is `fallback`.

## Vision and Capability Filtering

Declare capabilities in `models` and pass `require_vision=True` when a request needs image input.

```python
from aiyallm import Aiyallm

client = Aiyallm(
    providers=[
        {
            "name": "qwen",
            "type": "openai_compatible",
            "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
            "api_key": "your-key",
            "models": [
                {"id": "qwen-vl-plus", "capabilities": ["text", "vision"]}
            ],
        }
    ]
)

response = client.chat(
    [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is in this image?"},
                {
                    "type": "image_url",
                    "image_url": {"url": "https://example.com/image.jpg"},
                },
            ],
        }
    ],
    model="qwen/qwen-vl-plus",
    require_vision=True,
)
```

## Usage Ledger

Every successful completion is recorded.

```python
response = client.chat("Hello", model="deepseek/deepseek-chat")

print(response.usage.input_tokens)
print(response.usage.output_tokens)
print(response.usage.input_cache_hit_tokens)
print(response.usage.input_cache_miss_tokens)
print(response.usage.output_cache_hit_tokens)
print(response.usage.output_cache_miss_tokens)
print(response.usage.account)
print(response.usage.reasoning_tokens)

print(client.ledger.total())
print(client.ledger.by_model())
print(client.ledger.by_provider())
print(client.ledger.by_account())

client.ledger.export_jsonl("usage.jsonl")
```

For persistent accounting, pass a SQLite path.

```python
from aiyallm import UsageLedger

ledger = UsageLedger(sqlite_path="data/usage.sqlite3")
client = Aiyallm(providers=providers, ledger=ledger)
```

## Cost Calculation

Token counts are always recorded. Cost is calculated only when the application supplies a price.

```python
from aiyallm import Aiyallm, Price, PriceBook

price_book = PriceBook(
    {
        "deepseek-chat": Price(
            input_per_mtok=0.27,
            output_per_mtok=1.10,
        )
    }
)

client = Aiyallm(providers=providers, price_book=price_book)
response = client.chat("Hello", model="deepseek/deepseek-chat")

print(response.usage.cost_usd)
```

Prices are US dollars per million tokens.

## Streaming and Async

```python
for chunk in client.stream("Hello", model="deepseek/deepseek-chat"):
    print(chunk.content, end="", flush=True)
```

```python
import asyncio


async def main():
    response = await client.achat("Hello", model="deepseek/deepseek-chat")
    print(response.text)

    async for chunk in client.astream("Hello", model="deepseek/deepseek-chat"):
        print(chunk.content, end="", flush=True)


asyncio.run(main())
```

## Extending Providers

For a non-standard provider, subclass `aiyallm.BaseProvider` and implement `complete`, `acomplete`, `stream`, and `astream`.

```python
from aiyallm import Aiyallm, BaseProvider


class MyProvider(BaseProvider):
    def complete(self, request):
        ...
```

Then register the instance directly.

```python
client = Aiyallm(providers=[MyProvider(name="my-provider", models=["my-model"])])
```
