Metadata-Version: 2.4
Name: ai-switch
Version: 0.3.0
Summary: Free-tier AI provider router: automatic fallback across Groq, OpenRouter, NVIDIA and more.
Author: Dhinakaran Thangaraj
License-Expression: MIT
Project-URL: Homepage, https://github.com/Dheena731/llmswitch
Project-URL: Documentation, https://dheena731.github.io/llmswitch/
Project-URL: Repository, https://github.com/Dheena731/llmswitch
Project-URL: Issues, https://github.com/Dheena731/llmswitch/issues
Keywords: llm,ai,groq,openrouter,nvidia,fallback,router,rate-limit
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx>=0.27
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Dynamic: license-file

# LLMSwitch

Free-tier AI provider router for developers. Write AI code once; LLMSwitch handles provider fallback across Groq, OpenRouter, NVIDIA, and more.

## Install

```bash
pip install ai-switch
```

The distribution is named `ai-switch`; the import stays `llmswitch`.

## Configure

Set an API key for any provider you want LLMSwitch to use — it only tries providers with a key present:

```bash
export GROQ_API_KEY="..."
export CEREBRAS_API_KEY="..."
export GEMINI_API_KEY="..."
export OPENROUTER_API_KEY="..."
export NVIDIA_API_KEY="..."
export MISTRAL_API_KEY="..."
export HF_TOKEN="..."
```

| Provider | Env var | Order |
|---|---|---|
| Groq | `GROQ_API_KEY` | 1 |
| Cerebras | `CEREBRAS_API_KEY` | 2 |
| Gemini | `GEMINI_API_KEY` | 3 |
| OpenRouter | `OPENROUTER_API_KEY` | 4 |
| NVIDIA | `NVIDIA_API_KEY` | 5 |
| Mistral | `MISTRAL_API_KEY` | 6 |
| Hugging Face | `HF_TOKEN` | 7 |
| Ollama (local) | `OLLAMA_HOST` | 8 — last resort |

### Ollama as a local last resort

Ollama needs no API key and never rate-limits, so it makes a good final fallback when every
cloud free tier is spent. It is **opt-in** — set `OLLAMA_HOST` to enable it, so users without a
local server don't pay for a failed attempt on every request:

```bash
export OLLAMA_HOST="http://localhost:11434/v1"
```

## Usage

```python
from llmswitch import AIClient

client = AIClient()
response = client.chat("Write a Python web scraper")
print(response)
```

Providers are tried in order (Groq, OpenRouter, NVIDIA by default). If one is rate-limited, unreachable, or errors, LLMSwitch automatically falls back to the next.

### Model names

`model` accepts either a friendly alias (`"fast-model"`, `"smart-model"` — see [llmswitch/registry.py](llmswitch/registry.py)) that's resolved per-provider, or a raw provider-specific model name passed straight through:

```python
client.chat("hello", model="fast-model")
client.chat("hello", model="llama-3.1-8b-instant")  # Groq-specific name
```

### Health tracking

LLMSwitch remembers which providers just failed, so a rate-limited provider isn't retried on every call — it's rested and moved to the back of the queue:

```python
client.chat("hi")      # groq is rate-limited -> falls back to openrouter
client.chat("hi")      # groq skipped entirely; openrouter served directly
```

Cooldowns respect the provider's own `Retry-After` header when present, and otherwise back off exponentially (60s, 120s, 240s… capped at 15 min for rate limits; shorter for transient outages). A success clears the streak.

If *every* provider is resting, LLMSwitch still tries them — closest-to-recovery first — rather than failing without an attempt.

Inspect current state at any time:

```python
client.status()
# {'groq': {'available': False, 'cooldown_remaining': 57.0, ...},
#  'openrouter': {'available': True, 'total_successes': 2, ...}}
```

### Choosing providers explicitly

```python
from llmswitch import AIClient
from llmswitch.providers import GroqProvider, NVIDIAProvider

client = AIClient(providers=[GroqProvider(), NVIDIAProvider()])
```

See [examples/](examples/) for more.

## Tests

```bash
pip install -e ".[dev]"
pytest
```

## Status

Pre-release (v0.1, MVP). See [roadmap.md](roadmap.md) and [todo.md](todo.md) for current progress.
