Metadata-Version: 2.4
Name: orkestra-router
Version: 0.0.1
Summary: Smart LLM routing across providers - automatically picks the most cost-efficient model for your prompt
Author: Orkestra Team
License: MIT
Project-URL: Homepage, https://github.com/orkestra-ai/orkestra
Project-URL: Repository, https://github.com/orkestra-ai/orkestra
Project-URL: Issues, https://github.com/orkestra-ai/orkestra/issues
Keywords: llm,routing,ai,cost-optimization,gemini,claude,openai
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: scikit-learn>=1.3.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: torch>=2.0.0
Requires-Dist: transformers>=4.30.0
Provides-Extra: google
Requires-Dist: google-genai>=1.0.0; extra == "google"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40.0; extra == "anthropic"
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == "openai"
Provides-Extra: all
Requires-Dist: google-genai>=1.0.0; extra == "all"
Requires-Dist: anthropic>=0.40.0; extra == "all"
Requires-Dist: openai>=1.0.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-mock>=3.0; extra == "dev"
Dynamic: license-file

# orkestra

Smart LLM routing across providers. Automatically selects the most cost-efficient model for your prompt using KNN-based routing.

## Install

```bash
pip install orkestra

# With provider extras
pip install orkestra[google]
pip install orkestra[anthropic]
pip install orkestra[openai]
pip install orkestra[all]
```

## Quick Start

```python
import orkestra as o

provider = o.Provider("google", "YOUR_GEMINI_API_KEY")
response = provider.chat("Explain quantum computing")

print(response.text)
print(f"Provider: {response.provider}")
print(f"Model: {response.model}")
print(f"Cost: ${response.cost:.6f}")
print(f"Saved: {response.savings_percent:.1f}%")
```

## Multi-Provider Routing

Combine multiple providers and let orkestra pick the best one:

```python
import orkestra as o

google = o.Provider("google", "GOOGLE_KEY")
anthropic = o.Provider("anthropic", "ANTHROPIC_KEY")
openai = o.Provider("openai", "OPENAI_KEY")

multi = o.MultiProvider([google, anthropic, openai])

# Route to cheapest option across all providers
response = multi.chat("What is 2+2?", strategy="cheapest")

# Route to smartest option
response = multi.chat("Prove the Riemann hypothesis", strategy="smartest")

# Balanced: prefer mid-tier models, break ties by cost
response = multi.chat("Write a Python function", strategy="balanced")
```

## Streaming

```python
provider = o.Provider("google", "YOUR_KEY")
for chunk in provider.stream_text("Write a poem"):
    print(chunk, end="")
```

## Explore
```
======================================================================
PROVIDER MODEL CATALOG
======================================================================

  GOOGLE
    gemini-2.5-flash-lite        budget     $0.10/$0.40 per 1M tokens
    gemini-3-flash-preview       balanced   $0.50/$3.00 per 1M tokens
    gemini-3-pro-preview         premium    $2.00/$12.00 per 1M tokens

  ANTHROPIC
    claude-haiku-4               budget     $0.80/$4.00 per 1M tokens
    claude-sonnet-4-5            balanced   $3.00/$15.00 per 1M tokens
    claude-opus-4                premium    $15.00/$75.00 per 1M tokens

  OPENAI
    gpt-4o-mini                  budget     $0.15/$0.60 per 1M tokens
    gpt-4o                       balanced   $2.50/$10.00 per 1M tokens
    o3                           premium    $10.00/$40.00 per 1M tokens

======================================================================
ROUTER PREDICTIONS (which model gets picked per prompt)
======================================================================

  [Simple] What is the capital of Japan?
    google       → gemini-3-flash-preview (balanced)
    anthropic    → claude-sonnet-4-5 (balanced)
    openai       → gpt-4o (balanced)

  [Moderate] Explain how a hash table works with collision handling
    google       → gemini-3-flash-preview (balanced)
    anthropic    → claude-sonnet-4-5 (balanced)
    openai       → gpt-4o (balanced)

  [Complex] Implement a B-tree with insert and search in Python. Handle node ...
    google       → gemini-3-pro-preview (premium)
    anthropic    → claude-opus-4 (premium)
    openai       → o3 (premium)

======================================================================
COST SIMULATION (500 input tokens, 1000 output tokens)
======================================================================

  [Simple] What is the capital of Japan?
    google       gemini-3-flash-preview       $0.003250  (saves $0.009750, 75% vs gemini-3-pro-preview)
    anthropic    claude-sonnet-4-5            $0.016500  (saves $0.066000, 80% vs claude-opus-4)
    openai       gpt-4o                       $0.011250  (saves $0.033750, 75% vs o3)

  [Moderate] Explain how a hash table works with collision handling
    google       gemini-3-flash-preview       $0.003250  (saves $0.009750, 75% vs gemini-3-pro-preview)
    anthropic    claude-sonnet-4-5            $0.016500  (saves $0.066000, 80% vs claude-opus-4)
    openai       gpt-4o                       $0.011250  (saves $0.033750, 75% vs o3)

  [Complex] Implement a B-tree with insert and search in Python. Handle node ...
    google       gemini-3-pro-preview         $0.013000  (saves $0.000000, 0% vs gemini-3-pro-preview)
    anthropic    claude-opus-4                $0.082500  (saves $0.000000, 0% vs claude-opus-4)
    openai       o3                           $0.045000  (saves $0.000000, 0% vs o3)

======================================================================
STRATEGY COMPARISON (multi-provider selection)
======================================================================

  [Simple] What is the capital of Japan?
    Per-provider routes: google→gemini-3-flash-preview, anthropic→claude-sonnet-4-5, openai→gpt-4o
    cheapest   → google/gemini-3-flash-preview (balanced) $0.003250
    smartest   → google/gemini-3-flash-preview (balanced) $0.003250
    balanced   → google/gemini-3-flash-preview (balanced) $0.003250

  [Moderate] Explain how a hash table works with collision handling
    Per-provider routes: google→gemini-3-flash-preview, anthropic→claude-sonnet-4-5, openai→gpt-4o
    cheapest   → google/gemini-3-flash-preview (balanced) $0.003250
    smartest   → google/gemini-3-flash-preview (balanced) $0.003250
    balanced   → google/gemini-3-flash-preview (balanced) $0.003250

  [Complex] Implement a B-tree with insert and search in Python. Handle node ...
    Per-provider routes: google→gemini-3-pro-preview, anthropic→claude-opus-4, openai→o3
    cheapest   → google/gemini-3-pro-preview (premium) $0.013000
    smartest   → google/gemini-3-pro-preview (premium) $0.013000
    balanced   → google/gemini-3-pro-preview (premium) $0.013000

======================================================================
PUBLIC API
======================================================================

  import orkestra as o

  # Single provider (routes within one provider's model family)
  provider = o.Provider("google", "API_KEY")
  response = provider.chat("your prompt")
  stream   = provider.stream_text("your prompt")

  # Multi-provider (picks best provider+model using a strategy)
  multi = o.MultiProvider([provider1, provider2])
  response = multi.chat("your prompt", strategy="cheapest")
  response = multi.chat("your prompt", strategy="smartest")
  response = multi.chat("your prompt", strategy="balanced")

  # Response fields
  response.text             # generated text
  response.model            # model used (e.g. "gemini-2.5-flash-lite")
  response.provider         # provider name (e.g. "google")
  response.cost             # total cost in dollars
  response.input_tokens     # input token count
  response.output_tokens    # output token count
  response.savings          # dollars saved vs premium model
  response.savings_percent  # savings as percentage
```

## How It Works

Orkestra uses a KNN (K-Nearest Neighbors) router trained on benchmark query embeddings to predict which model tier will perform best for your specific prompt. Simple queries get routed to cheaper models, complex ones to premium models.

Each call:
1. Embeds your prompt using Longformer (768-dim)
2. KNN finds the 5 nearest training queries
3. Routes to the model that performed best on similar queries
4. Calls the selected model via the provider's API
5. Returns the response with cost and savings info

Router models are downloaded automatically on first use and cached in `~/.orkestra/routers/`.

## Supported Providers

### Google Gemini

| Tier | Model | Input $/1M | Output $/1M |
|------|-------|-----------|------------|
| Budget | `gemini-2.5-flash-lite` | $0.10 | $0.40 |
| Balanced | `gemini-3-flash-preview` | $0.50 | $3.00 |
| Premium | `gemini-3-pro-preview` | $2.00 | $12.00 |

### Anthropic Claude

| Tier | Model | Input $/1M | Output $/1M |
|------|-------|-----------|------------|
| Budget | `claude-haiku-4` | $0.80 | $4.00 |
| Balanced | `claude-sonnet-4-5` | $3.00 | $15.00 |
| Premium | `claude-opus-4` | $15.00 | $75.00 |

### OpenAI

| Tier | Model | Input $/1M | Output $/1M |
|------|-------|-----------|------------|
| Budget | `gpt-4o-mini` | $0.15 | $0.60 |
| Balanced | `gpt-4o` | $2.50 | $10.00 |
| Premium | `o3` | $10.00 | $40.00 |

## API Reference

### `orkestra.Provider(name, api_key)`

Create a provider with automatic routing.

- `name`: `"google"`, `"anthropic"`, or `"openai"`
- `api_key`: Your API key for the provider

### `provider.chat(prompt, *, max_tokens=8192, temperature=1.0)`

Generate a response with automatic model routing. Returns a `Response`.

### `provider.stream_text(prompt, *, max_tokens=8192, temperature=1.0)`

Stream text chunks with automatic model routing. Yields `str`.

### `orkestra.MultiProvider(providers)`

Combine multiple `Provider` instances for cross-provider routing.

### `multi.chat(prompt, *, strategy="cheapest", max_tokens=8192, temperature=1.0)`

Generate with strategy-based provider selection. Strategies: `"cheapest"`, `"smartest"`, `"balanced"`.

### `orkestra.Response`

| Field | Type | Description |
|-------|------|-------------|
| `text` | `str` | Generated response text |
| `model` | `str` | Model that was selected |
| `provider` | `str` | Provider that was used |
| `cost` | `float` | Actual cost in dollars |
| `input_tokens` | `int` | Input token count |
| `output_tokens` | `int` | Output token count |
| `savings` | `float` | Dollars saved vs base model |
| `savings_percent` | `float` | Percentage saved vs base model |
| `base_model` | `str` | Comparison baseline model |
| `base_cost` | `float` | What it would have cost with base model |

## License

MIT
