Metadata-Version: 2.4
Name: agent-smart-router
Version: 0.1.2
Summary: A lightweight CLI tool for delegating LLM tasks to expert models across multiple providers.
Author-email: Kerem Barbaros Karnabat <kbarbaros@hotmail.com>
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: openai>=1.0.0
Requires-Dist: filelock>=3.12.0
Requires-Dist: anthropic>=0.30.0
Dynamic: license-file

# Agent Smart Router (CLI Delegation Tool)

> A lightweight, fault-tolerant CLI tool for delegating LLM tasks to expert models across multiple providers (Nvidia NIM, Groq, OpenAI, Anthropic Claude, Gemini).

[![PyPI version](https://img.shields.io/pypi/v/agent-smart-router.svg)](https://pypi.org/project/agent-smart-router/)
![Python](https://img.shields.io/badge/Python-3.8%2B-blue)
![License](https://img.shields.io/badge/License-MIT-green)
![CI](https://github.com/cadakerem/agent-smart-router/actions/workflows/ci.yml/badge.svg)

## ⚡ Features

- **Multi-Provider Support:** Seamlessly route requests to `nvidia`, `groq`, `openai`, `anthropic`, or `gemini`.
- **Automatic Fallbacks:** Provide a comma-separated list of models. If one fails, it instantly falls back to the next.
- **Circuit Breaker:** Built-in health tracking and cooldowns to prevent spamming dead endpoints.
- **Reasoning Extraction:** Automatically extracts and formats hidden `<thought>` or `reasoning` blocks (e.g., from Nemotron or DeepSeek).
- **Streaming Native:** Built on the official OpenAI SDK for fast and reliable streaming chunks.

## 📦 Installation

Since the package is published on PyPI, you can install it globally via `pip`:

```bash
pip install agent-smart-router
```

Alternatively, you can clone the repository for local development:
```bash
git clone https://github.com/cadakerem/agent-smart-router.git
cd agent-smart-router
pip install -e .
```

### Setup API Keys
The router checks for a `keys.json` file in your working directory, or falls back to system environment variables.

**Option 1: keys.json**
Create a `keys.json` file:
```json
{
  "NVIDIA_API_KEY": "nvapi-...",
  "GROQ_API_KEY": "gsk_...",
  "OPENAI_API_KEY": "sk-...",
  "ANTHROPIC_API_KEY": "sk-ant-...",
  "GEMINI_API_KEY": "AIza..."
}
```

**Option 2: Environment Variables**
```bash
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export GEMINI_API_KEY="AIza..."
export NVIDIA_API_KEY="nvapi-..."
export GROQ_API_KEY="gsk_..."
```

## 💻 Usage

Once installed, you can use the `agent-smart-router` command globally from your terminal! 
Provide your model fallback chain using `-m`, and your prompt using `-p`.

```bash
agent-smart-router -m "<provider:model1>,<provider:model2>" -p "<your prompt>"
```

You can also read prompts from a file or via standard input:
```bash
agent-smart-router -m "nvidia:nemotron,groq:llama3" -f prompt.txt
cat logs.txt | agent-smart-router -m "groq:llama3"
```

### Examples

**Heavy Coding Task (Nvidia Laguna -> Groq Fallback):**
```bash
agent-smart-router -m "nvidia:poolside/laguna-xs-2.1,groq:groq/compound" -p "Write a python script to parse logs."
```

**Custom Cooldown and Project ID:**
```bash
agent-smart-router -m "groq:llama3" -p "Hello" --project "agent-core" --max-failures 3 --cooldown 300
```

## 🏗️ Architecture Overview

The router uses a `FileLock`-backed JSON state (`circuit_breaker.json`) to track failures across concurrent runs. 
If an endpoint times out or returns a 5xx error more than `MAX_FAILURES` times, the circuit trips and forces the router to skip that endpoint for the next 120 seconds, immediately trying the next fallback model.
