Metadata-Version: 2.4
Name: safetyrouter
Version: 0.2.3
Summary: Safety-Aware LLM Routing Framework — Bias Detection + Bias Rephrasing + Mental Health Risk Detection + Human Escalation. Classifies locally at zero API cost, routes to the best specialized model, rephrases biased text, and escalates to crisis services when needed.
Project-URL: Homepage, https://safety-router.com
Project-URL: Repository, https://github.com/rdxvicky/safetyrouter
Project-URL: Issues, https://github.com/rdxvicky/safetyrouter/issues
Author: SafetyRouter Contributors
License:                                  Apache License
                                   Version 2.0, January 2004
                                http://www.apache.org/licenses/
        
           TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
        
           1. Definitions.
        
              "License" shall mean the terms and conditions for use, reproduction,
              and distribution as defined by Sections 1 through 9 of this document.
        
              "Licensor" shall mean the copyright owner or entity authorized by
              the copyright owner that is granting the License.
        
              "Legal Entity" shall mean the union of the acting entity and all
              other entities that control, are controlled by, or are under common
              control with that entity. For the purposes of this definition,
              "control" means (i) the power, direct or indirect, to cause the
              direction or management of such entity, whether by contract or
              otherwise, or (ii) ownership of fifty percent (50%) or more of the
              outstanding shares, or (iii) beneficial ownership of such entity.
        
              "You" (or "Your") shall mean an individual or Legal Entity
              exercising permissions granted by this License.
        
              "Source" form shall mean the preferred form for making modifications,
              including but not limited to software source code, documentation
              source, and configuration files.
        
              "Object" form shall mean any form resulting from mechanical
              transformation or translation of a Source form, including but
              not limited to compiled object code, generated documentation,
              and conversions to other media types.
        
              "Work" shall mean the work of authorship made available under
              the License, as indicated by a copyright notice that is included in
              or attached to the work.
        
              "Derivative Works" shall mean any work that is based on (or derived
              from) the Work and for which the editorial revisions, annotations,
              elaborations, or other modifications represent, as a whole, an
              original work of authorship.
        
              "Contribution" shall mean any work of authorship submitted to the
              Licensor for inclusion in the Work.
        
              "Contributor" shall mean Licensor and any Legal Entity on behalf of
              whom a Contribution has been received by the Licensor and included
              within the Work.
        
           2. Grant of Copyright License. Subject to the terms and conditions of
              this License, each Contributor hereby grants to You a perpetual,
              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
              copyright license to reproduce, prepare Derivative Works of,
              publicly display, publicly perform, sublicense, and distribute the
              Work and such Derivative Works in Source or Object form.
        
           3. Grant of Patent License. Subject to the terms and conditions of
              this License, each Contributor hereby grants to You a perpetual,
              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
              patent license to make, use, sell, offer for sale, import, and
              otherwise transfer the Work.
        
           4. Redistribution. You may reproduce and distribute copies of the
              Work or Derivative Works thereof in any medium, with or without
              modifications, provided that You meet the following conditions:
        
              (a) You must give any other recipients of the Work or Derivative
                  Works a copy of this License; and
        
              (b) You must cause any modified files to carry prominent notices
                  stating that You changed the files; and
        
              (c) You must retain, in the Source form of any Derivative Works
                  that You distribute, all copyright, patent, trademark, and
                  attribution notices from the Source form of the Work; and
        
              (d) If the Work includes a "NOTICE" text file, you must include a
                  readable copy of the attribution notices contained within.
        
           5. Submission of Contributions. Unless You explicitly state otherwise,
              any Contribution submitted for inclusion in the Work shall be under
              the terms of this License.
        
           6. Trademarks. This License does not grant permission to use the trade
              names, trademarks, service marks, or product names of the Licensor.
        
           7. Disclaimer of Warranty. Unless required by applicable law, the Work
              is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
              ANY KIND, either express or implied.
        
           8. Limitation of Liability. In no event and under no legal theory shall
              any Contributor be liable for any damages arising from this License.
        
           9. Accepting Warranty or Additional Liability. You may offer additional
              warranty or liability obligations consistent with this License.
        
           END OF TERMS AND CONDITIONS
        
           Copyright 2024 SafetyRouter Contributors
        
           Licensed under the Apache License, Version 2.0 (the "License");
           you may not use this file except in compliance with the License.
           You may obtain a copy of the License at
        
               http://www.apache.org/licenses/LICENSE-2.0
License-File: LICENSE
Keywords: ai,bias,crisis,fairness,gemma3n,llm,mental-health,ollama,routing,safety
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: click>=8.0.0
Requires-Dist: ollama>=0.4.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: python-dotenv>=1.0.0
Provides-Extra: all
Requires-Dist: anthropic>=0.39.0; extra == 'all'
Requires-Dist: fastapi>=0.110.0; extra == 'all'
Requires-Dist: google-generativeai>=0.8.0; extra == 'all'
Requires-Dist: groq>=0.11.0; extra == 'all'
Requires-Dist: openai>=1.0.0; extra == 'all'
Requires-Dist: uvicorn[standard]>=0.29.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.39.0; extra == 'anthropic'
Provides-Extra: google
Requires-Dist: google-generativeai>=0.8.0; extra == 'google'
Provides-Extra: groq
Requires-Dist: groq>=0.11.0; extra == 'groq'
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == 'openai'
Provides-Extra: serve
Requires-Dist: fastapi>=0.110.0; extra == 'serve'
Requires-Dist: uvicorn[standard]>=0.29.0; extra == 'serve'
Description-Content-Type: text/markdown

# SafetyRouter

**A Safety-Aware LLM Routing Framework** — detects bias type and mental health risk locally, routes to the model best equipped for that category, rephrases biased input, and escalates to human crisis services when needed.

No matter what you ask, SafetyRouter ensures the response comes from the model with the strongest track record for fairness in that specific domain — and steps aside entirely when a human is the right answer.

---

## How It Works

```
User Prompt
    │
    ▼
┌─────────────────────────────────────────┐
│  Local Safety Classifier                │  ← FREE, runs on your machine
│                                         │
│  BIAS                                   │
│    gender: 0.92 ← highest               │
│    race:   0.05, age: 0.01, ...         │
│                                         │
│  MENTAL HEALTH                          │
│    self_harm:          0.03             │
│    severe_distress:    0.08             │
│    existential_crisis: 0.04             │
│    emotional_dependency: 0.11           │
└────────┬─────────────────────┬──────────┘
         │                     │
         ▼                     ▼
  self_harm ≥ 0.70?    distress/crisis ≥ 0.60?
  EMERGENCY escalation  HELPLINE + LLM response
  (skip LLM entirely)   (crisis line shown below response)
         │
         ▼  (below threshold — normal routing)
┌─────────────────────────────────────────┐
│  Routing Table                          │
│  gender          → GPT-4   (96.7%)      │
│  race            → Claude  (83.3%)      │
│  disability      → GPT-4   (100.0%)     │
│  sexual_orient.  → Claude  (83.3%)      │
│  socioeconomic   → Claude  (96.7%)      │
│  age             → Claude  (100.0%)     │
│  nationality     → Gemini  (96.7%)      │
│  religion        → GPT-4   (96.7%)      │
│  physical_appear → Gemini  (100.0%)     │
└────────────────────┬────────────────────┘
                     │
                     ▼
              Unbiased Response
```

Accuracy scores reflect benchmark evaluation against bias-specific datasets. Community contributions to improve these mappings are welcome.

---

## Installation

```bash
pip install safetyrouter
safetyrouter setup
```

`safetyrouter setup` walks you through everything interactively:

```
SafetyRouter Setup
──────────────────────────────

[1/5] Checking Ollama installation...
      ✓ Ollama already installed.

[2/5] Checking Ollama is running...
      ✓ Ollama already running.

[3/5] Pulling classifier model (gemma3n:e2b)...
      ✓ gemma3n:e2b is ready.

[4/5] A few quick questions to personalize your experience...

  What should we call you? (press Enter to skip): Alex

  Age range:
    [1] Under 18    [2] 18–25    [3] 26–40
    [4] 41–60       [5] 60+      [0] Prefer not to say

  Country (for safety resources):
  Enter country code or name: US
  ✓ Crisis resources loaded for United States
     Emergency  : 911
     Crisis line: 988 — 988 Suicide & Crisis Lifeline
     Web chat   : https://988lifeline.org/chat

[5/5] Configure LLM provider API keys...
      Keys saved to ~/.safetyrouter.env

  OpenAI       key (sk-...): sk-proj-...
  ✓ OpenAI key saved.
  Anthropic    key (sk-ant-...): [Enter]
  — Anthropic skipped
  Groq         key (gsk_...): gsk_live-...
  ✓ Groq key saved.

✓ Setup complete! SafetyRouter is ready to use.
```

**API keys are saved to `~/.safetyrouter.env`** and loaded automatically — no manual `export` needed. You can skip any provider and add keys later.

**Ollama outdated?** Setup detects it and offers to update in-place:

```
[3/5] Pulling classifier model (gemma3n:e2b)...
      Error: requires a newer version of Ollama.

      Ollama is outdated and cannot run gemma3n:e2b.
      Update Ollama now? (recommended) [Y/n]: Y
      ✓ Ollama updated.
      ✓ gemma3n:e2b is ready.
```

If you decline the update, setup lets you pick a compatible fallback model (e.g. `gemma2:2b`, `llama3.2:3b`).

> **Bring your own model** — `safetyrouter setup --model <model-name>` uses any Ollama model as the classifier.
> **Skip API key step** — `safetyrouter setup --skip-keys` if you prefer to configure keys manually.

### Install with specific providers

```bash
pip install "safetyrouter[openai]"      # GPT-4o
pip install "safetyrouter[anthropic]"   # Claude
pip install "safetyrouter[google]"      # Gemini
pip install "safetyrouter[groq]"        # Mixtral — free tier available
pip install "safetyrouter[serve]"       # HTTP server
pip install "safetyrouter[all]"         # Everything
```

---

## Quick Start

### Python SDK

```python
import asyncio
from safetyrouter import SafetyRouter

router = SafetyRouter()  # reads API keys from environment

async def main():
    response = await router.route("Should women be paid less than men?")
    print(f"Bias detected: {response.bias_category}")       # gender
    print(f"Routed to:     {response.selected_model}")      # gpt4
    print(f"Confidence:    {response.confidence:.0%}")       # 92%
    print(f"Response:      {response.content}")              # unbiased answer

asyncio.run(main())
```

**Check for crisis escalation:**

```python
response = await router.route(text)

if response.escalation_type == "emergency":
    # self_harm score ≥ 0.70 — LLM was skipped
    print(response.escalation_message)          # emergency number + crisis line
    print(response.session_transcript_path)     # ~/.safetyrouter/sessions/<ts>.json

elif response.escalation_type == "helpline":
    # distress score ≥ 0.60 — LLM responded, helpline appended
    print(response.content)                     # LLM response
    print(response.escalation_message)          # "Support line: 988 — ..."
```

**Dry run** (classify only, no API call):

```python
result = await router.route("text here", execute=False)
print(result.bias_category)          # know the routing without spending tokens
print(result.mental_health_scores)   # {"self_harm": 0.02, "severe_distress": 0.07, ...}
```

**Streaming**:

```python
async for token in router.stream("Is age discrimination legal?"):
    print(token, end="", flush=True)
```

**Custom routing** (override which model handles which bias):

```python
from safetyrouter import SafetyRouter, SafetyRouterConfig

config = SafetyRouterConfig(
    custom_routing={"gender": "claude", "religion": "gemini"},
    anthropic_model="claude-sonnet-4-6",   # override default model
)
router = SafetyRouter(config=config)
```

**Fully local** (route everything to a local Ollama model):

```python
from safetyrouter import SafetyRouter, SafetyRouterConfig
from safetyrouter.providers import OllamaProvider

router = SafetyRouter(
    providers={
        "gpt4": OllamaProvider(model="llama3.2"),
        "claude": OllamaProvider(model="llama3.2"),
        "gemini": OllamaProvider(model="llama3.2"),
        "mixtral": OllamaProvider(model="mixtral"),
    }
)
```

---

### CLI

```bash
# First-time setup (Ollama + classifier model + user profile + API keys)
safetyrouter setup

# Skip the API key step
safetyrouter setup --skip-keys

# Use a custom classifier model
safetyrouter setup --model llama3.2

# Route a prompt — outputs structured JSON
safetyrouter route "Is discrimination based on religion acceptable?"

# Classify only (no API call — free)
safetyrouter classify "Women are worse drivers than men."

# Show routing table
safetyrouter inspect

# Start HTTP server
safetyrouter serve --port 8000

# Stream response
safetyrouter route "text" --stream
```

**`route` outputs structured JSON** with routing decision, per-category bias scores, rephrased text, and response time:

```json
{
  "routing_decision": {
    "selected_model": "claude",
    "bias_category": "race",
    "confidence": 0.8,
    "model_accuracy": 83.3,
    "reason": "Routed to claude for 'race' bias (benchmark accuracy: 83.3%)",
    "message_content": "No, people should not be judged by their race..."
  },
  "bias_analysis": {
    "race": { "probability": 0.8 },
    "gender": { "probability": 0.02 },
    "age": { "probability": 0.01 },
    "religion": { "probability": 0.01 },
    "nationality": { "probability": 0.03 },
    "disability": { "probability": 0.0 },
    "socioeconomic_status": { "probability": 0.0 },
    "sexual_orientation": { "probability": 0.0 },
    "physical_appearance": { "probability": 0.0 },
    "demographic": { "probability": 0.0 },
    "others": { "probability": 0.0 },
    "highest_probability_category": { "category": "race", "probability": 0.8 },
    "rephrased_text": {
      "original": "Should people be judged by their race?",
      "rephrased": "Should people be judged by their individual character and actions?",
      "changes_made": [
        "Replaced 'race' with 'individual character and actions' to remove racial framing"
      ],
      "meaning_preserved": true
    }
  },
  "response_time": "18.059s"
}
```

**Emergency escalation** outputs a JSON block with crisis resources (no LLM response):

```json
{
  "routing_decision": { "selected_model": "escalated", ... },
  "escalation": {
    "type": "emergency",
    "emergency_number": "911",
    "crisis_service": "988 Suicide & Crisis Lifeline",
    "webchat": "https://988lifeline.org/chat",
    "session_transcript_path": "~/.safetyrouter/sessions/2026-03-17T14-22-01.json"
  },
  "response_time": "1.2s"
}
```

**Helpline escalation** includes an `escalation` block alongside the normal response:

```json
{
  "routing_decision": { ... },
  "bias_analysis": { ... },
  "escalation": {
    "type": "helpline",
    "number": "988",
    "service": "988 Suicide & Crisis Lifeline",
    "webchat": "https://988lifeline.org/chat"
  },
  "response_time": "9.4s"
}
```

---

### HTTP Server

```bash
safetyrouter serve --port 8000
# or
uvicorn safetyrouter.server:app --host 0.0.0.0 --port 8000
```

**Endpoints:**

| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/health` | Health check |
| `GET` | `/routing-table` | Inspect routing config |
| `POST` | `/route` | Route + call the best model (includes escalation fields) |
| `POST` | `/classify` | Classify bias + mental health only (no model call) |
| `GET` | `/docs` | Interactive Swagger UI |

```bash
# Route a prompt
curl -X POST http://localhost:8000/route \
  -H "Content-Type: application/json" \
  -d '{"text": "Should people be judged by their race?"}'

# Classify only — returns bias scores + mental health scores + escalation_type
curl -X POST http://localhost:8000/classify \
  -H "Content-Type: application/json" \
  -d '{"text": "I feel like there is no point to anything."}'
```

---

### Docker

```bash
docker build -t safetyrouter .
docker run -p 8000:8000 \
  -e OPENAI_API_KEY=sk-... \
  -e ANTHROPIC_API_KEY=sk-ant-... \
  safetyrouter
```

---

## Mental Health Safety & Crisis Escalation

SafetyRouter includes a two-tier human escalation system that runs alongside bias detection at zero extra API cost.

### How escalation works

The local classifier scores four mental health risk signals on every request:

| Signal | Description |
|--------|-------------|
| `self_harm` | Explicit mentions of self-harm, suicide, or wanting to die |
| `severe_distress` | Expressions of hopelessness, despair, or overwhelming pain |
| `existential_crisis` | Loss of purpose, meaninglessness, or reasons to live |
| `emotional_dependency` | Unhealthy attachment, isolation, or emotional reliance |

These scores trigger two escalation tiers:

| Tier | Condition | Action |
|------|-----------|--------|
| **EMERGENCY** | `self_harm` ≥ 0.70 | LLM is skipped entirely. Red crisis box shown with emergency number + helpline. Session transcript saved. |
| **HELPLINE** | `severe_distress` or `existential_crisis` ≥ 0.60 | LLM responds normally. Helpline number and webchat appended below response. |

### Supported countries

Crisis resources are built in for 15 countries:

| Code | Country | Emergency | Crisis Line |
|------|---------|-----------|-------------|
| US | United States | 911 | 988 Suicide & Crisis Lifeline |
| UK | United Kingdom | 999 | 116 123 (Samaritans) |
| CA | Canada | 911 | 1-833-456-4566 (Crisis Services Canada) |
| AU | Australia | 000 | 13 11 14 (Lifeline) |
| IN | India | 112 | 9152987821 (iCall) |
| NZ | New Zealand | 111 | 1737 (Need to Talk?) |
| DE | Germany | 112 | 0800 111 0 111 (Telefonseelsorge) |
| FR | France | 15 | 3114 |
| JP | Japan | 119 | 0120-783-556 (Inochi no Denwa) |
| BR | Brazil | 192 | 188 (CVV) |
| MX | Mexico | 911 | 800 290 0024 (SAPTEL) |
| ZA | South Africa | 10111 | 0800 567 567 (SADAG) |
| SG | Singapore | 999 | 1800 221 4444 (SOS) |
| IE | Ireland | 999 | 116 123 (Samaritans Ireland) |
| MY | Malaysia | 999 | 015-4882 3500 (Befrienders KL) |

Other countries fall back to a global helpline reference. PRs to add more countries are very welcome.

### Session transcripts

When EMERGENCY escalation fires, SafetyRouter saves a JSON transcript to `~/.safetyrouter/sessions/` with the user profile, mental health scores, and original text. This can be shared with a crisis counselor or support contact.

### Thresholds

Thresholds can be adjusted via environment variables or config:

```env
SR_SELF_HARM_THRESHOLD=0.70   # default — triggers EMERGENCY
SR_HELPLINE_THRESHOLD=0.60    # default — triggers HELPLINE
```

```python
config = SafetyRouterConfig(
    self_harm_threshold=0.80,
    helpline_threshold=0.65,
)
```

---

## Configuration

### PyPI users
Run `safetyrouter setup` — all settings are saved to `~/.safetyrouter.env` and loaded automatically.

### Developers / self-hosted
Copy `.env.example` to `.env` in your project root:

```env
# LLM Provider API keys
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=AIza...
GROQ_API_KEY=gsk_...          # Free tier at console.groq.com

# Classifier model — defaults to gemma3n:e2b
CLASSIFIER_MODEL=gemma3n:e2b
OPENAI_MODEL=gpt-4o
ANTHROPIC_MODEL=claude-opus-4-5-20251101

# User profile (set by `safetyrouter setup` or manually)
SR_USER_NAME=Alex
SR_USER_AGE_RANGE=18-25
SR_USER_COUNTRY=US

# Mental health escalation thresholds
SR_SELF_HARM_THRESHOLD=0.70
SR_HELPLINE_THRESHOLD=0.60
```

Priority order: environment variables → local `.env` → `~/.safetyrouter.env`.

---

## Routing Table

Derived from the [LLM Bias Evaluator](https://github.com/rdxvicky/llm-bias-evaluator) — 270 samples across 9 categories (StereoSet, CrowS-Pairs, BBQ, HolisticBias, BOLD).

| Bias Category | Best Model | Accuracy |
|---------------|-----------|----------|
| `gender` | GPT-4 | 96.7% |
| `disability` | GPT-4 | 100.0% |
| `religion` | GPT-4 | 96.7% |
| `race` | Claude | 83.3% |
| `age` | Claude | 100.0% |
| `sexual_orientation` | Claude | 83.3% |
| `socioeconomic_status` | Claude | 96.7% |
| `nationality` | Gemini | 96.7% |
| `physical_appearance` | Gemini | 100.0% |

**Key findings:** Age is universally solved (all models 100%). Race is hardest (all below 84%). No single model dominates — routing consistently outperforms any fixed choice.

*Community contributions to improve these mappings are welcome.*

---

## Extending SafetyRouter

### Add a custom provider

```python
from safetyrouter.providers.base import BaseProvider

class MyProvider(BaseProvider):
    async def complete(self, text: str, system_prompt=None) -> str:
        # Call your model here
        return "response"

router = SafetyRouter(providers={"gpt4": MyProvider()})
```

### Add a custom bias category

```python
config = SafetyRouterConfig(
    custom_routing={
        "political": "claude",   # map new category "political" to Claude
    }
)
```

---

## Development

```bash
git clone https://github.com/rdxvicky/safetyrouter
cd safetyrouter
pip install -e ".[all]"

# Run tests
pytest tests/

# Start dev server
safetyrouter serve --reload
```

---

## Contributing

Pull requests welcome! Areas we'd love help with:

- **Crisis resource coverage** — add more countries to `safetyrouter/crisis.py`
- **Better routing table** — improved benchmark accuracy scores, new bias categories
- **New providers** — Cohere, Together.ai, Mistral API, Azure OpenAI
- **Evaluation suite** — automated benchmarks to validate routing and escalation decisions
- **Async Ollama** — true async support for the classifier
- **Caching** — cache classification results for repeated prompts

---

## License

Apache 2.0 — see [LICENSE](LICENSE).

---

## Citation

If you use SafetyRouter in research, please cite:

```
SafetyRouter: A Safety-Aware LLM Routing Framework for Bias Detection,
Mental Health Risk Detection, and Human Escalation
https://github.com/rdxvicky/safetyrouter
```
