Metadata-Version: 2.5
Name: ai-gen1
Version: 0.1.1
Summary: The open-source typed decision engine for LLMs. Generate only 1 thing: the outcome.
Project-URL: Homepage, https://gen1-ai.web.app
Project-URL: Documentation, https://gen1-ai.web.app/docs/
Project-URL: Repository, https://github.com/breejesh/gen1.git
Project-URL: Issues, https://github.com/breejesh/gen1/issues
Author-email: gen1 Authors <breejesh2212@gmail.com>
License: Apache-2.0
License-File: LICENSE
Keywords: classification,decision-engine,fastapi,gen1,llm,logits,outcome-engine,pydantic,system-1,transformers,vllm
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: torch>=2.0.0
Requires-Dist: transformers>=4.40.0
Provides-Extra: benchmark
Requires-Dist: fastapi>=0.110.0; extra == 'benchmark'
Requires-Dist: matplotlib>=3.8.0; extra == 'benchmark'
Requires-Dist: rich>=13.7.0; extra == 'benchmark'
Requires-Dist: uvicorn>=0.29.0; extra == 'benchmark'
Provides-Extra: dev
Requires-Dist: black>=24.0.0; extra == 'dev'
Requires-Dist: mypy>=1.10.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.4.0; extra == 'dev'
Provides-Extra: vllm
Requires-Dist: vllm>=0.4.0; extra == 'vllm'
Description-Content-Type: text/markdown

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="resources/assets/logo-dark.svg">
    <source media="(prefers-color-scheme: light)" srcset="resources/assets/logo-light.svg">
    <img src="resources/assets/logo-dark.svg" width="220" alt="gen1">
  </picture>
</p>

<p align="center">
  <strong>Generate only 1 thing: the outcome.</strong><br>
  <em>High-throughput, reflexive semantic branching for agents and LLM applications without unnecessary autoregressive generation.</em>
</p>

<p align="center">
  <a href="https://pypi.org/project/ai-gen1/"><img src="https://img.shields.io/badge/pypi-v0.1.1-E5007D?style=flat-square&logo=pypi&logoColor=white" alt="PyPI"></a>
  <a href="https://www.npmjs.com/package/ai-gen1"><img src="https://img.shields.io/badge/npm-v0.1.0-CB3837?style=flat-square&logo=npm&logoColor=white" alt="npm"></a>
  <a href="https://gen1-ai.web.app/docs/"><img src="https://img.shields.io/badge/docs-live-10B981?style=flat-square&logo=gitbook&logoColor=white" alt="Docs"></a>
  <a href="https://gen1-ai.web.app"><img src="https://img.shields.io/badge/website-gen1--ai.web.app-6366F1?style=flat-square" alt="Website"></a>
  <a href="#"><img src="https://img.shields.io/badge/output-0%20tokens-22C55E?style=flat-square" alt="0 Tokens"></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-Apache--2.0-blue?style=flat-square" alt="License"></a>
  <a href="https://www.python.org/"><img src="https://img.shields.io/badge/python-3.10%2B-3776AB?style=flat-square&logo=python&logoColor=white" alt="Python 3.10+"></a>
</p>

<p align="center">
  <strong>No waitlist. Open-source & local-first.</strong> Run open weights locally on consumer GPUs, vLLM, llama.cpp, or Apple Silicon today.
</p>

---

## ⚡ The Problem: The Generative Tax on Agent Decisions

Most agent and application decisions are small: *route this prompt*, *verify citations*, *check guardrails*, or *score customer intent*. Standard generative LLMs answer these by spending **1,500ms to 3,000ms** decoding verbose JSON strings that your application immediately parses back into an `if` statement:

```
Generative LLM (Autoregressive Decode):
User Input ──► [Prefill Pass] ──► Decode Token 1 ──► Decode Token 2 ──► ... ──► Decode Token 80 ──► JSON Parse ──► if (route == "billing")
Latency: 1,840 ms | Tokens Decoded: 77 | Failure Mode: Syntax errors, truncated JSON, hallucinated keys
```

### The `gen1` Solution: Terminal Logit Projection

Named after Daniel Kahneman's **System 1** (fast, reflexive, instinctive thinking), `gen1` eliminates the decode loop entirely. It reads the model's terminal logit distributions (`logits[:, -1, :]`) during the initial prefill forward pass and directly projects them into normalized candidate choice probabilities:

```
gen1 (Outcome Only):
User Input ──► [Single Prefill Forward Pass] ──► Terminal Logit Projection ──► Outcome: "billing" (p=0.985)
Latency: 188 ms (9.8x faster) | Tokens Decoded: 0 | Failure Mode: Mathematically 0% schema errors
```

- **0 Tokens Decoded:** No conversational filler, no token streaming, and no autoregressive decoding overhead.
- **10x &ndash; 25x Speedup:** Sub-200ms decision latency on ordinary consumer desktop GPUs and laptops.
- **100% Schema Guarantee:** Invalid JSON, syntax errors, and schema violations are mathematically impossible.
- **Native Typed Interface:** First-class support for Python `Enum`, Pydantic models, and Shannon entropy uncertainty scores.

---

## 📊 Empirical Benchmarks (Consumer Hardware)

All benchmarks below are measured on **real consumer hardware** with open weights (Gemma 4-E4B 4-bit quantized) under empirical execution &mdash; zero cloud mocks.

### 1. Latency & Token Generation Comparison

<p align="center">
  <img src="resources/assets/benchmark_latency_tokens.svg" width="100%" alt="Per-Decision Latency and Token Decode Comparison">
</p>

### 2. Decision Throughput & Accuracy Agreement

<p align="center">
  <img src="resources/assets/benchmark_throughput_decisions.svg" width="49%" alt="Decision Throughput on Consumer Hardware">
  <img src="resources/assets/benchmark_accuracy_agreement.svg" width="49%" alt="Accuracy and Decision Agreement">
</p>

### Empirical Results Table

#### NVIDIA GeForce GTX 1660 Super (6 GB GDDR6 &middot; CUDA)
| Decision Primitive / Task | Normal LLM (Chat + JSON) | `gen1` (Outcome Only) | Tokens Decoded | Speedup |
| :--- | :---: | :---: | :---: | :---: |
| **Binary Guardrail (`Noul`)** | 1,780.2 ms | **185.4 ms** | 68 &rarr; **0 tokens** | **9.6x faster** |
| **Multi-Class Routing (`Choice`)** | 1,840.5 ms | **188.1 ms** | 77 &rarr; **0 tokens** | **9.8x faster** |
| **High-Context 8k Routing (`Choice`)** | 3,120.0 ms *(14.2s cold)* | **342.0 ms** *(560ms cold)* | 102 &rarr; **0 tokens** | **9.1x &ndash; 25.3x faster** |
| **Rubric Scoring (`Score`)** | 2,140.0 ms | **181.6 ms** | 91 &rarr; **0 tokens** | **11.8x faster** |
| **In-Process Direct Logits** | ~450.0 ms | **2.1 ms** | 50 &rarr; **0 tokens** | **214x faster** |

#### Apple MacBook Pro (M4 Pro &middot; 24 GB Unified Memory)
| Decision Primitive / Task | Normal LLM (Chat + JSON) | `gen1` (Outcome Only) | Tokens Decoded | Speedup |
| :--- | :---: | :---: | :---: | :---: |
| **Binary Guardrail (`Noul`)** | 1,463.9 ms | **168.1 ms** | 68 &rarr; **0 tokens** | **8.7x faster** |
| **Multi-Class Routing (`Choice`)** | 1,663.0 ms | **170.7 ms** | 77 &rarr; **0 tokens** | **9.7x faster** |
| **High-Context 8k Routing (`Choice`)** | 2,419.6 ms *(10.8s cold)* | **271.8 ms** *(423ms cold)* | 102 &rarr; **0 tokens** | **8.9x &ndash; 25.6x faster** |
| **Rubric Scoring (`Score`)** | 1,959.5 ms | **163.7 ms** | 91 &rarr; **0 tokens** | **12.0x faster** |

> **Accuracy Agreement:** `gen1` achieves **99.2% empirical decision agreement** with 100-token autoregressive JSON outputs while eliminating 100% of schema failures.

---

## 🚀 Quickstart

### Installation

```bash
# Python (PyPI)
pip install ai-gen1

# JavaScript / TypeScript (Node.js)
npm install ai-gen1
```

---

## 💡 Code Examples

### 1. Native Typed Decision Engine (Python `Enum` + Entropy)

Compile-time enum safety and Shannon entropy uncertainty scores:

```python
from enum import Enum
import gen1

class Route(str, Enum):
    BILLING  = "Invoices, payment failures, refunds, and subscriptions"
    SECURITY = "Compromised credentials, vulnerabilities, or auth errors"
    SUPPORT  = "General product troubleshooting and how-to questions"

engine = gen1.load("Qwen/Qwen2.5-7B-Instruct", backend="hf")

decision = engine.decide(
    context="Customer received unauthorized 2FA SMS prompt.",
    options=Route,
)

print(decision.winner)            # Route.SECURITY
print(decision.confidence)        # 0.9912 (calibrated logit probability)
print(decision.entropy)           # 0.08 bits (low entropy = high certainty)
print(decision.tokens_generated)  # 0
```

### 2. Multi-Primitive Semantic Decision Client (Python)

Evaluate multiple routing questions, guardrails, and rubrics in a single direct pass:

```python
from gen1 import Client, Choice, Noul, Score

with Client() as client:
    response = client.decide(
        state={"ticket": "Charged $49 twice for invoice #1042. Please refund ASAP!"},
        questions={
            "is_billing": Noul(instructions="Is this ticket a billing dispute?"),
            "priority":   Choice(instructions="Priority tier", criteria={"HIGH": None, "LOW": None}),
            "sentiment":  Score(instructions="Customer sentiment", criteria=["angry", "neutral", "happy"]),
        }
    )

print(response.nouls["is_billing"].passed)      # True
print(response.choices["priority"].choice)      # "HIGH" (confidence = 0.985)
print(response.scores["sentiment"].score)       # 1.2
print(response.usage.output_tokens)             # 0 (Outcome only, 185ms)
```

### 3. TypeScript / Node.js Client

```typescript
import { Gen1Client } from "ai-gen1";

const client = new Gen1Client({
  endpoint: "http://127.0.0.1:8080",
});

const res = await client.choice({
  state: "Kubernetes worker node crashed with OOMKilled 137.",
  question: {
    prompt: "Route alert to responding engineering team",
    options: ["DevOps", "Billing", "Security", "Sales"],
  },
});

console.log(res.choice);          // "DevOps"
console.log(res.confidence);      // 0.984
console.log(res.tokensGenerated); // 0 (168ms)
```

---

## 🧩 Core Decision Primitives

| Primitive | Purpose | Use Cases | Return Type |
| :--- | :--- | :--- | :--- |
| **`Noul`** | Binary True/False assertion | Guardrails, safety gates, escalation triggers | `bool` + calibrated probability |
| **`Choice`** | Multi-class semantic routing | Intent routing, triage, tool dispatch | `str` / `Enum` + confidence distribution |
| **`Score`** | Normalized rubric rating | Sentiment scoring, RAG relevance, citation check | `float` on calibrated continuous scale |
| **`Advanced`** | Multi-question atomic evaluation | Run all questions in a single forward pass | Combined typed response dictionary |

---

## 📁 Repository Structure

```
gen1/
├── sdks/                      # Multi-language client SDKs
│   ├── python/               # Python SDK package (ai-gen1 on PyPI)
│   │   ├── gen1/             # Engine, DecisionResult, Client, Choice, Noul, Score
│   │   ├── tests/            # Python unit & integration test suite
│   │   └── pyproject.toml
│   └── typescript/           # TypeScript / JavaScript client package (ai-gen1 on npm)
│       ├── dist/             # Compiled CJS, ESM, and types
│       ├── test/             # Local client tests
│       └── package.json
│
├── resources/                 # Production resources & evaluations
│   ├── assets/               # Logos & benchmark vector graphics (SVG/PNG)
│   ├── benchmarks/           # Hardware benchmark scripts & datasets
│   │   ├── run_benchmark_1660_super.py
│   │   ├── compare_llm_speed.py
│   │   └── generate_benchmark_charts.py
│   ├── cookbooks/            # 11 runnable production recipes
│   │   ├── 01_classification_confidence.py
│   │   ├── 03_llm_guardrails.py
│   │   ├── 04_rag_passage_classification.py
│   │   ├── 05_semantic_rerank.py
│   │   └── 06_function_calling_router.py
│   └── examples/             # Production integration patterns
│       ├── 01_fastapi_router.py
│       ├── 02_guardrail_gate.py
│       └── 03_pydantic_branch.py
│
└── website/                  # Production documentation & benchmark site
    ├── src/                  # Astro + Starlight components & interactive demos
    └── tests/                # Documentation route verification tests
```

---

## 📚 Production Cookbooks

Explore 11 end-to-end runnable recipes located in [`resources/cookbooks/`](resources/cookbooks/):

1. [**Hierarchical Classification**](resources/cookbooks/02_hierarchical_classification.py) &mdash; Two-tier topic classification with early exit.
2. [**LLM Guardrails & Safety Gating**](resources/cookbooks/03_llm_guardrails.py) &mdash; Zero-token input moderation before expensive LLM calls.
3. [**RAG Passage Filtering**](resources/cookbooks/04_rag_passage_classification.py) &mdash; Filter retrieved chunks with sub-millisecond precision.
4. [**Semantic Reranking**](resources/cookbooks/05_semantic_rerank.py) &mdash; Calibrated rubric scoring across candidate documents.
5. [**Function Calling Router**](resources/cookbooks/06_function_calling_router.py) &mdash; Route user queries directly to internal tools.
6. [**Speculative Cascade**](resources/cookbooks/09_sde_cascade.py) &mdash; Fast small model routing with fallback to large LLM on low confidence.
7. [**Citation Fact-Checking**](resources/cookbooks/07_citation_fact_check.py) &mdash; Verify LLM claim consistency against source documents.

---

## 📄 License

Apache-2.0 License. Free for commercial and research use.
