Metadata-Version: 2.4
Name: llamajuice
Version: 0.1.2
Summary: Squeeze every last drop from your NVIDIA GPU.
Project-URL: Homepage, https://github.com/michaelkenealy/llamajuice
Project-URL: Repository, https://github.com/michaelkenealy/llamajuice
Project-URL: Issues, https://github.com/michaelkenealy/llamajuice/issues
License: MIT
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: System :: Hardware
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.100
Requires-Dist: httpx>=0.25
Requires-Dist: rich>=13.0
Requires-Dist: tqdm>=4.60
Requires-Dist: typer>=0.9
Requires-Dist: uvicorn>=0.20
Provides-Extra: agent
Requires-Dist: chromadb>=0.4; extra == 'agent'
Requires-Dist: faster-whisper>=1.0; extra == 'agent'
Requires-Dist: langgraph>=0.2; extra == 'agent'
Requires-Dist: python-telegram-bot>=21.0; extra == 'agent'
Requires-Dist: sentence-transformers>=2.0; extra == 'agent'
Provides-Extra: all
Requires-Dist: chromadb>=0.4; extra == 'all'
Requires-Dist: faster-whisper>=1.0; extra == 'all'
Requires-Dist: langgraph>=0.2; extra == 'all'
Requires-Dist: python-telegram-bot>=21.0; extra == 'all'
Requires-Dist: sentence-transformers>=2.0; extra == 'all'
Description-Content-Type: text/markdown

# llamajuice

**Squeeze every last drop from your NVIDIA GPU.**

One command catches every local LLM setup issue. Another serves your model with full GPU offload. A third routes requests to the right model automatically.

Born from 16 hours of GPU detection failures, model incompatibilities, and streaming bugs while building an AI agent on an RTX 5050 8GB laptop.

```
pip install llamajuice
```

---

## juice doctor

The killer feature. Catches every issue that wastes hours.

```
$ juice doctor

  [ OK  ] GPU: NVIDIA GeForce RTX 5050 Laptop GPU (8.0 GB VRAM)
  [ OK  ] Driver: v572.97
  [ OK  ] CUDA: v12.8
  [ OK  ] VRAM free: 7.4 GB free of 8.0 GB
  [ OK  ] CUDA_VISIBLE_DEVICES: set to '0' (registry)
  [ OK  ] Ollama: running on :11434
  [ OK  ] LM Studio: running on :1234
  [ OK  ]   qwen2.5:7b: 4.4 GB, tool calling supported
  [ WARN]   gemma3:4b: 3.1 GB, no tool calling support
            For agents, use qwen2.5:7b or llama3.1:8b instead.
  [ OK  ] Local GGUFs: 12 models found
  [ OK  ] llama-server: installed, CUDA backend found
  [ OK  ] faster-whisper: installed
  [ OK  ] cuBLAS (Whisper): found via pip (nvidia-cublas-cu12)

  16 passed, 1 warnings, 0 failures
```

Every warning includes an actionable fix. Use `--json` for machine-readable output that AI agents can parse and act on.

---

## juice serve

Serve any model with full GPU offload. Finds models from LM Studio and Ollama automatically.

```
$ juice serve qwen2.5:7b

  GPU:    NVIDIA GeForce RTX 5050 Laptop GPU  8.0 GB
  Model:  qwen2.5:7b  (4.4 GB, ollama)
  Loading model to GPU (first load takes a few seconds)...
  VRAM:   4.7/8.0 GB  [#########---]  59%
  Server: http://localhost:11434/v1  (Ollama)
```

Works with Ollama models, LM Studio GGUFs, or direct paths:

```
juice serve qwen2.5:7b            # Ollama model
juice serve gemma-3-4b            # fuzzy matches LM Studio GGUF
juice serve ~/models/model.gguf   # direct path, launches llama-server
```

---

## juice combo

Multi-model routing behind one endpoint. Simple questions go to a fast 1.5B model. Complex reasoning and tool calls go to a capable 7B. Your app sees one endpoint.

```
$ juice combo

  Fast model:  qwen2.5:1.5b
  Main model:  qwen2.5:7b
  Backend:     http://localhost:11434/v1
  Proxy:       http://localhost:8080/v1

  Simple tasks -> qwen2.5:1.5b (instant)
  Complex/tools -> qwen2.5:7b (accurate)
```

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="none")
# That's it. Your existing code works.
# Juice routes to the right model automatically.
```

---

## juice bench

Benchmark on YOUR hardware. Compare servers side by side.

```
$ juice bench

  Hardware: NVIDIA GeForce RTX 5050 Laptop GPU (8.0 GB)

  +---------------------------------------------------------------+
  | Server | Model      | tok/s |  TTFT |             VRAM | Tools |
  |--------+------------+-------+-------+------------------+-------|
  | Ollama | qwen2.5:7b |  13.2 | 652ms | 4.7/8.0 GB (59%) | pass |
  +---------------------------------------------------------------+

  Ollama: 13 tok/s / 652ms TTFT / 59% GPU / tools:pass on RTX 5050
```

Use `--json` for machine-readable output.

---

## All commands

| Command | What it does |
|---------|-------------|
| `juice doctor` | Full diagnostic — GPU, drivers, models, dependencies. `--json` for agents. |
| `juice serve <model>` | Launch with full GPU offload. Ollama, LM Studio, or GGUF path. |
| `juice combo` | Multi-model proxy — fast + main model, one endpoint. |
| `juice bench` | Benchmark tok/s, TTFT, VRAM, tool calls. `--json` for agents. |
| `juice status` | GPU info + detected servers. |
| `juice models` | Your local models (LM Studio + Ollama). |
| `juice recommend` | Suggested models for your VRAM tier. |
| `juice setup` | Download llama-server binary. |

---

## Why not just use Ollama?

Ollama is great. Use it. Juice sits alongside it and fixes the gaps:

- **GPU detection fails silently** on RTX 50-series. `juice doctor` catches this.
- **No way to verify** your model is actually on GPU. `juice doctor` checks VRAM.
- **No multi-model routing.** `juice combo` gives you a fast model + a smart model behind one endpoint.
- **Models without tool support** silently fail. `juice doctor` tells you which models support tool calling.

Juice doesn't replace Ollama. It makes it reliable.

---

## For AI agents

`juice doctor --json` outputs structured diagnostics that agents can parse and fix:

```json
{
  "check": "CUDA_VISIBLE_DEVICES",
  "status": "WARN",
  "message": "not set (RTX 50-series detected)",
  "fixes": [
    {
      "command": "setx CUDA_VISIBLE_DEVICES 0",
      "platform": "win32",
      "requires_restart": true,
      "destructive": false
    }
  ]
}
```

Add to your project's CLAUDE.md:
```
Run `juice doctor --json` and fix any warnings where destructive=false.
```

---

## Install

```
pipx install llamajuice
```

This puts `juice` on your PATH automatically. If you don't have pipx:

```
pip install llamajuice
```

If `juice` isn't found after pip install, either add Python's Scripts directory to PATH or use:

```
python -m llamajuice.cli doctor
```

Requires:
- Python 3.10+
- NVIDIA GPU
- Ollama or LM Studio (for models)

On first `juice setup`, the correct llama-server binary for your CUDA version is downloaded automatically (~220 MB, one time).

---

## Licence

MIT
