Metadata-Version: 2.4
Name: yaarbal
Version: 0.1.7
Summary: BitTorrent-inspired peer-to-peer distributed LLM inference
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/hannanabdul55/distributed-full-gpt-inference
Project-URL: Issues, https://github.com/hannanabdul55/distributed-full-gpt-inference/issues
Project-URL: Documentation, https://github.com/hannanabdul55/distributed-full-gpt-inference/blob/main/README.md
Project-URL: Source, https://github.com/hannanabdul55/distributed-full-gpt-inference
Project-URL: Known Limitations, https://github.com/hannanabdul55/distributed-full-gpt-inference/blob/main/KNOWN_LIMITATIONS.md
Keywords: llm,p2p,distributed,inference,bittorrent,ai,ml
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Distributed Computing
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: cryptography<47,>=46.0.7
Requires-Dist: flask<4,>=3.1.3
Requires-Dist: python-dotenv<2,>=1.2.1
Requires-Dist: requests<3,>=2.32.5
Requires-Dist: torch<3,>=2.9
Requires-Dist: numpy<3,>=2.1
Requires-Dist: transformers<5,>=4.57.6
Requires-Dist: safetensors>=0.7.0
Requires-Dist: lz4<5,>=4.4.5
Provides-Extra: dev
Requires-Dist: pytest==8.4.2; extra == "dev"
Requires-Dist: pytest-cov==7.1.0; extra == "dev"
Requires-Dist: psutil>=5.9.0; extra == "dev"
Provides-Extra: gguf
Requires-Dist: llama-cpp-python==0.3.20; extra == "gguf"
Dynamic: license-file

<!-- generated-by: gsd-doc-writer -->
# Yaarbal

Yaarbal is a BitTorrent-inspired peer-to-peer system for running LLM inference across distributed nodes. Each peer hosts a complete model; a tracker brokers peer discovery; clients route inference to available seeders. Hosting earns credits, querying spends them — no central GPU pool, just peers.

> **Status:** v2.1 milestone in progress. Core whole-model P2P inference, credit economy, browser chat UI, and the `yaarbal` CLI are working. Treat the public reference network as demo-grade — production hardening (TLS, automatic failover, DHT) is on the roadmap.

Visit [yaarbal.app](https://yaarbal.app) for the public reference network. <!-- VERIFY: yaarbal.app demo network availability -->

---

## Installation

```bash
pip install yaarbal
yaarbal --version
```

Requires **Python 3.11 or newer**. The wheel pulls in PyTorch, Transformers, Flask, and the rest of the runtime stack — no manual model setup needed.

For a source/development install:

```bash
git clone https://github.com/hannanabdul55/distributed-full-gpt-inference.git
cd distributed-full-gpt-inference
python -m venv venv && source venv/bin/activate
pip install -e ".[dev]"
```

See [docs/distribution/](docs/distribution/) for Docker and other distribution paths.

---

## Quick Start

The fastest path is **single-node mode** — one machine, one model, no tracker. Best for trying the system or running a local assistant.

### 1. Start a seeder

```bash
yaarbal seed --model qwen2.5-0.5b-instruct --port 7001
```

Loads the full model weights and serves inference on port 7001.

### 2. Chat with it

In a second terminal:

```bash
YAARBAL_CHAT_BACKEND=single \
YAARBAL_SINGLE_NODE_URL=http://127.0.0.1:7001 \
yaarbal chat --model qwen2.5-0.5b-instruct
```

Type messages and press Enter. Use `/health`, `/metrics`, `/models`, `/reset`, or `/quit` inside the REPL.

For the full quickstart — including the distributed swarm path, Docker profiles, and the browser UI — see **[QUICKSTART.md](QUICKSTART.md)**.

---

## How It Works

```
   prompt
     │
     ▼
  ┌─────────┐         ┌────────────┐
  │ Client  │ ──────▶ │  Tracker   │   discover seeders for model X
  │ (CLI /  │ ◀────── │            │
  │ web UI) │         └────────────┘
  └────┬────┘                │
       │ POST /inference     ▼
       ▼               ┌──────────┐
  ┌─────────┐  ◀────── │  Seeder  │   hosts complete model X
  │  text   │          │  (full   │   earns credits per request
  │ output  │          │   model) │
  └─────────┘          └──────────┘
```

- **Tracker** — peer discovery, swarm management, credit settlement.
- **Seeder** — loads the full model weights and serves `POST /inference`. One seeder = one model on one host.
- **Client / leecher** — discovers a seeder via the tracker, routes the prompt, returns text.

The whole-model design (post-2026-04-12 pivot) means each peer hosts an entire model rather than a layer-shard. Layer-split inference was removed in Phases 51–52; whole-model is the only paradigm.

For the protocol design, credit economy, and trust tiers, see **[PROJECT_OVERVIEW.md](PROJECT_OVERVIEW.md)** and **[docs/architecture/](docs/architecture/)**.

---

## Common Commands

```bash
# CLI
yaarbal seed --model <name> --port 7001        # host a model
yaarbal run <model> "<prompt>"                  # one-shot inference via the swarm
yaarbal chat --model <name>                     # interactive REPL
yaarbal tracker 8000                            # host a tracker
yaarbal ui                                      # browser chat UI on localhost:5173
yaarbal status                                  # credits, hosted models, peers
yaarbal --list-models                           # show all 30+ supported models

# Docker
docker compose --profile single-node up single-node    # tracker + one seeder
docker compose up                                       # tracker only
docker compose --profile query run --rm leecher         # one-shot inference

# Tests
make test                                       # full suite
make test-unit                                  # unit tests only
make test-integration                           # integration tests
python -m pytest tests/unit/test_tracker.py -v  # single file

# Memory / benchmarks
make verify-memory
```

> Environment variables use the canonical `YAARBAL_*` prefix. Legacy `DGPT_*` aliases still work in v2.1 but emit a one-time deprecation warning.

---

## Supported Models

30+ models are registered out of the box, including:

| Family | Variants | Sizes |
|--------|----------|-------|
| GPT-2 | 5 variants | 82M – 1.5B |
| LLaMA | 3.2-1B, 3.2-3B, 3.1-8B, 3.3-70B | 1B – 70B |
| Mistral / Mixtral | 7B, 8x7B (MoE) | 7B – 46.7B |
| Phi | 3-Mini (4K/128K), 3-Medium | 3.8B – 14B |
| Qwen 2.5 | 7 sizes | 0.5B – 72B |
| DeepSeek | V3 (671B MoE), R1-Distill-Qwen-1.5B | 1.5B – 671B |
| MiniGPT | tiny test model | <100M |

LLaMA 8B and larger require a Hugging Face access token (`HF_TOKEN`).

See **[MODELS.md](MODELS.md)** for the full catalog with hardware requirements and per-model notes.

---

## Browser UI

```bash
yaarbal ui
```

Opens the chat UI at `http://127.0.0.1:5173`. Paste either a seeder URL or a tracker URL into the Server URL field — the SettingsDialog auto-detects the server type via `GET /health`. Configure model, backend, and connection in the Connection panel.

The UI is a Vite + React + TypeScript app under `web/`. Run `npm install && npm run dev` from `web/` for local development.

---

## Configuration

Configure Yaarbal via environment variables (canonical `YAARBAL_*` prefix). See `.env.example` and `config.py` for the complete list. Common ones:

| Variable | Purpose |
|----------|---------|
| `YAARBAL_MODEL` | Default model name |
| `YAARBAL_TRACKER_URL` | Tracker URL (default: `https://tracker.yaarbal.app`) |
| `YAARBAL_CHAT_BACKEND` | `single` (direct seeder) or `tracker` |
| `YAARBAL_SINGLE_NODE_URL` | Seeder URL when `CHAT_BACKEND=single` |
| `YAARBAL_AUTH_USER` / `YAARBAL_AUTH_PASS` | Optional HTTP Basic Auth |
| `HF_TOKEN` / `HUGGINGFACE_HUB_TOKEN` | Required for gated models (LLaMA 8B+) |

Full reference: **[docs/configuration/overview.md](docs/configuration/overview.md)**.

---

## Documentation

| Doc | What it covers |
|-----|----------------|
| [QUICKSTART.md](QUICKSTART.md) | Two-minute guide: single-node, swarm, Docker, browser UI |
| [PROJECT_OVERVIEW.md](PROJECT_OVERVIEW.md) | Engineering deep-dive: protocol, credit economy, code layout |
| [MODELS.md](MODELS.md) | Catalog of 30+ models with hardware requirements |
| [KNOWN_LIMITATIONS.md](KNOWN_LIMITATIONS.md) | Open issues (P0–P3), security caveats, roadmap items |
| [CHANGELOG.md](CHANGELOG.md) | Version history |
| [docs/architecture/](docs/architecture/) | Protocol design, BitTorrent mechanisms, topology |
| [docs/api/](docs/api/) | HTTP API reference (tracker + seeder) |
| [docs/configuration/](docs/configuration/) | Environment variables and tuning |
| [docs/deployment/](docs/deployment/) | Production deployment guides |
| [docs/guides/](docs/guides/) | Getting started, development, operations |
| [docs/research/](docs/research/) | Academic paper draft and related work |

---

## Known Limitations

- **No TLS** — tensor data is plaintext over HTTP unless fronted by a TLS-terminating proxy.
- **HTTP Basic Auth only** — optional, off by default; not a substitute for TLS.
- **Central tracker is a single point of failure** — DHT is on the roadmap, not yet implemented.
- **Sequential per-request latency** — single-request latency does not benefit from peer parallelism (only throughput does).
- **Gated models require HF auth** — LLaMA 8B+ needs `HF_TOKEN`.

Full list with severity and tracking: **[KNOWN_LIMITATIONS.md](KNOWN_LIMITATIONS.md)**.

---

## License

Apache License 2.0. See [LICENSE](LICENSE).
