Metadata-Version: 2.4
Name: slm-forge-studio
Version: 0.1.0
Summary: Local fine-tuning studio for small language models targeting edge deployment
Author: SLM Forge Contributors
License: MIT
Project-URL: Homepage, https://github.com/rs1990/slm-forge
Project-URL: Repository, https://github.com/rs1990/slm-forge.git
Project-URL: Issues, https://github.com/rs1990/slm-forge/issues
Keywords: LLM,fine-tuning,edge,MLX,GGUF,LoRA
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: gradio<7,>=4.44.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: psutil>=6.0.0
Requires-Dist: huggingface_hub>=0.24.0
Requires-Dist: datasets>=2.20.0
Requires-Dist: pandas>=2.2.0
Requires-Dist: pyarrow>=15.0.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: numpy<2.0
Requires-Dist: keyring>=25.0.0
Requires-Dist: torch>=2.2.0
Requires-Dist: transformers<6,>=4.40.0
Requires-Dist: peft<1,>=0.10.0
Requires-Dist: trl<2,>=0.8.0
Requires-Dist: accelerate<2,>=0.28.0
Requires-Dist: rouge_score>=0.1.2
Requires-Dist: ddgs>=9.0.0
Requires-Dist: trafilatura>=1.12.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: pypdf>=5.0.0
Requires-Dist: python-docx>=1.1.0
Provides-Extra: apple
Requires-Dist: mlx>=0.21.0; extra == "apple"
Requires-Dist: mlx-lm>=0.20.0; extra == "apple"
Requires-Dist: mlx-vlm>=0.1.10; extra == "apple"
Provides-Extra: export
Requires-Dist: coremltools>=8.0; extra == "export"
Requires-Dist: optimum[onnxruntime]>=1.20.0; extra == "export"
Requires-Dist: onnx>=1.16.0; extra == "export"
Requires-Dist: ai-edge-torch>=0.2.0; extra == "export"
Provides-Extra: gguf
Requires-Dist: llama-cpp-python>=0.3.0; extra == "gguf"
Provides-Extra: mcp
Requires-Dist: mcp>=1.0.0; extra == "mcp"
Provides-Extra: agents
Requires-Dist: anthropic>=0.30.0; extra == "agents"
Requires-Dist: apscheduler>=3.10.0; extra == "agents"
Requires-Dist: PyGithub>=2.1.0; extra == "agents"
Provides-Extra: rag
Requires-Dist: sentence-transformers>=3.0.0; extra == "rag"
Provides-Extra: run
Requires-Dist: slm-forge-studio[agents,rag]; extra == "run"
Provides-Extra: full
Requires-Dist: slm-forge-studio[agents,apple,export,gguf,rag]; extra == "full"
Requires-Dist: rouge_score>=0.1.2; extra == "full"
Requires-Dist: lm-eval>=0.4.3; extra == "full"
Provides-Extra: dev
Requires-Dist: pyinstaller>=6.0.0; extra == "dev"
Requires-Dist: pytest>=8.0.0; extra == "dev"

# SLM Forge

**Local fine-tuning studio for small language models — hardware-agnostic, edge-deployment ready.**

SLM Forge is an end-to-end workbench for taking a small open-weight model
(135 M – 7 B parameters), adapting it to your data, compressing it, and shipping it
to a phone, Raspberry Pi, Jetson Nano, or any low-RAM target — all from one
local Gradio UI, with no cloud account required.

It exists because most of the fine-tuning ecosystem assumes a CUDA server farm
and a "deploy to a GPU box" finish line. SLM Forge trains on **whatever you have**
— a $400 base-model Mac, a workstation with a 4090, a 32 GB Linux box, even
CPU-only — and aims at a finish line that says *"this model now runs at 30
tokens/sec on a $35 Pi."*

> **Platform support, honestly:** the full pipeline (train → fuse → export →
> chat) runs on every platform. On Apple Silicon it's MLX end to end,
> including MLX 4-bit packing and TurboQuant. On NVIDIA/CPU the adapter is
> fused via PyTorch (`peft merge_and_unload`) to fp16, compression happens at
> GGUF export (Q4_K_M etc.), and chat uses the GGUF backend
> (`llama-cpp-python`). MLX-runtime artifacts and TurboQuant remain Mac-only.

### Hardware-agnostic by default

The app probes your machine on launch (`core/system_probe.py` →
`core/system_tiers.py` → `core/backend.py`) and adapts:

| Detected | What changes |
|---|---|
| Apple Silicon (M1–M4, any RAM) | Picks **MLX/Metal** trainer; tier from unified RAM (S ≥ 32 GB, A ≥ 16 GB, B ≥ 7 GB, C otherwise) |
| NVIDIA CUDA (single or multi-GPU) | Picks **PyTorch + PEFT + TRL**; tier from per-GPU VRAM; auto-enables 4-bit / 8-bit QLoRA when VRAM is tight |
| Apple Silicon without MLX installed | Falls back to **PyTorch MPS** |
| CPU only | Refuses unrealistic models, recommends the smallest viable catalog entry |
| < 4 GB free | Marks as inference-only; refuses to start a training run |

You see the detected tier and a recommended starting model in the banner the
moment the app opens. The Wizard tab then refines the entire pipeline (LoRA
params, quantization, export targets) to that exact hardware — no manual
"what tier am I" guessing.

---

## Who it's for

- **Indie developers** shipping an on-device LLM feature in a mobile / desktop app.
- **Hardware / robotics teams** putting a domain-tuned assistant on a Jetson or Pi.
- **Researchers** who want to try LoRA + distillation experiments without renting GPUs.
- **Privacy-sensitive teams** (legal, medical, defense) who can't send data to a cloud trainer.
- **Educators** teaching modern fine-tuning without per-student cloud bills.

Whether you're on a base-model MacBook Air, a workstation with a 4090, or a
32 GB Linux box, the app picks a model and training config that fits your
machine — you don't need to know what "tier" you are before opening it.

---

## What you can build with it

| Application | Pipeline |
|---|---|
| Offline chat assistant for a phone app | Dataset → LoRA → 4-bit quant → Core ML / GGUF export |
| Domain-specific code completion (firmware, niche DSL) | Code dataset → LoRA on Qwen2.5-Coder → GGUF for VS Code extension |
| On-device document classifier (e.g. triage emails) | Tabular CSV → classify head → ONNX/TFLite export |
| Robot / kiosk vision Q&A | Imagefolder → SmolVLM LoRA → MLX or GGUF on Jetson |
| Speech-to-text for a Pi-based voice gadget | Whisper-tiny LoRA on accented data → quantize → run via `whisper.cpp` |
| Distilled tutor model from a 7 B teacher | Distill tab generates synthetic data → train 360 M student → export |
| Privacy-preserving RAG over local docs | Index folder → retrieve top-k → augment chat context |
| Knowledge-baked SLM (no RAG at runtime) | LoRA-bake your corpus into a 1.5 B model so the device works fully offline |

---

## The pipeline

```
[Dataset]  →  [Distill]  →  [Train]  →  [Quantize]  →  [Export]  →  [Validate]  →  [Chat]
                                                            │
                                                            ├── GGUF   (llama.cpp, Ollama, LM Studio)
                                                            ├── MLX    (Apple Silicon native)
                                                            ├── Core ML (iOS / macOS apps)
                                                            ├── ONNX   (cross-platform inference)
                                                            └── TFLite (Android / microcontrollers)
```

Each step is a tab. You can stop after any step. Everything for one job is
written under `runs/<timestamp>/`, so reproducibility is automatic.

---

## Features

### Data
- **Auto-detection**: drop in JSONL / CSV / Parquet / HF Hub id / image folder; the Dataset tab
  classifies modality (text / vision / audio / code / reasoning / time-series), suggests a column mapping, and normalizes to one canonical chat format.
- **Two-tier modality detection**: keyword scoring first, LLM fallback if ambiguous.
- **Dataset packing** for short-example efficiency (Axolotl-style).
- **Vision sandbox**: image paths sandboxed inside the dataset root — no traversal.

### Training
- **LoRA / QLoRA / DoRA / PiSSA / LoRA+** via MLX-LM (Apple) or HF Transformers + PEFT + TRL (CUDA).
- **DPO** for preference alignment in addition to SFT.
- **Knowledge distillation**: any local model or HF Inference API model can act as the teacher.
- **Classification head trainer**: skip generative LoRA and train a tiny classifier head directly.
- **Multi-GPU** when launched on CUDA via `accelerate launch` (single-node).
- **Live training log**: streams loss / val loss / peak RAM / tokens per second; cancellable.
- **Parameter advisor**: suggests batch size, LoRA rank, layers based on detected RAM.
- **Pre-flight checks**: refuses to start a training run that won't fit available memory.

### Quantization & Compression
- **MLX 4-bit / 8-bit weight quantization** with configurable group size.
- **TurboQuant** (opt-in, experimental on Apple Silicon) — KV cache compressed
  ~5–6× via random rotations + Lloyd-Max codebooks. Useful for ≥ 2 k context on edge.
- **QAT-aware catalog entries** (e.g. Gemma 3 QAT 4-bit) preferred when available.

### Export (multi-target)
| Target | Why |
|---|---|
| **GGUF** (Q4_K_M, Q5_K_M, Q8_0, F16) | Most portable edge format. Works with `llama.cpp`, Ollama, LM Studio. |
| **MLX** | Native on iPhone/iPad/Mac via MLX-Swift. Fastest on Apple Silicon. |
| **Core ML** | iOS/macOS app store apps via the Core ML framework. |
| **ONNX** | Cross-platform inference on Windows, Linux, mobile via ONNX Runtime. |
| **TFLite** | Android, ARM microcontrollers (TFLite Micro on supported chips). |
| **Ollama Modelfile** | One-line `ollama create` for local serving. |
| **Hugging Face Hub publish** | Auto-generated model card from your run metadata. |

### Validation
- Bench your exported model on a **target device profile** (Pi 4, Pi 5, Jetson Orin Nano,
  M2 MacBook, generic ARM, etc.). Reports peak RAM and inference tokens/sec
  before you ship it to hardware.
- Refuses unrealistic targets — e.g. won't pretend a 1 B generative LLM fits on
  an ESP32; recommends a sentence-embedding model instead.

### Chat / Eval / Experiments
- **Chat tab** with both MLX and GGUF backends — try the actual edge artifact, not just the trained one.
- **Eval tab** — perplexity + exact match + ROUGE-L on a held-out set.
- **Experiments tab** — kanban of past runs with metrics, model card preview, one-click publish to HF Hub.

### RAG
- Local file indexing (sentence-transformers).
- Augments chat context with retrieved chunks at inference time.
- Useful as an alternative to baking knowledge into LoRA when freshness matters.

### Infrastructure
- **Curated catalog** (`catalog.yaml`) of edge-friendly models, RAM-tagged
  (🟢 comfortable / 🟡 tight / 🔴 won't fit) against your machine.
- **Weekly auto-refresh** from `mlx-community` HF trending; hand-edited entries preserved.
- **Telemetry is opt-in** and asks once on first run. Crash reports are sanitized
  (paths stripped, secrets redacted) before posting.
- **Background scheduler** for hourly issue triage / weekly digest if you wire
  in the admin dashboard.

---

## Comparison

The honest version. SLM Forge wins on **end-to-end + Apple Silicon + edge focus**
and loses on **raw training throughput on a CUDA box**.

| | SLM Forge | Ollama / LM Studio | Axolotl | Unsloth | LLaMA Factory | HF AutoTrain | MLX-LM CLI |
|---|---|---|---|---|---|---|---|
| **Local UI** | ✅ Gradio | ✅ inference UI | ❌ YAML | ❌ Python | ✅ Web UI | ❌ cloud | ❌ |
| **Trains models** | ✅ | ❌ inference only | ✅ | ✅ | ✅ | ✅ paid | ✅ CLI |
| **Apple Silicon native (MLX)** | ✅ primary | ✅ inference | ⚠️ via PT | ❌ CUDA-only | ❌ | ❌ | ✅ |
| **CUDA training** | ✅ secondary | n/a | ✅ primary | ✅ primary | ✅ primary | ✅ | ❌ |
| **Multi-format edge export** | ✅ 5 formats | ❌ GGUF only | ❌ | ❌ | ❌ | ❌ | ❌ |
| **Pre-flight RAM check** | ✅ | ❌ | ❌ | ❌ | ❌ | n/a | ❌ |
| **Distillation built-in** | ✅ | ❌ | ⚠️ manual | ❌ | ⚠️ | ❌ | ❌ |
| **Vision LoRA** | ✅ MLX-VLM | ❌ | ✅ | ⚠️ | ✅ | ✅ | ⚠️ |
| **Classification head** | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ |
| **Cloud account required** | ❌ never | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ |
| **Cost** | free | free | free | free | free | $$ | free |
| **Data leaves your machine** | ❌ never (default) | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ |

### Where each tool is best

- **Use Ollama / LM Studio** if you only want to *run* an existing model. They don't train.
- **Use Unsloth** if you have a beefy NVIDIA GPU and want the fastest possible LoRA training of a 7B–70B model. SLM Forge is slower in that regime by design — it targets MLX.
- **Use Axolotl** if you live in YAML configs, want every esoteric training recipe, and run on rented A100s / H100s.
- **Use LLaMA Factory** if you want a similar all-in-one UI but on CUDA and don't care about edge export.
- **Use HF AutoTrain** if you have a budget but no hardware and don't mind your data hitting their cloud.
- **Use SLM Forge** if you want to take a model from raw dataset to a quantized artifact running on a Pi, on your own laptop, with a UI that won't let you OOM yourself, and without a cloud bill.

---

## Pros / cons

### Pros
- **End-to-end in one app** — no juggling Axolotl + llama.cpp + a separate quantizer + a separate Core ML converter.
- **Apple Silicon native** — uses MLX, not a Python+CUDA shim emulating MLX. Trains and infers on the unified memory chip.
- **Multi-format export pipeline** — same trained adapter exports to GGUF, MLX, Core ML, ONNX, TFLite from one button.
- **Memory-honest** — every model in the catalog is tagged with its real train + inference RAM cost, and the trainer pre-flights to refuse impossible runs.
- **Local by default** — no telemetry without consent, no cloud account, no data egress.
- **Curated catalog auto-refreshed** from `mlx-community` so you don't have to hunt for "is this 4-bit converted yet?"
- **Reproducible runs** — every job persists `run_meta.json` with seed, hyperparams, dataset hash, metrics, and exports.
- **Hardened**: env-var whitelist, secret-masked logs, sandboxed image paths, opt-in `trust_remote_code`, optional HTTPS + basic auth in server mode. See `SECURITY.md`.

### Cons (be honest)
- **Single-machine.** No multi-node distributed training. If you're training a 70B model, this is the wrong tool.
- **LoRA / QLoRA / DPO only — no full fine-tuning.** Full FT of even a 1B model is impractical on 8 GB unified memory anyway.
- **MLX path is more battle-tested than CUDA today.** CUDA training is fully wired up (single + multi-GPU via `accelerate launch`, auto-QLoRA) and the subprocess code path is exercised end-to-end on every push (`cuda-e2e-cpu` job in `.github/workflows/tests.yml`), but live GPU runs still need a self-hosted CUDA runner — treat as beta on real cards. Catalog RAM calibrations and most benchmark numbers were collected on Apple Silicon.
- **Catalog-driven.** Free-text HF model IDs work but skip the RAM safety badges. A model not in `catalog.yaml` may load but blow past your RAM budget without warning.
- **Export coverage by format.** GGUF (text models) and MLX are verified end-to-end on a real model. ONNX is verified via its isolated converter env. Core ML builds via its isolated env (decoder-only text models; vision LLMs are not supported by that path). TFLite (ai-edge-torch) is Linux-first and best-effort — on macOS use GGUF. See `scripts/validate_export_e2e.py`.
- **No RLHF / PPO** — only DPO is supported for preference learning.
- **No live-GPU CI** — 167 unit tests + a CPU-mode CUDA subprocess validator run on every push (`.github/workflows/tests.yml`), and a 5-min UI smoke run is still the recommended pre-release check on Apple Silicon. Live NVIDIA validation requires a self-hosted runner that doesn't exist yet.
- **TurboQuant is experimental** on Apple Silicon and depends on a community fork of `turboquant_plus`.

---

## What runs where

SLM Forge is a local app: it trains on *your* hardware, so what you get depends
on the machine, not on a plan tier. The full pipeline — train → fuse → export →
chat — works on all three rows below.

| | Apple Silicon (macOS) | Linux / Windows + NVIDIA | Linux / Windows, CPU only |
|---|---|---|---|
| Train (LoRA / LoRA+ / DPO) | MLX | CUDA (peft + trl) | works, slow |
| Fuse adapter | MLX packing, 2–8 bit | PyTorch `merge_and_unload` → fp16 | same |
| Export GGUF / ONNX | yes | yes | yes |
| Export Core ML | yes | — | — |
| Export TFLite | prefer GGUF | yes (ai-edge-torch is Linux-first) | yes |
| Chat | MLX or GGUF | GGUF | GGUF |
| RAG, Eval, Classify, Distill | yes | yes | yes |
| TurboQuant KV-cache | yes | — | — |

Apple-only pieces are MLX N-bit packing and TurboQuant. Everywhere else you pick
your compression at GGUF export (`Q4_K_M` and friends) instead — the pipeline
does not dead-end.

The hosted demo at [slm-forge.onrender.com](https://slm-forge.onrender.com) is a
showcase, not the product: it runs with `SLMFORGE_MEMORY_CONSTRAINED=1`, which
refuses build jobs. Install locally for anything real.

## Quickstart

### One-line install (recommended)

No Python setup required — the installer fetches its own (via [`uv`](https://docs.astral.sh/uv/)),
creates an isolated environment, installs the right backend for your machine
(MLX on Apple Silicon, PyTorch elsewhere), and adds a `slm-forge` launcher.

**macOS / Linux**
```bash
curl -fsSL https://raw.githubusercontent.com/rs1990/slm-forge/master/installer/install.sh | bash
```

**Windows** (PowerShell)
```powershell
irm https://raw.githubusercontent.com/rs1990/slm-forge/master/installer/install.ps1 | iex
```

Then run `slm-forge` (or use the Start Menu / desktop shortcut) and open
http://localhost:7860. Re-run the installer any time to upgrade. It installs to
`~/.slm-forge` (`%LOCALAPPDATA%\slm-forge` on Windows); uninstall by deleting
that folder and the launcher.

### pip / uv

The distribution is published as **`slm-forge-studio`** (PyPI rejects
`slm-forge` — it folds `l`/`i` to `1` and strips separators, which collides with
an unrelated `simforge` project). The command it installs is still `slm-forge`.

```bash
pip install "slm-forge-studio[apple]"   # macOS, Apple Silicon (MLX)
pip install "slm-forge-studio[run]"     # Linux / Windows (PyTorch, RAG, agents)
slm-forge
```

Runs and a user-editable model catalog live in `~/.slmforge` (override with
`SLMFORGE_HOME`). GGUF export needs `pip install llama-cpp-python` or a system
`llama.cpp`; see the note below.

### From source (developers)
```bash
git clone https://github.com/rs1990/slm-forge.git
cd slm-forge
pip install -r requirements.txt
python app.py
```
Open http://localhost:7860.

For GGUF export (the most common edge target):
```bash
pip install llama-cpp-python gguf sentencepiece
# or, if you prefer the system binary:
brew install llama.cpp     # macOS
```

**Core ML / ONNX / TFLite** don't need a manual install. Their converter
toolchains (coremltools / optimum / ai-edge-torch) pin an *older*
`transformers`/`torch` that conflicts with the trainer, so the Export tab builds
each one a **cached isolated virtualenv** on demand (the "Install tooling for
&lt;format&gt;" button, or automatically on first export). One-time, ~2-4 GB per
format, under `~/.cache/slm-forge/export-envs/`. ai-edge-torch (TFLite) is
Linux-first; on macOS prefer GGUF for Android/edge.

### One-command launcher (main app + admin + scheduler)
```bash
./launch.sh                # macOS / Linux
launch.bat                 # Windows
python launch.py           # any platform
```
Loads `.env.local` (whitelisted vars only), starts the main UI on `:7860` and
the admin dashboard on `:7861`.

### Server mode (multi-user, internet-facing)
```bash
export SLMFORGE_SERVER=1
export SLMFORGE_PASSWORD='a-strong-password'
# optional HTTPS:
export SSL_CERTFILE=cert.pem SSL_KEYFILE=key.pem
python app.py
```
Without `SLMFORGE_PASSWORD` the app refuses to start in server mode. Always
front this with a reverse proxy and rate-limiting in real deployments.

### Docker
```bash
docker build -t slm-forge .
docker run -it -p 7860:7860 \
  -e SLMFORGE_PASSWORD='strong-password' \
  -v $(pwd)/runs:/app/runs slm-forge
```
Container runs as non-root user `slmforge` (uid 10001).

---

## Smoke run (≈ 5 minutes on most hardware)

The app's banner picks a starter model that fits your machine, so you don't
need to memorize sizes. As a generic walkthrough:

1. **Wizard tab** → look at the detected tier + recommended model → type a one-line description ("summarize support tickets on a Pi 5") → **Plan** → **Apply to pipeline**.
2. **Dataset tab** → upload `assets/smoke_dataset.jsonl` → **Detect** → **Normalize**.
3. **Train tab** → the recommended model is already selected → **Start training**.
4. **Quantize tab** → defaults are tier-appropriate (4-bit weights, group 64) → **Quantize**.
5. **Export tab** → check `gguf` (or `mlx` / `coreml` / `onnx` / `tflite` depending on your edge target) → **Export**.
6. **Validate tab** → pick the device profile you'll deploy to → reports peak RAM and tok/s on that target.
7. **Chat tab** → talk to the result.

If you want to skip the wizard, the Train tab tags each catalog model with a fit badge:

| green: comfortable | yellow: tight | red: won't fit |
|---|---|---|

These badges are computed live against your *actual* free memory (and per-GPU VRAM on CUDA), not against any assumed reference machine.

---

## Hardware tiers

The tier classifier (`core/system_tiers.py`) maps detected hardware → coarse tier → defaults. **The app does this for you on launch**; the table is here so you know what to expect.

| Tier | Apple Silicon (unified RAM) | NVIDIA CUDA (per-GPU VRAM) | Recommended start | Trainable range |
|---|---|---|---|---|
| **S** — workstation | ≥ 32 GB | ≥ 40 GB (A100 / H100) | Phi-3.5-mini (3.8 B) | up to 7 B comfortably, 13 B tight |
| **A** — pro laptop | 16 – 32 GB | 20 – 40 GB (3090 / 4090 / A6000) | Qwen2.5-1.5B | up to 3 B comfortably, 7 B tight |
| **B** — entry laptop | 7 – 16 GB | 10 – 20 GB (3080 / 4070) | Qwen2.5-1.5B | 135 M – 1.5 B at 4 LoRA layers |
| **C** — minimal | < 7 GB free, no accelerator, or 4 – 10 GB VRAM | same | SmolLM2-360M | 135 M – 500 M only |
| **D** — too small | < 4 GB free | < 4 GB VRAM | (refused) | inference only |

CUDA tiers use the *smallest* GPU in the job as the bottleneck. 4-bit QLoRA is auto-enabled when per-GPU VRAM is < 8 GB, 8-bit when < 16 GB.

---

## MCP server

`mcp_server.py` exposes SLM Forge over the [Model Context Protocol](https://modelcontextprotocol.io)
(stdio), so an MCP client — Claude Desktop, Claude Code, or any MCP host — can
drive it programmatically. It's a thin wrapper over the same CLI/core, kept
**torch-free at import time**: heavy operations (train, eval, publish, index)
shell out to `python -m cli`, so the server starts instantly and an ML import
failure can't take it down.

### Tools

| Tool | Does |
|---|---|
| `list_models(modality?)` | Curated base models from `catalog.yaml` (RAM, LoRA layers, edge targets). |
| `list_runs()` | All runs with headline metrics, newest first. |
| `get_run(run_id, metrics_tail?)` | Full `run_meta.json` + tail of the metric stream. |
| `train_start(model, data, iters?, batch?, num_layers?, lora_rank?, learning_rate?)` | Starts a LoRA fine-tune **in the background**, returns `run_id` immediately. |
| `train_status(run_id)` | Concise live status: state + latest loss/iter. |
| `eval_run(run_id, data?, n_gen?)` | Score a finished run (blocks — loads a model). |
| `publish_run(run_id, repo, format?, private?)` | Push a run to the HF Hub (needs `HF_TOKEN`). |
| `build_index(corpus, name, run_id?, model?, chunk_size?, chunk_overlap?)` | Build a RAG index from a corpus. |

Training is long-running, so it's asynchronous: `train_start` returns a `run_id`
right away; poll `train_status(run_id)` (or `get_run`) until `status` is
`completed` or `failed`.

### Install & run

```bash
pip install -e ".[mcp]"     # installs the `mcp` SDK
slm-forge-mcp               # stdio server (or: python mcp_server.py)
```

### Client config (Claude Desktop / Claude Code)

```json
{
  "mcpServers": {
    "slm-forge": {
      "command": "slm-forge-mcp",
      "env": { "HF_TOKEN": "hf_..." }
    }
  }
}
```

Or, without installing the console script:

```json
{
  "mcpServers": {
    "slm-forge": {
      "command": "python",
      "args": ["/absolute/path/to/slm-forge/mcp_server.py"]
    }
  }
}
```

`HF_TOKEN` is only needed for `publish_run` and for pulling gated base models.

---

## Project layout

```
slm-forge/
├── app.py                    Gradio entrypoint, tab wiring
├── cli.py                    Command-line interface (train/eval/publish/index)
├── mcp_server.py             MCP server (stdio) wrapping the CLI/core
├── admin.py                  Issue triage / kanban dashboard (port 7861)
├── launch.{sh,py,bat}        One-command launchers (whitelist .env.local)
├── core/
│   ├── data/catalog.yaml     Curated edge-friendly model catalog (ships in the wheel)
│   ├── paths.py              Checkout vs installed layout (runs dir, catalog)
│   ├── trainer.py            MLX-LM LoRA trainer
│   ├── trainer_cuda.py       CUDA / accelerate trainer
│   ├── trainer_classify.py   Tiny classifier head
│   ├── distiller.py          Local + HF API distillation
│   ├── quantizer.py          4/8-bit weight quant + TurboQuant config emit
│   ├── exporter/             gguf, mlx, coreml, onnx, tflite, ollama_modelfile
│   ├── dataset_*.py          Detection + normalization (text/vision/tabular)
│   ├── rag_*.py              Indexing + retrieval
│   ├── evaluator.py          Perplexity + ROUGE + exact match
│   ├── target_validator.py   Pre-flight + post-export device checks
│   ├── secrets.py            Keyring-first HF token resolution
│   ├── telemetry.py          Opt-in logging with secret masking
│   ├── reporter.py           Sanitized GitHub Issue crash reports
│   ├── agents.py             Claude API agents for triage / planning
│   └── scheduler.py          Background hourly + weekly tasks
├── ui/                       One module per Gradio tab
├── runs/<timestamp>/         data/, adapter/, quantized/, exports/, train.log, run_meta.json
└── third_party/turboquant/   Vendored community fork (created on demand)
```

---

## Roadmap

- **Phase 2 (shipped)**: System-aware auto-recommendation. Probes RAM, accelerator, OS, empirical-benchmark cache, then auto-picks model + config so non-experts skip the badges. Use the **Wizard** tab.
- **Phase 3**: Per-format export verification harness (load each artifact in its target runtime, generate, compare against MLX baseline).
- **Backlog**: streaming speech (TTS), multi-step agent fine-tuning, federated LoRA, ESP32-S3 generative path (likely impossible without a custom kernel).

---

## Security

- All hardening fixes from the SECURITY.md audit are implemented (env whitelist, secret-masking logs, sandboxed image paths, server auth, sanitized crash reports, opt-in `trust_remote_code`).
- The Render Docker image runs as non-root.
- Server mode refuses to launch without `SLMFORGE_PASSWORD`.
- Report security issues by email — see `SECURITY.md` § "Vulnerability reporting".

---

## Contributing

Run `pytest tests/ -v` before submitting a PR (167 tests, no GPU required). The CI workflow at `.github/workflows/tests.yml` runs the same suite plus the CUDA subprocess validator (`scripts/validate_cuda_e2e.py`) on every push. The 5-min UI smoke flow above is still the recommended end-to-end check before tagging a release.
For larger changes, open an issue first — the admin dashboard agents will triage it.

---

## License

Apache 2.0. See `LICENSE`.

Trained model artifacts inherit the upstream model's license (most catalog
entries are Apache 2.0 / MIT; check `catalog.yaml` and the source HF model card).
