Mithril — Getting Started & Architecture Guide

What

Multi-Model Engine

Combine N models into one API

Providers

Local + Cloud + CLI

Tools

built-in (file, git, web, code)

Models

∞

Gemini, GPT, Claude, GGUF, Copilot, Junie

What is Mithril?

Mithril is a multi-model orchestration engine. You define a team of AI models (a "fellowship"), and Mithril exposes them as a single Ollama-compatible API endpoint.

Any tool that speaks Ollama — Junie, OpenCode, Open WebUI, LangChain — connects to Mithril and gets access to your orchestrated team.

How It Works

flowchart LR
    J(Junie) --> API(Mithril API)
    O(OpenCode) --> API
    W(Open WebUI) --> API
    API --> ORCH(Orchestrator)
    ORCH --> GEM(Gemini)
    ORCH --> GPT(OpenAI)
    ORCH --> COP(Copilot CLI)
    ORCH --> LOC(Local GGUF)

1. Universal Static Installer (Linux & macOS)

# Universal static zero-dependency binary with automatic PATH configuration:
curl -fsSL https://raw.githubusercontent.com/GiacomoSaccaggi/mithril/main/install.sh | bash

2. Python / Jupyter / Conda (PyPI)

# Direct pip install into active Python / Conda environment or Jupyter Notebook:
pip install mithril-cli
mithril --version

3. Homebrew (macOS & Linux)

# Install via official tap:
brew install GiacomoSaccaggi/tap/mithril
# or:
brew tap giacomosaccaggi/tap && brew install mithril

Complete setup

# 1. Install (via curl one-liner, pip, or brew)
curl -fsSL https://raw.githubusercontent.com/GiacomoSaccaggi/mithril/main/install.sh | bash
# Or in Python / Jupyter environments:
# pip install mithril-cli
# Or via Homebrew:
# brew install GiacomoSaccaggi/tap/mithril

# 2. Set API key
mithril config set gemini "AIzaSy..."

# 3. Download local model (optional, for free routing)
mithril download-model --model qwen-1.5b
# Or for local coding: mithril download-model --model qwen-14b

# 4. Create fellowship
cat > .mithril/fellowship.yaml << 'EOF'
name: "smart-team"
controller:
  provider: local
  model: qwen-1.5b
agents:
  - name: coder
    provider: gemini
    model: gemini-2.5-flash
    when: "coding tasks"
    can_call: [specialist]
    tools: ["*"]
  - name: specialist
    provider: copilot
    model: gpt-5.4
    when: "complex architecture, hard bugs"
    tools: ["*"]
  - name: local-coder
    provider: local
    model: qwen-14b
    when: "quick offline edits"
    tools: ["read_psi", "write_file", "edit_file"]
EOF

# 5. Start
mithril serve

# 6. Connect Junie: Settings → Ollama → http://localhost:16180

Docker deployment

# Local Docker
./docker-setup.sh
echo "MITHRIL_KEY_GEMINI=AIzaSy..." > .env
docker compose up -d
# → http://localhost:16180

# Remote server
ssh user@server
git clone https://github.com/GiacomoSaccaggi/Mithril.git && cd Mithril
echo "MITHRIL_KEY_GEMINI=AIzaSy..." > .env
docker compose up -d

# Connect: Junie → Ollama → http://server:16180

Provider	Type	Auth	Cost	Best For
`local`	GGUF	None	Free	Routing, offline coding (qwen-14b)
`gemini`	Cloud	API key	$$$	Main workhorse (cheap + fast)
`openai`	Cloud	API key	$$$$	GPT-4o, o1 reasoning
`anthropic`	Cloud	API key	$$$$	Claude for careful analysis
`groq`	Cloud	API key	$$	Fast inference (Llama, Mixtral)
`kiro`	CLI	AWS Builder ID	Credits	Claude Opus/Sonnet via Kiro
`copilot`	CLI	gh auth	2000/mo	GPT-5.4, cross-platform
`junie`	CLI	JetBrains	10 credits	Fable-5, integrated tools

Request Flow

Client sends chat request
GGUF classifier picks the right agent (free, ~100ms)
Agent executes with its provider + 24 tools
Auto-RAG injects relevant codebase context
Response streams back

Smart Budget Strategy

Use Gemini for 90% of work (cheap). Local qwen-14b for offline/private. Copilot/Junie only for tasks that need GPT-5.4 or Fable-5. Your 2000 Copilot credits last the whole month.