Metadata-Version: 2.4
Name: mlx-mcp-server
Version: 0.7.0
Summary: MCP server bridging Codex to local MLX LM (and any OpenAI-compatible backend)
Project-URL: Repository, https://github.com/deresolution20/mlx-mcp-server
License: MIT
License-File: LICENSE
Keywords: ai,claude,codex,llm,mcp,mlx
Requires-Python: >=3.11
Requires-Dist: httpx>=0.27.0
Requires-Dist: mcp[cli]>=1.0.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Description-Content-Type: text/markdown

# MLX MCP Server

[![PyPI version](https://img.shields.io/pypi/v/mlx-mcp-server.svg)](https://pypi.org/project/mlx-mcp-server/)
[![CI](https://github.com/deresolution20/mlx-mcp-server/actions/workflows/ci.yml/badge.svg)](https://github.com/deresolution20/mlx-mcp-server/actions/workflows/ci.yml)
[![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://pypi.org/project/mlx-mcp-server/)
[![License: MIT](https://img.shields.io/github/license/deresolution20/mlx-mcp-server.svg)](https://github.com/deresolution20/mlx-mcp-server/blob/main/LICENSE)
[![Publish to PyPI](https://github.com/deresolution20/mlx-mcp-server/actions/workflows/publish.yml/badge.svg)](https://github.com/deresolution20/mlx-mcp-server/actions/workflows/publish.yml)
[![AI Automation Consultant](https://img.shields.io/badge/AI%20Automation-Consultant-blueviolet)](https://github.com/deresolution20)

> MCP server bridging Codex to local MLX LM (and any OpenAI-compatible backend)

_AI automation consulting — I help businesses replace painful manual processes with LLM-powered pipelines and workflow automation._

---

## Overview

`mlx-mcp-server` is a [Model Context Protocol](https://modelcontextprotocol.io) server that bridges Codex to a **local** LLM (MLX LM on Apple Silicon, or any OpenAI-compatible `/v1` backend) so routine work can run on-machine when that is the right trade-off.

The core value is convenience and privacy, not a promise of measurable API-bill savings. The headline `iterate` tool runs a self-correcting local ladder — retry against a gate, step up to a bigger local model, and hand back to Codex when the local rungs are exhausted. This is useful for routine work (summarizing, extracting, classifying, boilerplate, single-file review) while Codex stays focused on multi-file reasoning, tool-heavy debugging, and judgment calls. See `POSTMORTEM.md` for the accounting behind that scope.

## Features

- **Local routine-work bridge** — routes eligible work (summarize, extract, classify, boilerplate, single-file review, first drafts) to a private local model when local quality is good enough.
- **Self-correcting `iterate` ladder** — retries locally, then uses a bigger model for that request only, then hands back to Codex; local rungs are exhausted before Codex takes over.
- **Gated retries** — structural gates (`require_json` / `schema_keys` / `contains` / `regex` / `min_len`) and an executable gate (run a linter or test against `$CANDIDATE_FILE`) let the local model fix its own output.
- **Category profiles and telemetry** — `summarize`, `extract`, `explain`, `boilerplate`, `review`, and `other` tune local retry behavior and log content-free routing data.
- **Runtime model switching** — swap the active model by name or fuzzy fragment with no Codex restart; choice persists across restarts.
- **Work-hours guard** — optionally blocks large (>22 GB RAM) models during work hours to avoid swap thrashing.
- **One-step install** — writes the Codex MCP entry to `~/.codex/config.toml` by default, with legacy Claude Code/Desktop support still available.
- **Backend-agnostic** — works with MLX LM or any OpenAI-compatible `/v1` endpoint; content-free usage metrics, MIT licensed, tested and CI'd.
- **Built-in observability package** — Prometheus exporter, Alloy pipeline, Terraform dashboards, and rollout docs for tracking local routing activity, routing discipline, and local inference health.

## Getting Started

Requires Python 3.11+ and a running OpenAI-compatible LLM backend (MLX LM on Apple Silicon is recommended).

```bash
pip install mlx-mcp-server
mlx-mcp-server install   # wire into Codex, then restart Codex or check /mcp
```

See [Usage](#usage) for backend setup, configuration, and the full tool reference.

## Usage

### 1. Start a local LLM backend

The server talks to any OpenAI-compatible `/v1` endpoint. On Apple Silicon, [MLX LM](https://github.com/ml-explore/mlx-lm) is recommended:

```bash
pip install mlx-lm
mlx_lm.server --model mlx-community/Qwen2.5-Coder-14B-Instruct-4bit   # serves on http://localhost:8080
```

Any other OpenAI-compatible backend works too (e.g. Ollama on Linux/Intel — point `MLX_BASE_URL` at `http://localhost:11434`).

### 2. Install and wire it into Codex

```bash
pip install mlx-mcp-server

# One-step setup for Codex user config
mlx-mcp-server install

# ...or project-local Codex config
mlx-mcp-server install --codex-project

# Preview the config without writing anything
mlx-mcp-server install --dry-run
```

The installer writes an `mlx` entry into Codex config (`~/.codex/config.toml` by default) with `[mcp_servers.mlx]`, environment variables, and approval-mode entries for `iterate`, `chat`, `quick_test`, `health_check`, and `list_models`. Because `iterate` can execute a shell gate, it is configured to prompt for approval; the other listed tools are approved by default. Restart Codex or run `/mcp`, then call `health_check` to verify the backend is reachable. Run `mlx-mcp-server help` for the full CLI reference.

### 3. How Codex uses it — local-first for routine work

The intended workflow is simple: route eligible routine work (summarize, extract, classify, reformat, boilerplate, single-file review, simple refactors, first drafts) to your private local model, then verify or polish the result in Codex. This can avoid remote generation for those delegated tasks, but it should not be treated as a general solution for cache-heavy API costs. The headline tool is `iterate`, which runs a self-correcting escalation ladder:

> **local model retries** (feeding each gate failure back in) → **a bigger local model** → **hand back to Codex**

Local rungs are exhausted before Codex takes over. You attach a **gate** so the local model can self-correct:

- **Structural gates** (cheap, content-free): `require_json`, `schema_keys`, `contains`, `regex`, `min_len`
- **Executable gate**: `check_command` — an arbitrary shell command that sees the candidate at `$CANDIDATE_FILE` and exits `0` to pass (e.g. a linter or test). Run only trusted commands and review the Codex approval prompt.

With no gate, `iterate` runs a single local attempt and asks Codex to verify.

```text
# Generate boilerplate, gated by a linter — retries locally until ruff is happy
iterate(message="write a Python slugify() function",
        category="boilerplate",
        check_command="ruff check $CANDIDATE_FILE")

# Extract structured data, gated on valid JSON with required keys
iterate(message="extract name, email, company from this signature: ...",
        category="extract",
        require_json=true,
        schema_keys=["name", "email", "company"])

# Quick one-off to the local model, no iteration
chat(message="explain what this regex does: ^\\d{3}-\\d{4}$")
```

### Tools

| Tool | What it does |
|------|--------------|
| `iterate` | Offload a task with a gate; retries locally, then a bigger local model, then hands back to Codex |
| `chat` | Send a single prompt to the local model and get the response + token usage |
| `quick_test` | Run a canned diagnostic (`hello` / `math` / `creative` / `code_review`) to sanity-check the model |
| `list_models` | List loaded models with speed/quality descriptions and the active marker |
| `set_model` | Switch the active model at runtime by name or fuzzy fragment — no restart needed |
| `health_check` | Confirm the backend is reachable |
| `set_work_hours_guard` | Block large (>22 GB RAM) models Mon–Fri 8am–5pm MT to avoid swap thrashing |
| `get_config` | Show current config (URL, active model, guard state) — resource `config://settings` |

Codex can inspect the server with `/mcp` and overall session state with `/status`. The bundled `/switch-model` and `/mlx-help` slash commands remain available for legacy Claude Code installs.

### Configuration

Set as env vars in the MCP server entry (the installer scaffolds these):

| Variable | Default | Notes |
|----------|---------|-------|
| `MLX_BASE_URL` | `http://localhost:8080` | Backend `/v1` URL |
| `MLX_DEFAULT_MODEL` | _(empty)_ | Optional — auto-detected from `/v1/models` if unset |
| `MLX_API_KEY` | _(empty)_ | Optional, for secured backends |
| `MLX_TIMEOUT` | `30` | Request timeout in seconds |

### AGENTS.md offload policy

Codex reliably follows local offload when the project gives it explicit guidance. This repository includes `AGENTS.md` with the expected policy:

- call `mcp__mlx__iterate` before eligible local work
- call `mcp__mlx__chat` for quick one-shot summarize/explain/extract/review tasks
- always pass `category` as one of `review`, `boilerplate`, `summarize`, `extract`, `explain`, or `other`
- keep multi-file architecture, tool-heavy work, and final user-facing synthesis on Codex

### Legacy Claude support

Claude Code and Claude Desktop config writers remain available for compatibility, but Codex is the primary target:

```bash
# Claude Code plus legacy slash commands
mlx-mcp-server install --claude-code --with-commands

# Claude Desktop
mlx-mcp-server install --claude-desktop
```

## Observability

The `observability/` package turns the server's content-free local telemetry into
operator-grade dashboards:

- local Prometheus exporter over `~/.omlx/mlx-call-log.jsonl`,
  `~/.omlx/hook-decisions.jsonl`, and `~/.omlx/stats.json`
- Alloy scrape and Grafana Cloud remote_write configuration
- Terraform-managed Grafana folder and dashboards
- docs for architecture, telemetry privacy, rollout gates, and PDF report
  artifacts

Phase 1 dashboards cover local routing activity, routing discipline, and local inference
health. Phase 2 latency metrics are deliberately gated: the exporter only emits
call duration histograms when `--enable-call-latency` is set after source-writer
validation and log cutover.

Start here:

```bash
python3 -m unittest discover -s observability/exporter -p 'test_*.py'
python3 observability/exporter/mlx_mcp_exporter.py --once
python3 observability/verify_runtime.py --strict
terraform -chdir=observability/terraform validate
```

See [observability/README.md](observability/README.md) for the architecture,
privacy contract, rollout checklist, and Grafana/Terraform workflow.

## Development

```bash
# Install the locked development environment
uv sync --locked
# Run the complete test suite and build distributions
uv run pytest -q
uv build
# Verify observability and Terraform
python3 -m unittest discover -s observability/exporter -p 'test_*.py'
uv run python observability/verify_runtime.py --strict
terraform -chdir=observability/terraform fmt -check -diff
terraform -chdir=observability/terraform validate
```

## License

MIT — see [LICENSE](LICENSE).

---

<div align="center">

<sub>Built and maintained by **Brice** — Observability Engineer at Grafana Labs / AI Automation Consultant. See more at [github.com/deresolution20](https://github.com/deresolution20).</sub>

</div>
