Metadata-Version: 2.4
Name: rustygate
Version: 0.1.16
Classifier: Programming Language :: Rust
Classifier: License :: Other/Proprietary License
Requires-Dist: fastcore>=2.2.11
Requires-Dist: pytest ; extra == 'dev'
Requires-Dist: pytest-timeout ; extra == 'dev'
Requires-Dist: httpx ; extra == 'dev'
Requires-Dist: websockets ; extra == 'dev'
Requires-Dist: jupyter-client ; extra == 'dev'
Requires-Dist: jupygate ; extra == 'dev'
Requires-Dist: jupyasyncclient>=0.2.16 ; extra == 'dev'
Requires-Dist: ipymini ; extra == 'dev'
Provides-Extra: dev
Summary: Jupyter kernel gateway with terminals, files, and cells APIs, as a single binary
Author: Jeremy Howard
Maintainer-email: "fast.ai" <infos@fast.ai>
License: Proprietary
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/AnswerDotAI/rustygate
Project-URL: Issues, https://github.com/AnswerDotAI/rustygate/issues
Project-URL: Repository, https://github.com/AnswerDotAI/rustygate

# rustygate

A Jupyter kernel gateway as one binary: kernels, terminals, files, search, and per-cell notebook operations, over the same REST and websocket API shapes `jupyter_server` uses.

## Running

    rustygate [--host H] [--port N] [--token T | --token-file PATH] [--trust-loopback] [--root DIR] [--tls] [--autorun DIR]

The defaults are 127.0.0.1, port 8787, no auth, and the working directory as the files root. With a token set, requests carry `Authorization: Bearer T`; Jupyter's `Authorization: token T` form and `?token=T` are also accepted. `--token-file` reads the token without putting it in the process arguments. `--trust-loopback` lets clients whose actual TCP peer is loopback omit the token. Do not use it behind an untrusted same-host reverse proxy, since the proxy is the TCP peer. Forwarded headers never affect authentication. `--tls` serves https and wss from a self-signed certificate generated at startup; clients opt out of verification (jupyasyncclient's `verify=False`).

The Python helper `rustygate.tools.start_gateway()` starts a background gateway with inherited stdout and stderr. Gateway and kernel startup errors appear in the calling process's console.

### Working directories

Notebook-bound kernels default to the notebook's parent directory under the served root. Folder-bound terminals default to their folder. Explicit `cwd` overrides either binding; relative values use the served root. Restart recomputes a kernel's default cwd from its current binding. See [DEV.md](DEV.md#kernel-lifecycle) for unbound processes and path rules.

`GET /api/kernels/{kid}/realpath?path=...` returns an on-disk path as a JSON string. Empty and relative paths use the bound notebook's parent, regardless of the kernel's cwd. A leading slash selects the gateway root. The target need not exist. Notebook-relative paths require a notebook binding; root-qualified paths need only an existing kernel. Existing symlinks and `..` resolve within the root; escapes and dangling symlinks are rejected. Notebook bindings separately require one canonical spelling, without symlink aliases.

### Notebook state and connections

Clients sharing a notebook share its kernel and authoritative notebook state. Accepted edits broadcast to every subscriber, including their author. Saving is debounced; contents reads, copies, search and command execution first save pending changes in their file or directory scope, then operate on disk normally.

`POST /api/kernels` with a `path` returns its existing non-dead kernel unchanged (200), ignoring startup options, or starts a new kernel (201). A concurrent creation still starting returns 409. Without a path, each request starts a new kernel.

A kernel id identifies the logical kernel across rename, restart and failed-restart retry. `GET/POST /api/kernels/{kid}/cells` operates on its current notebook binding. `PATCH /api/kernels/{kid}` with `{"path":"new.ipynb"}` renames that notebook. Deletion and new creation produce a new id. Path-addressed file and cell operations remain available for unopened notebooks.

Cell reads support `section=ID` for a heading and its descendants, and `ancestors=ID` for enclosing headings. Both use ordinary cell IDs and ignore collapse flags. `fields=id` returns IDs in cell dictionaries; `fields=id,source` returns IDs and sources. Omit `fields` for full cells without attachments, or use `fields=*` for all cell fields including attachments. `fields=*,meta` also returns notebook metadata. See [DEV.md](DEV.md#cells-api) for heading detection and selector composition.

Notebook settings use the same ordered ops: `{"ops":[{"op":"meta","metadata":{"solveit":{"mode":"learn"}}}]}` merge-patches notebook metadata without transferring or replacing cells and outputs. Accepted metadata broadcasts carry the resulting metadata, so subscribers need no follow-up read.

File and cell edits use last-accepted-edit-wins semantics, without hashes or stale-write checks. File writes and copies can require a new destination with `overwrite=false`; renames never overwrite. The watcher reloads valid external changes when the file's mtime changes.

Kernel traffic uses bounded queues with backpressure before notebook updates. A slow frontend's full queue closes its whole connection rather than silently dropping individual messages. Disconnected messages are not replayed: reconnecting clients load current notebook state. Kernel execution and persistence continue without connected clients.

### Startup notebooks

`--autorun AUTORUN` runs the direct `*.ipynb` children of `AUTORUN` under the files root once per gateway startup. Each notebook gets its own path-bound kernel, with its parent directory as the working directory. Nonempty code cells run in notebook order; outputs persist normally. Other cell types and CRAFT inheritance are not processed. Shared initialization belongs in explicit imports in the notebook.

Notebooks run independently in the background, without delaying HTTP readiness. A cell error stops the remaining cells in that notebook; errors are logged and its kernel remains available for inspection. Other notebooks and the gateway keep running. A missing directory is empty; unreadable directories or malformed notebooks produce an error. Attaching a client or restarting an individual kernel does not rerun AUTORUN. Omit the flag to disable it. `rustygate service install` accepts the same flag.

## Running as a service

`rustygate service` installs and controls a user service using launchd on macOS and systemd on Linux:

    rustygate service install --root ~/git --workdir ~/git/project
    rustygate service status
    rustygate service restart
    rustygate service stop
    rustygate service start
    rustygate service uninstall

`install` starts the service immediately; running it again replaces the configuration. Add `--host 0.0.0.0` for direct network access, `--token-file ~/.config/rustygate/token` for authentication, `--trust-loopback` when same-host clients should not need that token, and `--tls` when wanted. A missing token file is generated with user-only permissions. `--env-file` sources environment variables inherited by kernels. The service PATH is the rustygate binary's own directory (the venv bin, under a pip install) plus the standard system directories, so kernels find `python` despite the minimal launchd and systemd defaults. The default loopback service can instead be reached remotely through an SSH tunnel.

## Built-in Luau kernel

Start with `help()` in a Luau cell (or `lua(code="help()")` through MCP). It
returns a bundled quick-start guide and API index; `help("ex.edit_file")` gives
function details, using the same documentation as Jupyter inspection. Inside a
longer cell use `print(help())` to display it. No external docs/install needed.

Create with `POST /api/kernels` and `{"name":"luau"}` (optionally `cwd` and
`path`). The gateway starts its own executable as
`rustygate native-kernel -f CONNECTION_FILE`; no Python, Lua installation, or
external search command is needed. Omit `name` to retain the existing Python
default. An explicit Luau selection cannot be combined with custom `argv`.

Native APIs include `rg.search`, `rg.find`, `fs.read_text`, `ex.edit_text`,
file/cell viewing and editing, `os.execute`, `io.popen`, `print`, and `display`. For example, in successive cells:

```lua
hits = rg.search("TODO", {root=".", exts={"rs"}, max_results=100})
```

```lua
#hits.matches
```

```lua
original = fs.read_text("notes.txt")
edited = ex.edit_text(original, "%s/TODO/DONE/g")
return edited.text
```

`ex.edit_text` edits private in-memory text. To edit files or notebook cells:

```lua
print(ex.view_file("notes.txt")) -- use addresses from this view
-- replace ADDRESS with a verified address from the view
-- ex.edit_file("notes.txt", {{ADDRESS, "s", "TODO", "DONE"}})
-- ex.edit_cell("analysis.ipynb", "cell-id-prefix", {{ADDRESS, "c", "new source"}})
```

File commands use arrays of strings (nested arrays for global subcommands),
like Python command tuples. `path:address` and `path.ipynb:cellid:address`
prefixes allow cross-file/cell moves and copies. `ex.edit_cell` also accepts
a compact script. Options are `{inplace=false}` for preview and `{sw=4}` for
shift width; writes are enabled by default. `ex.edit_file` returns
`{files, changed, diff}`; `ex.edit_cell` returns one result. Every command is
validated/applied before any writes; this is **not** an atomic multi-file
transaction. Notebook metadata, outputs, source form and trailing newlines
are preserved.

```lua
local success, kind, code = os.execute("printf hello | tr a-z A-Z")
local p = assert(io.popen("git status --short"))
local output = p:read("*a")
local success, kind, code = p:close()
return output
```

Both launch `/bin/sh -c` in the kernel cwd/environment; Bash-specific syntax
requires explicitly calling `bash -c`. `os.execute` publishes stdout/stderr
and returns Lua 5.4-style success/nil, exit/signal, and code. No-command
`os.execute()` checks shell availability. Commands are noninteractive: stdin
is closed unless using `io.popen(command, "w")`. Programs you invoke must
exist, but Python or a separate Lua install is not needed.

Pipes support `read`, `lines`, `write`, `flush`, `close`, and `seek` (which
returns a pipe-not-seekable error). Reads accept byte counts or `l`, `L`, `a`,
`n` (also `*l`, etc.), with multiple formats and binary-safe strings. Writes
are unbuffered and return the handle; `close` returns the command status.
This is not the full Lua `io` library: no `io.open`, default streams, or
`setvbuf`. `io.type` and `io.close(pipe)` are available. stderr and write-mode
stdout are forwarded during pipe operations. A pipe can persist across cells;
close it explicitly. GC, interruption and kernel shutdown terminate outstanding
processes; an interrupt closes all kernel-owned pipes.

Search records carry path,
line number, lnhash and match spans. Search/discovery return all results by
default; an optional `max_results` truncates them and sets `complete=false` when
results are omitted. There are no imposed byte or result-count caps. This is
trusted local scripting, not an OS sandbox or memory-bounded execution.

`fs.read_text` uses a standard UTF-8 file read with normal filesystem/encoding
errors and no custom size limit. Interrupts are checked before and after the
read; they do not abort an OS call in progress.

Globals/functions persist; `local` variables remain cell-local. A cell may be a
whole expression or an ordinary chunk with an explicit `return`. Stream output,
MIME display, errors, user expressions, basic host-API completion/help, and the
tail retrieval of session input history are supported. `display` takes a MIME bundle,
for example `display({['text/markdown']='**hello**'})`.

SIGINT and control-channel interruption share an execution-specific token with
the VM and native operations. Text edits run normally, with interrupt checks
before and after the call; an interrupt waits for the edit to finish and discards
its result (file writes that completed are not rolled back). Shell process groups
are killed and their direct children reaped on interruption. Search workers are cancelled and joined before execution completes. Earlier
statements are not rolled back. Individual regex/compiler/filesystem calls and
allocation/free remain cooperative boundaries, not hard real-time preemption.
Restart/delete still terminate the child independently.

This initial implementation has no IPython magics, debugger, typechecker/LSP,
subshells, or alternate interpreter backend. MCP exposes `lua`
alongside `py`, with Python remaining the default for new `create` calls. Standalone executable
release artifacts and Linux validation are follow-up work. The design and
experiment record are in `meta/luau.md`.

## MCP

Use `lua(code="...")` for the bundled Luau kernel, or
`create(dlgname="work", language="luau")` to create/select a named one. `py` and
`lua` share execution, cancellation, and lifecycle handling. Either auto-creates
its language's kernel when none is current. There is one current kernel per MCP
session: a mismatched execution tool errors without running code or switching.
An omitted creation language reuses an existing binding unchanged, or defaults a
new kernel to Python; an explicit language (`python` or `luau`) must match an
existing binding. Creation, selection, and listing report the language, taken
from the kernel's startup `kernel_info_reply`, not inferred from argv.

Python startup/inspectors run only for Python kernels, including on restart.
The path-scoped endpoint also offers `lua(code)` alongside `py(code)`: the first
execution chooses the language and later calls must match. Clikernel forwards
these tools and arguments directly, including creation on named gateways.

`POST /mcp` is a stateful Streamable HTTP MCP server for Claude Code and other MCP clients. Its tools list, create, select, restart, interrupt, and delete kernels, plus execute Python/IPython cells (`py`) and Luau cells (`lua`); code runs exactly as written, and Python/IPython magics such as `%%bash` remain available through `py`. The MCP transport session remembers its current kernel server-side, so the model does not need to carry an id between calls. Executing with no kernel current starts one for the session, and ending the session (`DELETE /mcp`, which Claude Code sends on shutdown) stops every kernel the session created with autoclose. `create(dlgname)` is get-or-create by kernel path and autocloses its kernel by default (`autoclose=false` keeps it); an execution may name another existing binding for that call without changing the current selection. A client may also send per-session creation defaults (cwd, env, startup code, quiet) in its initialize params; see [DEV.md](DEV.md#mcp-api).

`POST /mcp/{path}` is the smaller stateless alternative for clients configured for one dialog. The URL-decoded relative path is the fixed kernel binding, so no MCP session id or model-visible kernel selector is needed. It exposes `py`, `lua`, and `interrupt`; the first execution lazily creates a kernel of the requested language, or reuses a matching bound kernel, while interrupt never creates one. Its tool executions use kernel priority 1, ahead of queued normal-priority notebook runs without preempting a cell already running.

Kernels actually spawned through MCP receive `RUSTYGATE_LAUNCHER=mcp`, available during their startup profiles and preserved across restart. Selecting or get-or-creating a kernel that was already running does not alter its environment. See [DEV.md](DEV.md#mcp-api) for the tool and execution contract.

