Metadata-Version: 2.4
Name: esoul
Version: 0.14.0
Summary: Build, run, and durably resume multi-agent networks on ExternalSoul from Python — event-sourced, audit-logged workspace automation.
Project-URL: Homepage, https://github.com/vyomkeshj/esoul-python
Project-URL: Documentation, https://externalsoul.com/sdk-docs/llm-reference.md
Project-URL: Repository, https://github.com/vyomkeshj/esoul-python
Project-URL: Changelog, https://github.com/vyomkeshj/esoul-python/blob/main/CHANGELOG.md
Author-email: ExternalSoul <hey@vyomkeshj.com>
License: MIT License
        
        Copyright (c) 2026 ExternalSoul
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: agent-builder,agents,ai-agents,automation,esoul,event-sourcing,externalsoul,kinetic,llm,multi-agent,workspace
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.9
Requires-Dist: httpx<1.0,>=0.27
Requires-Dist: pydantic<3.0,>=2.0
Requires-Dist: typing-extensions>=4.7; python_version < '3.11'
Provides-Extra: dev
Requires-Dist: build>=1.0; extra == 'dev'
Requires-Dist: mcp>=1.2; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Provides-Extra: mcp
Requires-Dist: mcp>=1.2; extra == 'mcp'
Description-Content-Type: text/markdown

# esoul — build agent networks in Python

`esoul` is the official Python SDK for the **[ExternalSoul](https://externalsoul.com)**
platform.

Two headline surfaces:

- **[Track training runs](#track-pytorch-training--esoultrack)** — use a workspace
  as your TensorBoard. One `run.log({...})` in your PyTorch loop (~1.5 µs, never
  blocks), live curves with min/max bands, per-epoch sample images, 50-run
  comparison, crash-resume, DDP-aware — and an agent that can read the curves and
  tell you the run has plateaued.
- **The agent-network builder** — declare a multi-agent network in a few lines of
  Python, get a clean auto-laid-out graph, create it in a workspace, run it, fan
  it out over your data, and **durably resume** it if it fails partway.

```bash
pip install esoul
```

```python
import esoul
from esoul.agent_builder import AgentNetwork

client = esoul.Esoul()                         # auto-detects credentials

# Declare an agent network — nodes + edges, no positions, no JSON blob.
net = AgentNetwork("Researcher", model="anthropic/claude-3-5-haiku-20241022")
trigger = net.trigger()                        # fires on agent_builder/run
agent   = net.agent("Researcher", system_prompt="Answer concisely, then save to Notes.")
notes   = net.tool_binding("My Notes", application_type="note_editor")
trigger >> agent                               # wire edges with `>>`
agent   >> notes

# Auto-laid-out (no overlapping cards) + created as a workspace app.
app = net.create(client, workspace_id="ws_abc", instance_name="researcher")

# Run it and wait for the result.
result = client.agents.invoke(
    app.node_id, workspace_id="ws_abc",
    input="Summarise the Q3 metrics into My Notes.",
).wait()
print(result.text)
```

---

## The builder — `esoul.agent_builder.AgentNetwork`

`AgentNetwork` produces the exact state the agent-builder UI persists, so a
code-built network **opens, edits, and runs identically to a hand-built one**.

### Nodes

Each constructor returns a `NodeRef` (its `.id` is the generated node id):

| Method | Node |
|---|---|
| `net.trigger(event=…, prompt_template=…, run_mode=…)` | a **trigger** — what the network waits for (default `agent_builder/run`) |
| `net.agent(name, system_prompt=…, model=…, description=…, context_apps=…)` | an **agent** |
| `net.tool_binding(app, application_type=…)` | a **Tool Binding** — gives an agent a workspace app's full toolkit |
| `net.workspace_tool(tool_names, label=…)` | a **Workspace Tool** node (e.g. `["return_result_to_router"]`) |
| `net.sub_agent_mapper(sub_agent_app_node_id=…, source_mode=…, source_config=…)` | a **Sub-Agent Mapper** (fan-out — see below) |
| `net.add_node(node_type, data, node_id=…)` | generic escape hatch for any other node type (`image_edit_agent`, `search`, `open_app`, …) |

Node ids are **unique per network** (a short token is stamped in), so two
code-built networks never collide.

### Edges

```python
trigger >> agent >> mapper        # chainable; `a >> b` returns b
net.connect(router, billing, routing_prompt="billing questions")   # agent→agent routing label
net.connect(driver, mapper, mapper_meta={"iterationMode": "per_row", "label": "fill each row"})
```

### Models

`model=` accepts a gateway string (`"anthropic/claude-3-5-haiku-20241022"`), a
`{"provider", "model"}` dict, or `None` (network default). Set it per-network
(`AgentNetwork(..., model=…)`) or per-agent (`net.agent(..., model=…)`).

### Layout, serialise, create

```python
net.auto_layout()          # clean, non-overlapping positions (also runs inside to_state)
state = net.to_state()     # the raw dict — pass to apps.create(initial_state=...) yourself
app   = net.create(client, workspace_id=ws, instance_name="my-net")   # …or one call
```

The layout is a pure, deterministic **layered (Sugiyama-style)** placement;
`from esoul.agent_builder import assert_no_overlap` lets your tests prove a
graph is clean.

---

## Fan out over data — the Sub-Agent Mapper

Point a `sub_agent_mapper` node at a spreadsheet column and it runs one
sub-agent per cell, in parallel:

```python
sub  = AgentNetwork("Filler", model="anthropic/claude-3-5-haiku-20241022")
t = sub.trigger(); a = sub.agent("Filler", system_prompt="Fill one cell.")
t >> a; a >> sub.tool_binding("Cities", application_type="spreadsheet")
sub_app = sub.create(client, workspace_id=ws, instance_name="filler")

main = AgentNetwork("Driver", model="anthropic/claude-3-5-haiku-20241022")
driver = main.agent("Driver", system_prompt="Hand off to the mapper.")
mapper = main.sub_agent_mapper(
    sub_agent_app_node_id=sub_app.node_id,
    source_mode="spreadsheet",
    source_config={"spreadsheetName": "Cities", "inputColumnName": "City", "filters": []},
)
main.connect(driver, mapper, mapper_meta={"iterationMode": "per_row"})
main_app = main.create(client, workspace_id=ws, instance_name="driver")

client.agents.invoke(main_app.node_id, workspace_id=ws, input="go").wait()
inv = client.mapper.get_latest(workspace_id=ws, mapper_node_id=mapper.id)
print(inv.completed_count, "/", inv.expected_count)   # e.g. 100 / 100
```

## Durable resume — recover an interrupted run without redoing work

A 100-sub-agent run that drops mid-flight (network error, a batch that
errored) doesn't have to start over. Per-item state lives server-side, so
resume **re-dispatches only the unfinished items**:

```python
res = client.mapper.resume(inv.id, retry="incomplete")
#                                  ^ heuristic for which items re-run:
#   "incomplete" (default): everything not completed  — resume without loss
#   "errored":   only failures — retry, leave good work
#   "stalled":   only queued/running — a run that died mid-flight
print(res.resumed, res.pending)        # pending = items still not completed
```

Completed work is never redone; safe to call repeatedly.

---

## Test agents — `esoul.evals`

Hermetically test an agent network before you publish: run it on graded inputs,
grade with `contains` / `regex` / `jsonSchema` / `llmJudge`, and assert
structural invariants over the event log. Same engine as the in-UI EVALS tab.

```python
from esoul.evals import CastMember, run_eval, make_agent_judge

res = run_eval(
    client, workspace_id=ws,
    cast=[CastMember("pos", agent.node_id, "I love this!")],
    cases=[{"id": "p", "target": "pos", "expected": [{"graderId": "contains", "expected": "POSITIVE"}]}],
)
print(res.summary())   # PASS/FAIL + per-case ✓/✗
```

Use Haiku for tests; one case = one `CastMember` + one case spec. Agent output
that ends on `return_result_to_router` is read from `router_handoff` (handled for
you). To surface results in the in-UI EVALS tab, persist `Eval`/`EvalRun`/
`EvalCaseResult` rows keyed by `agentRef.agentNodeId`. **Full how-to (graders,
judge, invariants, persistence, the EVALS-tab hollow-circle meaning, ops):
`docs/esoul-llm-reference.md` → "Testing harness".** Worked example:
`tests/fairy_comprehensive.py` + `../../scripts/eval-tests/persist-fairy-evals.ts`.

---

## Track PyTorch training — `esoul.track`

Use an ExternalSoul workspace as your TensorBoard: live loss curves, per-epoch
sample images, hyperparameter comparison across runs, and an agent that can read
the curves and tell you whether the run has plateaued.

```bash
pip install esoul
export ESOUL_TOKEN=esoul_pat_...          # Settings → Access Tokens
```

```python
import esoul.track as track

run = track.init(
    workspace="ml",                        # by NAME — omit if your token is workspace-scoped
    monitor="Training",                    # the monitor app; created on first use
    config={"lr": 3e-4, "batch_size": 64, "arch": "resnet50"},
    total_steps=len(loader) * EPOCHS,      # drives the progress ring + ETA
)

for epoch in range(EPOCHS):
    for i, batch in enumerate(loader):
        loss = train_step(batch)
        run.log({"train/loss": loss, "lr": sched.get_last_lr()[0]}, step=global_step)

    run.log({"val/loss": val_loss, "val/acc": val_acc}, step=global_step, epoch=epoch)
    run.log_images("samples", model.sample(8), step=global_step, epoch=epoch)

run.finish()
```

That is the whole integration. `loss` may be a live CUDA tensor — it is detached
on the spot and resolved later, off the training thread.

### What it costs your training loop

`log()` is a queue push: **~1.5 µs**, no formatting, no `.item()` (which would
synchronise the GPU), no I/O. A background thread batches, writes a
crash-recovery journal, and posts. Measured on a real run: **5,516 points in 2
HTTP requests, 1.3 µs per `log()` call.**

The reason it batches so hard is architectural, not lazy: each workspace write
is a transactional append, and app state is folded on every one. So the SDK
sends **one event per flush** and the platform folds each metric into a bounded
sketch — a fixed ~5 KB per metric whether the run logs 500 points or 5 million.
A 50-run sweep at 20k steps × 6 metrics (6 million datapoints) lives in **under
900 KB** of workspace state, and the charts open instantly at any scale.

### Crash, resume, and DDP

| Situation | What happens |
|---|---|
| Network drops mid-run | Frames queue on disk (`~/.esoul/runs/<id>/wal.jsonl`), retry automatically. Training never blocks or fails. |
| Process is killed | Unsent frames stay on disk. `track.init(resume="<run id>")` replays them and continues the same run. |
| Exception in your loop | `with track.init(...) as run:` marks the run **failed** and records the traceback on the run card. |
| `torchrun` / DDP | Rank 0 logs; other ranks get an inert handle, so **no `if rank == 0:` guards** are needed. Pass `all_ranks=True` to override. |
| Duplicate delivery | Each frame carries a monotonic `seq`; the platform folds it exactly once. |

```python
run = track.init(monitor="Training", resume=os.environ.get("RESUME_RUN_ID"))
print(run.id)      # save this to resume after a preemption
```

### Design the dashboard from Python

Omit this entirely and the monitor infers a layout from your metric names
(losses together, accuracies together, LR on a log axis, a sample strip if you
log images). When you want control, declare it:

```python
from esoul.track import Dashboard, Line, Tiles, Images, Config, System

run = track.init(
    monitor="Training",
    dashboard=Dashboard(
        title="ImageNet sweep",
        hero=["train/loss", "val/acc"],          # the two big readouts up top
        panels=[
            Tiles(["train/loss", "val/acc", "lr", "grad_norm"]),
            Line("Loss", ["train/loss", "val/loss"], span=2, height=240),
            Line("Accuracy", ["train/acc", "val/acc"]),
            Line("Learning rate", ["lr"], y="log"),
            Images("samples", title="Samples per epoch"),
            System(),                             # throughput / ETA / GPU
            Config(),                             # the hyperparameter table
        ],
    ),
)
```

`Line(...)` accepts a glob: `Line("All losses", ["*/loss"])`. Panels re-render
live as points arrive; the spec is stored with the app, so it survives reloads
and applies to every run in that monitor.

### The rest of the API

```python
run.log({"train/loss": 0.42}, step=i, epoch=e)   # scalars (float / np / torch)
run.log_images("samples", imgs, step=i)          # PIL, numpy HWC/CHW, bytes or paths
run.system(loss_scale=65536, temperature_c=71)   # ONLY what the SDK can't know:
#   throughput, ETA and (when torch is already imported) GPU util + memory are
#   derived from the step count and the clock — no calls needed in your loop.
run.note("lr warmup was too short")              # sticks to the run, agents read it
run.finish()                                     # or finish(status="killed")
run.stats                                        # points emitted, frames sent, µs/log
```

### Ask an agent about your run

The monitor is a real workspace app, so an agent in that workspace — or your own
script — can read the curves and reason about them:

```python
import json

tools = client.workspaces.apps.tools(app_id=app_id)          # discover per-app tools
verdicts = json.loads(
    client.workspaces.apps.call_tool(app_id=app_id, tool_name="analyze_run_Training")
)
# {"name": "resnet-baseline", "status": "finished",
#  "verdicts": [{"series": "val/loss", "verdict": "plateaued",
#                "best": 0.2726, "bestStep": 880, "changeFraction": -0.004}, ...]}
```

`analyze_run` compares the last quarter of the run against the quarter before it
and returns `improving` / `plateaued` / `worsening` per metric, with the best
value and the step it happened at. Other tools: `list_runs`, `read_run`,
`read_series` (the actual curve), `compare_runs` (overlays them and reports the
differences), `add_run_note`, `set_dashboard`.

So "wake me when val_loss plateaus" is a workspace trigger, not a cron job you
have to write.

### Files in your workspace

```python
f = client.files.upload(name="checkpoint-best.pt", content=open(path, "rb").read())
data = client.files.download(f.blob_url)     # ranged; a plain GET is 100× slower
```

### Streaming anything else — `esoul.stream`

`track` is a thin facade over `esoul.stream.EventStream`, which is generic: an
in-process queue, a batching daemon thread, a write-ahead log, retries, and one
event per flush. Any high-rate producer (robot telemetry, a crawler's progress,
throughput counters) can use it directly with its own reducer.


## Beyond agents: events + typed resources

ExternalSoul workspaces are **event-sourced** — every mutation is a typed
event on a per-workspace timeline. The UI, agents, and your scripts all
dispatch through the **same** reducers, so audit logging is automatic. Drop to
the raw event layer or the typed resources whenever you need to:

```python
client.describe()                               # what workspaces / apps / events exist
client.dispatch_event(app_id=…, event_name="spreadsheet_add_row", event_data={…})
```

| Resource | What it does |
|---|---|
| `client.spreadsheet` | Create / seed / read spreadsheets — `create`, `add_column`, `add_rows`, `set_cell`, `get_rows`. |
| `client.workspaces.apps` | App lifecycle — `list`, `create` (with `initial_state`), `rename`, `delete`. |
| `client.agents` | Invoke `agent_builder` runs — `invoke(...).wait()`. |
| `client.mapper` | Sub-Agent Mapper introspection (`list_invocations`, `get_latest`) **+ resume** (`resume`, `resume_latest`). |
| `client.questions` | Human-in-the-loop — `ask(...)` against the workspace question queue. |
| `client.drive` | Google Drive ops on the workspace's connected Drive. |
| `client.revert` | Run-revert kernel (`preview`). |
| `client.files` | Workspace files — `upload`, `list`, ranged `download`. |
| `client.workspaces.overview` / `.find` | The named project → workspace hierarchy; resolve a workspace BY NAME instead of pasting a UUID. |

## MCP server — drive ExternalSoul from Claude Desktop / Cursor / any MCP client

The whole SDK is also available as a [Model Context Protocol](https://modelcontextprotocol.io)
server, so any MCP client can read app state, dispatch events, run and build
agent networks, search, manage the human-in-the-loop queue, and scrub the
timeline — using your PAT.

```bash
pip install "esoul[mcp]"
ESOUL_TOKEN=esoul_pat_… esoul-mcp                    # stdio (default)
ESOUL_TOKEN=… esoul-mcp --transport streamable-http --port 8765
ESOUL_TOKEN=… ESOUL_MCP_READONLY=1 esoul-mcp         # browse, don't touch
```

Add to your client (`claude_desktop_config.json` → `mcpServers`):

```json
{
  "mcpServers": {
    "externalsoul": {
      "command": "esoul-mcp",
      "env": {
        "ESOUL_TOKEN": "esoul_pat_…",
        "ESOUL_WORKSPACE_ID": "6d735936-…"
      }
    }
  }
}
```

49 tools across every resource (21 in read-only mode). Set `ESOUL_WORKSPACE_ID`
to omit `workspace_id` on most calls. Long-running work (agent runs, HIL
questions) is fire-then-poll so calls stay short. Full reference:
[`.cursor/rules/docs/esoul-mcp-server.md`](../../.cursor/rules/docs/esoul-mcp-server.md).

## Authentication

`esoul.Esoul()` auto-detects credentials in this order:

1. Explicit `token=` kwarg
2. `ESOUL_TOKEN` environment variable
3. `/var/run/esoul/token` (the 0600 token every E2B sandbox writes at boot —
   `Esoul()` inside a sandbox Just Works)
4. `~/.config/esoul/credentials` (TOML with a `[default]` section: `token = "esoul_pat_..."`)

For off-platform use (laptop, CI), create a **Personal Access Token** in
workspace settings → "Access Tokens", pick the workspaces it can mutate, then:

```bash
export ESOUL_TOKEN="esoul_pat_..."
```

## Workspace isolation

A credential is scoped to specific workspaces. Cross-workspace dispatch is
**structurally impossible** — a server invariant, not a check in your script.

## Idempotency

Every write is automatically idempotent — the SDK generates a UUID per call
and reuses it across retries; the server caches the response under
`(session, key)` for 24h. Override with `idempotency_key="…"` for
application-level retry control.

## Async client

`esoul.AsyncEsoul()` mirrors the sync API exactly:

```python
import asyncio, esoul

async def main():
    async with esoul.AsyncEsoul() as client:
        await client.mapper.resume_latest(workspace_id="ws_abc", mapper_node_id="…")

asyncio.run(main())
```

## Typed errors

Failures surface as exception classes mapped from the server's `error.code`;
`esoul.EsoulError` catches everything:

```python
try:
    client.dispatch_event(...)
except esoul.WorkspaceAccessDenied as e:
    print(f"Token cannot reach workspace {e.details['workspaceId']}")
except esoul.RateLimitError as e:
    time.sleep(e.retry_after_seconds or 1)
except esoul.AuthError:
    print("Token expired or revoked")
except esoul.EsoulError as e:
    print(f"{e.code}: {e.message}")
```

## Status

**Beta.** The agent-builder, mapper, spreadsheet, and low-level
`dispatch_event` / `read_state` surfaces are stable and used in production
automations. See [CHANGELOG.md](./CHANGELOG.md) for release notes.

## License

MIT.
