Metadata-Version: 2.4
Name: multimodal-mllog
Version: 1.2.5
Summary: Experiment logbook pipeline: automatic per-run capture + on-demand reporting
License: MIT
Keywords: experiment-tracking,logbook,coding-agents,mlflow
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: License :: OSI Approved :: MIT License
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: pydantic>=2
Requires-Dist: click
Provides-Extra: mlflow
Requires-Dist: mlflow>=2; extra == "mlflow"
Provides-Extra: pdf
Requires-Dist: reportlab>=4.0; extra == "pdf"
Provides-Extra: dashboard
Requires-Dist: flask>=3.0; extra == "dashboard"
Requires-Dist: reportlab>=4.0; extra == "dashboard"
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == "mcp"

# multimodal-mllog

[![Python](https://img.shields.io/pypi/pyversions/multimodal-mllog)](https://pypi.org/project/multimodal-mllog/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

Experiment logbook pipeline for researchers training ML models with coding agents.

An external observer that **automatically captures** per-run facts (agent events, git state, optional MLflow info) as JSONL records via a sensor/adapter/sink architecture, and **on demand** generates a PDF experiment report from them via the coding agent.

```
coding agent  --> mllog capture (automatic via hook)   -->  JSONL store
coding agent  --> /logbook --from yesterday            -->  PDF report
researcher    --> mllog dashboard                       -->  local web UI
```


## Architecture

![Architecture](resources/architecture.drawio.png)

Record envelope (stored as one JSONL line per record in `./mllog/records/<YYYY-MM-DD>.jsonl`):

```jsonc
{
  "schema_version": "0.1",
  "record_id": "<ULID>",
  "display_name": "Fix LR scheduler bug",  // short title for UI (≤60 chars)
  "activity_type": "train | eval | analysis | attempt_failed",
  "status": "ok | failed",
  "started_at": "<iso8601>",
  "ended_at": "<iso8601>",
  "sources": {
    "agent":  { "event_count": 12, "edit_ledger": {}, "config_deltas": [], "commands": [] },
    "git":    { "commit": "abc123", "dirty": false, "changed_files": [] },
    "mlflow": { "run_id": "...", "params": {}, "metrics": {} },  // present only if active
    "env":    { "python_version": "3.13", "platform": "..." }
  },
  "digest": null,           // optional, evidence-gated
  "transcript_ref": null    // pointer to session JSONL, never raw content
}
```

## Installation

```bash
pip install multimodal-mllog              # core (capture, query)
pip install "multimodal-mllog[mlflow]"    # + optional MLflow sensor/sink
pip install "multimodal-mllog[dashboard]" # + local web dashboard (Flask + PDF rendering)
```

## Quickstart

1. Open your ML project in a coding agent (Claude Code, Codex, etc.).
2. Run your experiment: train, evaluate, or analyze.
3. The `Stop` hook captures the run automatically, or run `/mllog` manually.
4. Run `/logbook --from yesterday` to generate a PDF experiment report from stored records.
5. Run `mllog dashboard` to browse records in a local web UI.

## CLI reference

```bash
# Capture a run (sensors gather git, agent events, optional MLflow automatically)
mllog capture --type {train|eval|analysis|attempt_failed} --status {ok|failed} \
              [--name "short title"] [--session-path <path>] [--mlflow-run-id <id>] [--auto]

# Query records
mllog get-logs --from <when> [--to <when>] [--json]
# <when>: ISO date (2026-07-01), 'yesterday', 'today'

# Local dashboard
mllog dashboard [--port 8265] [--mllog-dir ./mllog]

# Sensor / adapter / sink status
mllog doctor

# Checkpoint
mllog checkpoint [--show | --advance]
```

## Reports

Reports are **agent-generated**: the `/logbook` skill instructs the coding agent to read
session records, analyze them, and produce a professional PDF report using reportlab. This
produces rich narrative documents with tables, cross-run comparisons, failure analysis, and
recommendations — not a mechanical data dump.

```
/logbook --from 2026-07-01 --to 2026-07-19
```

The dashboard also offers a quick summary generator for simple tabular overviews.

## Storage

- Records: `./mllog/records/<YYYY-MM-DD>.jsonl` (append-only, one line per record, fsynced)
- Checkpoint: `./mllog/checkpoint.json`
- Logbooks: `./mllog/logbooks/<from>_<to>.pdf`
- Override root with `MLLOG_DIR` env var.
- Override logbooks dir with `MLLOG_LOGBOOKS_DIR` env var.

## Dashboard

A local read-only web UI for browsing records and viewing experiment details.

```bash
pip install "multimodal-mllog[dashboard]"
mllog dashboard
# opens http://localhost:8265
```

Two views: Sessions (table, filterable) and Record Detail (metrics, config deltas, agent activity).

## Development

```bash
git clone https://github.com/jean-johnson-zwix/multimodal_mllogger
cd multimodal_mllogger
make install         # create venv + install deps
make test            # run pytest
make lint            # check core purity (no banned imports)
make frontend        # rebuild dashboard frontend
```

## License

MIT
