Metadata-Version: 2.4
Name: pi-line
Version: 1.0.0
Summary: Zero-dependency parallel script runner.
Author-email: Maximilian Todea <demon.and.max@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/importt-ant/pi-line
Project-URL: Repository, https://github.com/importt-ant/pi-line
Project-URL: Issues, https://github.com/importt-ant/pi-line/issues
Keywords: python,automation,pipeline,devtools,parallel,task-runner,process-pool,zero-dependencies
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Provides-Extra: dev
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: pytest-cov>=4.1; extra == "dev"

# π〽️❗: pi-line

A zero-dependency parallel script runner for Python.

Define units of work (**Pi**), execute them in parallel with a **Runner**, or feed them continuously through a **Line**.

## Install

```bash
pip install pi-line
```

## Quick start

### Run scripts in parallel

```python
from piline import Pi, Runner

pis = [
    Pi(name="train", script="train.py", args=["--epochs", "10"]),
    Pi(name="eval", script="eval.py"),
]

runner = Runner(max_workers=4)
results = runner.run(pis)

for r in results:
    print(f"{r.pi_name}: {'PASS' if r.succeeded else 'FAIL'}")
```

### Continuous feed with a Line

```python
from piline import Pi, Runner, Line

runner = Runner(max_workers=4)

with Line(runner, max_results=5000) as line:
    line.put(Pi(name="job1", script="job.py"))
    line.put(Pi(name="job2", script="job.py"))

    # look up a result by id
    result = line.get(pi_id)

    # flush results to external storage
    batch = line.drain_results()
# line.stop() called automatically
```

### Argument templates

Use `{artefact_dir}` and `{task_dir}` placeholders in args — they get resolved to the real paths before execution:

```python
Pi(
    name="train",
    script="train.py",
    args=["--output", "{artefact_dir}/model.pt"],
)
```

## Concepts

- **Pi** — A unit of work. Wraps a script with args, env vars, and an optional timeout. Gets a unique ID on creation. `.py` files run with Python; anything else runs directly.
- **Result** — Outcome of running a Pi. Exit code, timing, paths to stdout/stderr logs and artefacts.
- **Runner** — Executes a batch of Pi's in parallel using `ProcessPoolExecutor`.
- **Line** — Thread-safe queue that feeds Pi's to a Runner in batches. Results stored in a capped dict keyed by `pi_id`. Supports context manager, per-Pi and per-batch callbacks, and `drain_results()` for periodic flushing.

## Output layout

```
.piline/runs/<pi_name>/<pi_id>/
    stdout.log
    stderr.log
    artefact/
```

## Environment variables

Scripts receive these env vars automatically:

- `PILINE_PI_ID` — The Pi's unique ID
- `PILINE_PI_NAME` — The Pi's name
- `PILINE_TASK_DIR` — Path to the task directory
- `PILINE_ARTEFACT_DIR` — Path to the artefact subdirectory

## Callbacks

```python
Line(
    runner,
    on_pi_complete=lambda r: print(f"Done: {r.pi_name} ({'PASS' if r.succeeded else 'FAIL'})"),
    on_batch_complete=lambda results: print(f"Batch of {len(results)} finished"),
)
```

## API reference

See [docs/index.md](docs/index.md).
