Metadata-Version: 2.5
Name: plural
Version: 0.13.3
Summary: Build environments, evaluate agents, compare models, and capture traces.
Project-URL: Homepage, https://github.com/lastlabs-ai/plural
Project-URL: Documentation, https://lastlabs-ai.github.io/plural/
Project-URL: Repository, https://github.com/lastlabs-ai/plural
Project-URL: Issues, https://github.com/lastlabs-ai/plural/issues
Project-URL: Changelog, https://github.com/lastlabs-ai/plural/blob/main/CHANGELOG.md
Author: Plural
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: ai,benchmarks,environments,llm,openrouter,routing,tracing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: httpx<1,>=0.27
Requires-Dist: pydantic<3,>=2.7
Requires-Dist: pyyaml<7,>=6.0
Requires-Dist: tomli<3,>=2.0; python_version < '3.11'
Requires-Dist: typer<1,>=0.12
Provides-Extra: all
Requires-Dist: daytona<1,>=0.207; extra == 'all'
Requires-Dist: keyring<26,>=25; extra == 'all'
Requires-Dist: opentelemetry-api<2,>=1.27; extra == 'all'
Requires-Dist: opentelemetry-sdk<2,>=1.27; extra == 'all'
Provides-Extra: daytona
Requires-Dist: daytona<1,>=0.207; extra == 'daytona'
Provides-Extra: keyring
Requires-Dist: keyring<26,>=25; extra == 'keyring'
Provides-Extra: otel
Requires-Dist: opentelemetry-api<2,>=1.27; extra == 'otel'
Requires-Dist: opentelemetry-sdk<2,>=1.27; extra == 'otel'
Description-Content-Type: text/markdown

# Plural

Plural 0.13.3 is a Python SDK and CLI for building reproducible agent
evaluations and keeping the evidence behind every score.

## Evaluation in seven objects

1. **Environment** — world, actions, State, Observation, resources, and Runtime.
2. **Verifier** — deterministic, Agent, or Human scoring.
3. **Task** — instructions bound to one Environment and its Verifiers.
4. **Benchmark** — immutable versioned Task pins.
5. **Agent** — catalog model, instructions, and optional Harness.
6. **Harness** — native loop, a built-in name, or a custom executable.
7. **Job** — Agents × Tasks × attempts, with retries and evidence.

Python is the semantic source of truth. YAML serializes the same objects
losslessly, and the CLI uses the same resolver and defaults.

## Install

```bash
pip install "plural==0.13.3"
```

## First evaluation

```python
from pathlib import Path
from plural import Agent, Benchmark, Job, Task
from plural.verifiers import DeterministicVerifier

verify = Path("verify.py").read_text(encoding="utf-8")
verifier = DeterministicVerifier(
    name="resolved",
    check=("python", "-c", verify),
)
task = Task(
    name="ticket-1",
    instructions="Resolve the support ticket.",
    environment=environment,
    verifiers=[verifier],
)
benchmark = Benchmark(name="support", version="1.0.0", tasks=(task,))
agent = Agent(model="openai/gpt-5.6-luna", instructions="Use the available actions.")
job = Job(benchmark, agents=(agent,))

print(job.plan.trial_count)
```

Here `environment` is an `Environment` instance. Typed Python actions run
from the class directory:

```python
environment = SupportQueue(runtime=runtime)
```

Validate and run the same graph:

```bash
plural init support-eval
cd support-eval
plural validate project.py:job
plural export project.py:job --output job.yaml
plural inspect job.yaml
plural run job.yaml --dry-run
```

After implementing the Environment and Verifier, `plural run job.yaml` uses
local orchestration by default. The Environment Runtime still selects trusted
local subprocess, Docker, Daytona, or an installed provider plugin. Hosted
submission is explicit:

```bash
plural run job.yaml --hosted
```

Start with the [documentation](docs/index.md), the executable
[support queue](examples/first-project/), and [Wordle](examples/wordle/).

## Development

```bash
uv sync --group dev --group docs
uv run ruff check .
uv run mypy --strict src/plural tests/typing/consumer.py
uv run pytest
uv run python scripts/check_docs.py
uv run mkdocs build --strict
```

## License

Apache-2.0
