Metadata-Version: 2.4
Name: trade-study
Version: 0.1.0
Summary: Multi-objective trade-study orchestration: scoring, Pareto optimization, and Bayesian stacking
Author-email: "Joshua C. Macdonald" <jmacdo16@jh.edu>
License-Expression: MIT
Project-URL: Homepage, https://github.com/jcm-sci/trade-study
Project-URL: Documentation, https://jcm-sci.github.io/trade-study/
Project-URL: Repository, https://github.com/jcm-sci/trade-study
Project-URL: Changelog, https://github.com/jcm-sci/trade-study/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/jcm-sci/trade-study/issues
Keywords: trade-study,model-selection,scoring-rules,pareto,bayesian-stacking,design-of-experiments,sensitivity-analysis,multi-objective-optimization
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Provides-Extra: scoring
Requires-Dist: scoringrules>=0.8; extra == "scoring"
Provides-Extra: pareto
Requires-Dist: pymoo>=0.6; extra == "pareto"
Provides-Extra: stacking
Requires-Dist: arviz>=0.20; python_version < "3.12" and extra == "stacking"
Requires-Dist: arviz>=1.0; python_version >= "3.12" and extra == "stacking"
Requires-Dist: scipy>=1.10; extra == "stacking"
Provides-Extra: design
Requires-Dist: pyDOE3>=1.5; extra == "design"
Requires-Dist: SALib>=1.5; extra == "design"
Requires-Dist: scipy>=1.10; extra == "design"
Provides-Extra: adaptive
Requires-Dist: optuna>=4.0; extra == "adaptive"
Provides-Extra: parallel
Requires-Dist: joblib>=1.3; extra == "parallel"
Provides-Extra: all
Requires-Dist: trade-study[adaptive,design,parallel,pareto,scoring,stacking]; extra == "all"
Provides-Extra: examples
Requires-Dist: scikit-learn>=1.3; extra == "examples"
Requires-Dist: trade-study[all]; extra == "examples"
Provides-Extra: test
Requires-Dist: pytest>=8.1; extra == "test"
Requires-Dist: pytest-cov>=6.0; extra == "test"
Requires-Dist: trade-study[all]; extra == "test"
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.27; extra == "docs"
Provides-Extra: dev
Requires-Dist: mypy>=1.18; extra == "dev"
Requires-Dist: ruff>=0.13; extra == "dev"
Requires-Dist: trade-study[docs,test]; extra == "dev"
Dynamic: license-file

# trade-study

[![CI](https://github.com/jcm-sci/trade-study/actions/workflows/ci.yml/badge.svg)](https://github.com/jcm-sci/trade-study/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![Code style: ruff](https://img.shields.io/badge/code%20style-ruff-000000.svg)](https://github.com/astral-sh/ruff)
[![jcm-sci](https://img.shields.io/badge/jcm--sci-jcmacdonald.dev-blue)](https://jcmacdonald.dev/software/)

Multi-objective trade-study orchestration: define factors, build
parameter grids, run hierarchical study phases, and extract Pareto
fronts — for any domain where you compare alternatives against
competing objectives.

## Statement of need

Comparing design alternatives against multiple objectives is a common
task across scientific simulation, engineering trade-offs, and ML
hyperparameter tuning. Researchers typically glue together separate
tools for grid construction, execution, scoring, and Pareto analysis,
writing ad-hoc scripts that are hard to reproduce or extend to
multi-phase studies (screening → refinement → benchmark).

`trade-study` provides a single orchestration layer that composes these
steps into a reproducible, protocol-driven workflow. Users supply a
`Simulator` (generates data) and a `Scorer` (evaluates it); the
framework handles grid construction, parallel execution, Pareto
extraction, and phase chaining. All components are modular and
optional — use only what you need.

This package targets researchers and practitioners who need:

- structured multi-phase experimental design (screen → refine → benchmark),
- multi-objective Pareto analysis across heterogeneous factors, and
- a reproducible Python API that separates domain logic from study orchestration.

## Why trade-study?

| Need | Without trade-study | With trade-study |
|------|---------------------|------------------|
| Parameter grid | Manual `itertools.product` or one-off scripts | `build_grid(factors, method="sobol")` — full factorial, LHS, Sobol, Halton |
| Multi-objective ranking | Call pymoo directly, handle direction normalization | `extract_front(scores, directions)` — direction-aware |
| Phased studies | Custom loop with manual filtering between stages | `Study(phases=[Phase(..., filter_fn=top_k_pareto_filter(k=20)), ...])` |
| Adaptive search | Set up optuna study from scratch | `run_adaptive(world, scorer, factors, observables, n_trials=600)` |
| Reproducibility | Scattered scripts, no standard protocol | `Simulator` / `Scorer` protocols + `save_results()` / `load_results()` |

Existing tools solve pieces of this problem — [optuna](https://optuna.org/) for adaptive optimization, [pymoo](https://pymoo.org/) for multi-objective solvers, [SALib](https://salib.readthedocs.io/) for sensitivity analysis — but none provide the **hierarchical phase orchestration** that connects them into a single study.

## Quick start

```python
from trade_study import (
    Direction,
    Factor,
    FactorType,
    Observable,
    Phase,
    Study,
    build_grid,
    top_k_pareto_filter,
)

# 1. Define objectives
accuracy = Observable("accuracy", Direction.MAXIMIZE)
latency = Observable("latency_ms", Direction.MINIMIZE)
cost = Observable("cost_usd", Direction.MINIMIZE)

# 2. Define factors and build a design grid
factors = [
    Factor("learning_rate", FactorType.CONTINUOUS, bounds=(1e-4, 1e-1)),
    Factor("backend", FactorType.CATEGORICAL, levels=["A", "B", "C"]),
]
grid = build_grid(factors, method="lhs", n_samples=500)

# 3. Run a hierarchical study
study = Study(
    world=MySimulator(),  # implements Simulator protocol
    scorer=MyScorer(),  # implements Scorer protocol
    observables=[accuracy, latency, cost],
    phases=[
        Phase(
            "screening",
            grid=grid,
            filter_fn=top_k_pareto_filter(k=20),
        ),
        Phase("benchmark", grid="carry", filter_fn=None),
    ],
)
study.run(n_jobs=-1)

# 4. Inspect results
print(study.summary())
front = study.front()  # non-dominated configs
hv = study.front_hypervolume(ref=...)  # hypervolume indicator
```

### Protocols

Users implement two protocols to plug in their domain:

```python
from trade_study import Scorer, Simulator


class MySimulator:
    """Implements the Simulator protocol."""

    def generate(self, config: dict) -> tuple:
        """Return (truth, observations) for a given config."""
        ...


class MyScorer:
    """Implements the Scorer protocol."""

    def score(self, truth, observations, config: dict) -> dict[str, float]:
        """Return {observable_name: value} for a single trial."""
        ...
```

## Installation

```bash
pip install trade-study[all]
```

Or install only the extras you need:

```bash
pip install trade-study[design,pareto]
```

| Extra | Packages | Purpose |
|-------|----------|---------|
| `design` | [pyDOE3](https://github.com/relf/pyDOE3), [SALib](https://github.com/SALib/SALib), [scipy](https://scipy.org/) | Grid construction and sensitivity screening |
| `pareto` | [pymoo](https://pymoo.org/) | Non-dominated sorting and indicators |
| `scoring` | [scoringrules](https://github.com/frazane/scoringrules) | Proper scoring rules (CRPS, WIS, etc.) |
| `stacking` | [arviz](https://github.com/arviz-devs/arviz), scipy | Bayesian and score-based ensemble weights |
| `adaptive` | [optuna](https://optuna.org/) | Multi-objective Bayesian optimization |
| `parallel` | joblib | Parallel grid execution |
| `all` | All of the above | |

**Core dependency**: numpy only.

## API overview

### Design

```python
from trade_study import Factor, FactorType, build_grid, screen

factors = [
    Factor("x", FactorType.CONTINUOUS, bounds=(0.0, 1.0)),
    Factor("method", FactorType.CATEGORICAL, levels=["a", "b", "c"]),
]

grid = build_grid(factors, method="sobol", n_samples=1024)
si = screen(run_fn, factors, method="morris", n_eval=3000)
```

### Execution

```python
from trade_study import run_grid, run_adaptive

# Grid mode: evaluate all configs (optional parallelism)
results = run_grid(world, scorer, grid, observables, n_jobs=-1)

# Adaptive mode: multi-objective Bayesian optimization (NSGA-II)
results = run_adaptive(world, scorer, factors, observables, n_trials=600)
```

### Pareto analysis

```python
from trade_study import extract_front, hypervolume, pareto_rank

front_mask = extract_front(results.scores, directions)
ranks = pareto_rank(results.scores, directions)
hv = hypervolume(results.scores[front_mask], directions, ref=ref_point)
```

### Stacking

```python
from trade_study import stack_scores, stack_bayesian, ensemble_predict

weights = stack_scores(score_matrix)  # simplex-constrained optimization
weights = stack_bayesian(idata_dict)  # arviz stacking (Yao et al. 2018)
combined = ensemble_predict(predictions, weights)
```

### I/O

```python
from trade_study import save_results, load_results

save_results(results, "study_results")
results = load_results("study_results")
```

## Related packages

| Package | Description |
|---------|-------------|
| [TradeStudy.jl](https://github.com/jcm-sci/TradeStudy.jl) | Julia implementation of the same framework |

## Development

```bash
uv sync --extra dev
just ci          # lint → mypy --strict → pytest with coverage
just format      # auto-format
just check       # auto-fix lint
```

## License

MIT
