Metadata-Version: 2.4
Name: bravorl
Version: 0.1.0
Summary: A modular, SOTA, YAML-driven Reinforcement Learning framework built on PyTorch and Hydra.
Author: BravoRL Contributors
License: MIT
Keywords: reinforcement-learning,pytorch,hydra,ppo
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.0
Requires-Dist: hydra-core>=1.3.0
Requires-Dist: omegaconf>=2.3.0
Requires-Dist: gymnasium>=0.29.0
Requires-Dist: tensordict>=0.3.0
Requires-Dist: numpy>=1.23.0
Provides-Extra: wandb
Requires-Dist: wandb>=0.16.0; extra == "wandb"
Provides-Extra: tensorboard
Requires-Dist: tensorboard>=2.14.0; extra == "tensorboard"
Provides-Extra: mujoco
Requires-Dist: gymnasium[mujoco]>=0.29.0; extra == "mujoco"
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: mypy>=1.5.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: build>=1.2.0; extra == "dev"
Dynamic: license-file

# BravoRL

A modular, SOTA reinforcement learning framework built on **PyTorch** and **Hydra**, driven entirely by declarative YAML. Train any combination of algorithm, network architecture, and environment without writing code:

```bash
bravorl train --config /path/to/config.yaml runner.agent.learning_rate=0.0001
```

## Installation

```bash
pip install -e .
# optional extras
pip install -e ".[wandb]"       # Weights & Biases logging
pip install -e ".[tensorboard]" # TensorBoard logging
pip install -e ".[mujoco]"      # MuJoCo continuous-control environments
```

## Quickstart

Two ready-to-run configs live in `configs/`:

```bash
# Discrete control, CPU, ~seconds: PPO on CartPole-v1
bravorl train --config configs/ppo_cartpole.yaml

# Continuous control PPO on HalfCheetah-v4 (needs the [mujoco] extra)
bravorl train --config configs/ppo_halfcheetah.yaml runner.device=cpu
```

Config files may live anywhere on disk -- they don't need to be inside the package. Any positional argument after `--config` is a Hydra dotlist override addressed by its path in the config (e.g. `runner.total_timesteps=500000`, `runner.agent.clip_epsilon=0.1`).

## Architecture

```text
bravorl/
├── cli.py          # argparse <-> Hydra bridge (hydra.initialize_config_dir)
├── runner.py        # Trainer: instantiation + the env interaction loop
├── core/
│   ├── interfaces.py # cross-cutting ABCs (BaseLogger) and shared types
│   └── registry.py   # optional name -> class registries
├── algorithms/
│   ├── base.py       # BaseAlgorithm ABC
│   └── ppo.py         # PPO: GAE, clipped surrogate, clipped value loss
├── networks/
│   ├── base.py        # BaseNetwork ABC + shape-inference helpers
│   ├── mlp.py          # ActorCriticMLP (vector observations)
│   └── cnn.py           # ActorCriticCNN (image observations)
├── buffers/
│   ├── base.py          # BaseBuffer ABC
│   └── replay_buffer.py  # StandardReplayBuffer (on- and off-policy)
├── envs/
│   └── wrappers.py        # NormalizeObservation, ClipAction, RewardScaling, ...
└── utils/
    └── logger.py           # TensorBoardLogger, WandbLogger
```

Every component is instantiated from config via Hydra's `_target_` mechanism -- there is no hardcoded registry lookup and no `if env_id == "..."` branching anywhere. Network input/output shapes are always inferred at runtime from `env.observation_space` / `env.action_space`.

### The network-shaping problem, and how BravoRL solves it

A network needs to know the environment's observation/action shapes before it
can be built, but a naive recursive Hydra instantiation would try to build
`agent.network` before the environment even exists. `Trainer` sidesteps this
by being instantiated with `_recursive_=False` (see `cli.py`) and then doing
the instantiation itself, in the right order:

1. `env = instantiate(cfg.env)`
2. `network = instantiate(cfg.agent.network, obs_space=env.observation_space, action_space=env.action_space)`
3. `agent = instantiate(cfg.agent, network=network)` -- the explicit `network=` kwarg overrides the (still un-instantiated) `network:` key in the config.

## Extending BravoRL

* **New algorithm**: subclass `bravorl.algorithms.base.BaseAlgorithm`, implement `select_action`, `compute_loss` and `update`. Set `on_policy = True` if your algorithm must consume rollouts in collection order (like PPO); leave it `False` for off-policy algorithms sampled randomly from a replay buffer.
* **New network**: subclass `bravorl.networks.base.BaseNetwork`, implement `forward` and `get_action_distribution`.
* **New buffer**: subclass `bravorl.buffers.base.BaseBuffer`, implement `push`, `sample` and `__len__`.

Point any config's `_target_` at your new class's dotted path -- no other code needs to change.

## Non-functional notes

* Fully type-hinted, `mypy`-compliant.
* Google-style docstrings throughout.
* `pytest tests/` runs a fast CPU smoke test that trains PPO on CartPole-v1 for a handful of steps end-to-end.
