Metadata-Version: 2.1
Name: pynball-rl
Version: 1.0.2
Summary: A python implementation of the classic reinforcement learning domain pinball: http://irl.cs.brown.edu/pinball/
Home-page: https://github.com/tmssmith/pynball-rl
License: MIT
Author: Tom Smith
Author-email: tmssmith@gmail.com
Requires-Python: >=3.10,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: matplotlib (>=3.8.3,<4.0.0)
Requires-Dist: pygame (>=2.5.2,<3.0.0)
Requires-Dist: tomli (>=2.0.1,<3.0.0) ; python_version < "3.11"
Project-URL: Repository, https://github.com/tmssmith/pynball-rl
Description-Content-Type: text/markdown

# PynBall
Python implementation of the classic Pinball domain. 

The goal is to navigate a physically modelled ball around a number of obstacles to reach a target. The ball will bounce elastically off obstacles. The agent can apply small forces to the ball, accelerating it in the $x$ or $y$ axes. 

### Dynamics
The domain has a 4-dimensional continuous state space and a 1-dimensional discrete action space. Transition dynamics are stochastic with configuration.
#### State space:
State is representated as the ball position and velocity: $(x, y, \dot{x}, \dot{y})$.
#### Action space:
There are five integer actions available to the agent in each state:
- 0: Increase velocity in $x$,
- 1: Increase velocity in $y$,
- 2: Decrease velocity in $x$,
- 3: Decrease velocity in $y$,
- 4: No-Operation (configurable).

Changes to velocity are stochastic, modelled as normal distribution centered around the requested change, with a configurable standard deviation.

 #### Rewards:
- -1 for No-Operation action,
- -5 for all other actions,
- +10,000 for reaching the goal.

### Have a go
To play interactively run `python -m pynball_rl` and select a difficulty between 1 and 3. 

### Configurations
A number of configuration files are provided in  `pynball_rl.configs`. Configuration parameters are:
- `seed`: Seed for random number generator
- `step_duration`: Number of dynamics calculations per step. A larger value will improve robustness but reduce FPS.
- `drag`: Drag coefficient. The ball velocity is multiplied by this at the end of each step. Setting to 0.0 will effectively make the state space 2-dimensional $(x,y)$.
- `stddev_x`: The standard deviation of the normal distribution from which the change in $x$-velocity is sampled. Set to 0.0 for deterministic dynamics. 
- `stddev_y`: The standard deviation of the normal distribution from which the change in $y$-velocity is sampled. Set to 0.0 for deterministic dynamics. 
- `allow_noop` : Whether to include the no-operation action in the state space.

Additionally ball start location and radius, target location and radius, and obstacle placements can be set through configuration.

### Acknowledgements
The pinball domain was introduced in:

    G.D. Konidaris and A.G. Barto. Skill Discovery in Continuous Reinforcement Learning Domains using Skill Chaining. Advances in Neural Information Processing Systems 22, December 2009.

This implementation is based on:
- Original Java implementation at http://irl.cs.brown.edu/pinball/
- Python2 implementation at https://github.com/amarack/python-rl

