Metadata-Version: 2.1
Name: snapper-ml
Version: 0.4.0
Summary: A framework for reproducible machine learning
Author-email: Antonio Molner Domenech <antonio.molner@correo.ugr.es>, Yeray López Ramírez <ylopezramirez@correo.ugr.es>
License: MIT License
Project-URL: homepage, https://github.com/SnapperML/SnapperML
Keywords: machine learning,reproducibility,automation
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: docker>=7.1.0
Requires-Dist: EasyProcess>=1.1
Requires-Dist: mlflow>=2.17.2
Requires-Dist: gorilla>=0.4.0
Requires-Dist: optuna>=4.0.0
Requires-Dist: docstring-parser>=0.16
Requires-Dist: pydantic>=2.9.2
Requires-Dist: pydantic-settings>=2.6.1
Requires-Dist: python-dotenv>=1.0.1
Requires-Dist: ray>=2.38.0
Requires-Dist: PyYAML>=6.0.2
Requires-Dist: pytictoc>=1.5.3
Requires-Dist: coloredlogs>=15.0.1
Requires-Dist: py-cpuinfo>=9.0.0
Requires-Dist: typer>=0.13.0
Requires-Dist: pystache>=0.6.5
Requires-Dist: shellingham>=1.5.4
Requires-Dist: ruamel.yaml>=0.18.6
Requires-Dist: knockknock>=0.1.8.1
Requires-Dist: colorama>=0.4.6
Requires-Dist: termcolor>=2.5.0
Requires-Dist: mysqlclient>=2.2.5
Requires-Dist: numpy>=2.1.3
Requires-Dist: psycopg2>=2.9.10
Requires-Dist: flask_cors>=5.0.0
Requires-Dist: google-api-python-client
Requires-Dist: optuna-integration
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: tox; extra == "dev"
Requires-Dist: black; extra == "dev"

# SnapperML

[![](https://readthedocs.org/projects/snapperml/badge/?style=for-the-badge&version=latest)](https://snapperml.readthedocs.io/en/latest/?badge=latest)
![License: MIT](https://img.shields.io/badge/License-MIT-green.svg?style=for-the-badge)
[![PyPI version](https://img.shields.io/pypi/v/snapper-ml.svg?style=for-the-badge)](https://pypi.org/project/snapper-ml/)

<div style="text-align: center;">
  <img src="docs/assets/banner.png" alt="SnapperML">
</div>

SnapperML is a comprehensive framework for experiment tracking and machine learning operationalization (MLOps), built using well-supported technologies like [Mlflow](https://mlflow.org/), [Ray](https://github.com/ray-project/ray/), Docker, and more. It provides an opinionated workflow designed to facilitate both local and cloud-based experimentation.

## Key Features

- **Automatic Tracking**: Seamless integration with MLflow for parameter and metric tracking.
- **Distributed Training**: First-class support for distributed training and hyperparameter optimization using Optuna and Ray.
- **CLI-Based Execution**: Easily package and execute projects within containers using our intuitive Command Line Interface (CLI).
- **Web Interface**: A modern web interface developed with Vite, React, TypeScript, and Bootstrap for managing experiment configurations.

## Project Goals

SnapperML aims to:

1. **Enhance Maintainability**: By addressing technical debt and improving the codebase, making it cleaner and more efficient.
1. **Improve Scalability**: Ensure the system can handle large-scale experiments and concurrent requests smoothly.
1. **Provide a Robust Web UI**: A user-friendly interface that simplifies the setup and execution of ML experiments.
1. **Ensure Reproducibility**: Leverage MLOps principles to ensure experiments can be replicated easily.

## Architecture

### Overview

SnapperML integrates several components to streamline machine learning workflows:

- **CLI Framework**: Facilitates command-based interactions and logging for experiment execution.
- **Flask API**: Manages requests from the frontend and interfaces with backend processes.
- **Vite-Powered Web UI**: An accessible and intuitive web application that handles experiment configurations and tracks real-time logs.
- **Containerized Databases**: Securely stores experiment results using containerized MLflow and Optuna databases.
  > [!IMPORTANT]
  > Be sure to configure your databases and network settings carefully to ensure the security and integrity of your experiment data.

![Architecture Overview](./docs/assets/snapperml_architecture.png)

## Installation

### Prerequisites

- docker
- python 3.12+
- node.js (for UI development)

### Install

The python package can be install using **pip**:

```
pip install snapper-ml
```

Or from this repo:

```
pip install .
```

> [!NOTE]
> Python 3.12 or later is required. Ensure that Docker is installed and running on your system for full functionality.

## Deploy

To run SnapperML, you first need to deploy MLflow and Optuna databases. Execute:

> [!TIP]
> To use the SnapperML web interface, deploy it with:

```
snapper-ml make docker
```

Once the deploy finished you can execute `snapper-ml` in the CLI. For an ilustrative example, check the [example section](#Example).

To use snapperML web interface you need to deploy it too.

```
snapper-ml make UI
```

Open [localhost:4000](http://localhost:4000/) and upload your firsts experiments!

![](docs/assets/UI.png)

To stop snapper UI just execute:

```
make stop_UI
```

And to stop mlflow and optuna databases execute:

```
make stop_docker
```

> [!CAUTION]
> Running make stop_UI also stops the Docker containers for the databases, so ensure you have saved all necessary data.

## Documentation

The documentation is available [here](https://snapperml.readthedocs.io/en/latest/)

> [!TIP]
> Visit the documentation for more examples and detailed instructions.

## Example

```python
# train_svm.py

from snapper_ml import job

@job
def main(C, kernel, gamma='scale'):
    np.random.seed(1234)
    X_train, X_val, y_train, y_val = load_data()
    model = SVC(C=C, gamma=gamma, kernel=kernel)
    model.fit(X_train, y_train)
    accuracy = model.score(X_val, y_val)
    return {'val_accuracy': accuracy}


if __name__ == '__main__':
    main()
```

```yaml
# train_svm.yaml

name: "SVM"
kind: "group"
num_trials: 12
sampler: TPE

param_space:
  C: loguniform(0.01, 1000)
  gamma: choice(['scale', 'auto'])

metric:
  name: val_accuracy
  direction: maximize

ray_config:
  num_cpus: 4

data:
  folder: data/
  files: ["*QGSJet.txt"]

run:
  - train_svm.py
```

```

snapper-ml run --config_file=train_svm.yaml
```

> [!WARNING]
> Make sure the configuration files are correctly set to avoid runtime errors. Misconfigured parameters could lead to unexpected behavior.

There are more examples in the [examples folder](https://github.com/yerasiito/SnapperML/tree/master/examples).
