Metadata-Version: 2.4
Name: preflight-ml
Version: 0.1.1
Summary: Pre-flight checks for PyTorch pipelines. Catch silent failures before they waste your GPU.
Project-URL: Homepage, https://github.com/Rusheel86/preflight
Project-URL: Repository, https://github.com/Rusheel86/preflight
Project-URL: Issues, https://github.com/Rusheel86/preflight/issues
Author: Rusheel Sharma
License: MIT License
        
        Copyright (c) 2026 Rusheel Sharma
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: data-validation,debugging,deep-learning,machine-learning,mlops,pytorch
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Requires-Dist: click>=8.0
Requires-Dist: numpy>=1.20
Requires-Dist: rich>=12.0
Requires-Dist: torch>=1.9
Provides-Extra: dev
Requires-Dist: mypy>=1.0; extra == 'dev'
Requires-Dist: pre-commit>=3.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Description-Content-Type: text/markdown

# preflight

> Pre-flight checks for PyTorch pipelines. Catch silent failures before they waste your GPU.

[![CI](https://github.com/Rusheel86/preflight/actions/workflows/ci.yml/badge.svg)](https://github.com/Rusheel86/preflight/actions/workflows/ci.yml)
[![PyPI version](https://badge.fury.io/py/preflight-ml.svg)](https://pypi.org/project/preflight-ml/)
[![Python](https://img.shields.io/pypi/pyversions/preflight-ml.svg)](https://pypi.org/project/preflight-ml/)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![PyPI Downloads](https://img.shields.io/pypi/dm/preflight-ml.svg)](https://pypi.org/project/preflight-ml/)

---

Most deep learning bugs don't crash your training loop, they silently produce a garbage model.
NaNs in your data, labels leaking between train and val, wrong channel ordering, dead gradients.
You won't know until hours later, after the GPU bill has landed.

**preflight** is a pre-training validation tool you run in 30 seconds before starting any training job.
It's not a linter. It's a pre-flight check, similar to the kind pilots run before the expensive thing takes off.

---

## Install

```bash
pip install preflight-ml
```

## Quickstart

Create a small Python file that exposes your dataloader:

```python
# my_dataloader.py
import torch
from torch.utils.data import DataLoader, TensorDataset

x = torch.randn(200, 3, 224, 224)
y = torch.randint(0, 10, (200,))
dataloader = DataLoader(TensorDataset(x, y), batch_size=32)
```

Run preflight:

```bash
preflight run --dataloader my_dataloader.py
```

Output:

```
preflight — pre-training check report
╭────────────────────────┬──────────┬────────┬──────────────────────────────────────────────────╮
│ Check                  │ Severity │ Status │ Message                                          │
├────────────────────────┼──────────┼────────┼──────────────────────────────────────────────────┤
│ nan_inf_detection      │ FATAL    │ PASS   │ No NaN or Inf values found in 10 sampled batches │
│ normalisation_sanity   │ WARN     │ PASS   │ Normalisation looks reasonable (mean=0.001)      │
│ channel_ordering       │ WARN     │ PASS   │ Channel ordering looks correct (NCHW)            │
│ label_leakage          │ FATAL    │ PASS   │ No val_dataloader provided — skipped             │
│ split_sizes            │ INFO     │ PASS   │ train=200 samples                                │
│ vram_estimation        │ WARN     │ INFO   │ No CUDA GPU detected — skipped                   │
│ class_imbalance        │ WARN     │ PASS   │ Class distribution looks balanced                │
│ shape_mismatch         │ FATAL    │ PASS   │ No model provided — skipped                      │
│ gradient_check         │ FATAL    │ PASS   │ No model+loss provided — skipped                 │
╰────────────────────────┴──────────┴────────┴──────────────────────────────────────────────────╯

  0 fatal  0 warnings  9 passed

Pre-flight passed. Safe to start training.
```

## Checks

preflight runs 10 checks across three severity tiers. A **FATAL** failure exits with code 1 and blocks CI.

| Check | Severity | What it catches |
|---|---|---|
| `nan_inf_detection` | FATAL | NaN or Inf values anywhere in sampled batches |
| `label_leakage` | FATAL | Samples appearing in both train and val sets |
| `shape_mismatch` | FATAL | Dataset output shape incompatible with model input |
| `gradient_check` | FATAL | Zero gradients, dead layers, exploding gradients |
| `normalisation_sanity` | WARN | Data that looks unnormalised (raw pixel values etc.) |
| `channel_ordering` | WARN | NHWC tensors when PyTorch expects NCHW |
| `vram_estimation` | WARN | Estimated peak VRAM exceeds 90% of GPU memory |
| `class_imbalance` | WARN | Severe class imbalance beyond configurable threshold |
| `split_sizes` | INFO | Empty or degenerate train/val splits |
| `duplicate_samples` | INFO | Identical samples within a split |

## With a model

Pass a model file to enable shape, gradient, and VRAM checks:

```python
# my_model.py
import torch.nn as nn
model = nn.Sequential(nn.Flatten(), nn.Linear(3 * 224 * 224, 10))
```

```python
# my_loss.py
import torch.nn as nn
loss_fn = nn.CrossEntropyLoss()
```

```bash
preflight run \
  --dataloader my_dataloader.py \
  --model my_model.py \
  --loss my_loss.py \
  --val-dataloader my_val_dataloader.py
```

## Configuration

Add a `.preflight.toml` to your repo root to configure thresholds and disable checks:

```toml
[thresholds]
imbalance_threshold = 0.05
nan_sample_batches = 20

[checks]
vram_estimation = false

[ignore]
# check = "class_imbalance"
# reason = "intentional: rare event dataset"
```

## CI integration

Add to your GitHub Actions workflow:

```yaml
- name: Install preflight
  run: pip install preflight-ml

- name: Run pre-flight checks
  run: preflight run --dataloader scripts/dataloader.py --format json
```

The `--format json` flag outputs machine-readable results. Exit code is `1` if any FATAL check fails, `0` otherwise.

## List all checks

```bash
preflight checks
```

## What preflight does NOT do

- It does not replace unit tests. Use pytest for code logic.
- It does not guarantee a correct model. Passing preflight is a minimum safety bar, not a certification.
- It does not run your full training loop. Use it as a gate before training starts.
- It does not modify your code unless you pass `--fix`.

## Roadmap

- [ ] `--fix` flag — auto-patch common issues (channel ordering, normalisation)
- [ ] Dataset snapshot + drift detection (`preflight diff baseline.json new_data.pt`)
- [ ] Full dry-run mode (one batch through model + loss + backward)
- [ ] Jupyter magic command (`%load_ext preflight`)
- [ ] `preflight-monai` plugin for medical imaging checks
- [ ] `preflight-sktime` plugin for time series checks

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md). New checks are welcome. Each one needs a passing test,
a failing test, and a fix hint.

## License

MIT — see [LICENSE](LICENSE).
