Metadata-Version: 2.4
Name: mrsnappy
Version: 0.2.2
Summary: 3D fluorescent puncta detection for microscopy z-stacks.
Author: Marco Rojas-Cessa
License-Expression: MIT
Project-URL: Homepage, https://github.com/marcorojas-cessa/SNAPpy
Project-URL: Source, https://github.com/marcorojas-cessa/SNAPpy
Project-URL: Issues, https://github.com/marcorojas-cessa/SNAPpy/issues
Keywords: microscopy,fluorescence,puncta,spot-detection,bioimage-analysis
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Scientific/Engineering :: Image Processing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: joblib>=1.3
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: PyYAML>=6.0
Requires-Dist: scikit-image>=0.21
Requires-Dist: scikit-learn<1.8,>=1.7.2
Requires-Dist: scipy>=1.10
Requires-Dist: tifffile>=2023.7.10
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Dynamic: license-file

# SNAPpy

SNAPpy is a Python implementation of a two-stage workflow for detecting fluorescent puncta in 3D microscopy z-stacks. It first finds candidate puncta with configurable 3D image-processing recipes, then uses local 3D features and an SVM classifier to remove false candidates.

The package includes bundled ch1, ch2, and ch3 models optimized for Rothstein lab z-stack images. These models are intended to make routine lab processing simple while preserving the lower-level API for training, benchmarking, and custom model development.

SNAPpy does not require a GPU. The primary publication benchmark ran SNAPpy on a small CPU-only AWS `r7i.large` worker with 2 vCPUs and 16 GiB RAM.

## Installation

Install SNAPpy from PyPI with:

```bash
python -m pip install mrsnappy
```

Then check that the command-line tool is available:

```bash
mrsnappy --help
```

To install the current GitHub version instead of the PyPI release:

```bash
python -m pip install "git+https://github.com/marcorojas-cessa/SNAPpy.git"
```

If you are installing from a local checkout:

```bash
git clone https://github.com/marcorojas-cessa/SNAPpy.git
cd SNAPpy
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install .
```

For development:

```bash
python -m pip install -e ".[dev]"
```

### Naming

- The repository and project are named `SNAPpy`.
- The PyPI distribution name is `mrsnappy`.
- The terminal command is `mrsnappy`.
- The Python import package is `mrsnappy`.

Python packages are installed with `pip`, so the install command is `python -m pip install ...`, not `python install ...`. The PyPI name `snappy` is already used by an unrelated project, so SNAPpy uses `mrsnappy` as both its install name and command-line tool.

## Quick Start

Run the optimized ch1 model on one 3D TIFF stack:

```bash
mrsnappy detect \
  --model ch1 \
  --input /path/to/zstack.tif \
  --output /path/to/detections.csv
```

Run the optimized ch2 or ch3 models by changing `--model`:

```bash
mrsnappy detect --model ch2 --input image.tif --output ch2_detections.csv
mrsnappy detect --model ch3 --input image.tif --output ch3_detections.csv
```

The output CSV contains `detection_id`, `x`, `y`, `z`, and `score` columns. Coordinates are written in image voxel coordinates, with `z` corresponding to the stack axis.

Typical output:

```csv
detection_id,x,y,z,score
1,42.3,88.1,12.0,1.74
2,51.9,91.5,13.2,0.62
```

## Batch Detection

For a folder arranged as:

```text
images/
  image_001.tif
  image_002.tif
```

run:

```bash
mrsnappy detect \
  --model ch1 \
  --input /path/to/images \
  --output /path/to/detections
```

For an explicit image list, pass `--input-list image_paths.txt` instead of `--input`.

## Custom Model Detection

Use this only if you trained or optimized your own model:

```bash
mrsnappy detect \
  --model /path/to/model.joblib \
  --input /path/to/zstack.tif \
  --output /path/to/detections.csv
```

Optimized SNAPpy models embed the selected processing recipe. Use `--config /path/to/config.yaml` only when intentionally overriding that recipe.

## Model Optimization

Create an editable optimization config, then run optimization:

```bash
mrsnappy init-config --output config.yaml
mrsnappy optimize \
  --train-dir /path/to/labeled_dataset \
  --out-dir /path/to/model \
  --config config.yaml
```

The config controls Stage 1 candidate-generation sweeps, Stage 1 pass/fail guardrails, the number of Stage 1 recipes shortlisted into Stage 2, Stage 2 feature-pack sweeps, fitting choices, and SVM tuning.

The `train/` and `val/` folders have distinct roles. SNAPpy uses `train/` to fit the Stage 2 SVM candidate classifier. It uses `val/` to screen and rank Stage 1 recipes, select Stage 2 feature/SVM settings, tune the final decision threshold, and choose the winning model. Final held-out test scoring should be done outside SNAPpy with `mrsnappy detect`.

The default Stage 1 background-correction sweep uses `rolling_box_3d`, a SciPy 3D grayscale-opening approximation that is substantially faster than exact 3D rolling-ball correction on large z-stacks. Custom configs can instead request `slice_opening_2d` for slice-wise 2D morphological opening, `rolling_ball_2d` for slice-wise scikit-image rolling-ball correction, or `rolling_ball_3d` for scikit-image n-dimensional rolling-ball correction.

### Expected Dataset Layout

`mrsnappy optimize` expects a labeled dataset root containing `train/` and `val/` directories. Each 3D TIFF image must have a same-stem CSV file with ground-truth coordinates:

```text
labeled_dataset/
  train/
    image_001.tif
    image_001.csv
    image_002.tif
    image_002.csv
  val/
    image_101.tif
    image_101.csv
```

Ground-truth CSV files must contain case-insensitive `x`, `y`, and `z` columns in voxel coordinates. TIFF files must contain finite numeric voxel values; images with `NaN` or `Inf` are rejected.

### Optimizer Outputs

By default, optimization writes only the essential model files:

```text
model/
  model.joblib
  model_manifest.json
  effective_config.yaml
  optimizer_plan.json
  dataset_profile.json
  summary.json
  summary.md
```

`model.joblib` is the trained model used by `mrsnappy detect`. `model_manifest.json` records the winning Stage 1 recipe, Stage 2 recipe, selected feature pack, SVM parameters, decision threshold, and validation metrics. `effective_config.yaml` is the exact resolved config used for the run. `optimizer_plan.json` records the number of Stage 1 configs, Stage 2 recipes after shortlist, and SVM settings before computation.

To export concise optimizer evidence, add:

```bash
mrsnappy optimize ... --export-optimize-report
```

This creates:

```text
model/
  export_optimize_report/
    stage1_recipes.csv
    stage1_by_image.csv
    stage1_summary.csv
    stage2_recipes.csv
    stage2_summary.csv
    selection_decision.json
    selection_decision.md
```

`stage1_recipes.csv` lists every Stage 1 candidate-generation recipe and its ID. `stage1_by_image.csv` reports per-validation-image Stage 1 TP, FP, FN, candidate count, GT count, precision, recall, and F1 before SVM classification. `stage1_summary.csv` reports aggregate Stage 1 metrics, guardrail pass/fail status, preflight score, rank, and shortlist status. `stage2_recipes.csv` lists every Stage 2 recipe expanded from the shortlisted Stage 1 recipes. `stage2_summary.csv` reports the finalized Stage 1 plus Stage 2 validation outcome, including TP, FP, FN, precision, recall, F1, SVM parameters, decision threshold, runtime, and winner status.

To export candidate-level validation features for the winning pipeline, add:

```bash
mrsnappy optimize ... --export-candidate-features
```

This creates:

```text
model/
  export_candidate_features/
    val_candidates.csv
    candidate_features_manifest.json
```

`val_candidates.csv` includes validation candidates from only the winning Stage 1 recipe and winning Stage 2 feature pack. It includes `image_id`, `candidate_id`, subpixel `x/y/z`, `maxima_score`, `svm_score`, `model_score`, `decision_threshold`, `accepted_by_model`, `accepted_detection_id`, candidate ground-truth `label`, matched GT ID and coordinates, nearest GT distance, and the selected feature columns. `maxima_score` is the detector response value at the candidate maximum before Gaussian fitting and SVM classification. For LoG detection, this is the robust-z-normalized LoG response. For h-max detection, this is the processed-image value at the accepted h-max local maximum.

The candidate feature export uses the fitted validation candidates generated during Stage 2 evaluation for the winning recipe and does not export non-winning recipes.

### Custom Optimization Configs

The default config is only a readable YAML file. You can use it as a template:

```bash
mrsnappy init-config --output config.yaml
```

To test a small custom optimizer, edit `config.yaml` so it contains only the Stage 1 recipe, Stage 2 feature pack, fit variant, and SVM settings you want. The command stays the same:

```bash
mrsnappy optimize \
  --train-dir /path/to/labeled_dataset \
  --out-dir /path/to/custom_model \
  --config config.yaml
```

Before running an expensive optimization, inspect the planned search:

```bash
mrsnappy optimize \
  --train-dir /path/to/labeled_dataset \
  --out-dir /path/to/model \
  --config config.yaml \
  --dry-run
```

The dry run writes `optimizer_plan.dry_run.json` and prints the Stage 1 recipe count, Stage 2 recipe count after shortlist, SVM grid size, and safety caps without training a model.

## Bundled Lab Models

| Channel | Source run | Validation main F1 | Test main F1 | Notes |
|---|---:|---:|---:|---|
| ch1 | `native_ch1_replay_v12` | 0.941 | 0.919 | Strong performance on the ch1 validation and held-out test split. |
| ch2 | `native_ch2_replay_v5` | 0.862 | 0.885 | Balanced precision/recall on the ch2 held-out test split. |
| ch3 | `native_ch3_replay_v5` | 0.935 | 0.858 | High recall but lower strict localization F1; review detections for quantitative localization-sensitive analyses. |

These bundled models were optimized on existing Rothstein lab channel-specific data. They should be validated on any new microscope, acquisition settings, sample preparation, or staining condition before being used for final quantitative conclusions.

## Python API

The command-line interface is the simplest way to process routine lab images. Use the Python API when integrating SNAPpy into another analysis script.

```python
from mrsnappy import detect, init_config, optimize, optimize_dry_run

detect(model="ch1", input_path="image.tif", output="detections.csv")
detect(model="/path/to/model.joblib", input_path="/path/to/images", output="/path/to/detections")

init_config("config.yaml")
optimize_dry_run(
    train_dir="/path/to/labeled_dataset",
    out_dir="/path/to/model",
    config="default",
)

optimize(
    train_dir="/path/to/labeled_dataset",
    out_dir="/path/to/model",
    config="default",
    export_optimize_report=True,
    export_candidate_features=False,
)
```

The API expects the same dataset layout and produces the same output files as the CLI.
See [docs/cli_api.md](docs/cli_api.md) for the complete CLI/API reference.

## How It Works

SNAPpy uses a two-stage workflow:

1. It processes the 3D image with optional background correction, mandatory global robust z-score normalization, and optional smoothing, then searches for local maxima in 3D.
2. It refines each candidate with local Gaussian-style fitting and measures intensity, shape, contrast, and background features.
3. It applies a trained classifier to keep candidates that resemble true puncta and reject likely artifacts.
4. It writes final 3D puncta coordinates and decision scores.

During optimization, SNAPpy first screens candidate-generation recipes on validation images for enough recall without excessive candidate burden. It then ranks the candidate-generation recipes that pass this screen and sends the top three Stage 1 configurations to Stage 2 by default. Stage 2 fits each SVM configuration on `train/`, evaluates it on all `val/` images, and chooses the final recipe from validation performance.

See [docs/workflow.md](docs/workflow.md) for a more detailed explanation.

## Repository Scope

This repository contains the installable SNAPpy package, bundled lab-use models, documentation, and lightweight tests. Raw microscopy data, benchmark result trees, cluster logs, and manuscript-generation artifacts are intentionally excluded.

## More Documentation

- [Installation guide](docs/installation.md)
- [CLI/API reference](docs/cli_api.md)
- [Bundled lab models](docs/models.md)
- [Hardware guidance](docs/hardware.md)
- [Workflow overview](docs/workflow.md)

## Citation

If you use SNAPpy in a publication, cite the associated manuscript once available.
