Metadata-Version: 2.4
Name: vesuvius
Version: 0.2.4
Project-URL: Homepage, https://github.com/ScrollPrize/villa
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Scientific/Engineering
Requires-Python: <3.14,>=3.10
Description-Content-Type: text/markdown
Requires-Dist: numpy>=2.0.2
Requires-Dist: pillow>=11.3.0
Requires-Dist: pynrrd>=1.1.3
Requires-Dist: requests>=2.32.5
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: fsspec>=2025.9.0
Requires-Dist: numcodecs<0.16,>=0.12.1
Requires-Dist: zarr<3,>=2
Requires-Dist: s3fs>=2025.9.0
Requires-Dist: cachetools>=6.2.4
Provides-Extra: volume-only
Requires-Dist: numpy>=2.0.2; extra == "volume-only"
Requires-Dist: pillow>=11.3.0; extra == "volume-only"
Requires-Dist: pynrrd>=1.1.3; extra == "volume-only"
Requires-Dist: requests>=2.32.5; extra == "volume-only"
Requires-Dist: pyyaml>=6.0.2; extra == "volume-only"
Requires-Dist: fsspec>=2025.9.0; extra == "volume-only"
Requires-Dist: numcodecs<0.16,>=0.12.1; extra == "volume-only"
Requires-Dist: zarr<3,>=2; extra == "volume-only"
Requires-Dist: s3fs>=2025.9.0; extra == "volume-only"
Provides-Extra: models
Requires-Dist: torch<2.9,>=2.6; extra == "models"
Requires-Dist: pytorch-lightning>=2.5.5; extra == "models"
Requires-Dist: pytorch-optimizer>=3.8.0; extra == "models"
Requires-Dist: monai>=1.5.1; extra == "models"
Requires-Dist: huggingface-hub>=0.35.0; extra == "models"
Requires-Dist: timm>=1.0.20; extra == "models"
Requires-Dist: dynamic-network-architectures>=0.3.1; extra == "models"
Requires-Dist: nnunetv2; extra == "models"
Requires-Dist: numba>=0.60.0; extra == "models"
Requires-Dist: batchgenerators>=0.25.1; extra == "models"
Requires-Dist: batchgeneratorsv2; extra == "models"
Requires-Dist: einops>=0.8.1; extra == "models"
Requires-Dist: fft-conv-pytorch>=1.2.0; extra == "models"
Requires-Dist: fvcore>=0.1.5.post20221221; extra == "models"
Requires-Dist: scikit-image>=0.24.0; extra == "models"
Requires-Dist: scipy>=1.13.1; extra == "models"
Requires-Dist: pandas>=2.3.2; extra == "models"
Requires-Dist: tensorstore>=0.1.69; extra == "models"
Requires-Dist: tqdm>=4.67.1; extra == "models"
Requires-Dist: typed-argument-parser>=1.11.0; extra == "models"
Requires-Dist: wandb[media]>=0.22.0; extra == "models"
Requires-Dist: psutil>=7.1.0; extra == "models"
Requires-Dist: aiohttp>=3.12.15; extra == "models"
Requires-Dist: blosc2>=2.5.1; extra == "models"
Requires-Dist: dask>=2024.8.0; extra == "models"
Requires-Dist: dask-image>=2024.5.3; extra == "models"
Requires-Dist: lxml>=6.0.2; extra == "models"
Requires-Dist: cucim-cu13>=25.12; extra == "models"
Provides-Extra: render
Requires-Dist: open3d<0.19,>=0.18; extra == "render"
Requires-Dist: trimesh>=4.8.2; extra == "render"
Requires-Dist: opencv-python-headless>=4.11.0.86; extra == "render"
Requires-Dist: opencv-contrib-python>=4.11.0.86; extra == "render"
Requires-Dist: libigl>=2.6.1; extra == "render"
Requires-Dist: alphashape>=1.3.1; extra == "render"
Requires-Dist: connected-components-3d>=3.24.0; extra == "render"
Requires-Dist: simpleitk>=2.5.2; extra == "render"
Requires-Dist: tifffile>=2024.8.30; extra == "render"
Provides-Extra: gui
Requires-Dist: napari>=0.5.6; extra == "gui"
Requires-Dist: magicgui>=0.10.1; extra == "gui"
Requires-Dist: magic-class>=0.7.17; extra == "gui"
Requires-Dist: pyqt6; extra == "gui"
Provides-Extra: notebooks
Requires-Dist: jupyterlab>=4; extra == "notebooks"
Requires-Dist: ipykernel>=6; extra == "notebooks"
Requires-Dist: matplotlib; extra == "notebooks"
Requires-Dist: nest-asyncio>=1.6.0; extra == "notebooks"
Provides-Extra: blending
Requires-Dist: scipy>=1.13.1; extra == "blending"
Requires-Dist: tqdm>=4.67.1; extra == "blending"
Requires-Dist: nest-asyncio>=1.6.0; extra == "blending"
Provides-Extra: tests
Requires-Dist: pytest>=8.4.2; extra == "tests"
Provides-Extra: all
Requires-Dist: torch<2.9,>=2.6; extra == "all"
Requires-Dist: pytorch-lightning>=2.5.5; extra == "all"
Requires-Dist: pytorch-optimizer>=3.8.0; extra == "all"
Requires-Dist: monai>=1.5.1; extra == "all"
Requires-Dist: huggingface-hub>=0.35.0; extra == "all"
Requires-Dist: timm>=1.0.20; extra == "all"
Requires-Dist: numba>=0.60.0; extra == "all"
Requires-Dist: dynamic-network-architectures>=0.3.1; extra == "all"
Requires-Dist: nnunetv2; extra == "all"
Requires-Dist: batchgenerators>=0.25.1; extra == "all"
Requires-Dist: batchgeneratorsv2; extra == "all"
Requires-Dist: einops>=0.8.1; extra == "all"
Requires-Dist: fft-conv-pytorch>=1.2.0; extra == "all"
Requires-Dist: fvcore>=0.1.5.post20221221; extra == "all"
Requires-Dist: scikit-image>=0.24.0; extra == "all"
Requires-Dist: scipy>=1.13.1; extra == "all"
Requires-Dist: pandas>=2.3.2; extra == "all"
Requires-Dist: tensorstore>=0.1.69; extra == "all"
Requires-Dist: tqdm>=4.67.1; extra == "all"
Requires-Dist: typed-argument-parser>=1.11.0; extra == "all"
Requires-Dist: wandb[media]>=0.22.0; extra == "all"
Requires-Dist: psutil>=7.1.0; extra == "all"
Requires-Dist: open3d<0.19,>=0.18; extra == "all"
Requires-Dist: trimesh>=4.8.2; extra == "all"
Requires-Dist: opencv-python-headless>=4.11.0.86; extra == "all"
Requires-Dist: opencv-contrib-python>=4.11.0.86; extra == "all"
Requires-Dist: libigl>=2.6.1; extra == "all"
Requires-Dist: alphashape>=1.3.1; extra == "all"
Requires-Dist: connected-components-3d>=3.24.0; extra == "all"
Requires-Dist: simpleitk>=2.5.2; extra == "all"
Requires-Dist: tifffile>=2024.8.30; extra == "all"
Requires-Dist: napari>=0.5.6; extra == "all"
Requires-Dist: magicgui>=0.10.1; extra == "all"
Requires-Dist: magic-class>=0.7.17; extra == "all"
Requires-Dist: pyqt6; extra == "all"
Requires-Dist: nest-asyncio>=1.6.0; extra == "all"
Requires-Dist: build; extra == "all"
Requires-Dist: pybind11>=3.0.1; extra == "all"
Requires-Dist: setuptools>=80.9.0; extra == "all"
Requires-Dist: wheel; extra == "all"
Requires-Dist: pytest>=8.4.2; extra == "all"
Requires-Dist: aiohttp>=3.12.15; extra == "all"
Requires-Dist: blosc2>=2.5.1; extra == "all"
Requires-Dist: dask>=2024.8.0; extra == "all"
Requires-Dist: dask-image>=2024.5.3; extra == "all"
Requires-Dist: lxml>=6.0.2; extra == "all"

# vesuvius
From [Vesuvius Challenge](https://scrollprize.org), a Python library for : 
- Accessing CT data of Herculaneum Scrolls
- Training 2d or 3d semantic segmentation models, with support for multi-task and multi-class
- Training 2d or 3d models on regression tasks
- Inferring with models trained with the trainers provided in the package or with pretrained nnUNetv2 models
  - Inference can be performed on remote data (http, s3) stored as Zarr arrays
- Rendering .obj segments with local or remote data
- Voxelizing large .obj segmentations for use as 3d labels
- Preprocessing labels of fiber-like structures
- Interactive labeling and model training through a Napari based trainer
- Proofreading large arrays of image/label pairs and saving approved chunks
- Computing structure tensors on large Zarr arrays, and deriving eigenvalues and eigenvectors from them

`vesuvius` also comes prepackaged with: 
- extensive data augmentation
- Dataset orchestrator with streaming adapters for image (tif/png/jpg), zarr, and napari-backed sources

_this package is in active development_


### Entrypoints: 

| Name                          | Script                             | Description                                                                                                                                                                                               |
| ----------------------------- | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `vesuvius.predict`            | `models.run.inference`             | outputs logits from a pretrained nnUNet-v2 model or one trained within the Vesuvius training framework; currently only works on Zarr data and can be fully distributed with `--num_parts` and `--part_id` |
| `vesuvius.blend_logits`       | `models.run.blending`              | blends the logits from `vesuvius.predict` using Gaussian blending                                                                                                                                         |
| `vesuvius.finalize_outputs`   | `models.run.finalize_outputs`      | performs softmax / argmax / none on the blended array and writes a final `uint8` volume                                                                                                                   |
| `vesuvius.compute_st`         | `structure_tensor.run_create_st`   | computes structure tensors on input data and derives eigen-values/vectors                                                                                                                                 |
| `vesuvius.napari_trainer`     | `napari_trainer.main_window`       | launches a Napari window for interactive training and inference                                                                                                                                           |
| `vesuvius.proofreader`        | `utils.vc_proofreader.main`        | opens a Napari window that loads local / remote image-label arrays and extracts training patches                                                                                                          |
| `vesuvius.voxelize_obj`       | `scripts.voxelize_objs`            | converts input `.obj` meshes to voxel grids and outputs `.tif` stacks                                                                                                                                     |
| `vesuvius.refine_labels`      | `scripts.edt_frangi_label`         | refines surface or fibre labels with a custom Frangi-based filter                                                                                                                                         |
| `vesuvius.render_obj`         | `rendering.mesh_to_surface`        | renders `.obj` meshes and outputs their surface-volume layers                                                                                                                                             |
| `vesuvius.flatten_obj`        | `rendering.slim_uv`                | flattens an `.obj` mesh using `slim_uv`                                                                                                                                                                   |
| `vesuvius.train`              | `models.run.train`                 | main entry point for training models                                                                                                                                                                      |




## 📓 Introductory notebooks
To get started, we recommend these notebooks that jump right in:

1. 📊 [Scroll Data Access](https://colab.research.google.com/github/ScrollPrize/vesuvius/blob/main/notebooks/example1_data_access.ipynb): an introduction to accessing scroll data using a few lines of Python!

2. ✒️ [Ink Detection](https://colab.research.google.com/github/ScrollPrize/vesuvius/blob/main/notebooks/example2_ink_detection.ipynb): load and visualize segments with ink labels, and train models to detect ink in CT.

3. 🧩 [Volumetric instance segmentation cubes](https://colab.research.google.com/github/ScrollPrize/vesuvius/blob/main/notebooks/example3_cubes_bootstrap.ipynb): how to access instance-annotated cubes with the `Cube` class, used for volumetric segmentation approaches.

## `vesuvius` does:
- **Data retrieval**: Fetches volumetric scroll data, surface volumes of scroll segments, and annotated volumetric instance segmentation labels. Remote repositories and local files are supported.
- **Data listing**: Lists the available data on [our data server](https://dl.ash2txt.org).
- **Data caching**: Caches fetched data to improve performance when accessing remote repositories.
- **Normalization**: Provides options to normalize data values.
- **Multiresolution**: Accesses and manages data at multiple image resolutions.

## `vesuvius` doesn't do:
- **Remote data modification**: The read-only library does not support modifying the original data.
- **Complex analysis**: While it provides access to data, it does not include built-in tools for complex data analysis or visualization.

## Installation

`vesuvius` can be installed with `pip`.
Then, before using the library for the first time, accept the license terms:
```sh
$ pip install vesuvius
$ vesuvius.accept_terms --yes
```

___
**Note:** 

The model framework utilized by `vesuvius` is _heavily_ inspired by [nnUNetv2](https://github.com/MIC-DKFZ/nnUNet). The default configuration will use the same blocks and construct the same encoders/decoders as the default nnUNetv2 ResEncUNet. As such, a significant portion of the code from the nnUNetv2 repository is duplicated here, with modifications to enable multi-task support with dynamic decoder branches, as well as other data formats. 

Additionally, the augmentations provided within this package are from another of [MIC-DKFZ's](https://github.com/MIC-DKFZ) fantastic array of machine learning libraries, in this case [batchgeneratorsv2](https://github.com/MIC-DKFZ/batchgeneratorsv2). 

Copying the modules directly into `vesuvius` was a choice of end-user friendliness, as we were using highly modified branches of both libraries, which created conflicts if an end user were to attempt to run any of our models.

_**Detailed documentation for training and inference are located in [the docs folder](docs/docs)**_
___


### Napari based training

1. Run `vesuvius.napari_trainer`
2. Add an image to the viewer
3. Add a label layer, with the suffix as the name of your target (ex : `32_ink`)
4. Optionally, add a label layer with the suffix `_mask` (ex : `32_mask`)
5. Set your training configuration and hit `run training`
6. Infer on the same layer, or another by importing an image, selecting it in the inference widget, and hitting `Run Inference`


![alt text](docs/docs/images/napari_trainer.png)

___

### Proofreading labels

1. Update the config in `/utils/vc_proofreader/config.py` with the proper paths
2. Run `vesuvius.proofreader`
3. Select your desired patch size and min labeled percentage 
4. Click `run` 
5. Approve patches with `a` or by checking the box
6. Skip patches or continue with `spacebar` or `next pair`
7. Patches are saved in the output dir, and their locations in the .json progress file

![alt text](docs/docs/images/proofreader.png)

### Training with `vesuvius.train`

Supported model types:
- Single task, single class semantic segmentation / regression
- Single task, multi-class semantic segmentation / regression
- Multi-task, single-class semantic segmentation / regression
- Multi-task, multi-class semantic segmentation / regression

By default, when provided with a single channel (binary) input label, the model will output 2 channels (fg/bg), but can adapt to any number of input channels.

**Place your data in the following format:** 
```
data/
  images/
    volume1.zarr (or .tif, .png, .jpg)
    volume2.zarr
  labels/
    volume1_ink.zarr (or tif, .png, .jpg) 
    volume1_hz_fiber.zarr ( if you want an additional task for the same volume )
    volume2_ink.zarr
   ```
**Begin training with:**
```bash
 vesuvius.train -i /path/to/data --batch-size 4 --patch-size 128,128,128 --model-name /path/to/save/checkpoints --config-path /vesuvius/models/configuration/multi-task_config.yaml
```
There are a number of other optional parameters, which you can find with `--help`

When this command is run: 
- a ConfigManager class will be instantiated, which will take the arguments given and the configuration file, and store these as properties.
- The trainer class will execute its class method `__build_model`, which will create an instance of `NetworkFromConfig`
- `NetworkFromConfig` will dynamically determine the number of pooling operations, stages, feature map sizes, operation dimensionality, and other specified parameters
- The trainer class will execute its `_configure_dataset` method, instantiating the `DatasetOrchestrator` with the adapter that matches `data_format`
- The trainer class will execute the rest of the setup required for training, through the following additional class methods:
  - `_build_loss`
  - `_get_optimizer`
  - `_get_scheduler`
  - `_get_scaler`
  - `_configure_dataloaders`

- The training loop will begin

Training will output the current losses in the `tqdm` progress bar, and will save a "debug" png or gif (depending on dimensionality) in the checkpoint directory. 

By default, the last 10 checkpoints are saved. This is not a smart way to do it, and will be changed to just store the last 3 + last 2 best validation. 

Training will run for 1,000 epochs by default, with 200 batches/epoch. This can be modified through the configuration file, which can be optionally provided to `vesuvius.train`, and some examples are provided in the [models folder](src/vesuvius/models/configuration/)

### Rendering and Flattening objs
Documentation is provided in [the rendering folder](src/vesuvius/rendering/README.md)
