Metadata-Version: 2.4
Name: distmetrics
Version: 1.0.8
Summary: Measure Disturbances in OPERA RTC-S1 Stacks
Author-email: "Charlie Marshak, Talib Oliver-Cabrera, Richard West, Jungkyo Jung and OPERA Project Science Team" <opera-pst-dev@jpl.nasa.gov>
License: Apache-2.0
Project-URL: Homepage, https://github.com/opera-adt/distmetrics
Project-URL: Bug Tracker, https://github.com/opera-adt/distmetrics/issues
Classifier: Intended Audience :: Science/Research
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE.txt
Requires-Dist: affine
Requires-Dist: asf_search
Requires-Dist: astropy
Requires-Dist: dem_stitcher>=3.2
Requires-Dist: einops
Requires-Dist: geopandas
Requires-Dist: matplotlib
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: pydantic
Requires-Dist: torch
Requires-Dist: rasterio
Requires-Dist: scikit-image
Requires-Dist: scipy
Requires-Dist: shapely
Requires-Dist: tenacity
Requires-Dist: tqdm
Provides-Extra: develop
Requires-Dist: pytest; extra == "develop"
Requires-Dist: pytest-cov; extra == "develop"
Requires-Dist: ruff; extra == "develop"
Requires-Dist: jupyterlab; extra == "develop"
Dynamic: license-file

# distmetrics 

[![PyPI license](https://img.shields.io/pypi/l/distmetrics.svg)](https://pypi.python.org/pypi/distmetrics/)
[![PyPI pyversions](https://img.shields.io/pypi/pyversions/distmetrics.svg)](https://pypi.python.org/pypi/distmetrics/)
[![PyPI version](https://img.shields.io/pypi/v/distmetrics.svg)](https://pypi.python.org/pypi/distmetrics/)
[![Conda version](https://img.shields.io/conda/vn/conda-forge/distmetrics)](https://anaconda.org/conda-forge/distmetrics)
[![Conda platforms](https://img.shields.io/conda/pn/conda-forge/distmetrics)](https://anaconda.org/conda-forge/distmetrics)

The `distmetrics` (or informally `dist-lib`) provides a set of python tools and metrics to identify generic disturbances within OPERA RTC-S1 time-series including a transformer-based metric proposed in Hardiman-Mostow et al., 2024 [[1]](#1).
The transformer metric and its application occupies most of this library in order to effectively, efficiently apply this deep-learning based model (using a visual transformer architecture).
Also, there are:
- tools for despeckling
- GIS tools to merge burst-wise products after metrics have been computed
- downloading burst time series
"Generic land disturbances" refer to any land disturbances observable with OPERA RTC-S1 including land-use changes, natural disasters, deforestation, etc.
A disturbance metric is a per-pixel function that quantifies land disturbances between a set of baseline images (pre-images) and a new acquisition (post-image).
This library is specific to the dual-polarization VV $+$ VH OPERA RTC-S1 data.

# Usage

See the [`notebooks/`](notebooks/). 
These notebooks show how to:

1. download the necessary, publicly available time series of OPERA RTC-S1 data (see setup below)
2. despeckle the time-series and 
3. calculate the disturbance metrics for delineating areas of disturbances

## Setup

Please make sure to ensure you have an ASF account (https://search.asf.alaska.edu/#/) and set up a `~/.netrc` with:

```
machine urs.earthdata.nasa.gov
    login <username>
    password <password>
```


## Provenance of Transformer Models

Currently there are 4 models in this library (see `from distmetrics.model_load import ALLOWED_MODELS`), though you can load your own weights and config assuming the same architecture.
The models in this library are:
   - `transformer_original`
   - `transformer_optimized`
   - `transformer_optimized_fine`
   - `transformer_anniversary_trained`
   - `transformer_anniversary_trained_10`
   - `transformer_anniversary_trained_optimized`
   - `transformer_anniversary_trained_optimized_fine`
   - `transformer_v0_32`
   - `transformer_v1_32`

Please see the [dist-s1-model](https://github.com/opera-adt/dist-s1-model) [A] for training this transformer model and [dist-s1-training-data](https://github.com/opera-adt/dist-s1-training-data) [B] for curating a dataset from OPERA RTC-S1. There is a link in [A] to the existing dataset that was curated based on time-series with dense preimages and despeckled and masked water. In [B], you can understand how the data is curated. For the models above, here is some additional discussion:
- `transformer_original` - the model trained by Harris Hardiman-Mostow on the training data linked to in [A].
- `transformer_optimized` and `transformer_optimized_fine` are models with the same architecture as `transformer_original` and using the same dataset in [A] by [dmartinez05](https://github.com/dmartinez05) with reduced size. The `fine` refers to the fact that within the input size of the model (`16 x 16`) there are patches used within the input size that are `4 x 4` (as opposed to `8 x 8`).
- `transformer_anniversary_trained` and `transformer_anniversary_trained_10` were trained by [Jungkyo Jung](https://github.com/oberonia78) using a dataset similar to one found in [B] using despeckling and some landcover masking. The context length of the the former is 20 and the latter 10.
- `transformer_anniversary_trained_optimized` and `transformer_anniversary_trained_optimized_fine` were trained by [dmartinez05](https://github.com/dmartinez05) using the same dataset used for `transformer_anniversary_trained` and are optimized in that they use less parameters and the `fine` suffix refers to the internal model patches are `4 x 4`.
- `transformer_v0_32` and `transformer_v1_32` are trained on [A] and [B] respectively using a 32 x 32 input window with 8 x 8 patches by [dmartinez05](https://github.com/dmartinez05).

## Background on Metrics

This is a python implementation of disturbance metrics for OPERA RTC-S1 data. The intention is to use this library to quantify disturbance in the RTC imagery. Specifically, our "metrics" define distances between a set of dual polarizations "pre-images" and a single dual polarization "post-image". Some of the metrics only work on single polarization imagery.

The following metrics have been implemented in this library:

1. Transformer metric - mean and std estimated from a Vision Transformer [[1]](#1) inspired by [[2]](#2).
2. Mahalanobis 1d and 2d  - based on mean and std from sample statistics in patches around each pixel [[3]](#3), [[4]](#4).
3. Log-ratio - this is not a non-negative function just a difference of pre and post images in dB [[2]](#1). Only works on single polarization images.
4. CuSum metric - both absolute residuals and normalized residuals are computed in a per-pixel fashion. See [[5]](#5) and [[6]](#6).

It is worth noting that other metrics can be generated from the above using `+`, `max`, `min` or linear combinations (with positive scalars). As such, when the distmetric has some auxiliary meaning (e.g. as a probability), such combinations are easier as they are more meaningfully comparable.

## Installation

We recommend using the `conda/mamba` package manager to install this library.

```
mamba install -c conda-forge distmetrics
```

You can also use `pip`, although this doesn't ensure proper dependencies are installed.

### GPU support

To get the best performance of pytorch, you need to ensure pytorch recognizes the GPU.
Using `conda-forge` distributions, you may require you to ensure that `cudatoolkit` is installed (this is the additional library in `environment_gpu.yml`).
For our servers, we needed to install `cudatoolkit>=11.8` to get pytorch to recognize the GPU.
There are certain libraries that may downgrade `pytorch` to use CPU only (you can check this by looking at the distribution of pytorch before installing the library).
There may be different distributions of pytorch and cuda drivers that are compatible, but providing detailed instructions is beyond the scope of these instructions.


### For development

We use [pixi](https://pixi.sh/latest/#installation) to manage the development environment; all of its
configuration lives in `pyproject.toml` under `[tool.pixi.*]` and the resolved environment is committed
in `pixi.lock`.

```bash
git clone https://github.com/opera-adt/distmetrics.git
cd distmetrics
pixi install
pixi run python -c "import distmetrics"
```

`pixi install` installs `distmetrics` itself as an editable dependency, so no separate `pip install -e .` is
needed. Common tasks:

```bash
pixi run test          # pytest tests
pixi run lint          # ruff check
pixi run format        # ruff format
pixi run fix           # ruff check --fix
pixi run format-check  # ruff format --diff
```

For JupyterLab (with `jupyter-collaboration` for real-time collaborative editing):

```bash
pixi run jupyter lab
```

Python 3.12, 3.13, and 3.14 are supported. The default environment is the newest supported version (3.14) and
is the only one materialized locally; the `py312`/`py313`/`py314` environments back the CI matrix. To
reproduce a matrix failure locally, run e.g. `pixi run -e py312 pytest tests`, then reclaim the disk with
`pixi clean -e py312`.

The `lint` environment is python-free (~165 MB instead of the full ~1.3 GB stack) and is what CI's ruff job
installs; the same `lint` feature is folded into `default`, so the tasks above run the exact ruff the lock
file pins.

When using the transformer model, if you have `gpu` available, it adviseable to check that the output from the below snippet is indeed `cuda`:

```
from distmetrics import get_device

get_device() # should be `cuda` if GPU is available or `mps` if using mac M chips
```

# References

<a id=1>[1]</a> H. Hardiman Mostow et al., "Deep Self-Supervised Disturbance Mapping with Sentinel-1 OPERA RTC Synthetic Aperture Radar", [https://arxiv.org/abs/2501.09129](https://arxiv.org/abs/2501.09129).

<a id=2>[2]</a> O. L. Stephenson et al., "Deep Learning-Based Damage Mapping With InSAR Coherence Time Series," in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-17, 2022, Art no. 5207917, doi: 10.1109/TGRS.2021.3084209. https://arxiv.org/abs/2105.11544 

<a id="3">[3]</a> E. J. M. Rignot and J. J. van Zyl, "Change detection techniques for ERS-1 SAR data," in IEEE Transactions on Geoscience and Remote Sensing, vol. 31, no. 4, pp. 896-906, July 1993, doi: 10.1109/36.239913. https://ieeexplore.ieee.org/document/239913 

<a id=4>[4]</a> Deledalle, CA., Denis, L. & Tupin, F. How to Compare Noisy Patches? Patch Similarity Beyond Gaussian Noise. Int J Comput Vis 99, 86–102 (2012). https://doi.org/10.1007/s11263-012-0519-6. https://inria.hal.science/hal-00672357/

<a id=5>[5]</a> Sarem Seitz, "Probabalistic Cusum for Change Point Detection", https://web.archive.org/web/20240817203837/https://sarem-seitz.com/posts/probabilistic-cusum-for-change-point-detection/, Accessed September 2024.

<a id=6>[6]</a> Tartakovsky, Alexander, Igor Nikiforov, and Michele Basseville. Sequential analysis: Hypothesis testing and changepoint detection. CRC press, 2014.
