Metadata-Version: 2.4
Name: geotessera
Version: 0.10.2
Summary: Python library interface to the Tessera geofoundation model embeddings
License: MIT License
        
        Copyright 2025-2026 Anil Madhavapeddy <anil@recoil.org>
        Copyright 2025-2026 Frank Feng
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/ucam-eo/geotessera
Project-URL: Documentation, https://geotessera.readthedocs.io
Project-URL: Repository, https://github.com/ucam-eo/geotessera
Project-URL: Issues, https://github.com/ucam-eo/geotessera/issues
Project-URL: Changelog, https://github.com/ucam-eo/geotessera/blob/main/CHANGES.md
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Environment :: Console
Classifier: Operating System :: OS Independent
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: numpy>=1.24.0
Requires-Dist: geopandas
Requires-Dist: pandas
Requires-Dist: matplotlib
Requires-Dist: rasterio
Requires-Dist: rich
Requires-Dist: typer
Requires-Dist: geodatasets>=2024.8.0
Requires-Dist: scikit-learn>=1.7.1
Requires-Dist: scikit-image>=0.25.2
Requires-Dist: pyarrow>=17.0.0
Requires-Dist: urllib3>=2
Requires-Dist: xarray
Requires-Dist: rioxarray
Requires-Dist: zarr>=3.3
Requires-Dist: obstore
Requires-Dist: dask
Requires-Dist: fsspec
Requires-Dist: aiohttp
Requires-Dist: zarr-cm>=0.4.1
Requires-Dist: contextily
Provides-Extra: docs
Requires-Dist: sphinx>=8.2.3; extra == "docs"
Provides-Extra: s3
Requires-Dist: s3fs; extra == "s3"
Dynamic: license-file

# GeoTessera

Python library for accessing and working with Tessera geospatial foundation model embeddings.

> ## 🚀 TESSERA v2 is here
>
> **The TESSERA v2 code is now live** — model weights and inference code are
> available in the [`ucam-eo/tessera`](https://github.com/ucam-eo/tessera)
> repository. v2 is our next-generation pixel-wise Earth foundation model; see
> the preprint, [*TESSERA v2: Scaling Pixel-wise Earth Foundation Models*](https://arxiv.org/abs/2607.03949).
>
> **Want to try v2 embeddings early?** You can **pre-request** v2 embeddings for
> your region and become an early tester:
>
> - 👉 **[Submit a v2 Embedding Pre-Request](../../issues/new?template=v2-embedding-prerequest.yml&labels=v2-embedding-prerequest)**
>
> ⚠️ **Heads-up:** we are still ramping up the compute, storage, and release
> infrastructure for v2, so v2 embeddings will be produced **slowly** at first
> and there is **no guaranteed turnaround time**. **If you need embeddings soon,
> request v1.1 instead** — it is fully available today. Scroll down to
> [Request missing embeddings](#request-missing-embeddings) to open a standard
> v1.1 request.

## Overview

GeoTessera provides access to geospatial embeddings from the [Tessera
foundation model](https://github.com/ucam-eo/tessera), which processes
Sentinel-1 and Sentinel-2 satellite imagery to generate 128-channel
representation maps at 10m resolution. These embeddings compress a full year of
temporal-spectral features into dense representations optimized for downstream
geospatial analysis tasks. Read more details about [the model](https://github.com/ucam-eo/tessera).

The library offers two access paths. The [zarr backend](#cloud-native-zarr-access)
streams embeddings directly from the public cloud store — no downloads, queries
routed to the correct UTM zone, values returned dequantised on their native 10m
grid — and is the recommended interface for analysis. The
[tile download interface](#python-api) fetches embeddings as NPY or GeoTIFF
files for offline work and GIS export.

![Coverage map](https://github.com/ucam-eo/tessera-coverage-map/blob/main/map.png)

### Request missing embeddings

This repo provides **precomputed embeddings** for multiple years and regions.
Embeddings are generated by **randomly sampling tiles** within each region to ensure broad spatial coverage.

If some **years (2017–2025) / areas** are still missing for your use case, please submit an **Embedding Request**:

- 👉 **[Open an Embedding Request](../../issues/new?template=embedding-request.yml&labels=embedding-request)**
- Please include: **your organization, intended use, ROI as a bounding box with four points (lon,lat, 4 decimals), and the year(s)**.

After you submit the request, we will **prioritize your ROI** and notify you via a comment in the issue once the embeddings are ready. 

### Important Notice ⚠️
On 20th August 2025, we updated the data processing pipeline of GeoTessera to resolve the issue of tiling artifacts, as shown below. We have retained the embeddings generated before August 20, as they remain effective for use in small-scale areas. After the 2024 embedding generation is completed, we will reprocess the tiles affected by tiling artifacts. If you observe such artifacts during use and they significantly impact performance, please raise the issue **[here](../../issues/new?template=embedding-request.yml&labels=embedding-request)**, and we will prioritize reprocessing your request.

![Pipeline Change](https://github.com/ucam-eo/geotessera/blob/main/pipeline_change.png)

Please note that if the artifacts you observe are slanted, this is not a bug in the pipeline but rather a result of the Sentinel-1/2 satellite trajectories. Currently, Tessera cannot completely eliminate such artifacts, as they reflect the inherent characteristics of the raw data. However, we have observed that they have minimal impact on downstream tasks.

## Table of Contents

- [Installation](#installation)
- [Cloud-Native Zarr Access](#cloud-native-zarr-access)
- [Architecture](#architecture)
- [Quick Start](#quick-start)
- [Python API](#python-api)
- [CLI Reference](#cli-reference)
- [Registry System](#registry-system)
- [Data Organization](#data-organization)
- [Contributing](#contributing)

## Installation

Requires Python 3.12 or later.

```bash
pip install geotessera
```

For development:
```bash
git clone https://github.com/ucam-eo/geotessera
cd geotessera
uv sync --all-extras --dev   # or: pip install -e ".[docs]"
```

Sphinx lives in the `docs` extra and the test harness in the `dev` group, so a
plain `pip install geotessera` pulls in neither.

## Cloud-Native Zarr Access

The zarr backend streams embeddings directly from the public store. Nothing is
downloaded up front: each query is routed to the UTM zone that holds it, and
the values return dequantised as float32 on their native 10m UTM grid, never
resampled. This is the recommended interface for analysis.

```python
from geotessera import GeoTesseraZarr

gt = GeoTesseraZarr()
print(gt.years)  # [2017, ..., 2025]

# One embedding, with a status explaining any missing value
vec, status = gt.probe(0.12, 52.20, year=2024)
print(status)  # 'valid', 'water', 'nodata', or 'outside'

# Many points, one bulk read per UTM zone
X = gt.sample_points([(0.12, 52.20), (-2.97, 53.44)], year=2024)  # (2, 128)

# A lon/lat bounding box as a mosaic on the native UTM grid
bbox = (0.05, 52.15, 0.20, 52.25)
mosaic, transform, crs = gt.read_region(bbox, year=2024)

# A fixed-size patch centred on a point, merged across UTM zones when needed
patch, transform, crs = gt.read_patch(0.12, 52.20, year=2024, size_px=256)

# Stream a large region in row strips rather than holding it in memory
for block, transform, crs in gt.iter_region(bbox, year=2024, strip_rows=512):
    predictions = model.predict(block.reshape(-1, 128))
```

Other dataset versions are selected by store URL. v2 stores also publish
matryoshka prefixes of each embedding, so `depth=16` reads the first 16
dimensions for an eighth of the bytes:

```python
from geotessera.registry import zarr_store_url

gt = GeoTesseraZarr(zarr_store_url("v2"))
X16 = gt.sample_points(coords, year=2024, depth=16)  # (N, 16)
```

HTTP reads retry with exponential backoff, so a transient server error costs
one chunk rather than the whole read. The constructor also accepts any
`zarr.abc.store.Store`, so the store can be wrapped — for example in zarr's
experimental `CacheStore`, which keeps fetched chunks for reuse across
queries:

```python
from zarr.experimental.cache_store import CacheStore
from zarr.storage import MemoryStore
from geotessera.store import DEFAULT_STORE, zarr_store

store = CacheStore(zarr_store(DEFAULT_STORE), cache_store=MemoryStore(),
                   max_size=2 * 1024**3)
gt = GeoTesseraZarr(store)
```

For direct access to one UTM zone, `gt.open_zone(lon=0.15)` returns an xarray
dataset with a `.tessera` accessor that works in that zone's own eastings and
northings. The store layout follows the `geoemb:` convention for geospatial
embedding data; the
[zarr quickstart](https://geotessera.readthedocs.io/en/latest/zarr_quickstart.html)
and the [examples repository](https://github.com/ucam-eo/geotessera-examples)
walk through complete workflows.

### Agent Skill

The repository ships an [Agent Skill](https://agentskills.io) that teaches
coding agents the zarr interface. Claude Code users can install it as a
plugin:

```
/plugin marketplace add ucam-eo/geotessera
/plugin install geotessera@ucam-eo
```

The skill itself is [`skills/geotessera/SKILL.md`](skills/geotessera/SKILL.md),
in the open Agent Skills format, so it also works with other
SKILL.md-compatible agents when copied into their skills directory.

## Architecture

### Core Concepts

This section describes the tile download interface; the
[zarr backend](#cloud-native-zarr-access) above streams the same embeddings
without downloading files. The tile workflow has two steps:

1. **Retrieve embeddings**: Fetch raw numpy arrays for a geographic bounding box
2. **Export to desired format**: Save as raw numpy arrays or convert to georeferenced GeoTIFF files

### Coordinate System and Tile Grid

The Tessera embeddings use a **0.1-degree grid system**:

- **Tile size**: Each tile covers 0.1° × 0.1° (approximately 11km × 11km at the equator)
- **Tile naming**: Tiles are named by their **center coordinates** (e.g., `grid_0.15_52.05`)
- **Tile bounds**: A tile at center (lon, lat) covers:
  - Longitude: [lon - 0.05°, lon + 0.05°]
  - Latitude: [lat - 0.05°, lat + 0.05°]
- **Resolution**: 10m per pixel (variable number of pixels per tile depending on latitude)

### File Structure and Downloads

When you request embeddings, GeoTessera downloads files over HTTPS from the
public Source Cooperative repository into the output directory you specify,
where they persist for re-use:

#### Embedding Files (via `fetch_embedding`)
1. **Quantized embeddings** (`grid_X.XX_Y.YY.npy`):
   - Shape: `(height, width, 128)`
   - Data type: int8 (quantized for storage efficiency)
   - Contains the compressed embedding values

2. **Scale files** (`grid_X.XX_Y.YY_scales.npy`):
   - Shape: `(height, width)` or `(height, width, 128)`
   - Data type: float32
   - Contains scale factors for dequantization

3. **Dequantization**: `final_embedding = quantized_embedding * scales`

4. **Persistent Storage**: Files are downloaded into your chosen output
   directory and skipped on rerun, so interrupted downloads resume cleanly

#### Landmask Files (for GeoTIFF export)
When exporting to GeoTIFF, additional landmask files are fetched:
- **Landmask tiles** (`grid_X.XX_Y.YY.tiff`):
  - Provide UTM projection information
  - Define precise geospatial transforms
  - Contain land/water masks
  - Cached alongside the embedding tiles for re-use

### Data Flow

```
User Request (lat/lon bbox)
    ↓
Parquet Registry Lookup (find available tiles from manifest.parquet)
    ↓
HTTPS Downloads from Source Cooperative to Output Directory (integrity verified)
    ├── embedding.npy (quantized) → output dir
    └── embedding_scales.npy → output dir
    ↓
Dequantization (multiply arrays)
    ↓
Output Format
    ├── NumPy arrays → Direct analysis
    └── GeoTIFF → GIS integration
```

**Storage Note**: Only the Parquet manifests (tens to a couple of hundred
MB per dataset) are cached under `~/.cache/geotessera`. Embedding tiles are
downloaded on demand into the output directory you specify and persist there
for re-use across runs.

## Quick Start

### Check Available Data

Before downloading, check what data is available:

```bash
# Generate a coverage map showing all available tiles
geotessera coverage --output coverage_map.png

# Generate a coverage map for the UK
geotessera coverage --country uk

# View coverage for a specific year
geotessera coverage --year 2024 --output coverage_2024.png

# Customize the visualization
geotessera coverage --year 2024 --tile-color blue --tile-alpha 0.3
```

### Download Embeddings

Download embeddings as either numpy arrays or GeoTIFF files:

```bash
# Download as GeoTIFF (default, with georeferencing)
geotessera download \
  --bbox "-0.2,51.4,0.1,51.6" \
  --year 2024 \
  --output ./london_tiffs

# Download as raw numpy arrays (with metadata JSON)
geotessera download \
  --bbox "-0.2,51.4,0.1,51.6" \
  --format npy \
  --year 2024 \
  --output ./london_arrays

# Download using a GeoJSON/Shapefile region
geotessera download \
  --region-file cambridge.geojson \
  --format tiff \
  --year 2024 \
  --output ./cambridge_tiles

# Download specific bands only
geotessera download \
  --bbox "-0.2,51.4,0.1,51.6" \
  --bands "0,1,2" \
  --year 2024 \
  --output ./london_rgb
```

### Create Visualizations

Generate PCA visualizations and web maps from downloaded GeoTIFFs:

```bash
# Create a PCA mosaic from downloaded tiles
geotessera visualize ./london_tiffs pca_mosaic.tif

# Use histogram equalization for maximum contrast
geotessera visualize ./london_tiffs pca_balanced.tif --balance histogram

# Create web tiles and serve interactively
geotessera webmap pca_mosaic.tif --serve

# Serve existing web visualizations locally
geotessera serve ./london_web --open
```

## Python API

### Core Methods

The `GeoTessera` class downloads embedding tiles as files; for streaming
access use the [zarr backend](#cloud-native-zarr-access). The tile interface
provides two main methods for retrieving embeddings:

```python
from geotessera import GeoTessera

# Initialize the client
gt = GeoTessera()

# Method 1: Fetch a single tile
embedding, crs, transform = gt.fetch_embedding(lon=0.15, lat=52.05, year=2024)
print(f"Shape: {embedding.shape}")  # e.g., (1200, 1200, 128)
print(f"CRS: {crs}")  # Coordinate reference system from landmask

# Method 2: Fetch all tiles in a bounding box
bbox = (-0.2, 51.4, 0.1, 51.6)  # (min_lon, min_lat, max_lon, max_lat)
tiles_to_fetch = gt.registry.load_blocks_for_region(bounds=bbox, year=2024)
embeddings = gt.fetch_embeddings(tiles_to_fetch)

for year, tile_lon, tile_lat, embedding_array, crs, transform in embeddings:
    print(f"Tile ({tile_lat}, {tile_lon}): {embedding_array.shape}")
```

### Export Formats

#### Export as GeoTIFF

```python
# Export embeddings for a region as individual GeoTIFF files
# Step 1: Get the tiles for the region
bbox = (-0.2, 51.4, 0.1, 51.6)
tiles_to_fetch = gt.registry.load_blocks_for_region(bounds=bbox, year=2024)

# Step 2: Export those tiles as GeoTIFFs
files = gt.export_embedding_geotiffs(
    tiles_to_fetch=tiles_to_fetch,
    output_dir="./output",
    bands=None,  # Export all 128 bands (default)
    compress="lzw"  # Compression method
)

print(f"Created {len(files)} GeoTIFF files")

# Export specific bands only (e.g., first 3 for RGB visualization)
files = gt.export_embedding_geotiffs(
    tiles_to_fetch=tiles_to_fetch,
    output_dir="./rgb_output",
    bands=[0, 1, 2]  # Only export first 3 bands
)
```

#### Work with NumPy Arrays

```python
# Fetch and process embeddings directly
tiles_to_fetch = gt.registry.load_blocks_for_region(bounds=bbox, year=2024)
embeddings = gt.fetch_embeddings(tiles_to_fetch)

for year, tile_lon, tile_lat, embedding, crs, transform in embeddings:
    # Compute statistics
    mean_values = np.mean(embedding, axis=(0, 1))  # Mean per channel
    std_values = np.std(embedding, axis=(0, 1))    # Std per channel

    # Extract specific pixels
    center_pixel = embedding[embedding.shape[0]//2, embedding.shape[1]//2, :]

    # Apply custom processing
    processed = your_analysis_function(embedding)
```

### Visualization Functions

```python
from geotessera.visualization import (
    create_rgb_mosaic,
    visualize_global_coverage
)
from geotessera.web import (
    create_coverage_summary_map,
    geotiff_to_web_tiles
)

# Create an RGB mosaic from multiple GeoTIFF files
create_rgb_mosaic(
    geotiff_paths=["tile1.tif", "tile2.tif"],
    output_path="mosaic.tif",
    bands=(0, 1, 2)  # RGB bands
)

# Generate web tiles for interactive maps
geotiff_to_web_tiles(
    geotiff_path="mosaic.tif",
    output_dir="./web_tiles",
    zoom_levels=(8, 15)
)

# Create a global coverage visualization
visualize_global_coverage(
    tessera_client=gt,
    output_path="global_coverage.png",
    year=2024,  # Or None for all years
    width_pixels=2000,
    tile_color="red",
    tile_alpha=0.6
)
```

## CLI Reference

### download

Download embeddings for a region in your preferred format:

```bash
geotessera download [OPTIONS]

Options:
  -o, --output PATH         Output directory [required]
  --bbox TEXT              Bounding box: 'lon,lat' (single tile) or 'min_lon,min_lat,max_lon,max_lat'
  --tile TEXT              Single tile by any point within it: 'lon,lat'
  --region-file PATH       GeoJSON/Shapefile to define region
  --country TEXT           Country name (e.g., 'United Kingdom', 'UK', 'GB')
  -f, --format TEXT        Output format: 'tiff' or 'npy' (default: tiff)
  --year INT               Year of embeddings (default: 2024)
  --dataset-version TEXT   Tessera dataset version (e.g. v1, v1.1)
  --dataset-variant TEXT   Tessera dataset variant (default: the version's default variant; list with `geotessera info`)
  --bands TEXT             Comma-separated band indices (default: all 128)
  --compress TEXT          Compression for TIFF format (default: lzw)
  --dry-run                Calculate total download size without downloading
  --list-files             List all created files with details
  -v, --verbose            Verbose output
```

**Resume behaviour**: Both TIFF and NPY downloads automatically skip files that already exist on disk, so interrupted downloads can be resumed by re-running the same command.

Single tile examples:
```bash
# Download a single tile containing a specific point
geotessera download --tile "0.17,52.23" --year 2024 -o ./single_tile

# Same result using --bbox with 2 coordinates
geotessera download --bbox "0.17,52.23" --year 2024 -o ./single_tile
```

Output formats:
- **tiff**: Georeferenced GeoTIFF files with UTM projection
- **npy**: Raw numpy arrays with metadata.json file

### visualize

Create PCA visualization from multiband GeoTIFF or NPY format embeddings:

```bash
geotessera visualize INPUT_PATH OUTPUT_FILE [OPTIONS]

Options:
  --n-components INT       Number of PCA components (default: 3)
  --crs TEXT               Target CRS for reprojection (default: EPSG:3857)
  --balance TEXT            RGB balance method: histogram, percentile, or adaptive
  --percentile-low FLOAT   Lower percentile for percentile balance (default: 2.0)
  --percentile-high FLOAT  Upper percentile for percentile balance (default: 98.0)
```

### webmap

Create web tiles and interactive viewer from a PCA mosaic:

```bash
geotessera webmap RGB_MOSAIC [OPTIONS]

Options:
  -o, --output PATH        Output directory
  --min-zoom INT           Min zoom for web tiles (default: 8)
  --max-zoom INT           Max zoom for web tiles (default: 15)
  --serve/--no-serve       Start web server immediately
  -p, --port INT           Port for web server (default: 8000)
  --region-file PATH       GeoJSON/Shapefile boundary to overlay
  --force/--no-force       Force regeneration of tiles
```

### coverage

Generate a world map showing data availability:

```bash
geotessera coverage [OPTIONS]

Options:
  -o, --output PATH        Output PNG file, or a directory to also receive the
                           coverage.json/globe.html (default: tessera_coverage.png)
  --year INT               Specific year to visualize
  --bbox TEXT              Bounding box: 'lon,lat' (single tile) or 'min_lon,min_lat,max_lon,max_lat'
  --tile TEXT              Single tile by any point within it: 'lon,lat'
  --by-source              Render each (version, variant) source in a distinct colour
  --dataset-version TEXT   Tessera dataset version (e.g. v1, v1.1; or 'all' with --by-source)
  --dataset-variant TEXT   Tessera dataset variant (default: the version's default variant; or 'all' with --by-source)
  --region-file PATH       GeoJSON/Shapefile to focus on specific region
  --country TEXT           Country name to focus on (e.g., 'United Kingdom')
  --tile-color TEXT        Color for tiles (default: red)
  --tile-alpha FLOAT       Transparency 0-1 (default: 0.6)
  --tile-size FLOAT        Size multiplier (default: 1.0)
  --width INT              Output image width in pixels (default: 2000)
  --no-countries           Don't show country boundaries
  --no-multi-year-colors   Disable multi-year color coding
```

### serve

Serve web visualizations locally:

```bash
geotessera serve DIRECTORY [OPTIONS]

Options:
  -p, --port INT           Port number (default: 8000)
  --open/--no-open         Auto-open browser (default: open)
  --html TEXT              Specific HTML file to serve
```

### info

Display information about GeoTIFF files or the library:

```bash
geotessera info [OPTIONS]

Options:
  --tiles PATH             Analyze tile files/directory (GeoTIFF or NPY format)
  --dataset-version TEXT   Tessera dataset version (e.g. v1, v1.1)
  --dataset-variant TEXT   Tessera dataset variant (default: the version's default variant; list with `geotessera info`)
  -v, --verbose            Verbose output
```

## Registry System

### Overview

GeoTessera uses a Parquet-based registry system to efficiently manage and access the large Tessera dataset:

- **Per-version manifests**: Each dataset version has its own `manifest.parquet`
  listing every `(year, lon, lat)` tile available for that version's variants
- **Fast queries**: Uses pandas DataFrames for efficient spatial and temporal filtering
- **Block-based organization**: Internal 5×5 degree geographic blocks for efficient queries
- **Minimal storage**: Only manifest files (tens to a couple of hundred MB per dataset) are cached locally
- **Integrity checking**: Every download is verified against the response
  `Content-Length`, and against an MD5 computed over the streamed body whenever
  the server's `ETag` is a content MD5 (single-part uploads)
  - A mismatch rejects the download and triggers a retry, so corrupt or
    truncated files never reach the cache

### Dataset Versions and Variants

Tessera embeddings are published as dataset *versions* (e.g. `v1`, `v1.1`,
`v2`) and, within a version, as *variants* produced by different model runs.
Each `(version, variant)` pair — a *dataset* — has its own directory in the
repository's `npy/` tree:

| Version | Variant                | `npy/` directory | Status      |
|---------|------------------------|------------------|-------------|
| `1.0`   | `vultr` (default)      | `v1/`            | available   |
| `1.1`   | `cambridge` (default)  | `v1.1-cam/`      | available   |
| `1.1`   | `dclimate`             | —                | coming soon |
| `2.0`   | `2B-L~beta1` (default) | `v2-2B-L~beta1/` | available   |
| `2.0`   | `2B-L~beta2`           | `v2-2B-L~beta2/` | available   |

The v1 series predates the variant-suffix scheme, so all its variants share
the bare `v1/` directory. The library defaults remain `dataset_version="v1"`
and `year=2024` — the only combination with full global coverage today.
List the datasets at any time with `geotessera info`, and select them on the
CLI with `--dataset-version` and `--dataset-variant`, or in Python:

```python
gt = GeoTessera(dataset_version="v1.1", dataset_variant="cambridge")
```

Use `geotessera coverage --by-source` to render each `(version, variant)` source
in a distinct colour on the coverage map and globe viewer.

### Registry Sources

The registry can be loaded from multiple sources (in priority order):

1. **Local file** (via `registry_path` parameter)
2. **Local directory** (via `--registry-dir` or `registry_dir` parameter, looks for `manifest.parquet`, falling back to the legacy `registry.parquet`)
3. **Remote URL** (via `registry_url` parameter)
4. **Default remote** (from `https://data.source.coop/tessera/tessera/npy/{dataset}/manifest.parquet`, where `{dataset}` encodes the (version, variant) pair: `v1`, `v1.1-cam`, `v2-2B-L~beta1`)

```python
# Use local manifest file
gt = GeoTessera(registry_path="/path/to/manifest.parquet")

# Use local registry directory
gt = GeoTessera(registry_dir="/path/to/registry-dir")

# Use default remote manifest (downloads and caches automatically)
gt = GeoTessera()  # Default behavior
```

### Registry Structure

The Parquet manifest contains columns for:
- **Coordinates**: `lon`, `lat` (tile center coordinates)
- **Year**: `year` (data year, 2017-2025)
- **Size**: `file_size` (file size in bytes for download planning)

```python
# Example manifest query
import pandas as pd
manifest = pd.read_parquet("manifest.parquet")
print(manifest.head())
```

### Regenerating Manifests (Maintainers)

The per-version manifests can be rebuilt at any time by scanning the public
Source Cooperative repository itself — no local copy of the data is needed:

```bash
# Rescan every dataset and write one manifest per npy/ directory
# (./manifests/{v1,v1.1-cam,v2-2B-L~beta1}/manifest.parquet) plus a
# per-version landmasks.parquet (./manifests/{v1,v1.1,v2}/landmasks.parquet)
geotessera-registry s3scan s3://tessera/tessera/npy/ \
    --landmasks-uri s3://tessera/tessera/landmasks/ \
    --output ./manifests

# Or scope to a single dataset — the directory name encodes the
# (version, variant) pair, so no --variant flag is needed
geotessera-registry s3scan "s3://tessera/tessera/npy/v1.1-cam/" \
    --landmasks-uri s3://tessera/tessera/landmasks/v1.1/ \
    --output ./manifests
```

The scan uses anonymous S3 `ListObjectsV2` calls against
`https://data.source.coop`, so no credentials are required to regenerate.
Uploading the results does require source.coop write access, and the two
parquet files go to *different* trees (matching where clients fetch them —
note the npy/ tree is keyed by dataset directory, the landmasks/ tree by
plain version):

```bash
aws s3 cp manifests/v1.1-cam/manifest.parquet \
    s3://tessera/tessera/npy/v1.1-cam/manifest.parquet \
    --endpoint-url https://data.source.coop
aws s3 cp manifests/v1.1/landmasks.parquet \
    s3://tessera/tessera/landmasks/v1.1/landmasks.parquet \
    --endpoint-url https://data.source.coop
```

The `s3scan` summary panel prints these per-file upload commands for you.
Transient listing failures (Cloudflare 503s, timeouts) are retried with
exponential backoff; if a shard still fails after retries, the affected
manifest is not written and the command exits non-zero so an incomplete
manifest can never be uploaded. See the
[maintenance guide](https://geotessera.readthedocs.io/en/latest/maintenance.html)
for the full workflow, including caching caveats.

### How Registry Loading Works

1. **Load Parquet manifest** → Download and cache the dataset's manifest (if not local)
2. **Request tiles for bbox** → Query DataFrame for tiles in region
3. **Filter by year and variant** → Select tiles matching the requested year/variant
4. **Find available tiles** → Return list of matching tiles
5. **HTTPS download** → Fetch tiles on demand from the Source Cooperative mirror into the output directory, with integrity checks
6. **Persist** → Downloaded tiles stay in the output directory and are skipped on rerun

## Data Organization

### Tessera Data Structure

```
Remote Server (https://data.source.coop/tessera/tessera)
├── npy/                                       # NPY embeddings + scales
│   │                                          # (one dir per (version, variant) dataset)
│   ├── v1/                                    # 1.0 — all variants share this dir
│   │   ├── manifest.parquet                   # Per-dataset tile manifest
│   │   └── 2024/grid_0.15_52.05/grid_0.15_52.05{,_scales}.npy
│   ├── v1.1-cam/                              # 1.1 / cambridge
│   │   ├── manifest.parquet
│   │   └── 2024/grid_0.15_52.05/grid_0.15_52.05{,_scales}.npy
│   └── v2-2B-L~beta1/                         # 2.0 / 2B-L~beta1 (beta)
│       └── 2024/grid_0.15_52.05/grid_0.15_52.05{,_scales}.npy
├── landmasks/                                 # Landmask TIFFs (per version)
│   ├── v1/
│   │   ├── landmasks.parquet                  # Landmask manifest
│   │   └── grid_0.15_52.05.tiff               # Landmask with projection info
│   ├── v1.1/
│   │   ├── landmasks.parquet
│   │   └── grid_0.15_52.05.tiff
│   └── v2/
│       ├── landmasks.parquet
│       └── grid_0.15_52.05.tiff
└── zarr/                                      # Cloud-native zarr stores
    ├── v1/                                    # 60 UTM zone groups + RGB pyramid
    ├── v2-2B-L~beta1/                         # v2 stores add matryoshka
    └── v2-2B-L~beta2/                         # prefix arrays (d4, d16)
```

### Local Cache Structure

```
~/.cache/geotessera/                 # Default cache location (manifests only)
├── v1/                              # 1.0 dataset dir + v1 landmask registry
│   ├── manifest.parquet
│   └── landmasks.parquet
├── v1.1-cam/                        # 1.1/cambridge dataset dir
│   └── manifest.parquet
├── v1.1/                            # v1.1 landmask registry
│   └── landmasks.parquet
├── v2-2B-L~beta1/                   # 2.0 beta dataset dir
│   └── manifest.parquet
└── v2/                              # v2 landmask registry
    └── landmasks.parquet

# Note: Embedding and landmask tiles are NOT stored here. They are downloaded
# into the output directory you specify and persist there for re-use.
```

### Coordinate Reference Systems

- **Embeddings**: Stored in simple arrays, referenced by center coordinates
- **GeoTIFF exports**: Use UTM projection from corresponding landmask tiles
- **Web visualizations**: Reprojected to Web Mercator (EPSG:3857)

## Cache Configuration

GeoTessera caches only the per-dataset Parquet manifests (tens to a couple of hundred MB each). Embedding and landmask tiles are downloaded into the output directory you specify and persist there for re-use across runs.

### Python API

```python
from geotessera import GeoTessera

# Use custom cache directory for registry
gt = GeoTessera(cache_dir="/path/to/cache")

# Use default cache location (recommended)
gt = GeoTessera()
```

### CLI

```bash
# Specify custom cache directory
geotessera download --cache-dir /path/to/cache ...

# Use default cache location
geotessera download ...
```

### Default Cache Locations

When `cache_dir` is not specified, the registry is cached in platform-appropriate locations:
- **Linux/macOS**: `$XDG_CACHE_HOME/geotessera` or `~/.cache/geotessera`
- **Windows**: `%LOCALAPPDATA%/geotessera`

## Hash Verification

GeoTessera verifies every downloaded file (embeddings, scales, and landmasks) against the response `Content-Length`, and additionally against an MD5 computed over the streamed body whenever the server's `ETag` is a content MD5 (a single-part upload; this covers landmask TIFFs and scales files). Large multipart-uploaded embedding tiles carry a composite ETag that is not a content hash, so they are length-checked only. A mismatch rejects the download and triggers a retry with backoff, so corrupt or truncated files never reach the cache.

## Contributing

Contributions are welcome! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
This project is licensed under the MIT License - see the [LICENSE](LICENSE.md) file for details.

## Citation

If you use Tessera in your research, please cite the [arXiv paper](https://arxiv.org/abs/2506.20380):

```bibtex
@misc{feng2025tesseratemporalembeddingssurface,
      title={TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis}, 
      author={Zhengpeng Feng and Clement Atzberger and Sadiq Jaffer and Jovana Knezevic and Silja Sormunen and Robin Young and Madeline C Lisaius and Markus Immitzer and David A. Coomes and Anil Madhavapeddy and Andrew Blake and Srinivasan Keshav},
      year={2025},
      eprint={2506.20380},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2506.20380}, 
}
```

## Links

- [Tessera Foundation Model](https://github.com/ucam-eo/tessera)
- [Tessera Interactive Notebook](https://github.com/ucam-eo/tessera-interactive-map)
- [Tessera Examples](https://github.com/ucam-eo/geotessera-examples)
- [Documentation](https://geotessera.readthedocs.io/)
- [PyPI Package](https://pypi.org/project/geotessera/)
- [Issue Tracker](https://github.com/ucam-eo/geotessera/issues)


## Star History

[![Star History Chart](https://star-history.dera.page/svg?repos=ucam-eo/geotessera&type=Date)](https://star-history.dera.page/#ucam-eo/geotessera&Date)
