Metadata-Version: 2.4
Name: ps_eor
Version: 1.0
Summary: Power Spectra generation for 21-cm experiments and Foreground modeling/removal
Project-URL: Homepage, https://gitlab.com/flomertens/ps_eor
Project-URL: Documentation, https://ps-eor.readthedocs.io
Project-URL: Repository, https://gitlab.com/flomertens/ps_eor
Project-URL: Issues, https://gitlab.com/flomertens/ps_eor/-/issues
Author-email: Florent Mertens <florent.mertens@gmail.com>, Satyapan Munshi <satyapan.iiserm@gmail.com>
License-Expression: GPL-3.0-or-later
License-File: LICENSE
Requires-Python: <4.0,>=3.10
Requires-Dist: astropy>=6.0
Requires-Dist: click>=8.0
Requires-Dist: configparser>=6.0
Requires-Dist: fast-histogram>=0.11
Requires-Dist: h5py>=3.10
Requires-Dist: healpy>=1.16
Requires-Dist: joblib>=1.3
Requires-Dist: matplotlib>=3.8
Requires-Dist: numpy>=1.20
Requires-Dist: pyfftw>=0.13
Requires-Dist: reproject>=0.14
Requires-Dist: scikit-learn>=1.3
Requires-Dist: scipy>=1.10
Requires-Dist: tables>=3.9
Provides-Extra: dev
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff==0.16.1; extra == 'dev'
Provides-Extra: ml-gpr
Requires-Dist: corner>=2.2; extra == 'ml-gpr'
Requires-Dist: dynesty>=2.0; extra == 'ml-gpr'
Requires-Dist: emcee>=3.1; extra == 'ml-gpr'
Requires-Dist: gpytorch>=1.11; extra == 'ml-gpr'
Requires-Dist: libpipe>=0.1.5; extra == 'ml-gpr'
Requires-Dist: pandas>=2.0; extra == 'ml-gpr'
Requires-Dist: pyro-ppl>=1.9; extra == 'ml-gpr'
Requires-Dist: torch>=2.0; extra == 'ml-gpr'
Provides-Extra: ultranest
Requires-Dist: ultranest>=3.0; extra == 'ultranest'
Description-Content-Type: text/markdown

# ps_eor: Power-spectrum analysis and signal separation for 21-cm cosmology

`ps_eor` provides data containers, power-spectrum estimators, simulations, and
signal-separation tools for low-frequency interferometric 21-cm experiments.
It supports the path from gridded visibility data to spatial, cylindrical, and
spherical power spectra, including Gaussian-process foreground and systematic
separation.

📖 **Documentation:** <https://ps-eor.readthedocs.io/en/latest/> — getting
started, topic-based user guide, `pstool` command reference, and full API
reference.

**Upgrading from ps_eor 0.x to 1.0? Read the [ML-GPR migration
guide](https://ps-eor.readthedocs.io/en/latest/user_guide/ml_gpr/migration_1_0.html).**

## What the package does

**Data handling** — build analysis-ready visibility cubes from calibrated images:

- read frequency-ordered FITS images (Jy/PSF), using matching PSF files for an
  accurate Kelvin conversion, or inferring the PSF area from image metadata;
- select the field of view and baseline range, Fourier-transform to the *uv*
  plane, and store visibilities, weights, frequencies, and metadata as HDF5.

**Power spectra** — measure 21-cm power spectra:

- spatial, cylindrical, and spherical spectra, plus cross-spectra between
  independent observations, with the instrumental and cosmological
  normalizations applied.

**Signal separation** — suppress foregrounds and systematics, preserve the 21-cm signal:

- polynomial and PCA foreground models;
- ML-GPR, a Gaussian-process framework that infers posterior foreground,
  systematic, noise, and 21-cm components and propagates them to power spectra —
  for one observation or jointly across many, separating repeatable sky from
  observation-dependent excess variance.

**Simulation and sensitivity** — generate test data and forecast performance:

- draw Gaussian-process component cubes (foreground, 21-cm, systematic;
  single- or multi-epoch) from covariance kernels or an input power spectrum;
- generate thermal-noise data cubes from measured weights or a simulated
  telescope layout and observing strategy;
- forecast UV coverage and power-spectrum sensitivity, with built-in models for
  LOFAR, NenuFAR, MWA, SKA-Low, HERA, AARTFAAC, and OVRO-LWA.

The GPR approach follows [Mertens et al. (2018)](https://ui.adsabs.harvard.edu/abs/2018MNRAS.478.3640M/abstract);
the multi-observation extension relates to [Munshi et al. (2025)](https://ui.adsabs.harvard.edu/abs/2025A%26A...704A.205M/abstract).

## Installation

The base package contains the data and power-spectrum functionality:

```bash
pip install ps-eor
```

Install the Gaussian-process backend and its standard samplers, including NUTS
through Pyro, with:

```bash
pip install "ps-eor[ml-gpr]"
```

UltraNest is available separately:

```bash
pip install "ps-eor[ml-gpr,ultranest]"
```

For development:

```bash
git clone https://gitlab.com/flomertens/ps_eor.git
cd ps_eor
pip install -e ".[ml-gpr,dev]"
```

Python 3.10 or newer is required.

## Creating a visibility cube from FITS images

Given one calibrated image and one matching PSF image per frequency, `ps_eor`
can perform the Jy/PSF-to-Kelvin conversion and construct a weighted
`CartDataCube`. The files are sorted using their FITS frequency metadata:

```python
from pathlib import Path

import numpy as np

from ps_eor import datacube, psutil

image_files = [str(path) for path in Path("fits/images").glob("*.fits")]
psf_files = [str(path) for path in Path("fits/psfs").glob("*.fits")]

image_files = psutil.sort_by_fits_key(image_files, "CRVAL3")
psf_files = psutil.sort_by_fits_key(psf_files, "CRVAL3")

cube = datacube.CartDataCube.load_from_fits_image_and_psf(
    image_files,
    psf_files,
    umin=50,
    umax=250,
    theta_fov=np.deg2rad(4),
    int_time=10,
    total_time=10 * 3600,
)
cube.save("visibilities.h5")
```

Here, `umin` and `umax` are in wavelengths, `theta_fov` is in radians, and the
integration and total observing times are in seconds. The same operation is
available as `pstool gen_vis_cube`.

## Basic power-spectrum workflow

```python
import numpy as np

from ps_eor import datacube, pspec

cube = datacube.CartDataCube.load("visibilities.h5")

config = pspec.PowerSpectraConfig(
    el=2 * np.pi * np.arange(cube.ru.min(), cube.ru.max(), 10),
    window_fct="hann",
)
ps_gen = pspec.PowerSpectraBuilder(config).get(cube)

spatial = ps_gen.get_ps(cube)
cylindrical = ps_gen.get_ps2d(cube)

kbins = np.logspace(np.log10(ps_gen.kmin), np.log10(0.5), 10)
spherical = ps_gen.get_ps3d(kbins, cube)
```

## ML-GPR workflow

The high-level fitter is configured from TOML:

```python
from ps_eor.ml_gpr import MLGPRConfigFile, MLGPRForegroundFitter

config = MLGPRConfigFile.get_defaults()
fitter = MLGPRForegroundFitter(config)
noise_for_fit = fitter.process_noise_cube(noise_cube)
result = fitter.run(data_cube, noise_for_fit)
```

Advanced users can work directly with the lower-level building blocks:

```python
from ps_eor.ml_gpr.kernels import UVScaledKernel
from ps_eor.ml_gpr.multidata import MultiData
from ps_eor.ml_gpr.regressor import MultiGPRegressor
```

## Command-line interface

The package installs `pstool`, which can run the main analysis and sensitivity
workflows without writing Python code:

| Command | Purpose |
| --- | --- |
| `gen_vis_cube` | Read frequency-ordered FITS image and PSF files, convert Jy/PSF to Kelvin, select the field of view and baseline range, and write an HDF5 visibility cube. |
| `run_flagger` | Identify bad frequency and UV samples from Stokes I and V cubes, then save the flag mask and a diagnostic plot. |
| `even_odd_to_sum_diff` | Build signal and noise-proxy cubes from an even/odd data split. |
| `diff_cube` | Subtract two compatible visibility cubes. |
| `combine` | Combine observations or nights, with optional flagging and noise-based weighting. |
| `combine_sph` | Combine spherical-harmonic observations. |
| `make_ps` | Calculate spatial, cylindrical, and spherical power spectra and save their numerical results and diagnostic plots. |
| `run_ml_gpr` | Separate foreground, systematic, noise, and 21-cm components and produce their posterior power spectra and diagnostics. |
| `run_ml_gpr_inj` | Inject a simulated 21-cm signal and measure its recovery through ML-GPR. |
| `vis_to_sph` | Convert a Cartesian visibility cube into a spherical-harmonic cube. |
| `simu_uv` | Simulate telescope UV coverage for a pointing and observing setup. |
| `simu_noise_img` | Generate a thermal-noise FITS image cube from simulated UV coverage. |
| `simu_noise_ps` | Predict spatial, cylindrical, and spherical power-spectrum sensitivity at one redshift. |
| `simu_noise_ps_zrange` | Follow the sensitivity of a selected \(k\)-mode across a redshift range. |

Each command documents its inputs and options:

```bash
pstool --help
pstool COMMAND --help
```

## ML-GPR tutorials

The numbered notebooks form a focused ML-GPR tutorial series:

1. [Quick start](https://gitlab.com/flomertens/ps_eor/-/blob/master/notebooks/01-Quickstart.ipynb)
2. [Simulating data](https://gitlab.com/flomertens/ps_eor/-/blob/master/notebooks/02-Simulating-Data.ipynb)
3. [The covariance model](https://gitlab.com/flomertens/ps_eor/-/blob/master/notebooks/03-The-Covariance-Model.ipynb)
4. [Running a fit](https://gitlab.com/flomertens/ps_eor/-/blob/master/notebooks/04-Running-a-Fit.ipynb)
5. [Training the VAE](https://gitlab.com/flomertens/ps_eor/-/blob/master/notebooks/05-Training-the-VAE.ipynb)
6. [ML-GPR with the VAE kernel](https://gitlab.com/flomertens/ps_eor/-/blob/master/notebooks/06-ML-GPR-with-the-VAE-Kernel.ipynb)
7. [Config-driven runs](https://gitlab.com/flomertens/ps_eor/-/blob/master/notebooks/07-Config-Driven-Runs.ipynb)
8. [Multi-epoch GPR](https://gitlab.com/flomertens/ps_eor/-/blob/master/notebooks/08-Multi-Epoch-GPR.ipynb)
9. [Data in-painting](https://gitlab.com/flomertens/ps_eor/-/blob/master/notebooks/09-Data-In-Painting.ipynb)
10. [Validation and failure modes](https://gitlab.com/flomertens/ps_eor/-/blob/master/notebooks/10-Validation-and-Failure-Modes.ipynb)

Specialist examples, including Cross-GPR and sampler-prior comparisons, live in
the [advanced notebooks](https://gitlab.com/flomertens/ps_eor/-/tree/master/notebooks/advanced).

## Tests

```bash
pytest
```

Frozen numerical fixtures used to compare the GPyTorch implementation with the
former GPy backend are stored under `tests/data/gpr_golden/`.

## License

`ps_eor` is distributed under the GNU General Public License v3 or later.
