Metadata-Version: 2.4
Name: recip-slice
Version: 0.4.1
Summary: Physical reciprocal-space slice reconstruction from XDS rotation electron-diffraction data
Author-email: Yinlin Chen <yinlin.chen@su.se>
Project-URL: Organization, https://www.su.se/
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Physics
Classifier: Topic :: Scientific/Engineering :: Chemistry
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.24
Requires-Dist: matplotlib>=3.7
Requires-Dist: fabio>=2023.4
Requires-Dist: scipy>=1.10
Requires-Dist: PyQt5>=5.15
Provides-Extra: gui
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: mypy>=1.11; extra == "dev"
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-cov>=5; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"

# Recip Slice

Recip Slice reconstructs physically scaled crystallographic sections from raw
rotation electron-diffraction images processed with XDS. Each detector
observation is converted to a scattering vector, unrotated into a fixed crystal
frame, selected against one or more reciprocal-space slabs, and accumulated
directly onto an orthonormal Cartesian grid in Å⁻¹.

The principal use is to generate interpretable `hk0`, `h0l`, `0kl`, offset-layer,
and diagonal reciprocal-space sections from a rotation series without first
allocating a dense 3D reciprocal-space volume.

![Example physical h0l slice](docs/images/image.png)

## Main outputs

By default, reconstructions share one results directory; the result name is the
filename prefix:

```text
<XDS directory>/recip_slices/<result name>_*
```

The directory contains, for every selected plane:

- `*_sum.npy`: count-preserving accumulation;
- `*_mean.npy`: coverage-normalized intensity;
- `*_geometry_coverage.npy`: valid detector/rotation/plane sampling coverage,
  independent of intensity thresholding;
- `*_signal_coverage.npy`: sampling weight retained after signal selection;
- `*_coverage.npy`: deprecated compatibility alias for `signal_coverage`;
- `*_qx.npy` and `*_qy.npy`: Cartesian reciprocal-space coordinates;
- `*_manifest.json`: plane definitions, unit-cell and hkl grid-direction
  metadata, grid metadata, filenames, frame-selection provenance, and
  reconstruction diagnostics.

Each manifest plane also reports sampling diagnostics from
`geometry_coverage`: covered and uncovered requested-grid fractions,
contributing frame and detector-sample counts, and coverage binned by physical
`|q|` in Å^-1. These metrics describe the measured reciprocal-space support;
they do not interpolate or fill missing regions.

The saved arrays are never modified by display controls.

## Installation

The same source package is used on Linux, Windows, and macOS. Published binary
wheels contain the native exact-order sparse accumulator, so normal wheel
installation does not require a compiler:

```bash
python -m pip install recip-slice
```

For development from a checkout:

```bash
python -m pip install -e .
```

PyQt5 is installed as a core dependency because the program is GUI-first. The
legacy `.[gui]` extra remains accepted for compatibility.

Source installs attempt to compile the small native sparse bilinear accumulator
from the shipped generated C source. The default build policy is
`RECIP_SLICE_NATIVE=auto`: if compilation is unavailable, installation succeeds
with the scientifically equivalent serial NumPy fallback. Performance-critical
or release builds can set `RECIP_SLICE_NATIVE=require` to fail rather than
silently fall back; `RECIP_SLICE_NATIVE=disable` explicitly requests the NumPy
backend. Run with `--verbose` to see `Sparse accumulator backend: native` or
`numpy-serial`; the same value is stored in the manifest.

On an offline machine with NumPy and the build dependencies already installed,
`python -m pip install -e . --no-build-isolation` avoids fetching build
dependencies. See `docs/installation.md` for Linux, Windows, macOS, and wheel
release details.

`fabio` is required to read SMV detector frames. SciPy remains a core
dependency for supported detector/noise-processing paths.

## Command line

Reconstruct the three principal planes in one frame-reading pass:

```bash
recip-slice reconstruct /path/to/xds \
  --xparm GXPARM.XDS \
  --planes hk0 h0l 0kl \
  --q-max 1.0 \
  --dq 0.002 \
  --slice-thickness 0.008 \
  --central-beam-q-cutoff 0.02 \
  --intensity-cutoff 10 \
  --prefix sample_name \
  --output-dir /path/to/xds/recip_slices \
  --no-plot --verbose
```

By default the command reads `NAME_TEMPLATE_OF_DATA_FRAMES` and `DATA_RANGE`
from `XDS.INP`; frame numbers come from the `?` field in the XDS template.
Raw-pixel processing does not apply `EXCLUDE_DATA_RANGE`, XDS region/pixel
masks, or detector spatial distortion by default. Enable those explicitly with
`--apply-xds-frame-exclusions`, `--apply-xds-region-exclusions`, and
`--apply-detector-distortion`. `DATA_RANGE` remains the experimental image
range in either mode; the preflight and GUI report whether that range is
complete for the requested policy. Use `--glob` only for an explicit manual
image selection. Manual paths are first matched through the XDS template; an
incompatible filename must contain one unambiguous frame-number group, or the
run stops rather than guessing from a run/suffix number. A normal run requires
`XDS.INP`, a refined `GXPARM.XDS` or `XPARM.XDS`, and every frame required by
the selected mode.

Noise reduction is disabled by default. To enable the CMOS-aware cleanup, add
`--noise-reduction cmos-robust`; it removes an additive detector baseline,
local charge-spread halos, isolated column streaks, and a conservative radial
residual without averaging neighbouring reflection pixels. The `cmos-mixed`
mode applies the same cleanup but restores 10% of the removed detector
residual as a deterministic partial correction. Both modes leave the raw
detector frames untouched.

SMV reads use a bounded four-worker read-ahead queue by default. On a fast
local disk this can be tuned with `--prefetch-workers N`, or disabled with
`--no-prefetch` when memory is constrained. Temporal plane-candidate reuse is
disabled by default. It can be enabled explicitly with
`--temporal-plane-cache`; `--no-temporal-plane-cache` forces
the baseline selection path explicitly.

For a guided custom Miller-index plane, use `--custom-plane` with an optional
offset and in-plane orientation. The offset is the right-hand side of the
fractional equation and the slab thickness remains physical reciprocal-space
distance:

```bash
recip-slice reconstruct /path/to/xds \
  --custom-plane 1 1 0 \
  --plane-name hhl_layer \
  --plane-offset-hkl 2 \
  --in-plane-basis 1 -1 0 0 0 1 \
  --q-max 1.0 --dq 0.002 --slice-thickness 0.008 \
  --no-plot
```

This reconstructs `h + k = 2`. The resolved Cartesian normal and orthonormal
in-plane basis, equation, offset, units, and sampling settings are recorded
in the result manifest and survive result loading. A custom plane is a single
plane in this milestone slice; the GUI layer builder and offset-series
controls remain planned work.

Open the lightweight viewer:

```bash
recip-slice view /path/to/result/folder --prefix sample_name
```

Open the Qt workspace for a new analysis session (the default when no
subcommand is supplied):

```bash
recip-slice
```

Open the Qt workspace:

```bash
recip-slice gui /path/to/xds --prefix sample_name
```

From a checkout, the same GUI can be launched directly without installing the
package in editable mode:

```bash
python run.py /path/to/xds --prefix sample_name
```

Interactive GUI launches print a short ASCII welcome banner with the current
version and a pointer to `recip-slice --help` in the command window. The
reconstruction, preflight, viewer, and JSON-reporting commands keep their
output focused for scripts and pipelines.

## GUI workflow

The workspace keeps a simple three-step route:

1. **Data**: select one parent folder. Every usable nested XDS dataset is
   indexed and can then be switched from a compact selector without another
   scan.
2. **Planes**: select principal, layer, or diagonal sections. All selected
   planes share one detector-frame pass.
3. **Settings**: review the resolution limit and workload; optional sampling
   and filtering controls remain collapsed for beginners.

The run/open actions remain visible from every step. Expert mode removes the
guidance and exposes technical controls for a direct folder-to-run workflow.
The right-hand workspace prioritises the reciprocal-space image; optional
appearance and export controls are collapsed until requested.

Under **Settings → Advanced settings**, **Run all indexed datasets…** applies
the current reconstruction settings to every usable XDS dataset in the
selected folder. A confirmation summary appears first, followed by separate
dataset- and detector-frame progress bars.

Recent experiment folders, named plane/settings presets, and display preferences
are retained between sessions. Recent folders remain available from the Recent
Projects menu, but a new GUI instance never selects the previous folder
automatically. Saved reconstructions are selectable as individual runs, with
provenance and quality summaries available without opening the process log.
Re-running an occupied result name requires an explicit choice to create a
timestamped version or replace the existing output.

**Compare** opens a second viewer for another indexed XDS dataset. **Choose
dataset…** reuses the searchable **Find…** dataset browser and can browse to
another folder. Saved slices are loaded automatically; if none are available,
the GUI offers to reconstruct dataset B using the current A settings and shows a
cancellable progress bar. Side-by-side and blink views work for any loaded
pair; numerical difference and ratio views are available only when the plane
and physical q-coordinate grids match exactly, so the GUI never performs
hidden scientific resampling. Appearance controls apply to whichever pane was
last clicked, allowing A and B to be adjusted independently; the active pane
is indicated only by a coloured frame. Hovering either pane reports the nearest
hkl coordinate and both intensities. Comparison figures can be exported for
reporting.

## Scientific scope

The output is geometrically precession-like, but it is not a simulation of a
physical precession experiment. Intensities retain the sampling, detector
response, dynamical scattering, and other limitations of the original
rotation data. `sum` is the weighted sum of signal-selected detector
intensities. `mean` is exactly `sum / signal_coverage`; it is not a
crystallographic intensity correction. `geometry_coverage` describes valid
detector/rotation/plane sampling before intensity thresholding, while
`signal_coverage` describes samples retained after thresholding.

If XDS spatial correction tables (`X-CORRECTIONS.cbf` and
`Y-CORRECTIONS.cbf`) or `BKGPIX.cbf` are present, they are detected and
recorded, but they are not applied unless the corresponding opt-in flags are
used. The manifest records both available and applied corrections/masks, the
refined geometry actually used, frame selection, q axes, preflight warnings,
and saved filenames. It also records detector samples considered, samples
rejected by the central-beam and optional XDS masks, signal-threshold
rejections, processed frames, and per-plane geometry/signal-covered fractions.

See [docs/algorithm.md](docs/algorithm.md) for the coordinate model,
[docs/architecture.md](docs/architecture.md) for the software design,
[docs/gui.md](docs/gui.md) for the interaction model, and
[docs/reproducibility.md](docs/reproducibility.md) for reporting guidance.
The prioritized, cross-platform feature plan is in
[docs/roadmap.md](docs/roadmap.md).

For an external transform check against indexed XDS observations, run:

```bash
recip-slice validate-spot /path/to/xds --xparm GXPARM.XDS
```

This requires an indexed `SPOT.XDS` beside the XDS inputs; the package does
not fabricate experimental validation data. The diagnostic reports total,
indexed, unindexed, and validated-indexed records separately; XDS `(0, 0, 0)`
unindexed records are not included in hkl residuals. SPOT detector positions
may be in XDS's spatially corrected coordinate system, so this check does not
automatically enable detector distortion correction for reconstruction.

## Testing

```bash
python tools/project_cycle.py check
```

PyQt5 is a core dependency. GUI integration tests run headlessly in CI with
`QT_QPA_PLATFORM=offscreen`.

## Project production cycle

Project housekeeping is separate from dataset housekeeping. From a checkout,
run `python tools/project_cycle.py release` to validate the source, run tests,
check every Python file with Ruff and mypy, quarantine known generated
artefacts, build the wheel and source distribution, and verify that the
canonical packaged `style.qss` and SVG icons are included in both packages.
The standalone `clean`
command is a dry run unless `--apply` is supplied; it never targets experiment
folders or reconstruction results.
