Metadata-Version: 2.4
Name: scpreflight
Version: 1.5.0
Summary: The pre-flight check for single-cell analysis: predict what unsupervised methods will recover - before running them - from a count matrix and ~100 labels, with a cross-kingdom calibration, signature figures and a one-command HTML report.
Author-email: Taishan Li <litaishan.cns1day@gmail.com>
License: MIT
Keywords: single-cell,scRNA-seq,diagnostics,unsupervised,recoverability,ploidy,endopolyploidy
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.22
Requires-Dist: scipy>=1.8
Requires-Dist: pandas>=1.4
Requires-Dist: scikit-learn>=1.1
Provides-Extra: test
Requires-Dist: pytest>=7; extra == "test"
Provides-Extra: h5ad
Requires-Dist: anndata>=0.9; extra == "h5ad"
Provides-Extra: plots
Requires-Dist: matplotlib>=3.7; extra == "plots"
Dynamic: license-file

# scPreflight

**Predict what unsupervised single-cell analysis will recover — before running it.**

Almost every single-cell study starts by embedding cells in a low-dimensional space and
looking for structure. Nothing in that workflow tells you whether the target you care
about — a treatment effect, a donor difference, a developmental state, a ploidy level —
will appear in the embedding or be silently missed. scPreflight is the pre-flight check:
two numbers, computed from the count matrix and ~100 labelled cells, that say

1. **Is there anything to find?** — a cross-validated supervised ceiling, and
2. **Will an unsupervised method find it?** — a predicted unsupervised AUC, read off a
   calibration fitted on 97 contrasts from ten public datasets spanning five species
   (*Arabidopsis*, rice, human, mouse, zebrafish), two kingdoms and eight platforms.

Read together, they separate a **buried** signal (real, but outranked by batch, donor
and cell type — a small labelled subset will recover it) from an **absent** one
(redesign the experiment). Across the calibration set, 52% of real targets leave more
than 0.10 AUC unclaimed by unsupervised analysis: buried is the common case.

## Sixty seconds to a real result

```bash
pip install -e ".[test,h5ad]"     # from a repository checkout
scpreflight demo
```

The demo runs the diagnostic on a bundled real dataset — 1,468 *Arabidopsis* endosperm
nuclei with absolute flow-sorted ploidy labels (GSE157145) — and prints, in seconds:

```
"supervised_ceiling": 0.9405,          <- the ploidy signal is real and strong
"predicted_unsupervised_auc": 0.8064,  <- but unsupervised analysis won't reach it
"gap": 0.1341,                         <- > 0.10: a buried signal
"auc_total_counts": 0.692,             <- and it rides on library size,
"warnings": [ ... matched evaluation ... ]  <- so score it on a matched subset
"verdict": "buried signal: ... A small labelled subset (~100 cells) recovers most of the gap"
```

## On your own data

**From scanpy / AnnData** (labels can be strings; NaN / `''` / `-1` = unlabelled):

```python
from scpreflight import diagnose_adata
out = diagnose_adata(adata, "condition", covariate_keys=["batch"], layer="counts")
print(out["supervised_ceiling"], out["predicted_unsupervised_auc"], out["verdict"])
```

**From bare arrays** (cells × genes raw counts, dense or sparse):

```python
from scpreflight import spectral_diagnosis
out = spectral_diagnosis(counts, labels, covariates=meta[["batch", "stage"]])
```

**From the shell** (JSON on stdout; `.h5ad` / `.npz` / `.mtx[.gz]` / `.csv`):

```bash
scpreflight diagnose counts.h5ad --labels obs:condition --covariates cov.csv
scpreflight profile counts.npz --labels labels.csv:group   # full recoverability profile
scpreflight depth 950                                      # planning: what can this depth support?
```

`diagnose` uses the full matrix — unlabelled cells shape the spectrum, exactly as the
calibration defines it. See `scpreflight diagnose --help` for formats and `--genes-axis`
for `.mtx` orientation.

## How to read the two numbers

| Ceiling | Gap (ceiling − predicted) | Meaning | Action |
|---|---|---|---|
| < 0.60 | — | nothing to find | redesign the experiment, not the analysis |
| ≥ 0.60 | > 0.10 | **buried**: real but outranked | ~100 labels recover most of it; residualise known covariates first |
| ≥ 0.60 | ≤ 0.10 | recoverable | unsupervised analysis will capture most of what exists |

If the output warns that the target correlates with total counts, score any recovery on
a library-size-matched subset (`scpreflight.evaluation.matched_evaluation`) — otherwise
"recovery" may be sequencing depth, not biology.

**Where the prediction comes from.** The predicted AUC is
`0.5 + 0.5·σ(a + b·alignment)`, where alignment is the fraction of the target's
discriminant direction inside the leading-10 principal subspace. Fitted on the plant
contrasts alone, it predicts the 57 animal contrasts at MAE 0.047; held out one dataset
at a time, the fit on the remaining nine predicts the tenth at a median MAE 0.046
(range 0.019–0.076). Constants and provenance: `scpreflight.diagnostics.CALIBRATION`.

**Applicability limits.** Binary targets; raw counts (not log-normalised — you will be
warned); scRNA-seq geometry (the calibration spans Smart-seq2 to 10x to inDrops);
2,000 variable genes is part of the definition, not a tuning knob. Memory scales with
cells × 2,000 after sparse HVG selection (~1.6 GB at 100k cells).

## Sequencing depth comes first

```python
from scpreflight import depth_advisor
depth_advisor(950)
# {'unsupervised': 0.66, 'with_100_labels': 0.689, 'supervised_ceiling': 0.837,
#  'recommendation': 'too shallow: the information itself is depth-limited...'}
```

Measured by binomial thinning of the endosperm gold standard: below ~2,000 counts per
cell the *ceiling itself* collapses — no estimator can recover what is not there.

## The origin case: endopolyploidy

The diagnostic came out of a concrete failure: recovering per-nucleus ploidy from plant
single-cell data as a latent variable is impossible not for lack of signal (supervised
ceiling 0.943) but because the ploidy direction ranks ~18th in the variance spectrum.
The package therefore also ships the weakly supervised protocol that does work:

```python
from scpreflight import PloidyPipeline
p = PloidyPipeline(covariates=meta[["batch", "stage"]]).fit(counts, labels=y_partial)
score = p.score_          # continuous ploidy score; p.tiers(K) for discrete tiers
```

with the measured label ladder (0 labels → 0.751, 100 → 0.853, 400 → 0.910 on the gold
standard, library-size-matched) in `PloidyPipeline.label_budget_guide()`, and the
matched evaluation design in `scpreflight.evaluation`.

## Signature figures and the flight briefing

```bash
pip install -e ".[plots]"
scpreflight report --demo -o report.html     # one command, the whole story
```

`scpreflight.pl` draws the six signature figures — the **Preflight Map** (your
targets on the recovered-vs-ceiling plane, over the released calibration cloud),
the **Spectrum Needle**, the **Twin Gauge**, the Alignment Curve, the Calibration
Placement and the Depth Runway — with fixed layouts and a fixed semantic palette
(amber = buried, viridian = recoverable, slate = absent), all vector and
publication-ready:

```python
import scpreflight as spf
outs = [spf.spectral_diagnosis(X, adata.obs[k], keep_arrays=True) for k in keys]
spf.pl.preflight_map(outs).savefig("preflight_map.pdf")
```

`scpreflight report` wraps everything into a single self-contained HTML flight
briefing: verdict stamp, the two numbers as instrument cards, every figure with a
plain-language reading, and a provenance footer (no JavaScript, no external
assets, prints cleanly).

## Install

```bash
pip install -e ".[test,h5ad]"   # h5ad extra = anndata, for .h5ad / AnnData input
pytest                          # full test suite, a few seconds
```

Requires numpy, scipy, pandas, scikit-learn (Python ≥ 3.9). The research modules behind
the papers live in `scpreflight.research` (repository only, not in the wheel).

## Reproducing the paper

`scripts/` contains every analysis; `results/` every output JSON; `figs/source/` a
source-data table per figure panel. `scripts/check_consistency.py` verifies every
number in the manuscript against the file that produced it, and
`scripts/reproduce_calibration.py` refits the released calibration constants from the
exported contrast set in minutes. The test suite pins the shipped `CALIBRATION` and
`DEPTH_TABLE` constants to the released result files.

## Data

All public: GSE157145 (At endosperm gold standard), GSE261441 (At maturing root),
GSE166894 (At cotyledon pools), GSE226097 (At rosette atlas), GSE273875 (rice
root), GSE96583 (Kang PBMC), GSE132044 (Ding platform-comparison resource),
GSE84133 (Baron pancreas), GSE102827 (Hrvatin visual cortex), GSE112294 (Wagner
zebrafish embryogenesis). No new experimental data were generated.

## Citation

See `CITATION.cff`. If you use the bundled demo data beyond the demo, also cite
Picard et al., *Nature Plants* 2021 (GSE157145).
