Metadata-Version: 2.4
Name: trustworthy_survey
Version: 0.2.0
Summary: Design-aware reliability benchmarking and correction for ML on complex survey data (NHANES-first)
Author-email: Shengjie Zhang <2142024286@qq.com>
License: BSD-3-Clause
Project-URL: Repository, https://doi.org/10.57760/sciencedb.011ou
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.5
Requires-Dist: numpy>=1.23
Requires-Dist: scikit-learn>=1.2
Dynamic: license-file

# trustworthy_survey

A validated toolkit for evaluating and repairing machine learning under complex sampling designs — and
beyond surveys, wherever a serving population drifts from the training population.

Companion to Zhang, *Nature Computational Science* (Resource submission): calibration layer with
exactness guarantees, per-instance repairability certificate, DS-Mondrian prediction sets, certified
estimand search, and a four-rung ground-truth validation ladder.

## The Drift Law / Drift Audit (v0.2.0)

A characterization theorem stack for transport bias under label-mixture transport
(`theory/drift_law_theorems.tex`): transport bias equals the inner product of
margin drift with the model's channel error profile (Theorem 1); the level-wise
law bias = −(1−a)·drift holds **if and only if** the model's channel law is an
affine contraction of the outcome's channel law, and the slope estimates the
absorption index *a* with a calibrated bootstrap SE (Theorems 2–3); an exact
counterfactual margin forecast supports pre-deployment what-if audits
(Theorem 5). The estimation stack ships with an efficiency-optimal default
(count-weighted WLS = GLS = individual interaction regression), a domain
warning whose firing has a theorem-backed three-mechanism diagnosis, and two
empirically established boundary conditions (functional-form gap; second-order
terms rejected by LOO).

Three lines on your own transport:

```python
from trustworthy_survey import drift_law
r = drift_law(preds=model_preds, y=target_labels, channel=channel_levels)
r["beta_hat"], r["beta_se"], r["a_hat"], r["warning"]   # slope, ±SE, absorption, domain check
```

End-to-end worked audit on real cross-country data (ESS 2023, DE→HU/GR/PT):
`examples/drift_audit_quickstart.py`. Real-domain evidence: NHANES PHQ9,
bank-internal deployment, UCI specialty transport (boundary case), ESS
cross-country regime discrimination (domain warning: zero false alarms on 10
real transports).

Deep-model demo (ROUND 81): the same ESS audit with a small MLP shows the law
holds *more* cleanly for flexible models (|r| up to 0.98 vs 0.92 for logistic
regression) yet flexible models absorb *less* (â 0.46–0.75 vs 0.57–0.97) —
equal source fit, worse transport absorption, sharper regime discrimination.
Baseline comparison (exp_driftlaw_baseline_comparison.csv): ECE, Brier, and
the global calibration slope do **not** discriminate law-active from absorbed
transports (ECE is *lower* in the dangerous regime: 0.63 vs 1.50 pp); only
the channel-structured drift slope and its linearity evidence do
(|r| 0.94 vs 0.53).

## Install

```bash
# after unpacking the archived release
pip install -e .
pytest tests/test_reproduce_paper.py -q   # headline numbers regenerate as assertions
```

## Quickstart (30 lines)

See `notebooks/`: `quickstart_nhanes.ipynb`, `quickstart_uci.ipynb`, `quickstart_synthetic.ipynb`.

## Layers

Input (frame builders, design reporters) / Correction (weighted recalibration) / Audit (certificate,
conformal, causal, operating curve) / Validation (four-rung ladder) / Guidance (14 pitfalls, reporting
grid). See Box 1 of the manuscript.

## Citation

Zhang, S. trustworthy_survey: a validated toolkit for evaluating and repairing machine
learning under complex sampling designs. *Nat. Comput. Sci.* (submitted). Data deposit: ScienceDB
10.57760/sciencedb.011ou. License: BSD-3-Clause.
