Metadata-Version: 2.4
Name: kyle_out_zip
Version: 1.12.1
Summary: Multi-site clinical cohort preprocessing, predictive modeling, and model explainability pipeline
Author: Lewis Katz School of Medicine at Temple University Center for Biostatistics & Epidemiology
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: scikit-learn
Requires-Dist: xgboost
Requires-Dist: scipy
Requires-Dist: matplotlib
Requires-Dist: seaborn
Requires-Dist: shap
Requires-Dist: dalex
Requires-Dist: openpyxl
Dynamic: license-file

Clinical Site Statistics Rollup & Predictive Modeling Pipeline
================
Lewis Katz School of Medicine at Temple University Center for
Biostatistics & Epidemiology
2026-08-12

# Project Overview

This repository contains the multi-center pipeline, statistical models,
and manuscript deliverables consolidating longitudinal Electronic Health
Record (EHR) inpatient data across five clinical health systems: -
**TEMPLE** (Lewis Katz School of Medicine at Temple University) -
**PSU** (Penn State University) - **Hopkins** (Johns Hopkins
University) - **PITT** (University of Pittsburgh Medical Center) -
**Geisinger** (Geisinger Health System)

The pipeline standardizes encounter types and procedure coding, executes
longitudinal lookback window aggregations (1-year and 2-year), maps lab
values via LOINC codes, generates Table 1 descriptive statistics
rollups, and trains patient-level traditional machine learning models
(Logistic Regression, Random Forest, HistGBM, and XGBoost) using
Stratified 5-Fold Cross-Validation on $N = 273,755$ qualifying index
patient encounters.

------------------------------------------------------------------------

# ✅ Cohort-Construction Fix & Server Re-Validation (Completed August 2026)

A confirmed data-leakage bug in `Table1_Consolidated_Labs.py`’s cohort
construction (`load_and_process_site`) was identified and resolved. The
original outcome definition evaluated a **patient-level “ever, anywhere
in history”** flag rather than scoping readmission to 30 days
post-discharge following the selected index encounter. This structurally
entangled the outcome with `readmission_count_twelve_months` (feature \>
0 implied outcome = 1 with zero exceptions across all 270,565
non-readmitted patients in the original dataset).

The fix protocol (`select_index_encounters_and_outcome()`) combines
random index encounter sampling **among eligible candidates only**
(verifying a 30-day post-discharge follow-up window) with an outcome
strictly scoped to
$[t_{\text{discharge}}, t_{\text{discharge}} + 30 \text{ days}]$.
Because the lookback window
($[t_{\text{admit}} - 365 \text{ days}, t_{\text{admit}}]$) and outcome
window are strictly disjoint, zero data leakage is guaranteed regardless
of index selection.

**Server Re-Validation Completed**: The hardened pipeline was executed
on the `dassie` server
(`consolidated_out_no_n_px_ccs_20260811_154116.zip` +
`Table1_run_20260811_094355.log`). The final 5-health-system cohort
consists of **$N = 273,755$ unique patients** (73,786 excluded for
lacking verified follow-up) with a leak-free 30-day readmission rate of
**11.37%** (31,113 readmitted).

- **Headline Full-Model AUCs**: XGBoost = **0.706**, HistGBM =
  **0.705**, Random Forest = **0.691**, Logistic Regression = **0.682**.
- **2-Feature Utilization Benchmark** (`prior_IP_OS_ED_count` +
  `readmission_count_twelve_months`): AUCs = **0.581–0.583** across all
  architectures, retaining **82.3%–85.4%** of full-model discrimination
  capacity while remaining 100% leak-free.
- **Unit Tests**: All 97 unit tests across the test suite pass (95
  passed, 2 skipped by design).

------------------------------------------------------------------------

# ✅ Manuscript Content-Integrity Fix (v1.12.1, August 2026)

A v1.12.0 manuscript “re-compilation” pass silently dropped several
sections from `eDERRI_Traditional_ML_Models_Manuscript.qmd` with no
trace in `CHANGELOG.md`/`NEWS.md`: all 5 SHAP/DALEX explainability
figures, the Author Contributions & Acknowledgments section, the
Software/package-citations section, and Appendices A & B. A follow-up
restore pass brought those sections back structurally, but in doing so
reverted large parts of the narrative prose (Abstract, Introduction,
Methods, cohort flowchart, Discussion) to a **stale, pre-leakage-fix
version** — citing the old $N = 347{,}541$ cohort, pre-fix AUCs
(0.831/0.823/0.821/0.783), and several numbers (a Table 4 odds-ratio
example, a Table 2 Brier score) that did not match the actual exported
table CSVs, even though every table and figure in the same document was
already built from the corrected, leak-free $N = 273{,}755$ cohort.

All narrative numbers were re-verified directly against the exported
CSVs (`Manuscript_Table1_Descriptive_Stats.csv`,
`Manuscript_Table3_Model_Performance.csv`,
`Table3_2Feature_vs_FullModel_Comparison.csv`,
`Manuscript_Table2_Logistic_Regression.csv`) and corrected in place; the
CONSORT flowchart’s per-stage encounter/patient dual counts were
restored; a Limitations subsection addressing potential
survivorship/loss-to-follow-up bias (raised in the 2026-08-12 meeting)
was added; and `test_manuscript_content_integrity.py` was added to guard
against this exact failure mode recurring — it asserts required
sections/figures are present, that known stale pre-fix values are
absent, and that headline AUC/Brier numbers in the prose match the
exported CSVs.

------------------------------------------------------------------------

# Repository Structure & Extract Downloads Directory

To maintain a clean working repository, server extracts and download
archives are organized under the **`extracts/`** directory:

    kyle_out_zip/
    ├── extracts/
    │   ├── from_server/          # Drop newly downloaded server ZIP archives here (*.zip)
    │   └── archive/              # Archived historic server extracts and legacy datasets
    ├── old_logs/                 # Archived pipeline execution logs and raw run dumps
    ├── Dr_Rubin_Manuscript_Deliverables_20260729/   # Formatted manuscript tables & figures
    ├── Dr_Rubin_Manuscript_Deliverables_20260729.zip
    ├── Dr_Rubin_Data_Leakage_Autopsy_Deliverables_20260811.zip # Technical Autopsy & Distribution Suite
    ├── Dr_Rubin_Manuscript_Review.zip               # Lightweight manuscript & table package
    ├── eDERRI_Traditional_ML_Models_Manuscript.qmd  # Quarto manuscript draft
    ├── Table1_Consolidated_Labs.py                   # Cohort construction + model training
    ├── generate_manuscript_tables.py
    ├── generate_shap_4panel.py
    ├── generate_dalex_4panel.py
    ├── build_dr_rubin_deliverables.py
    ├── build_autopsy_deliverables.py
    ├── create_review_zip.py
    ├── pyproject.toml
    ├── test_table1.py
    ├── test_pipeline_and_paths.py
    ├── test_manuscript_data_fixes.py
    ├── test_table3_and_manuscript_hygiene.py
    ├── test_index_selection_outcome.py
    └── test_table1b_missingness_and_ordering.py    # Unit tests for Table 1b and logical row ordering

All Python helper scripts (`generate_manuscript_tables.py`,
`generate_shap_4panel.py`, `generate_dalex_4panel.py`,
`build_dr_rubin_deliverables.py`, `diagnose_lr_utilization.py`)
automatically search `extracts/from_server/` and `extracts/` recursively
to detect and process the latest extract ZIP files.

------------------------------------------------------------------------

# Deliverables & Manuscript Table Specifications

### 1. Formatted Manuscript Tables (`Dr_Rubin_Manuscript_Deliverables_20260729/01_Manuscript_Tables/`)

- **`Manuscript_Tables_Combined.xlsx`**: Holds sheets matching
  Dr. Rubin’s exact shells:
  - `Table 1. Descriptive`: Baseline characteristics for all **83
    features** (Whole cohort, Readmission subset, Non-readmission
    subset).
  - `Table 2. LogReg`: Multivariable Logistic Regression risk factors
    for all **93 features** (Dual Unscaled Marginal ORs & Normalized
    ORs, 95% CIs, P-values).
  - `Table 3. Model performance`: Model discrimination and calibration
    measures (AUC, F1, Sensitivity, Specificity, Brier Score across
    model types; all cross-validation $\text{SD} < 0.01$).
- **`Manuscript_Tables_By_Site.xlsx`**: Per-site Table 1s, Table 2s, and
  master site performance metrics.

### 2. Publication Figures (`Dr_Rubin_Manuscript_Deliverables_20260729/02_Manuscript_Figures/`)

- **`Figure_Top10_DALEX_4Panel.png`**: High-resolution 300 DPI 4-panel
  summary plot comparing top 10 feature importances via DALEX variable
  dropout loss across models.
- **`Figure_Top10_SHAP_4Panel.png`**: High-resolution 300 DPI summary
  plot comparing top 10 features via SHAP Shapley values across models.
- **`Figure_Prior_Utilization_SHAP_Blowup.png`**: 4-panel architectural
  comparison for the top utilization feature (`prior_IP_OS_ED_count`)
  showing how LR (linear tail), RF (bimodal butterfly), HistGBM (smooth
  saturation), and XGBoost (regularized step) process risk differently.
- **`Figure_SHAP_4Panel.png`**: Combined 4-panel full SHAP summary
  figure.
- **`Figure_HistGBM_SHAP.png`**: Standalone 300 DPI HistGBM SHAP summary
  plot.

------------------------------------------------------------------------

# Empirical Diagnostic Findings

A 2-feature Logistic Regression diagnostic experiment
(`diagnose_lr_utilization.py` / `benchmark_2feature_all_models.py`) was
conducted on the leak-free combined dataset ($N = 273,755$ encounters),
benchmarked against the canonical Table 2 full-model results: - **Full
93-Feature Model** (Logistic Regression): $\text{AUC-ROC} = 0.682$,
$\text{Brier Score} = 0.223$ - **2-Feature Utilization Sub-Model**
(`prior_IP_OS_ED_count` + `readmission_count_twelve_months`):
$\text{AUC-ROC} = 0.583$, $\text{Brier Score} = 0.099$ - **Finding**:
Historical healthcare utilization alone accounts for **85.4%** of the
total discriminatory power of Logistic Regression. The same 2-feature
benchmark repeated across Random Forest, HistGBM, and XGBoost captures
82.3%-84.1% of each architecture’s full-model AUC-ROC (see manuscript
Table 3), always benchmarked against the same canonical full-model
numbers reported in Table 2 rather than independently recomputed.

------------------------------------------------------------------------

# Running Pipeline Scripts

### 1. Generating Manuscript Tables & Figures

``` bash
python generate_manuscript_tables.py
python generate_shap_4panel.py
python generate_shap_blowup_analysis.py
python build_dr_rubin_deliverables.py
python benchmark_2feature_all_models.py  # Regenerates Table 3 (2-feature sub-model benchmark)
```

### 2. Running Unit Tests

``` bash
pytest -v
```

------------------------------------------------------------------------

# License & Authorship

Developed by the **Lewis Katz School of Medicine at Temple University
Center for Biostatistics & Epidemiology** in collaboration with the
**eDERRI Research Group**.
