Metadata-Version: 2.4
Name: lda4microbiome
Version: 1.0.0
Summary: A workflow for topic-model (LDA/NMF) analysis of microbiome data with interactive Sankey visualizations
Author-email: Peiyang Huo <peiyang.huo@kuleuven.be>
License: MIT
Project-URL: Homepage, https://gitlab.kuleuven.be/aida-lab/projects/LDA4Microbiome_Workflow
Keywords: microbiome,LDA,NMF,topic modeling,MALLET,gensim
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.3.0
Requires-Dist: numpy>=1.20.0
Requires-Dist: scipy>=1.7.0
Requires-Dist: matplotlib>=3.3.0
Requires-Dist: seaborn>=0.11.0
Requires-Dist: plotly>=5.0.0
Requires-Dist: gensim>=4.0.0
Requires-Dist: scikit-learn>=1.0.0
Requires-Dist: little-mallet-wrapper
Requires-Dist: anywidget
Requires-Dist: traitlets
Requires-Dist: kaleido>=1.0.0
Provides-Extra: clustering
Requires-Dist: fast-hdbscan; extra == "clustering"
Dynamic: license-file

# Microbiome LDA Topic Modeling Workflow

## Overview
This workflow implements topic modeling for microbiome data analysis.
Two LDA implementations (gensim/MALLET) and NMF are available.

For LDA in environmental microbiome:
Huo, P., Vargas Ribera, P., Rediers, H. et al. Latent Dirichlet Allocation reveals tomato root-associated bacterial interactions responding to hairy root disease. Environmental Microbiome 20, 161 (2025). (https://doi.org/10.1186/s40793-025-00822-2)

For LDA in pig enterotyping:
Comer L, Huo P, Colleluori C, et al. From forest to farm: the impact of a broad spectrum of lifestyles on the porcine gut microbiota[J]. Current Research in Microbial Sciences, 2026: 100576. (https://doi.org/10.1016/j.crmicr.2026.100576)

For NMF:
Enterosignatures define common bacterial guilds in the human gut microbiome, Frioux, Clémence et al., Cell Host & Microbe, Volume 31, Issue 7, 1111 - 1125.e6 (https://doi.org/10.1016/j.chom.2023.05.024)

## Workflow Steps
The workflow consists of four steps. We are using an IBD data set to demonstrate.

The demo data (`notebooks/data/`) is the gut microbiome 16S dataset from the Inflammatory Bowel Disease Multi-omics Database (IBDMDB / HMP2), obtained via the Bioconductor package [HMP2Data](https://bioconductor.org/packages/HMP2Data/) and filtered exactly as in the MIDASim paper (samples with library size ≥ 3000 and taxa present in ≥ 2 samples, yielding 146 samples × 614 taxa).

He, M., Zhao, N. & Satten, G.A. MIDASim: a fast and simple simulator for realistic microbiome data. Microbiome 12, 135 (2024). (https://doi.org/10.1186/s40168-024-01822-z)

All workflow functions are imported from the top-level package:

```python
from lda4microbiome import preprocess, training, k_evaluation, viz
```

### Step 1: Data Transforming 


- Input ASV abundance table and taxonomy file paths
- ASV abundance table requires a CSV file where the index contains samples and columns contain features (ASV IDs), with values as **raw counts** instead of relative abundance values
- Taxonomy table requires a CSV file where the index contains ASV IDs and columns include taxonomy levels (Domain, Kingdom, Phylum, Class, Order, Family, Genus, and Species)

```python
output_path = str(REPO_ROOT / "notebooks" / "demo_output")
prep = preprocess(
    base_directory=output_path,
    asv_df=asv,
    taxa_df=taxa,
    taxa_level="ASV",
)
```

### Step 2: Model Building
Select a range of models you want to evaluate.

```python
models = training(
    output_path,
    MC_range=range(2,11),
    implementation="lda_gensim",  # 'lda_mallet', 'nmf' are available
)
```

### Step 3: Model Evaluation
We provide two evaluations. One is basic metrics: perplexity and coherence score for LDA; squared R error and coherence score for NMF. The other is our designed StripeSankey, which directly illustrates how samples change across models.

```python
k_evaluation(output_path, plot="metrics")

k_evaluation(output_path, plot="stripesankey", min_flow_samples=5,
                            metadata_df=metadata,
                            hover_field=["diagnosis", "biopsy_location", "is_inflamed"],
                            high_threshold=0.67)

k_evaluation(output_path, plot="meta_sankey",
             metadata_df=metadata, hover_field=["diagnosis", "biopsy_location"],
             high_threshold=0.67, k_list=[6,7,8,9,10])
```

`plot="stripesankey"` — cluster-lineage view showing how samples move between MCs across K:

![StripeSankey](notebooks/figures/stripesankey_20260731_1506.png)

`plot="meta_sankey"` — the same flows encoded by a metadata field (here `diagnosis`):

![meta_sankey](notebooks/figures/meta_sankey_20260731_1506.png)

### Step 4: Single Result Visualization
Once k is decided, we provide publication-ready visualizations. For the sample-MC relationship: annotated heatmap and stacked bar chart. For the MC-Feature relationship: heatmap.

```python
k_selected = 4
v = viz(
    base_directory=output_path,
    k_value=k_selected,
    metadata_df=metadata,
    onestop=True
)

v.heatmap_sample(
    include=["diagnosis", "biopsy_location", "is_inflamed"],
    sort="diagnosis",
    save_plot=True
)

v.bar_sample(groupby="diagnosis")

v.heatmap_mc(
    feature_level="Genus",
    use_top_tokens=True,
    top_n=10,
    colorscale="Blues",
)
```

Sample–MC relationship. `heatmap_sample` — sample-MC probabilities with metadata annotation strips:

![heatmap_sample](notebooks/figures/heatmap_sample_20260731_1521.png)

`bar_sample` — stacked MC composition grouped by a metadata column:

![bar_sample](notebooks/figures/sample_bar.png)

MC–Feature relationship. `heatmap_mc` — top features per MC:

![heatmap_mc](notebooks/figures/MC_heatmap.png)


## Installation

Requires Python ≥ 3.9. You can install this package from PyPI:

```bash
pip install lda4microbiome
```

## Acknowledgements

The MALLET LDA implementation is run through [little-mallet-wrapper](https://github.com/maria-antoniak/little-mallet-wrapper), a Python wrapper around [MALLET](https://mimno.github.io/Mallet/) (MAchine Learning for LanguagE Toolkit) by Andrew McCallum.

## License

This project is licensed under the MIT License — see the [LICENSE](LICENSE) file for details.
