Metadata-Version: 2.4
Name: glyph-discovery
Version: 0.1.0
Summary: Domain-agnostic hypothesis generation using sparse autoencoders
Author-email: Robbie Holland <robbie.holland@stanford.edu>
License-Expression: MIT
Project-URL: Homepage, https://github.com/RobbieHolland/Glyph
Project-URL: Repository, https://github.com/RobbieHolland/Glyph
Project-URL: Paper, https://openreview.net/forum?id=rgpgukbeVf
Keywords: sparse autoencoders,mechanistic interpretability,hypothesis generation,phewas,medical imaging
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.0
Requires-Dist: torchvision>=0.15.0
Requires-Dist: pytorch-lightning>=2.0.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: scikit-learn>=1.3.0
Requires-Dist: scipy>=1.10.0
Requires-Dist: statsmodels>=0.14.0
Requires-Dist: matplotlib>=3.7.0
Requires-Dist: networkx>=3.0
Requires-Dist: umap-learn>=0.5.4
Requires-Dist: pillow>=9.5.0
Requires-Dist: pyarrow>=12.0.0
Requires-Dist: hydra-core>=1.3.0
Requires-Dist: omegaconf>=2.3.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: tqdm>=4.65.0
Provides-Extra: autointerp
Requires-Dist: google-generativeai>=0.8.0; extra == "autointerp"
Requires-Dist: google-cloud-aiplatform>=1.60.0; extra == "autointerp"
Provides-Extra: tracking
Requires-Dist: wandb>=0.15.0; extra == "tracking"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: flake8>=6.0.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Dynamic: license-file

# Glyph: Mechanistic Science

**Domain-Agnostic Hypothesis Generation using Sparse Autoencoders**

Glyph is a Python toolkit for discovering interpretable features and generating scientific hypotheses from multimodal data. It uses Sparse Autoencoders (SAEs) to learn interpretable features, then conducts Phenome-Wide Association Studies (PheWAS) to identify and rank hypotheses.

📄 **Paper**: [MechSci: Scaling Clinical Science via Mechanistic Interpretability of Multimodal Medical Foundation Models](https://openreview.net/forum?id=rgpgukbeVf#discussion) — *Agents4Science 2025*

[![Video Presentation](https://img.youtube.com/vi/s3t-Nu8HNFc/maxresdefault.jpg)](https://www.youtube.com/watch?v=s3t-Nu8HNFc)
*Click to watch the video presentation*

## Features

- **Sparse Autoencoders** with TopK activation and Matryoshka nested dictionaries
- **Ghost Gradient Recovery** to prevent dead features during training
- **PheWAS Analysis** for systematic hypothesis testing
- **Statistical Rigor** with odds ratios, confidence intervals, and AUC metrics
- **GPU Acceleration** for large-scale analysis
- **Domain Agnostic** - works with any structured data (images, tabular, etc.)

## Installation

```bash
pip install glyph-discovery
```

Optional extras:

```bash
pip install "glyph-discovery[autointerp]"   # LLM-based feature interpretation
pip install "glyph-discovery[tracking]"     # Weights & Biases logging
```

To work on Glyph itself:

```bash
git clone https://github.com/RobbieHolland/Glyph.git
cd Glyph
pip install -e ".[dev]"
```

## Quick Start

```python
from glyph import Glyph

# Define your configuration overrides
config = [
    "data.metadata=/path/to/metadata.csv",
    "data.input_cols=[image_path]",
    "data.outcome_cols=[disease]",
    "data.embedding_map={image_path: resnet18}",
    "data.cache_dir=./cache",
    "data.output_dir=./outputs",
    "seed=42",
]

# Initialize Glyph
ms = Glyph(config_path='config', config_overrides=config)

# Option 1: Run full pipeline
results = ms()  # Equivalent to ms.fit().hypothesis_search()

# Option 2: Run in stages
ms.fit()                # Extract embeddings, train SAE, compute linear probes
ms.hypothesis_search()  # Run PheWAS study

# Access results directly
print(f"Embeddings: {list(ms.embeddings.keys())}")
print(f"SAE models: {list(ms.sae_models.keys())}")
print(f"PheWAS hypotheses: {len(ms.phewas_results)}")
```

## Pipeline Overview

```
Raw Data → Embeddings → SAE Training → Sparse Features → PheWAS → Ranked Hypotheses
```

### `ms.fit()` - Steps 1-4

1. **Embedding Extraction**: Extract dense representations using pretrained models (e.g., ResNet18)
2. **Linear Probe**: Evaluate embedding quality for outcome prediction (AUC)
3. **SAE Training**: Learn sparse, interpretable features with TopK activation
4. **SAE Linear Probe**: Evaluate sparse feature quality for outcome prediction

### `ms.hypothesis_search()` - Step 5

5. **PheWAS Study**: Test all sparse feature-outcome associations
   - Computes odds ratios, confidence intervals, and AUC for each feature
   - Filters by minimum activation count for statistical reliability
   - Ranks hypotheses by odds ratio

## Example Output

```
[PheWAS Results]
  Total hypotheses: 162

  Outcome class balance (train):
    outcome: 2436/3295 positive (73.9%), 859/3295 negative (26.1%)

  Top 10 hypotheses (by train odds ratio):
  ──────────────────────────────────────────────────────────────────────────────────────────
  Feature           Train OR    Test OR      AUC    % AUC    Count
  ──────────────────────────────────────────────────────────────────────────────────────────
  feature_98            3.75       3.96    0.796    60.0%     2818
  feature_61            3.45       3.34    0.773    55.2%     2524
  feature_140           2.86       2.64    0.614    23.2%      674
  ...
  ──────────────────────────────────────────────────────────────────────────────────────────
```

- **Train/Test OR**: Odds ratio (how much the feature increases disease odds)
- **AUC**: Predictive power of this single feature
- **% AUC**: `(feature_auc - 0.5) / (embedding_auc - 0.5)` - what % of embedding's predictive power this feature captures
- **Count**: Number of samples where feature activates

## Configuration

Glyph uses [Hydra](https://hydra.cc/) for configuration management.

### Key Configuration Options

```yaml
# Data settings
data:
  metadata: /path/to/metadata.csv      # CSV with sample IDs and file paths
  input_cols: [image_path]             # Columns containing input data paths
  outcome_cols: [disease]              # Columns containing outcomes to predict
  embedding_map: {image_path: resnet18}  # Map input columns to embedding models
  cache_dir: ./cache                   # Cache directory for embeddings
  output_dir: ./outputs                # Output directory for results

# SAE settings
sae:
  top_ks: [20]                         # TopK sparsity values to try
  matryoshka: [2048]                   # Dictionary sizes (Matryoshka nesting)
  max_steps: 5000                      # Training steps
  learning_rate: 0.0003                # Learning rate

# PheWAS settings
phewas:
  min_activations: 25                  # Minimum feature activations for hypothesis
  top_n_print: 10                      # Number of top hypotheses to display
```

## Project Status

✅ **Core Pipeline Complete**

- [x] Phase 1: Project structure
- [x] Phase 2: Data loading (GlyphDataset, DataLoader)
- [x] Phase 3: Embedding extraction (ResNet18, caching)
- [x] Phase 4: SAE training (TopK, Matryoshka loss, ghost gradients)
- [x] Phase 5: Linear probe (binary classification, regression)
- [x] Phase 6: PheWAS analysis (odds ratios, AUC, ranking)
- [x] Phase 7: Main Glyph class with fit/hypothesis_search API
- [x] Phase 8: Examples
- [x] Phase 9: Unit tests (83 tests passing)
- [ ] Phase 10: AutoInterp (interpretation with LLMs)
- [ ] Phase 11: Documentation

## Running Tests

```bash
# Run all tests
python -m pytest tests/test*.py -v

# Run specific test module
python -m pytest tests/test6_sae.py -v
python -m pytest tests/test7_phewas.py -v
```

## Inspiration

Glyph distills core algorithms from a larger medical hypothesis generation research codebase. Key innovations preserved:

- **Matryoshka SAEs** for multi-resolution feature learning
- **Ghost Gradients** for dead feature recovery
- **TopK Activation** for interpretable sparse codes
- **Rigorous Statistics** with odds ratios and confidence intervals

## License

MIT License - see [LICENSE](LICENSE) for details.

## Citation

If you use Glyph in your research, please cite:

```bibtex
@software{glyph2025,
  title = {Glyph: Domain-Agnostic Hypothesis Generation},
  author = {Holland, Robbie},
  year = {2025},
  url = {https://github.com/RobbieHolland/Glyph}
}
```
