Metadata-Version: 2.4
Name: ebo-sieve
Version: 0.1.0
Summary: EBO-SIEVE - Screening Intelligence & Evaluation Engine: a config-driven, license-clean phenotypic-screen analysis platform with an MCP server.
Author: Ebovir
License: MIT
Project-URL: Homepage, https://github.com/Ziqiyang72960/ebo-sieve
Project-URL: Repository, https://github.com/Ziqiyang72960/ebo-sieve
Keywords: drug-discovery,screening,cheminformatics,mcp,dose-response,admet
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.26
Requires-Dist: pandas>=2.0
Requires-Dist: scipy>=1.11
Requires-Dist: scikit-learn>=1.3
Requires-Dist: statsmodels>=0.14
Requires-Dist: networkx>=3.0
Requires-Dist: matplotlib>=3.7
Requires-Dist: plotly>=5.18
Requires-Dist: openpyxl>=3.1
Requires-Dist: xlrd>=2.0
Requires-Dist: requests>=2.31
Requires-Dist: pydantic>=2.5
Requires-Dist: PyYAML>=6.0
Requires-Dist: rdkit>=2023.9
Provides-Extra: web
Requires-Dist: fastapi>=0.110; extra == "web"
Requires-Dist: uvicorn>=0.29; extra == "web"
Requires-Dist: python-multipart>=0.0.9; extra == "web"
Provides-Extra: enrichment
Requires-Dist: gseapy>=1.1; extra == "enrichment"
Provides-Extra: admet
Requires-Dist: admet-ai>=2.0; extra == "admet"
Provides-Extra: generate
Requires-Dist: reinvent>=4.0; extra == "generate"
Provides-Extra: mcp
Requires-Dist: mcp>=1.2; extra == "mcp"
Provides-Extra: claude
Requires-Dist: anthropic>=0.40; extra == "claude"
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == "openai"
Provides-Extra: test
Requires-Dist: pytest>=8.0; extra == "test"
Dynamic: license-file

# EBO-SIEVE — Screening Intelligence & Evaluation Engine

*PyPI package: **`ebo-sieve`** · import name: **`ebosieve`** · console command:
**`ebo-sieve`** · MCP server name: **`ebo-sieve`**.*

```bash
pip install ebo-sieve        # once published; for local dev use: pip install -e .
```

A **generalizable, config-driven, license-clean** platform that ingests any
phenotypic / reporter screen, computes normalized activity, calls hits with
proper plate-aware statistics, fits dose-response, runs full cheminformatics +
SAR, predicts targets and ADMET, proposes analogs, and links hits to biology.

**Nothing is hardcoded** to a specific compound, virus, assay, or file layout —
every study-specific choice lives in a `StudyConfig` (YAML). The scientific core
is **fully deterministic and runs headless with NO LLM**; the LLM/UI is a thin
layer on top, never a dependency of the science.

> **Build status:** the deterministic core (modules 1–13), the CLI, the tool
> layer, the **MCP server**, the **provider-agnostic LLM layer** with the
> scientific-integrity guardrail, the custom **blue/white web app** (FastAPI +
> interactive Plotly + 3Dmol structure viewer), a **Boltz-2 co-folding** plug-in,
> synthetic data, and the test suite are complete and passing (30/30). Packaged
> as **`ebo-sieve`** and push-ready for PyPI + Hugging Face Spaces (see
> [deploy/DEPLOY.md](deploy/DEPLOY.md)).

---

## What resembles LOWE (capability map)

| LOWE capability | SIEVE module | Commercial-clean basis |
|---|---|---|
| MatchMaker (drug→target) | `core/dti.py` | **PubChem BioAssay (CC0)** ECFP-Tanimoto kNN |
| Compound generation | `core/generate.py` | **REINVENT 4** (Apache-2.0) → RDKit-enum fallback |
| ADMET prediction | `core/admet.py` | **ADMET-AI** (MIT) → RDKit physchem fallback |
| Maps of Biology (pathways) | `core/enrichment.py` + `core/knowledgegraph.py` | Reactome/WikiPathways/GO (CC0/CC-BY), Open Targets (CC0), networkx |
| Compound procurement | `core/procure.py` | PubChem vendor lookup (public domain) |
| Assay analytics / hit ID | `core/normalize` + `hitcalling` + `doseresponse` + `condition_analysis` | RDKit/scipy/statsmodels (BSD) |
| NL orchestration | `tools/` + `mcp_server.py` + `llm/` | MCP server (Claude-native) + provider-adapter plugins |
| UI | `app/` *(roadmap)* | Streamlit / HTML endpoint |

---

## Quick start

```bash
pip install -r requirements.txt          # or: pip install -e .

# 1) generate synthetic demo data (no proprietary files needed)
ebo-sieve make-example --out examples/

# 2) run the full deterministic pipeline (NO LLM), primary screen
ebo-sieve run \
  --config config/luciferase_toa.yaml \
  --input-file examples/primary_screen.csv \
  --compounds examples/compounds.csv \
  --out results/primary/

# 3) dose-response study
ebo-sieve run \
  --config config/gfp_dose_response.yaml \
  --input-file examples/dose_response.csv \
  --compounds examples/compounds.csv \
  --out results/dose/

# propose a column/role mapping for an unknown file
ebo-sieve infer --input-file examples/primary_screen.csv

# launch the EBO-SIEVE web app (custom blue/white FastAPI UI)
ebo-sieve serve            # http://127.0.0.1:8000
```

## Interfaces at a glance

| Interface | Command | Best for |
|---|---|---|
| Headless CLI | `ebo-sieve run …` | reproducible batch runs, no LLM |
| **Web app** | `ebo-sieve serve` | a polished blue/white browser UI: pick/upload data, run, interactive Plotly, chat |
| MCP server | `ebo-sieve mcp` | Claude Desktop/Code plugin |
| Standalone chat | `ebo-sieve chat` | guardrailed NL over the tools |
| HTML report | written by every run | a self-contained, shareable `report.html` |

Every run also writes `report.html` — a self-contained, blue/white, interactive
EBO-SIEVE report you can open in any browser or email. To deploy the web app
publicly (Hugging Face Spaces + PyPI), see [deploy/DEPLOY.md](deploy/DEPLOY.md).

Full CLI:

```bash
ebo-sieve run --config config/study.yaml \
  [--inputs <dir> | --input-file <file>] \
  [--compounds compounds.csv] [--viability viability.csv] \
  [--crispr-hits genes.csv] [--online] [--no-generation] \
  --out results/
```

- `--online` enables PubChem network calls (lead-SMILES resolution, procurement
  CIDs). **Default is fully offline / hermetic** so runs are reproducible.

---

## Outputs

Every run writes to `--out`:

- **Tables:** `tidy_data.csv`, `per_condition_summary.csv`, `hit_calls.csv`,
  `dose_response.csv`, `ranked_hits.csv`, `condition_classification.csv`,
  `chem_descriptors.csv`, `scaffolds.csv`, `mmp.csv`, `similarity.csv`,
  `clusters.csv`, `predicted_targets.csv`, `admet.csv`, `generated_ideas.csv`,
  `enrichment.csv`, `procurement.csv`, `plate_qc.csv`, `knowledge_graph_edges.csv`
- **Interactive Plotly figures** (`figures/`): dose-response curves, hit
  waterfall, condition heatmap (lead highlighted), predicted-target bars,
  enrichment dot-plot, ADMET radar.
- **`results_summary.md`** — QC/Z'/SSMD, top hits, IC50s, profile matches,
  predicted targets/pathways, and **all caveats**.
- **`run_log.json`** — inputs, params, per-step trace, library + backend versions,
  global seed. Reproducible.

---

## Configuration (`StudyConfig`)

Everything generic lives in YAML. Three examples ship in `config/`:

- `luciferase_toa.yaml` — luminescence, time-of-addition, `lower_is_more_activity`
- `gfp_dose_response.yaml` — GFP %-positive, dose series, `higher_is_more_activity`
- `viability_counterscreen.yaml` — cytotoxicity counter-screen (selectivity index)

Key fields: `assay_readout {name, direction}`, `controls {background, vehicle,
positive}`, `normalization {method}`, `condition_axes [{name, is_concentration,
unit}]` (generic: `time_of_addition | dose | cell_line | MOI | ...`, multiple
allowed), optional `lead_compound` anchor, column-role mapping, and configurable
`hitcalling` / `doseresponse` thresholds.

---

## The science (deterministic core)

1. **adapters** — ingest (a) long CSV/TSV, (b) plate-map xlsx (values grid + key
   grid), (c) irregular two-block spreadsheets. Auto-detected; no hardcoded cell
   coordinates. `infer_schema()` proposes a mapping for confirmation.
2. **normalize** — background-subtract, normalize (`vehicle_ratio | pct_of_control
   | robust_z | none`), compute direction-aware percent-activity, aggregate
   mean/SD/CV/n.
3. **hitcalling** — robust Z (median/MAD), SSMD, percent-of-control, **B-score
   via two-way median polish** (row/column de-trending), per-plate **Z'-factor**
   + control SSMD + signal window QC, **Benjamini-Hochberg FDR**, configurable
   hit calls with edge/QC flags.
4. **doseresponse** — **4-parameter logistic** fit → IC50/EC50, Hill, top,
   bottom, R², **95% CI (bootstrap or covariance)**, QC flags (incomplete /
   ambiguous / no-plateau / poor-fit).
5. **condition_analysis** — pivot compound × condition; classify
   `condition_dependent | pan_active | inactive`; match to lead profile.
6. **chem** — RDKit canonicalization, ECFP4 (Morgan r=2, 2048-bit), Tanimoto to
   lead, **Butina clustering**, **Bemis-Murcko scaffolds**, **matched molecular
   pairs (RDKit-native single-cut — no mmpdb license dependency)**, descriptors
   (MW, cLogP, TPSA, HBD, HBA, RotB, aromatic rings), QED, Lipinski + Veber
   flags, SAR table. Lead-without-SMILES resolves via PubChem REST (`--online`).
7. **dti** — target prediction (MatchMaker analog): ECFP-Tanimoto kNN over an
   annotated **PubChem BioAssay (CC0)** reference → ranked targets + confidence +
   neighbor provenance. Stable plug-in interface `predict_targets(smiles)`. A
   small bundled reference ships for offline demos.
8. **admet** — ADMET-AI (MIT) when installed; transparent RDKit physchem proxies
   otherwise (backend always labelled).
9. **generate** — REINVENT 4 (Apache-2.0) when installed; RDKit scaffold-
   decoration enumeration fallback, ranked by QED + Lipinski/Veber + novelty.
10. **enrichment** — hypergeometric ORA (native, always available) / GSEA (gseapy,
    MIT, when installed) against **Reactome/WikiPathways/GO** gene sets.
11. **knowledgegraph** — networkx graph compound→target→pathway→phenotype;
    neighbors, shortest paths, shared-target clusters, evidence-linked
    "explain why X may act via Y" (**never fabricates an edge**).
12. **procure** — PubChem CID + vendor-page lookup (`--online`).
13. **report** — all CSVs, interactive Plotly figures, `results_summary.md`,
    `run_log.json`.

### Caveats the pipeline always states (never fabricates around)

- **SELECTIVITY_INDEX** is null unless a viability/cytotoxicity dataset is supplied.
- **CYTOTOXICITY-CONFOUND** flag for hits whose activity may be general cell shutdown.
- **Target/pathway predictions are hypotheses** requiring wet-lab (e.g. CRISPR) validation.

---

## Licenses & data sources (commercial-use declaration)

**All defaults are permissive / public-domain — $0 in licenses.**

### Python dependencies

| Package | License | Commercial use |
|---|---|---|
| numpy, pandas, scipy, scikit-learn, statsmodels, networkx | BSD-3-Clause | ✅ Yes |
| rdkit | BSD-3-Clause | ✅ Yes |
| matplotlib | PSF (BSD-compatible) | ✅ Yes |
| plotly, openpyxl, pydantic, PyYAML | MIT | ✅ Yes |
| xlrd | BSD | ✅ Yes |
| requests | Apache-2.0 | ✅ Yes |
| pytest | MIT | ✅ Yes |
| *(optional)* gseapy | MIT | ✅ Yes |
| *(optional)* admet-ai | MIT | ✅ Yes |
| *(optional)* reinvent (REINVENT 4) | Apache-2.0 | ✅ Yes |
| *(optional)* mcp, anthropic SDK | MIT | ✅ Yes |
| *(optional)* openai SDK, streamlit | Apache-2.0 | ✅ Yes |

### Data sources

| Source | License | Status |
|---|---|---|
| PubChem + **PubChem BioAssay** *(default DTI substrate)* | Public domain | ✅ Yes |
| Open Targets | CC0 | ✅ Yes |
| Reactome, WikiPathways | CC0 / CC-BY | ✅ Yes (attribute CC-BY) |
| Gene Ontology, UniProt | CC-BY | ✅ Yes (attribute) |
| **ChEMBL** *(OPTIONAL, flagged)* | CC-BY-**SA** | ⚠️ Share-alike — only via `use_chembl=True`, obligation surfaced in every result |
| Boltz-2 / ESMFold *(optional structure plug-in)* | MIT | ✅ Yes |

**Deliberately NOT used:** KEGG, full DrugBank, commercial-restricted STRING,
MSigDB `C2:KEGG`, and the Enrichr web service (commercial ambiguity). AlphaFold3
weights are **not** used. ChEMBL is an optional, clearly-flagged enrichment only;
**PubChem BioAssay is the default substrate** so the core stays share-alike-free.

---

## Natural-language layer — MCP server + provider plugins

The `tools/` layer is the **single surface** the LLM (or any MCP client) can
touch: thin JSON-schema wrappers over the deterministic core whose handlers
return already-computed numbers + artifact paths only. Two entry points consume
the same registry.

### 1. As an MCP server (Claude-native plugin, recommended)

```bash
pip install mcp
ebo-sieve mcp          # stdio transport
```

Register it with any MCP client. Claude Desktop
(`~/Library/Application Support/Claude/claude_desktop_config.json`) — once
`pip install ebo-sieve` has put the console script on the path:

```json
{
  "mcpServers": {
    "ebo-sieve": { "command": "ebo-sieve", "args": ["mcp"] }
  }
}
```

Zero-install form (no manual `pip`, via `uvx`):

```json
{
  "mcpServers": {
    "ebo-sieve": { "command": "uvx", "args": ["--from", "ebo-sieve", "ebo-sieve", "mcp"] }
  }
}
```

If a client launches with a minimal PATH and can't find the command, use the
**absolute** path to the console script (e.g. `/opt/anaconda3/bin/ebo-sieve`).

MCP is an open standard, so this is both the most intuitive (the chat UI comes
for free) and the most generalizable (any MCP-capable client works).

**stdio hygiene:** MCP's stdio transport requires stdout to carry *only*
JSON-RPC. SIEVE runs noisy scientific libraries (ADMET-AI/Lightning/RDKit/tqdm),
so at `mcp` startup it preserves the real stdout for the transport and redirects
fd 1 → stderr, diverting every stray library write (import banners + progress).
This keeps the plugin stable in a client; the headless CLI is unaffected.

### 2. Standalone guardrailed chat (provider-agnostic)

```bash
ebo-sieve chat                       # auto-detects a provider
ebo-sieve chat --message "Load examples/... run it, show the top hits"
```

**Provider adapters are plugins** (`llm/providers/`, implementing a one-method
`LLMProvider` protocol), so the backend is generalizable to everyone. Two ship:

| Provider | SDK (license) | Use |
|---|---|---|
| `AnthropicProvider` | `anthropic` (MIT) | **Claude — default when `ANTHROPIC_API_KEY` is set** |
| `OpenAICompatibleProvider` | `openai` (Apache-2.0) | Local **Ollama**/vLLM, plus OpenAI/DeepSeek/Together/… |

**Auto-detection policy (confirmed):** Claude when an Anthropic key is present,
otherwise a local/OpenAI-compatible endpoint (Ollama `http://localhost:11434/v1`
by default). If the `anthropic` SDK is absent but a key is set, SIEVE still
reaches Claude via its OpenAI-compatible endpoint through the `openai` SDK — so
"key present ⇒ Claude" holds without a silent downgrade. Override anything with
`LLM_PROVIDER`, `LLM_MODEL`, `LLM_BASE_URL`, `LLM_API_KEY`.

Open-weight models remain first-class: **Mistral (Apache-2.0), Qwen2.5
(Apache-2.0), DeepSeek (MIT)** via Ollama keep the fully-local, $0 story intact.
*Note: Llama ships under a restricted community license.*

**Commercial-clean note:** defaulting to Claude changes nothing about licensing
(the `anthropic`/`mcp` SDKs are MIT) and the deterministic core still computes
every number **locally and free**. The only shift is that the optional chat layer
may call a paid API and send tool inputs/outputs (compound ids, summary stats —
not raw data files, which never leave the local core). Flip to the local adapter
for privacy-sensitive screens.

### Scientific-integrity guardrail

The LLM **never computes, estimates, or invents** a numeric/scientific result —
it may only call the tools and narrate their returned values, always surfacing
caveats, citing the source tool for every number. The guardrail is **structural**
(the model's only lever is the tool layer, whose handlers return the numbers) and
reinforced by the system prompt in `llm/guardrail.py`. Demonstrated by
`test_guardrail_calls_tool_instead_of_estimating`: asked to "eyeball the IC50",
the model calls `get_dose_response` and reports the fitted value, not a guess.

---

## Testing

```bash
PYTHONPATH="$PWD" python -m pytest tests/ -q
```

18 unit + smoke tests cover all three input formats, direction-aware
normalization, hit-calling (active recovery, median-polish, BH-FDR), IC50
recovery, cheminformatics, target prediction, enrichment, the knowledge graph,
and an end-to-end reproducible run — all on a synthetic-data generator
(`tests/synthetic.py`), so **no proprietary files are needed**.

## Roadmap

Done: `tools/` (JSON-schema wrappers), `mcp_server.py` (MCP plugin), `llm/`
(provider-agnostic tool-calling loop + guardrail). Remaining:

- `app/` — Streamlit/Gradio GUI: upload, schema confirmation, chat, live Plotly,
  downloads, run-log viewer, HTML endpoint. (The MCP server already provides a
  full chat interface via Claude Desktop/Code today.)
