# OpenMix -- Technical Constitution

**Project:** OpenMix -- The Open-Source Framework for Computational Formulation Science
**Repo:** github.com/vijayvkrishnan/openmix (PUBLIC)
**Author:** Vijay Krishnan <vijayven.krishnan@gmail.com>
**License:** Apache 2.0
**Python:** 3.10+

---

## 0. What This Document Is

This is the authoritative reference for any AI agent or human working on OpenMix. It defines what the project is, how to write code for it, and what standards every contribution must meet.

If this document and any other file disagree, this document wins.

---

## 1. What OpenMix Is

**One sentence:** An open-source Python framework that brings computational intelligence to mixture and formulation science -- the missing layer between single-molecule tools (RDKit) and real-world formulations.

**Tagline:** "RDKit is for molecules. OpenMix is for mixtures."

**What it does:**
- Resolves any INCI ingredient to its molecular identity (SMILES, LogP, MW, charge)
- Observes formulations through physics -- reports what it sees, what it expected, and where they disagree
- Validates ingredient interactions against a curated knowledge base (273 rules, 6 domains)
- Runs autonomous experiments: an LLM proposes formulations, the physics engine evaluates, the agent iterates
- Learns across experiments: three-layer memory persists findings with confidence scores

**What it is NOT:**
- Not a product or SaaS (it's open infrastructure)
- Not a replacement for lab testing (it does the noticing, the lab does the testing)
- Not a black-box scorer (observations are structured and interpretable)
- Not limited to cosmetics (pharma, food, supplements, home care, materials)

**Target users (in adoption order):**
1. Formulation scientists (cosmetic, pharma, food) who lack computational tools
2. ML researchers who want benchmarks for mixture property prediction
3. AI agent builders who need formulation tools their agents can call (MCP)

**North star:** An open-source autonomous scientist for formulation chemistry. The AlphaFold moment for mixtures -- except open from day one.

---

## 2. Architecture

```
openmix/
  src/openmix/
    schema.py               # Formula, Ingredient -- core data model
    observe.py              # Physics observation engine (dual modes)
    validate.py             # Rule-based validation (3 modes)
    score.py                # Heuristic stability scoring (5 sub-scores)
    experiment.py           # Autonomous experiment runner
    memory.py               # Three-layer persistence (index, logs, discoveries)
    analysis.py             # Post-experiment insight extraction
    discover.py             # Hypothesis-driven rule discovery from data
    constraints.py          # Programmatic constraint enforcement
    matching.py             # Ingredient name matching (fuzzy)
    molecular.py            # RDKit integration (optional dependency)
    llm.py                  # Multi-provider LLM abstraction
    mcp_server.py           # MCP server (5 tools)
    resolver/               # INCI -> SMILES -> molecular properties
      resolve.py            #   Three-tier: seed cache -> user cache -> PubChem API
      cache.py              #   Local cache management (~/.openmix/)
      pubchem.py            #   PubChem API integration
      seed_ingredients.json #   Ships with package (2,400+ ingredients)
    knowledge/              # Domain knowledge (YAML, no code to contribute)
      data/                 #   Interaction rules, HLB values, aliases
      loader.py             #   Loads and indexes knowledge base
      constants.py          #   Preservative names, functional categories
      pka.py                #   Henderson-Hasselbalch pH calculations
    scorers/                # Pluggable evaluation for experiment runner
      base.py               #   Scorer interface + composite
      molecular.py          #   Physics-informed scorer (uses resolver)
      model.py              #   Trained ML model scorer
      lab.py                #   Cloud lab / manual feedback scorer
      shampoo_adapter.py    #   Adapter for shampoo stability dataset
    benchmarks/             # FormulaBench datasets + feature engineering
      shampoo.py            #   812 formulations, binary stability
      pharma_solubility.py  #   251 drug-excipient pairs
      mixture_solubility.py #   MixtureSolDB (146K records)
      drug_excipient.py     #   DE-INTERACT (4.2K interactions)
      features.py           #   Shared feature engineering
    cli/                    # CLI interface
      main.py               #   observe, validate, run, experiment, memory, demo, info
  tests/                    # Mirrors src/ structure
  experiments/              # YAML experiment definitions
  data/                     # Benchmark datasets (gitignored raw data)
  docs/                     # Specs, blog drafts
  examples/                 # Training scripts, demos (experimental, not production)
  models/                   # Saved ML models (gitignored)
```

### Key Design Principles

1. **Observations, not scores.** The physics engine reports what it sees, what it expected, and where they disagree. No arbitrary 0-100 scores as the primary output.
2. **Two modes, one engine.** Engineering mode minimizes concerns. Discovery mode investigates them. Same observations, different interpretation.
3. **Memory as hint, physics as truth.** The experiment memory suggests prior findings. The observation engine is always the source of truth.
4. **Layers are independently useful.** Validation works without the observer. The observer works without the experiment runner. Each layer adds value on its own.
5. **Pluggable everything.** LLM provider, evaluation function, knowledge base, ingredient resolver. The framework handles the loop; the components are swappable.

---

## 3. Development Workflow

### Plan Before Code

Before implementing any non-trivial feature:
1. State the problem in one sentence
2. Describe the approach (which files change, what the API looks like)
3. List what tests will verify it works
4. Get alignment before writing code

Do not start coding a feature without a plan. "Let me just try something" is how technical debt is born.

### Test-Driven Development

For new functionality:
1. Write the test first (or at least define what the test will assert)
2. Implement until the test passes
3. Refactor with tests green

For bug fixes:
1. Write a test that reproduces the bug
2. Fix the bug
3. Verify the test passes

**Every PR must maintain or increase test coverage.** No merging code that reduces the number of passing tests.

### Pre-Commit Checklist

Before any commit:
- [ ] `python -m pytest tests/ -q` -- all tests pass
- [ ] `python -m ruff check src/ tests/` -- zero lint errors (src AND tests)
- [ ] No `# type: ignore` without a comment explaining why
- [ ] No `Any` types without justification
- [ ] No dead code, no commented-out code, no TODO without a tracking issue
- [ ] If you changed an API: updated all callers in the same commit
- [ ] If you added a dependency: justified why, added to the right optional group

### Commit Discipline

- **This is a PUBLIC repo.** Every commit is visible. Commit messages matter.
- Author: `Vijay Krishnan <vijayven.krishnan@gmail.com>`
- No `Co-Authored-By: Claude` -- this is Vijay's project
- Commit messages: imperative mood, concise, explain the "why" not the "what"
- One logical change per commit. Don't bundle unrelated fixes.

---

## 4. Code Quality Standards

### The Bar

This codebase will be evaluated by:
- Senior engineers deciding whether to contribute
- ML researchers deciding whether to cite it
- Companies deciding whether to adopt it

Every file must read as if a senior engineer wrote it deliberately. No AI slop.

### SOLID Principles (applied pragmatically)

- **Single Responsibility:** Each module does one thing. `observe.py` observes. `validate.py` validates. Don't mix concerns.
- **Open/Closed:** New domains are added via YAML rules, not code changes. New scorers implement the `Scorer` interface.
- **Liskov Substitution:** Any `LLMProvider` subclass works anywhere a provider is expected. Any `Scorer` works in the experiment runner.
- **Interface Segregation:** Don't force callers to depend on things they don't use. `validate()` doesn't require the resolver. `observe()` doesn't require an LLM.
- **Dependency Inversion:** High-level modules (experiment runner) depend on abstractions (Scorer, LLMProvider), not concrete implementations.

### No Premature Abstraction

Three instances of similar code is better than one premature abstraction. Only abstract when:
- You have 3+ concrete cases that share the pattern
- The abstraction makes the code clearer, not just shorter
- You can name the abstraction clearly (if you can't name it, you don't understand it)

### What Good Code Looks Like Here

```python
# YES: clear, specific, self-documenting
def _extract_findings(log: ExperimentLog) -> list[Discovery]:
    """Extract ingredient preferences and avoidances from a completed experiment."""
    if not log.best_trial:
        return []
    ...

# NO: vague, over-abstracted, comments restating the code
def process(data):
    """Process the data."""  # what data? process how?
    results = []  # initialize results list
    for item in data:  # iterate over items
        results.append(transform(item))  # transform and append
    return results
```

### Security

- Never hardcode API keys or credentials. Use environment variables.
- Validate all external input (user-provided YAML, API responses, PubChem data).
- The resolver makes HTTP calls to PubChem -- handle timeouts, malformed responses.
- YAML loading: always use `yaml.safe_load()`, never `yaml.load()`.
- No `eval()`, no `exec()`, no `pickle.loads()` on untrusted data.
- The MCP server exposes tools -- ensure inputs are validated before processing.

---

## 5. Scientific & ML Standards

### Reproducibility

- Every experiment produces a complete JSON log with all trials, observations, and configuration
- Random seeds must be set and recorded when using stochastic methods
- Benchmark results must be reproducible from the committed code + data
- "It worked on my machine" is not acceptable. CI must pass on 3.10, 3.11, 3.12.

### Statistical Rigor

- Report confidence intervals, not just point estimates
- Use appropriate evaluation: leave-ingredients-out (LIO) for generalization, not random splits
- Fisher's exact test for rule discovery (not chi-squared on small samples)
- Never claim a result without the evaluation methodology alongside it
- Effect sizes matter more than p-values. A statistically significant but tiny effect is not a finding.

### Benchmarking Discipline

- FormulaBench datasets have defined train/test protocols. Follow them exactly.
- Baseline comparisons must use the same splits, same features, same evaluation metric
- Report both random-split AND leave-out performance. Random-split alone is misleading.
- New models must beat the established baseline on the defined metric to be worth reporting

### Honest Reporting

- If a method doesn't work, say so. The 0.52 R² ceiling on MixtureSolDB is an honest result.
- Coverage warnings: when the knowledge base has thin coverage for a domain, say so
- Confidence scores on discoveries: don't present a single-experiment finding as established fact
- "We don't know" is a valid and valuable scientific statement

### What Constitutes a "Finding"

A finding worth reporting must have:
1. A clear, falsifiable claim
2. Quantitative evidence (not just "it seems to work")
3. Comparison to a meaningful baseline
4. Acknowledgment of limitations
5. Reproducibility from committed code

"Our framework converges faster with memory" is not a finding without controlled experiments showing iteration counts with vs without memory, across multiple runs, with variance reported.

---

## 6. Current State (v0.2.1)

### What's Shipped
- Physics observation engine with dual modes (engineering/discovery)
- Ingredient resolver (2,429 seed ingredients, 100% SMILES, 94% LogP)
- 273 interaction rules (85 hard + 173 soft), 6 domains
- Henderson-Hasselbalch pH observations (25 ingredients, literature pKa)
- Autonomous experiment runner (pluggable LLM, YAML config, JSON logging)
- Experiment memory: three-layer persistence with consolidation and prior knowledge injection
- MCP server (5 tools: observe, validate, resolve, compatibility, pH check)
- FormulaBench: 4 datasets (shampoo 812, pharma 251, MixtureSolDB 146K, DE-INTERACT 4.2K)
- CLI: observe, validate, run, experiment, memory, demo, info
- CI: GitHub Actions, Python 3.10-3.12, ruff + pytest
- PyPI: `pip install openmix`

### Known Limitations
- Layer 2 ML hit a 0.52 R² ceiling on leave-solutes-out (MixtureSolDB). The bottleneck is mixture aggregation, not molecular representation.
- Discovery engine hypothesis generation is noisy (too many narrow thresholds, direction errors)
- Hansen solubility estimation was built but disabled (empirical estimation too crude)
- Knowledge base has thin coverage in pharma and food domains
- No embedding-based similarity search in memory retrieval (simple category matching)

### Key Numbers
- 165 tests, CI green on 3.10-3.12
- 273 interaction rules, 6 domains
- 2,429 seed ingredients with SMILES
- 4 benchmark datasets
- 5 MCP tools

---

## 7. Roadmap

| Version | Target | What Ships |
|---------|--------|------------|
| v0.2.1 | March 2026 | Physics observation engine, resolver, dual modes, experiment memory |
| v0.3 | Q2 2026 | Blog post + launch, CI hardening, more benchmark datasets, genuine finding |
| v0.5 | Q3 2026 | Trained stability models, HuggingFace pre-trained models |
| v1.0 | Q4 2026 | Mixture property prediction, published paper |

---

## 8. File Conventions

- **Source:** `src/openmix/` -- all production code
- **Tests:** `tests/test_*.py` -- mirrors source structure
- **Experiments:** `experiments/*.yaml` -- reproducible experiment definitions
- **Knowledge:** `src/openmix/knowledge/data/*.yaml` -- domain rules (YAML, no code)
- **Benchmarks:** `src/openmix/benchmarks/` -- dataset loaders + feature engineering
- **Examples:** `examples/` -- experimental scripts, NOT production code
- **Docs:** `docs/` -- specs, blog drafts

### Naming

- Files: `snake_case.py`
- Classes: `PascalCase`
- Functions/variables: `snake_case`
- Constants: `UPPER_SNAKE_CASE`
- Private functions: `_leading_underscore`

### Dependencies

- Core (always available): `pydantic`, `pyyaml`
- Optional groups: `rdkit`, `agent` (anthropic), `bench` (sklearn, xgboost, scipy), `openai`
- Guard optional imports: `try: import rdkit ... except ImportError: RDKIT_AVAILABLE = False`
- Never add a core dependency without strong justification

---

## 9. Common Pitfalls

- **YAML loading:** Always `yaml.safe_load()`. Never `yaml.load()`.
- **PubChem API:** Rate-limited. Always handle timeouts. Cache aggressively.
- **hash() is non-deterministic:** Python randomizes hash() across processes since 3.3. Use hashlib for stable hashes.
- **Ingredient name matching is fuzzy:** "Vitamin E" vs "Tocopherol" vs "DL-Alpha-Tocopherol" are the same thing. The matcher handles this, but tests must account for it.
- **Observation objects are dataclasses:** They're mutable. Don't mutate objects you loaded from disk -- make copies.
- **numpy in tests:** Not a core dependency. Tests that use numpy must be in `test_benchmarks.py` (which requires `[bench]`).

---

## 10. What Not To Do

- Don't add features nobody asked for. Solve the problem at hand.
- Don't refactor working code while implementing a feature. Separate commits.
- Don't add comments that restate the code. Comments explain WHY, not WHAT.
- Don't add error handling for impossible cases. Trust internal code.
- Don't create utility modules for one-time operations.
- Don't use `# type: ignore` to make type errors go away. Fix the types.
- Don't commit experimental scripts to `src/`. They go in `examples/`.
- Don't claim a benchmark result without the full evaluation pipeline committed.
- Don't inflate numbers in the README. If we have 273 rules, say 273 -- not "250+".

---

**Version:** 1.0
**Last Updated:** April 2026
