Metadata-Version: 2.4
Name: pyxel-config-core
Version: 0.5.0
Summary: Validation and knowledge core for ESA Pyxel configs, callable as a library or an MCP server.
Author: Doby Baxter
License-Expression: MIT
Project-URL: Homepage, https://pyxel-config-lab-ede25c.gitlab.io
Project-URL: Repository, https://gitlab.com/dobybaxter127/pyxel-config-lab
Project-URL: Issues, https://gitlab.com/dobybaxter127/pyxel-config-lab/-/issues
Project-URL: Documentation, https://gitlab.com/dobybaxter127/pyxel-config-lab/-/blob/main/pyxel-config-core/README.md
Project-URL: Pyxel (upstream), https://esa.gitlab.io/pyxel/
Keywords: pyxel,esa,detector-simulation,mcp,model-context-protocol,json-schema,validation,yaml,astronomy
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jsonschema>=4.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: pydantic>=2.0
Provides-Extra: mcp
Requires-Dist: mcp>=1.2; extra == "mcp"
Provides-Extra: rag
Requires-Dist: qdrant-client>=1.7; extra == "rag"
Requires-Dist: fastembed>=0.3; extra == "rag"
Requires-Dist: numpy>=1.24; extra == "rag"
Provides-Extra: checks
Requires-Dist: pyxel-sim<4,>=3.0; extra == "checks"
Requires-Dist: pyyaml>=6.0; extra == "checks"
Requires-Dist: pytest>=7; extra == "checks"
Dynamic: license-file

# pyxel-config-core 🪐🔭

[![PyPI](https://img.shields.io/pypi/v/pyxel-config-core?color=06B6D4&label=PyPI)](https://pypi.org/project/pyxel-config-core/)
[![Python](https://img.shields.io/pypi/pyversions/pyxel-config-core?color=9333EA)](https://pypi.org/project/pyxel-config-core/)
[![License](https://img.shields.io/badge/license-MIT-F5C518)](https://gitlab.com/dobybaxter127/pyxel-config-lab/-/blob/main/pyxel-config-core/LICENSE)

Validation and knowledge core for **ESA Pyxel** configurations — callable as a
plain Python library *or* as an **MCP server** that any LLM assistant can use as
a tool.

It reuses the same source-of-truth artifacts as
[Pyxel Config Lab](https://pyxel-config-lab-ede25c.gitlab.io): the Pyxel JSON
schema, the model-parameter catalogue, and the tutorial library. The browser GUI
and this core stay in lockstep because they read the same bundled contract.

---

## What it exposes

| Capability | Function | MCP tool | CLI |
|---|---|---|---|
| Validate a config against the schema | `validate_config(config)` | `validate_pyxel_config` | `pyxel-config validate` |
| Friendly diagnostics only | `explain_config(config)` | `explain_pyxel_config` | `pyxel-config explain` |
| Auto-repair a config (rename / re-stage) | `autofix(config)` | `fix_pyxel_config` | `pyxel-config fix` |
| List valid models (optionally per stage) | `list_models(stage=None)` | `list_pyxel_models` | `pyxel-config models` |
| Fetch a tutorial | `find_tutorial(group, model)` | `get_pyxel_tutorial` | `pyxel-config tutorial` |
| Keyword-search tutorials | `search_tutorials(query)` | `search_pyxel_tutorials` | `pyxel-config search --keyword` |
| Semantic (vector) tutorial search | `semantic_search_with_fallback(query)` | `semantic_search_pyxel_tutorials` | `pyxel-config search` |
| Model parameters + defaults | `model_parameters(model)` | `get_pyxel_model_parameters` | `pyxel-config params` |

Three front doors onto one implementation. Whatever the CLI says about a config,
the MCP tool says too.

Every input and output is a Pydantic model, so results are structured and
schema-checked on both sides of the boundary.

`validate_config`, `explain_config`, and `autofix` all accept either a config
`dict` or a raw **YAML string**, so an assistant can pass a user's pasted
`.yaml` straight through without parsing it first.

---

## Install

```bash
pip install pyxel-config-core              # library only
pip install "pyxel-config-core[mcp]"       # + MCP server
pip install "pyxel-config-core[rag]"       # + semantic search (Qdrant + fastembed)
pip install "pyxel-config-core[mcp,rag]"   # everything
```

Working on the package itself? Clone the repo and install it editable
instead — `pip install -e ".[mcp,rag]"` from `pyxel-config-core/`.

Requires Python 3.10+.

---

## Command line

<p align="center">
  <img src="https://gitlab.com/dobybaxter127/pyxel-config-lab/-/raw/main/assets/12-cli-overview.png" alt="The pyxel-config command line: the ESA-blue wordmark over the command list, with validate, explain, fix, models, params, tutorial, search, index, mcp, doctor and banner" width="900" />
</p>

```bash
pyxel-config                     # the wordmark, and what it can be asked to do
pyxel-config validate my.yaml    # check it against the bundled schema
pyxel-config explain  my.yaml    # the same checks, with did-you-mean suggestions
pyxel-config fix      my.yaml -w # repair mistyped and misplaced model names
pyxel-config models charge_generation
pyxel-config params  load_image
pyxel-config search  "dark current"
pyxel-config doctor              # what is installed, bundled, and missing
```

`pyxel-config mcp` runs the same server as `pyxel-config-mcp`, so an assistant
and a person are reaching identical code.

**The exit code is the verdict**, which is what makes this usable in CI without
parsing any output:

| Code | Meaning |
|---|---|
| `0` | valid, or nothing to complain about |
| `1` | the config has errors |
| `2` | the command line itself was wrong |
| `3` | could not run: unreadable file, unparseable YAML, missing extra |
| `4` | no such tutorial, model or stage |

```yaml
# .gitlab-ci.yml
validate-configs:
  image: python:3.12-slim
  script:
    - pip install --quiet pyxel-config-core
    - pyxel-config validate configs/*.yaml
```

**stdout carries the answer, stderr carries the furniture.** The banner and any
progress note go to stderr, so a pipe stays clean with no flag at all:

```bash
pyxel-config models --json | jq -r '.models[].func'
pyxel-config fix my.yaml -o - > repaired.yaml     # config out, report on stderr
```

Colour follows ESA: institutional blue and white carry the frame and the labels,
and red and amber are kept for the things that are actually wrong. Palette
entries are named for the job they do, not the hue, so a repaint never reaches
a call site. It steps down on its own: truecolor to 256 to 16 to none,
Unicode glyphs to ASCII when the output encoding cannot carry them, and a
wordmark that shrinks through two smaller lockups rather than wrapping. `--json`,
`--no-color`, `--ascii`, `--banner` and `-q` override any of that;
[`NO_COLOR`](https://no-color.org) is honoured.

---

## Library usage

```python
from pyxel_config_core import (
    validate_config, autofix, list_models, find_tutorial, model_parameters,
)

result = validate_config(open("my_config.yaml").read())
for d in result.diagnostics:
    print(d.severity, d.path, d.message)

# Discover what models exist before building a config
for m in list_models("charge_generation").models:
    print(m.func)

# Repair typo'd names and misplaced models in one call
fix = autofix(open("my_config.yaml").read())
for change in fix.changes:
    print(change.action, change.path, change.before, "->", change.after)

tut = find_tutorial("charge_generation", "dark_current")
params = model_parameters("load_image")
```

A HoloViz/Panel assistant (or any Python app) can `import` this directly — no
server, no network, no access to anything private.

---

## Semantic search (RAG)

The tutorial catalogue is split into ~850 heading-level passages, embedded with a
local **fastembed** model (ONNX, CPU, no API keys), and stored in an embedded
**Qdrant** index — a real vector database that runs in-process, with no server to
operate.

Build the index once (downloads the embedding model on first run):

```bash
pip install -e ".[rag]"
pyxel-config-index
```

Then query it:

```python
from pyxel_config_core import semantic_search

for m in semantic_search("how do I add dark current to the detector", limit=5):
    print(f"{m.score:.3f}  {m.group}/{m.model}  §{m.section}")
```

Once the index is built, `search_tutorials(query)` automatically uses it and
falls back to keyword search if the index or the RAG extra is absent. Point the
index elsewhere with `PYXEL_CONFIG_CORE_INDEX=/path/to/dir`.

Scale note: at ~850 passages a brute-force index would also work; Qdrant is used
because it is a standard vector store, scales cleanly, and here runs embedded —
so the retrieval design is production-shaped without a production dependency.

---

## MCP server usage

```bash
pyxel-config-mcp          # runs over stdio
pyxel-config mcp          # the same server, via the CLI
```

Point any MCP-compatible client at that command. The eight tools above become
callable by the model, so an assistant can *check* a config against the real
schema instead of guessing, *repair* it, *discover* the valid models, and
*retrieve* the real tutorial passages to cite.

The server imports cleanly on both the 1.x (`FastMCP`) and 2.x (`MCPServer`)
line of the MCP SDK.

### Building and fixing configs

Two tools turn the server from a read-only checker into an active assistant:

- **`list_pyxel_models(stage=None)`** enumerates every valid model function,
  optionally filtered to one pipeline stage. It reads the same schema the
  validator does, so the list is always the real, current set — an assistant can
  *discover then build* instead of guessing a name and validating it. Each entry
  carries the full `func:` path ready to drop into a config.

- **`fix_pyxel_config(config)`** applies the func diagnostics automatically. It
  renames a typo'd model to its nearest valid match, moves a valid-but-misplaced
  model into its owning stage, and returns the corrected config alongside a
  structured change log (`changes`), anything it could not fix (`unresolved`),
  and the diagnostics that remain after the repair (`remaining_diagnostics`).
  It fixes `func` names only — it never invents required arguments, so a rename
  onto a model with required args will still surface those in
  `remaining_diagnostics`. The input is never mutated; a corrected copy is
  returned.

### Graceful semantic search

`semantic_search_pyxel_tutorials` returns a `TutorialSearchResult` with a `mode`
(`"semantic"` or `"keyword"`), a `degraded` flag, and the `matches`. If the
vector index has not been built or the `[rag]` extra is not installed, it does
not error — it falls back to keyword search and flags the result as degraded, so
a client always gets usable matches with a clear signal about which engine
answered.

---

## The shared contract

Bundled under `pyxel_config_core/data/`:

- `pyxel_schema.json` — the validation contract
- `modelParameters.json` — model parameters and defaults
- `tutorials/` — the tutorial catalogue

These are copied from the Config Lab repo. Keep them refreshed with the same
freshness check the GUI already uses (`check_schema_version.py`,
`buildTutorialManifest.js`) so both consumers track upstream Pyxel together.
Rebuild the vector index (`pyxel-config-index`) whenever the tutorials change.

---

## Func diagnostics

Model-function names in the `pipeline` are checked against the schema itself
(the single source of truth), not a separate list, so every stage is covered.
When a `func` is wrong, `validate_config` now:

- collapses the schema's downstream argument explosion (one bad func could
  otherwise surface hundreds of misleading "X is a required property" errors
  from the branches the validator tried) down to a single diagnostic;
- distinguishes a **typo** from a **valid model in the wrong stage**, and points
  the latter at the stage it belongs to;
- offers a **"did you mean"** suggestion (Levenshtein over the leaf name) and a
  short list of valid models for that stage, carried as structured
  `suggestion` and `valid_options` fields on the `Diagnostic`.

This is the Python port of the browser GUI's `modelDiscriminator.js`, so both
consumers reason about `func` values the same way.

## What validation covers

The core validates exactly what the schema encodes — no more, no less — because
the schema is the single contract shared with the GUI. That already includes a
fair amount of value checking, not just structure:

- **types** and **required arguments** per model;
- **enums** — a bad choice string is rejected against the allowed set
  (e.g. `single` / `exponential` / `binomial` / `sbx`);
- **numeric bounds** (`minimum` / `maximum` / `exclusiveMinimum`) and
  **array-length bounds** (`minItems` / `maxItems`) wherever the schema declares
  them;
- **func discriminators**, with the friendly typo / wrong-stage diagnostics
  described above.

## Deliberately out of scope

Only checks that live *above* the schema are not attempted here — by design, so
there is never a second source of truth to drift from the GUI:

- **Super-schema constraints.** Cross-field consistency (e.g. one argument that
  must stay `<=` another), physical plausibility, and any bound Pyxel enforces
  at *runtime* but that isn't written into the JSON schema. These pass here and
  are caught only when Pyxel actually runs. The fix, if ever wanted, is to
  encode the rule in the schema so both consumers gain it at once — not to add a
  separate checker.
