Metadata-Version: 2.3
Name: queryglot
Version: 0.1.0
Summary: One question, many query languages. Schema-aware natural-language search over your observability stack, as an MCP server.
License: MIT
Keywords: mcp,prometheus,elasticsearch,promql,observability,rag,llm
Author: Divyanshu Sharma
Author-email: sharmadivyanshu24@gmail.com
Requires-Python: >=3.11,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Provides-Extra: serve
Requires-Dist: fastapi (>=0.115,<0.116) ; extra == "serve"
Requires-Dist: langgraph (>=1.2,<2.0)
Requires-Dist: mcp (>=1.0,<2.0)
Requires-Dist: pydantic (>=2.9,<3.0)
Requires-Dist: uvicorn (>=0.32,<0.33) ; extra == "serve"
Project-URL: Repository, https://github.com/sharmadivyanshu/queryglot
Description-Content-Type: text/markdown

# queryglot

**One question, many query languages.** Schema-aware natural-language search
over your observability stack — Prometheus, Elasticsearch — and any
OpenAPI-described API, shipped as an MCP server any agent can plug into.

> "p95 latency by route" is easy. Knowing YOUR latency metric is called
> `http_server_request_duration_seconds` and carries a `route` label — that's
> the actual problem. Frontier models write fluent PromQL over metric names
> that don't exist.

## How it works

```
question ──> retrieve            ──> compile        ──> validate           ──> execute
             (BM25 + synonyms       (LLM, schema        (the backend's OWN     (real data,
              over YOUR schema,      slice in the        parser + unknown-      query shown)
              introspected live)     prompt)             metric check)
                    │                      ▲                   │ parse error
                    │ nothing matches      └─── bounded repair ┘
                    ▼
                 ABSTAIN — refuses to guess a metric name
```

- **Retrieval owns facts** (your metric names, labels, index fields —
  introspected from the live backend, never hallucinated). An LLM reranker
  orders the candidates by intent — over a closed set it can never add to.
- **The model owns syntax** (PromQL / Query DSL — swap in your own fine-tune
  via any OpenAI-compatible endpoint, including `mlx_lm.server` on a Mac).
- **The backend owns truth**: every query is validated by the server's own
  parser (`format_query`, `_validate/query`) before execution, and parse
  errors drive a bounded repair loop.
- **Abstention is a feature**: off-schema questions get a refusal, not an
  invented metric. The eval scores this.

## Use it from any MCP client

```json
{
  "mcpServers": {
    "queryglot": {
      "command": "queryglot-mcp",
      "env": {
        "QUERYGLOT_PROMETHEUS": "http://localhost:9090",
        "QUERYGLOT_ELASTIC": "http://localhost:9200",
        "QUERYGLOT_OPENAPI": "http://localhost:8081/api/v3",
        "QUERYGLOT_LLM_URL": "http://localhost:11434/v1",
        "QUERYGLOT_LLM_MODEL": "qwen3.5:4b"
      }
    }
  }
}
```

Tools exposed: `search(question, backend?)`, `list_schema(query?)`,
`refresh_schema()`.

Or the CLI:

```bash
queryglot "p95 http request duration" --prometheus http://localhost:9090
```

Or run an HTTP server with the ask-widget and query playground:

```bash
pip install "queryglot[serve]"  # or: poetry install --extras serve
queryglot-serve --prometheus http://localhost:9090
```

`queryglot-serve` also serves the query playground at `/` and the embeddable
ask-widget bundle at `/widget.js` — see "Embed the ask-widget" below.

The playground is a small Discover-style console: your schema in a filterable
rail (type badges, prefix groups, and the items the last answer actually
used), a time-range picker whose window runs as a real `query_range` — the
window comes from the picker, never the model, so the compile prompt stays
byte-stable — and results as a bar chart or histogram with the raw rows one
toggle away. Answered questions also get a one-sentence conversational
summary, grounded strictly on the returned data (never computed, never
invented — an empty summary beats a wrong one), and repeat questions are
served from a short answer cache with an honest `cached Ns ago` tag and a
fresh re-run button.

Environment variables (serve only):
- `QUERYGLOT_SERVE_TOKEN` — bearer token for `/api/*` endpoints. Empty = open (intended for localhost/demo).
- `QUERYGLOT_CORS_ORIGINS` — comma-separated allowed origins for embedding.

Any OpenAI-compatible endpoint works as the model: OpenAI, Ollama, or your own
LoRA behind `mlx_lm.server` — that last one is the point of `finetune/`.

## Embed the ask-widget

`queryglot-serve` ships a self-contained widget: a floating "Ask" pill that
opens a search panel wired to the same schema-grounded engine as the CLI and
MCP server. Drop one script tag on any page:

```html
<script
  src="https://your-queryglot-host/widget.js"
  data-api="https://your-queryglot-host"
  data-theme="auto"
  data-token="optional-bearer-token"
  data-backend="optional-backend-name"
></script>
```

- `data-api` (required) — base URL of the queryglot HTTP API.
- `data-theme` — `light`, `dark`, or `auto` (default; follows the host page's `prefers-color-scheme`).
- `data-token` — bearer token, only needed when the server sets `QUERYGLOT_SERVE_TOKEN`.
- `data-backend` — pins searches to one backend instead of auto-routing.

See `frontend/README.md` for the build (`npm run build:all`) that packages
the widget and playground into the Python wheel.

## Evaluation — deterministic, no LLM judge

`eval/run_eval.py` scores golden questions against a live backend: the
outcome must match, required metrics must appear in the query, and the query
must actually execute. Abstention cases score correct only on refusal.
`eval/docker-compose.yml` brings up real backends; CI runs the full
integration suite against a real Prometheus and petstore on every push.

## Status

**v0.1.0** — the RAG arm, working end to end.

- [x] Prometheus + Elasticsearch backends (introspect / validate / execute)
- [x] BM25 + synonym schema retrieval, exact-name boosting
- [x] compile -> validate -> repair -> execute LangGraph with abstention
- [x] MCP server + CLI; 141 backend tests (live-Prometheus and
      live-petstore integration included, always exercised in CI) plus 82
      frontend tests; CI gates on all of it
- [x] Verified NL->PromQL dataset generator (parse+execute gated, metric-disjoint splits)
- [x] Bake-off complete — RAG 8/10, FT-only 3/10, FT+RAG 9/10 on the same
      golden set; full analysis in `finetune/README.md`, build history and
      bugs in `DESIGN_NOTES.md`
- [x] OpenAPI backend — read-only, GET-only by construction; validated
      against the spec's own contract; petstore-verified in CI
- [x] HTTP serve layer: answer cache, grounded conversational summaries,
      structured schema API, bearer auth — engine outcomes are always 200
      payloads (an abstention is an answer, not an error)
- [x] Embeddable ask-widget (~15 KB gz, Shadow-DOM, one script tag) +
      Discover-style playground: schema rail, time-range windows via
      `query_range`, histogram/bar-chart results with rows one toggle away
- [ ] Loki (LogQL) backend; Datadog connector

## Where this is going: apps that agents can actually use

The `Backend` protocol (introspect / validate / execute) is not
observability-specific. The same loop pointed at a product's own OpenAPI spec
or database turns any app into something an AI can query *safely*:

- **OpenAPI backend — shipped.** Introspects a product's own spec into the
  catalog; questions compile into validated, GET-only API calls. Existing
  OpenAPI->MCP generators dump every endpoint as a tool, which measurably
  degrades agents (arXiv 2411.15399) and executes whatever the model asks.
  queryglot's contribution is the missing layer: schema-grounded retrieval,
  server-side validation, and abstention.
- **Customer-facing ask widget** — an embeddable search box backed by the
  same engine: visitors' questions become validated queries against the
  app's data, never hallucinated ones.
- **`llms.txt` + MCP endpoint generation** — one schema catalog, two
  audiences: humans get the widget, agents get a typed, validated interface
  instead of scraping. Discoverability for the agentic web, with execution
  semantics — not just markup.

## Known limits

- `metric_candidates` (unknown-metric detection) is regex-based and
  best-effort; the backend parser owns syntax, this only improves error
  messages. Complex PromQL may slip past it — never through the parser.
- Backend auto-routing is retrieval-strength-based; ambiguous questions
  ("errors in checkout") can route to the wrong store. Pass `backend=` to pin.
- Synonym table is small and English-only, grown from eval failures.
- When a backend's catalog is smaller than the retrieval k (8), retrieval
  sends the whole catalog and the abstention gate rarely fires — abstention
  then rests on the validation layer.

## License

MIT

