Metadata-Version: 2.4
Name: navigator-cli
Version: 0.1.2
Summary: Navigator CLI + Data Navigator service: query public data sources from your agent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: fastapi<1,>=0.115
Requires-Dist: httpx<1,>=0.27
Requires-Dist: jinja2<4,>=3.1
Requires-Dist: keyring<26,>=25
Requires-Dist: markdown>=3.10.2
Requires-Dist: navigator-contracts==0.1.1
Requires-Dist: pydantic<3,>=2.7
Requires-Dist: pyyaml<7,>=6
Requires-Dist: uvicorn[standard]<1,>=0.30

# Data Navigator

Members-only catalogue of skill files that teach AI agents to query public
databases, plus the executor that runs the queries. Three faces — web UI, the
`navigator` CLI, and (Phase 3) MCP — all over one HTTP API. See `SPEC.md` for
the full design; this README covers running and validating the local app.

Data Navigator finds and queries individual structured public-record sources.
Workflows Navigator uses complex OSINT workflows to return structured data
assembled from several sources.

## Layout

The skill tree is the single source of truth:

```text
skills/<jurisdiction>/<source-slug>/
├── SKILL.md     # agent-facing playbook: inputs, output shape, gotchas
├── meta.yaml    # validated catalogue record (app/skills_registry.py)
└── adapter.py   # run(input, ctx) — executes the query
```

- `skills/_template/` — copy this to add a source; every field is documented.
- `skills/_meta/navigator-data/` — the meta-skill members drop into
  `~/.claude/skills/` so their agent knows the find → show → query loop.
- `app/skills_registry.py` — walks the tree, validates every `meta.yaml`
  (pydantic), builds the registry. Invalid skill dirs fail the app at startup.
- `app/runner.py` — dynamic adapter loading + `AdapterContext.get_key()`
  (OS keychain / `NAVIGATOR_KEY_<NAME>` env; keys never enter agent context).

Source runtimes: `local` (public or BYO-key, runs on the member's machine,
unmetered) and `hosted` (our key, runs server-side, metered — e.g.
OpenSanctions).

## Run locally

```bash
uv sync
uv run uvicorn app.main:app --reload --port 8000
```

Open `http://localhost:8000/data`. Set `OPENROUTER_API_KEY` to enable the
natural-language query bar (`/api/query/natural` returns 503 without it —
there is no fallback planner).

## Run with Docker

```bash
docker compose up --build
```

## CLI

One unified `navigator` CLI for the whole product: `navigator data …` is Data
Navigator, `navigator tools …` (Phase 3) is osint-navigator.

```bash
navigator auth login                   # magic link → PAT in the OS keychain
navigator auth status                  # tier + quota state
navigator data find company            # search the catalogue
navigator data show no/brreg/enheter   # print the source's SKILL.md + meta
navigator query no/brreg/enheter --input '{"navn":"Equinor","size":3}'
navigator query no/brreg/enheter --input '{"navn":"Equinor"}' --out companies.csv
navigator query <source-id> --input-file request.json --out response.json
navigator tools find "satellite imagery"   # OSINT tool search (osint-navigator API)
navigator tools show <tool-id>             # full tool record + documentation
navigator keys missing                 # BYO keys you still need to set
navigator keys set companies-house     # store a key in the OS keychain
```

For a headless, Docker, or CI installation, sign in at
`https://navigator.indicator.media/data`, open **Connect an Agent → API Keys**,
generate a key, and inject it through the runtime's secret manager as
`NAVIGATOR_PAT`. Do not put the key in an agent prompt, `CLAUDE.md`, shell
arguments, or source control. `NAVIGATOR_PAT` takes precedence over the OS
keychain. `navigator auth import --stdin` is an installer-to-keychain bridge,
not a keychainless authentication path; environment credentials must be
revoked from the API Keys modal rather than with `navigator auth logout`.

`--input-file` is the safe path for arbitrary text or larger request bodies;
it avoids interpolating user content into a shell command. `--out
<file>.json|.csv` writes the records to the file and prints a compact summary
instead of the full payload. Hosted sources route to the Data Navigator server
(`DATANAV_BASE_URL`) with the stored PAT.

There is no install step: in **remote** mode the CLI fetches skill bundles from
the authed API and caches them under `~/.navigator/cache/` (re-fetched when a
source's meta changes). Mode is **auto-detected** — a packaged install (no local
`skills/` tree) defaults to remote so `navigator query …` works out of the box
for members; an in-repo checkout defaults to local for dev and smoke tests.
Set `DATANAV_CATALOGUE=remote|local` to override. The only thing a member
installs is the meta-skill.

## Membership (Phase 2)

Everything is paid-member-gated in production (`DATANAV_AUTH=on`; default off
for local dev). Auth is owned by osint-navigator — this service validates
Bearer PATs / the shared `osint_session` cookie via its internal introspection
API and requires the shared `data_sources` capability for Data Navigator's web,
CLI/remote source-skill catalogue, and MCP surfaces. Through 2026-09-05 that
capability is promotional for Pro and Lab members; it then returns to Lab-only
alongside `desktop_download`. Metering: UI NL queries + try-it burn `data_ui`;
hosted-source queries burn `data_hosted`; public + BYO local execution is never metered.
Quota storage lives in osint-navigator.

Cross-service proof: `uv run python scripts/integration_check_membership.py`
(boots osint-navigator's internal API from the sibling checkout and verifies
401/403/tier/quota behavior end-to-end).

## MCP, meta-skill, rollout

- **MCP**: `data_find_source` / `data_show_source` / `data_query` live on
  osint-navigator's remote MCP and proxy here over the service channel. Local
  sources are unmetered over MCP (the member's agent reasons; upstream is free).
- **Meta-skill**: members download it from the UI ("Agent setup") or
  `GET /api/meta-skill` and drop it into `~/.claude/skills/navigator-data/`.
- **Rollout**: the full Tools/Data/Skills/Workflows toggle is public. Data
  remains fail-closed behind the member gate; Skills and Workflows are disabled
  as coming soon. `/data` is reverse-proxied from osint-navigator when
  `DATA_NAVIGATOR_URL` is set.
- **Deploy and September rollback**: `../docs/deploy-runbook.md`.

## API

```bash
curl -s "http://localhost:8000/api/registry"
curl -s "http://localhost:8000/api/sources/no/brreg/enheter"   # meta + skill_md
curl -s -X POST "http://localhost:8000/api/query/no/brreg/enheter" \
  -H "Content-Type: application/json" -d '{"navn":"Equinor","size":3}'
```

## Validation

First, validate the skill tree (both smoke scripts assume this passes):

```bash
uv run python scripts/validate_skills.py
```

The validation gate is three live public APIs returning records:

- `no/brreg/enheter` — Norway Brønnøysund companies
- `us/usaspending/awards` — USAspending federal awards
- `global/gleif/lei-records` — GLEIF LEI records

```bash
uv run pytest
DATANAV_BASE_URL=http://localhost:8000 uv run python scripts/smoke_public_apis.py
NAVIGATOR_CMD="uv run navigator" uv run python scripts/smoke_cli_public_apis.py
bash scripts/smoke_docker_image.sh
```

## Deployment

`render.yaml` defines a Docker web service (health check `/health`, port from
`$PORT`). Server secrets: `OPENROUTER_API_KEY` (NL planner) and
`NAVIGATOR_KEY_THINKPOL` (hosted partner source). Arbiter is a BYO-key source:
members register through Indicator's attributed partner link in the Arbiter
source metadata, then store their own key with `navigator keys set arbiter`.
`ARBITER_API_BASE` optionally selects the matching Arbiter deployment and
defaults to production.

Phase 2 (membership wiring: auth introspection against osint-navigator,
metered quotas, submissions form) and Phase 3 (MCP, catalogue growth) are
specified in `SPEC.md` §9.

The implemented `global/arbiter/case-studies` partner source runs locally with
the member's Arbiter key. Spotlight uses the Navigator CLI and saves its
top-level-compatible report response for deterministic Arbiter analytics
rendering. Charged or mutating Arbiter operations must use an explicit
structured query; the natural-language planner cannot approve them.
