# MeshVault Viewer — Control API (full reference for AI agents)

> A self-describing, JSON-driven API for controlling MeshVault's embeddable 3D viewer with
> no human and no server. This document is the authoritative, single-fetch reference. The
> API is also self-describing at runtime via `listCommands()`, which returns the same
> parameter schemas described here.

================================================================================
## 1. Setup

The viewer core ships as a standalone ES-module bundle (`meshvault-viewer.js`) that
includes Three.js and has NO backend dependency.

Browser / module:
```js
import { createViewer } from "/static/dist/meshvault-viewer.js";
const mv = createViewer(document.getElementById("app"));
```
Non-module / agent bridge (global):
```js
const mv = window.MeshVaultViewer.createViewer(document.getElementById("app"));
```
Live harness for experimentation: open `/static/viewer.html`; `window.mv` is a ready
instance.

MCP (Model Context Protocol): if you are an MCP-connected agent, you don't need a
browser at all — the `meshvault-mcp` server hosts this same viewer headlessly and
exposes 9 tools: `load_model` (http(s) URL or absolute local file path; loads AND
returns a scene description in one call; multi-file OBJ/FBX/gltf assets load
textured), `describe_scene`, `viewer_execute`
({action, params} passthrough to every command in this document),
`list_viewer_commands`, `get_state`, `compare_models` (geometric 1-vs-N shape
registration), `screenshot` (returns real MCP image content; `best_view:true` for a
one-call hero shot; `preset:"studio"|"neutral"|"dark"` pins lighting/background so
renders are comparable across sessions), `open_in_app` (push your current model +
camera into the human's running `meshvault` app for live co-review; the app also
honors `?path=`/`?dir=` deep links), and `get_app_state` (read what the HUMAN is
looking at — path + camera — to continue their session headless). Install:
`pip install "meshvault[mcp]"`, `playwright install chromium`, then wire
`meshvault-mcp` into your client config. Full setup + behavior notes: docs/mcp.md.
Without MCP, the local server also offers `GET /api/screenshot` (PNG over plain
authenticated HTTP; same render presets) — see docs/api.md.

`createViewer(container, options)` returns an object with:
- `execute(command)` → Promise<{ok, result|error}> — the one command entry point.
- `getState()` → JSON snapshot (also available as the `get_state` command).
- `getSceneInfo()` → per-mesh/material info (also the `get_scene_info` command).
- `listCommands()` → array of { action, description, params } for every command.
- `on(event, cb)` → subscribe to events: `loaded`, `error`, `animations`, `measurement`,
  `executed`, `navmodechange`, or `*` for all.
- `loadFile(File)` → load a local File (drag-drop / file input) with no server.
- `destroy()` → fully release the WebGL context and listeners.

================================================================================
## 2. The command contract

Call: `await mv.execute({ action: "<name>", params: { ... } })`
Returns: `{ ok: true, result: <any JSON> }` or `{ ok: false, error: "<message>" }`.

Rules an agent can rely on:
- It NEVER throws. Failures are always `{ ok: false, error }`.
- Unknown actions and unknown params are rejected with a helpful message.
- Params are type-checked/coerced against the schema (number/boolean/string/array, with
  min/max/enum/required/default). Out-of-range or wrong-enum values return an error.
- Commands that need a loaded model return `{ ok:false, error:"... requires a loaded model" }`
  when none is loaded — so "did nothing because empty" is distinguishable from success.
- `screenshot` / `capture_views` / `turntable` / `export_glb` return image/geometry data as
  strings (PNG data URLs / OBJ text / base64 GLB) — always JSON-safe.

================================================================================
## 3. Observing state (no vision required)

- `get_state` → {
    model: { loaded, name, vertices, faces, dimensions{width,height,depth}, bounds{min,max,center,size}, scale, modified },
    NOTE: get_state.model.vertices counts UNIQUE welded positions; describe_scene and
    get_scene_info count position-attribute entries (seam duplicates included) — the two
    numbers legitimately differ on any mesh with UV/normal seams.
    camera: { mode, position[3], target[3], fov, presets[] },
    display: { wireframe, grid, axes, normals, background, renderMode, clip, fog },
    animation: { hasAnimations, clips[], playing, time, duration }
  }
- `get_scene_info` → { meshes:[{name, vertices, faces, materials[]}], materials:[{name,type,color,roughness,metalness}] }
- `get_bounds` → { min[3], max[3], center[3], size[3] } (world units) or null.

- `describe_scene { maxItems?:1..50=8, checks?=true, views?=false }` — THE recommended
  first observation after `load`: one token-bounded snapshot with everything needed to
  reason without vision. Returns:
  {
    loaded, summary,                       // 2–4 plain sentences, safe to show a human
    model: { name, format, vertices, triangles, meshCount, materialCount, textureCount,
             animated, animationClips[], dimensions{width,height,depth},
             bounds{min,max,center,size}, sizeHint, userScale, modified },
    hierarchy: { nodes:[{name,kind,depth}], totalNodes, truncated },   // outline, depth<=4
    meshes:    { items:[{id,name,triangles,vertices,center[3],size[3],materials[],
                         hasUVs,hasVertexColors?,skinned?}], omitted },  // largest first
                // id = stable mesh id (pass to `focus {id}`); center/size = world-space
                // placement of the part (skinned meshes report the bind pose)
                // materials items also carry: textures {slot:{width,height,colorSpace}}
                // and, when the viewer adjusted PBR values for preview, the asset's
                // ORIGINAL values in `authored` + modifiedByViewer:true — audit the
                // authored values, not the displayed ones.
    materials: { items:[{name,type,color,metalness,roughness,maps[],transparent,doubleSided}], omitted },
    issues:    [{ severity:error|warning|info, code, message, meshes?[] }],
    view:      { camera{position,target,fov,mode}, renderMode, environment, clip, grid },
    suggestedViews?: [{azimuth, elevation, score}]   // only when views:true
  }
  Notes: counts and dimensions are LIVE (recomputed from the current buffers, correct
  after simplify/rotate/reset). `vertices` sums position-attribute counts, the same basis
  as meshes.items[].vertices — seam-duplicated vertices count once per duplicate.
  Materials describe the ASSET even while a solid/normals render-mode override is active.
  `views:true` renders ~24 offscreen scoring views — expect seconds on software GL.
  Issue codes: missing_normals, missing_uvs (textured but no UVs — error), empty_meshes,
  unindexed_geometry, scale_tiny/scale_huge, nan_positions, degenerate_faces (relative
  sliver test on raw positions), not_watertight (open edges; counted on position-welded
  vertices so UV seams don't false-positive), non_manifold_edges, normals_maybe_flipped
  (signed-volume test on closed meshes), and checks_skipped when the scene exceeds the
  300k-triangle QA budget (report stays fast).
  With no model loaded it returns { loaded:false, summary } instead of an error.

- `sample_points { count?:16..20000=4096, seed?=42 }` → { count, seed, surfaceArea,
  points:[[x,y,z],...] } — deterministic, area-weighted surface samples in WORLD space
  (the geometric fingerprint used for comparison; same model+seed = same points).
- `get_mesh_stats` → numeric surface-quality statistics: per-mesh + total surface area,
  volume (NULL for open meshes — not computable reliably when the surface isn't closed),
  edge-length distribution (min/median/p95/max), sliver %, dihedral roughness (mean/p95
  angle between adjacent faces — a RELATIVE indicator for comparing iterations of the
  same asset; hard-edged models legitimately score high, e.g. a cube is 60° mean),
  open/non-manifold/degenerate counts, and `issuePoints`: representative world locations
  of defects to `focus {point}` on. Multi-mesh totals carry `approx:true` on median/mean
  fields (triangle-weighted; read per-mesh entries for precision). USE THIS to compare
  mesh iterations: connectivity QA alone can mislead (a topologically perfect mesh can
  be visual garbage). Skipped with `skipped:true` above 300k triangles.

After any command, re-read `get_state` to verify the effect. To SEE the result, call
`screenshot` and read the returned PNG data URL.

================================================================================
## 4. Loading

- `load { url:string, extension?:string, name?:string }` — load from a URL (extension
  inferred if omitted). Resolves only when the model is render-ready. Returns {stats, state}.
  Formats: .obj .fbx .gltf .glb .stl .ply .dae .3mf .usdz.
  Compressed glTF is supported transparently: Draco geometry, KTX2/Basis textures, and
  Meshopt (EXT_meshopt_compression). Decoders are bundled locally — no CDN, works offline.
- `unload` — clear the model, reset to an empty scene.
- Local files (no URL): use `mv.loadFile(file)` (not an execute command).

================================================================================
## 5. Finding the "front" of a model  (IMPORTANT)

There is no universal geometric "front" for an arbitrary mesh — it is semantic, and many
models are baked in odd orientations (lying down, facing an unexpected axis). Therefore:

- The presets `front/back/left/right/top/bottom` are WORLD-AXIS directions
  (front = +Z). They are convenient but WRONG for mis-oriented models.
- To find the real, semantic front, MEASURE it:

  `score_views { azimuths?:number[], elevations?:number[], size?:number, fill?:number }`
    → ranked [{ azimuth, elevation, score, coverage }], best first.
    Scoring is LIGHTING-INDEPENDENT and blends geometric detail (normal-material edges,
    good for panels/bezels) with albedo/texture detail (good for faces' eyes/mouth), so it
    works for both mechanical parts and organic/scanned models.

  `find_best_view { apply?=true, upright?=true, fill?, size? }`
    → moves the camera to the top-scored angle AND auto-uprights it (corrects camera roll
    so a lying-down model appears the right way up, without modifying the model). Returns
    { azimuth, elevation, score, coverage, ranked }.

  `auto_upright` — correct only the camera roll for the CURRENT view (uses left-right
  symmetry of the framed subject). Useful after a manual `orbit`/`set_view`.

================================================================================
## 5b. Exploring parts of a model  (focus)

After `describe_scene` you know each mesh's `id`, `name`, world `center` and `size`.
`focus` points the camera at one of them (or any world point) — including parts far too
small to see in the whole-model view (it rescales clip planes and zoom limits; a 1 cm
part on a 10 m assembly frames correctly):

  `focus { id?:number, name?:string, point?:[x,y,z], radius?:number, fill?:0.1..1 }`
    → { target:{kind,id?,name?}, center[3], size[3], distance, camera, note }

  - PREFER `id` (from describe_scene/get_scene_info): real-world mesh names are often
    meaningless ("mesh_0", UUIDs) or absent. `name` matches meshes AND groups
    (exact > case-insensitive > substring) and errors with candidates when ambiguous.
  - The view DIRECTION is kept; only target/distance change. The part may be OCCLUDED
    by surrounding geometry — combine with `set_clip {axis:'camera'}` or
    `set_render_mode wireframe` to see through, then `screenshot` to verify.
  - `orbit`/`set_view`/`frame`/`find_best_view` re-frame the WHOLE model (they do not
    know about the focused part) — call `focus` again afterwards if needed.
  - `reset_camera` restores the whole-model view and the original clip planes.
  - Skinned meshes: positions are the bind pose, not the animated pose.

Exploration recipe: `describe_scene` → pick parts by size/name/issues → `focus {id}` →
`screenshot` → repeat. For interior parts: `focus` + `set_clip {enabled:true,
axis:'camera', position:0.3}`.

Recommended agent recipe for a hero shot of an unknown model:
```js
await mv.execute({ action: "load", params: { url } });
await mv.execute({ action: "find_best_view", params: { fill: 0.85 } }); // front + upright
await mv.execute({ action: "set_background", params: { color: "#33373f" } });
const shot = await mv.execute({ action: "screenshot", params: { width: 1024, height: 1024 } });
// shot.result is a PNG data URL
```

================================================================================
## 5b. Scene composition (multi-object scenes)

`load` REPLACES the whole scene; `add_model` COMPOSES (same params as load, plus
`transform` for immediate placement and `frame:false` to keep the camera). The newest
object becomes ACTIVE — every single-object command (describe_scene, get_mesh_stats,
center/ground/rotate, focus, animation) targets the active object; `describe_scene`
adds a `scene` section (per-object summaries + totals) whenever objectCount > 1.

- `list_objects` → per-object {id, name, active, visible, opacity, transform, source}.
- `set_active_object { id }` — retarget single-object commands.
- `set_object_transform { id, position?:[x,y,z], quaternion?:[x,y,z,w] |
  rotation?:[x,y,z] Euler°, scale?:number | scale_xyz?:[x,y,z] }` — PLACEMENT lives
  on a per-object wrapper, never baked into vertices; `get_object_transform`,
  `reset_object_transform` read/clear it.
- `set_object_visible { id, visible }`, `set_object_opacity { id, opacity:0..1 }`
  (ghosting — display-only, exports keep authored materials), `remove_object { id }`.
- `frame_all` — frame the union of visible objects (presets/orbit frame the ACTIVE
  object only).
- `get_scene_manifest` — version-1 JSON {objects:[{source, transform, visible,
  opacity}], lighting, environment, background} for .mvscene persistence.
- Vertex-bake ops normalize the active object in its OWN frame and refuse skinned
  models. GLB export = all visible objects with placements applied.

================================================================================
## 6. Camera

- `set_view { preset: front|back|left|right|top|bottom|iso, fill?:0.1..1 }` — world-axis preset. `fill` = framing tightness (higher = tighter).
- `orbit { azimuth:deg, elevation?:deg=15, fill? }` — spherical angle around the model (azimuth 0 = +Z).
- `set_camera { position:[x,y,z], target?:[x,y,z], fov?:1..179 }` — explicit camera placement; mirrors get_camera so a pose captured in one session reproduces exactly in another.
- `frame { fill?, keep_direction?=true }` — fit the model; keeps the current view direction by default.
- `reset_camera` — restore the initial framed view (orbit mode).
- `set_nav_mode { mode: orbit|fpv }`.
- `get_camera` → { position, target, fov, mode, presets }.

================================================================================
## 7. Display, mesh inspection, cross-section

- `set_render_mode { mode: textured|solid|wireframe|normals }` — HOW the model is drawn:
    textured = mesh + texture (lit PBR surface, the default); solid = the mesh only
    (uniform matte, no texture — read pure form/topology); wireframe = edges only.
    Also 'normals' = per-face normal colors for geometry inspection.
    Aliases: shaded→textured, clay→solid.
- `set_wireframe { enabled }`, `set_grid { visible }`, `set_axes { visible }`,
  `set_normals { visible }` (vertex-normal lines).
- `set_clip { enabled, axis?:x|y|z|camera=camera, position?:0..1=0.5, flip?=false }` —
  cutting plane. 'camera' cuts relative to the current view (keeps the near side, cutting
  away geometry farther from the camera — i.e. "see only the front mesh"); x/y/z cut along
  model axes for cross-sections. `position` is normalized across the model bbox. Set
  `enabled:false` to clear.
- `set_fog { enabled, density? }` — exponential scene fog (off in hero captures by default).
- `set_background { color: "#rrggbb" }`, `set_scale { scale }`.
- `set_lighting { azimuth?, elevation?, key_intensity?, fill_intensity?, ambient?, exposure? }`
  — studio light rig for a hero look (degrees / multipliers; only provided fields apply).
  NOTE: with IBL on (the default), light-direction changes are visually subtle — call
  `set_environment {enabled:false}` first when you need a directional lighting sweep.
- `set_environment { enabled?, intensity?:0..5, asBackground? }` — image-based lighting
  (IBL): a procedural studio environment that gives PBR/metallic materials realistic
  reflections. ON by default at intensity 1 (the studio light rig above stays active as
  the baseline). `intensity` scales the environment contribution; `asBackground:true`
  shows the environment image behind the model (disabling IBL also clears it). In the
  matte `solid` render mode the environment is automatically suspended so the clay
  surface stays readable. Turn IBL off to reproduce the plain light-rig-only look.
- `get_environment` → { enabled, intensity, asBackground } (also in
  `get_state().display.environment`).

================================================================================
## 8. Transforms (mutate the model geometry)

- `center` — center the centroid at the origin.
- `ground` — drop so the lowest point sits on Y=0.
- `auto_orient` — PCA orient (NOTE: can worsen orientation for heads/faces; prefer
  `find_best_view` + `auto_upright`, or `rotate`).
- `rotate { axis: x|y|z, degrees:number }`.
- `simplify { ratio: 0.01..1 }` → { before, after } vertex counts (UV-preserving).
- `recompute_normals` — merge vertices + recompute smooth normals (UV-preserving).
- `reset` — undo all transforms (restore original geometry).

================================================================================
## 9. Animation

- `play_animation { index?=0 }`, `pause_animation`,
  `set_animation_time { seconds }`, `set_animation_speed { multiplier }`.
- Check `get_state().animation.hasAnimations` first; commands error cleanly if none.

================================================================================
## 10. Measurement

- `measure { a:[x,y,z], b:[x,y,z] }` → { distance } and draws the line.
- `set_measure_mode { enabled }` — interactive click-to-measure; disabling clears the overlay.
- `clear_measurement` — remove the measurement markers/line/label (do this before a
  clean screenshot; a prior `measure` otherwise stays visible in every capture).

================================================================================
## 11. Capture / hero shots

- `screenshot { width?, height?, transparent?=false, fog?=false, hideGround?=false, ssao?=true }`
    → PNG data URL. Explicit resolution (one dimension may be omitted, derived from aspect).
    `transparent` = alpha cutout for compositing. Fog is suppressed by default for clean shots;
    `ssao:true` renders through the SSAO/tone-mapping composer for hero quality.
- `capture_views { views?=["front","left","right","back"], width?=1024, height?=1024, transparent?, fill?, hideGround? }`
    → { <label>: <PNG data URL> }. `views` accepts preset names and/or {azimuth,elevation}.
    Auto-hides grid/axes and suppresses fog.
- `turntable { frames?=8, elevation?=15, width?=512, height?=512, fill?, transparent?, hideGround? }`
    → { azN: <PNG data URL> } evenly spaced around the model.

================================================================================
## 12. Export

- `export_obj` → OBJ text (string).
- `export_glb` → GLB as a `data:model/gltf-binary;base64,...` URL.

================================================================================
## 13. Events

`mv.on("loaded", d => ...)`, `"error"`, `"animations"`, `"measurement"`, `"executed"`,
`"navmodechange"`, or `"*"`. Handlers receive `(data, eventName)`.

================================================================================
## 14. Notes & limitations

- The standalone viewer is server-less; loading a URL with external textures needs those
  resources reachable (or inject `resolveResource(ref)=>url` in `createViewer` options).
- Transparent captures bypass the SSAO composer, so their tonality differs slightly from
  opaque captures of the same view.
- `find_best_view` handles azimuth + camera roll (upright). It does not permanently
  re-orient the model; use `rotate` if you need the geometry itself re-oriented.
