Metadata-Version: 2.4
Name: specjam
Version: 0.4.0
Summary: A harness-neutral engineering meta-harness with governed memory, execution, diagnosis, replay, and safe evolution.
Author: Fernando Costa
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/nandoligeiro/SpecJam
Project-URL: Repository, https://github.com/nandoligeiro/SpecJam
Project-URL: Issues, https://github.com/nandoligeiro/SpecJam/issues
Keywords: agents,engineering-workflow,flow-graphs,rws,sqlite,vector-memory,cli
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: local
Requires-Dist: fastembed<0.9,>=0.8; extra == "local"
Requires-Dist: sqlite-vec<0.2,>=0.1.8; extra == "local"
Dynamic: license-file

# SpecJam

SpecJam is an installable, harness-neutral engineering meta-harness. It turns a natural-language request into a durable, auditable flow and coordinates isolated execution sessions instead of allowing an agent to jump directly from ambiguity to implementation.

The first release combines three ideas:

- **Flow graphs**: declarative stages, artifact gates, conditional routing, bounded reviewers, and a pure routing function.
- **RWSA skills**: Routing, Workflow, Semantics, and Attachments as an executable contract for reusable agent capabilities.
- **A local workspace**: a standard-library-only CLI that installs, verifies, inspects, updates, and scaffolds the method in any repository.
- **A meta-harness runtime**: increment-scoped sessions, harness adapters, versioned skill resolution, independent reviewer sessions, and auditable execution plans.
- **Governed memory**: migratable SQLite records, vector and lexical recall, explainable ranking, lifecycle feedback, bounded context, and mandatory provenance.
- **Governed learning**: evaluated experience, explicit reflection candidates, and deterministic promotion into episodic or semantic memory.
- **Adaptive harness composition**: task-aware memory routing, versioned runtime policies, bounded optimization, and regression-gated evolution.
- **Portable execution**: normalized Codex and Claude Code CLI adapters plus a Devin v3 API adapter, without vendor SDKs in the core.
- **Diagnosis and replay**: evidenced failure attribution, immutable trajectories, exact-request replay, and comparative harness benchmarks.

## Quick start

Run SpecJam directly from PyPI with `uvx`, or install it persistently with `uv tool`:

```bash
uvx specjam install
uv tool install specjam
specjam install
```

To pin the current release explicitly:

```bash
uvx --from 'specjam==0.4.0' specjam --help
```

For a source checkout, `uv run` keeps the package isolated and reproducible:

```bash
uv run specjam install
uv run specjam verify
uv run specjam inspect
uv run specjam classify "Add a new payment capability"
uv run specjam flow scaffold --flow delivery --slug payment-capability
uv run specjam memory init --db .specjam/memory/specjam.db --dimensions 3
uv run specjam memory calibrate --db .specjam/memory/specjam.db --dimensions 3 --cases examples/memory-calibration.json
```

For automatic, fully local embeddings and `sqlite-vec` acceleration:

```bash
uv tool install 'specjam[local]'
specjam memory prepare
specjam memory init
specjam memory add --kind procedure --content "Validate contracts before migration" \
  --source-ref trail://delivery-42/inc-4
specjam memory search --text "How should I migrate this API?"
```

`memory prepare` is the only step allowed to download the ONNX model. Normal
indexing and retrieval use cached model files and do not access the network.

The installer creates `.specjam/` and a minimal root `AGENTS.md` bridge. Existing bridge files are preserved unless `--force` is supplied. Runtime state is ignored; the lockfile and managed definitions remain inspectable.

## Architecture

```text
  natural language -----> discovery graph
       |
       +-----------------> delivery graph
       |
       +-----------------> postmortem graph

graph + state --> pure route --> decision --> append-only trail
                                      |
                                      +--> bounded read-only reviewers
                                      |
                                      +--> one synthesis writer

increment --> session manager --> execution harness
                   |                    |
                   +--> isolated reviews
                   +--> versioned skills

task + graph + experience --> harness planner --> versioned runtime harness
                                      |
                                      +--> candidate --> evolution gate

session request --> execution adapter --> normalized outcome --> diagnosis
      |                                      |
      +---------- immutable trajectory <-----+
                         |
                         +--> replay --> comparative benchmark
```

The route function never writes files, invokes tools, or calls a model. Persistence belongs to the trail adapter. This split makes the highest-risk policy easy to test.

## First-class flows

SpecJam keeps each flow explicit and data-driven:

- **Discovery** turns uncertainty into a problem statement, evidence, options, and a decision.
- **Delivery** runs SDD: `SPEC → DESIGN? → BUILD → VALIDATE`.
- **Postmortem** turns an incident record into root cause, actions, and follow-up.

The daily engineering loop is a supporting L0–L3 classification mechanism, not a fourth flow graph.

The three graphs are `graphs/discovery-graph.json`, `graphs/delivery-graph.json`, and `graphs/postmortem-graph.json`; organizations can add or replace graphs without changing the routing engine.

## Meta-harness runtime

Every verifiable increment may create one implementation session, zero or more isolated reviewer sessions, and one controlled synthesis decision. Session policy is declared in the graph:

```json
{
  "session_policy": {
    "strategy": "new_per_increment",
    "harness": "default",
    "max_retries": 2
  },
  "skills": [
    "ligeiro-mindware/learning-domain-driven-design@latest"
  ]
}
```

The core exposes an `ExecutionHarness` protocol rather than depending on a
vendor SDK. `CommandExecutionHarness` invokes Codex or Claude Code with explicit
argv and no shell interpolation. `DevinExecutionHarness` uses the Devin v3
organization-session API with an injected token provider, so credentials are
never stored in a trajectory. Every adapter returns the same `ExecutionOutcome`
contract with status, evidence, usage, duration, and summary.

Before creating a session, `HarnessPlanner` classifies the task, routes memory
proportionally, then composes a content-addressed runtime configuration. The
session records the exact harness version and policy. Proposed changes are
bounded and must pass `EvolutionGate`; no execution may rewrite the accepted
harness merely because it produced a plausible reflection. See
[Harness evolution](docs/harness-evolution.md).

Supported session strategies are `reuse`, `new`, `new_per_increment`, `isolated`, `parallel`, and `exclusive`.

## Execution, diagnosis, and replay

`specjam execution run` accepts a serialized `SessionRequest` and runs it with
`--provider codex`, `claude`, or `devin`. The generated prompt preserves the
objective, role, policy, resolved skills, required artifacts, and cited memory
without copying arbitrary environment variables or credentials into the result.

Failed outcomes pass through `DiagnosisEngine`. Its typed classes distinguish
context, tool, planning, implementation, validation, constraint, transient, and
harness failures. A diagnosis is explicitly a confidence-scored hypothesis; it
can create a reflection candidate but cannot write memory directly.

Evaluated executions can be captured in an append-only `TrajectoryStore` and
replayed through a different adapter. Candidate trajectories retain `replay_of`
lineage. `specjam benchmark compare` matches this lineage and reports quality,
constraint, regression, generalization, cost, latency, and human-intervention
deltas. See [Execution, diagnosis, and replay](docs/execution-replay.md).

## Selective SQLite vector memory

SpecJam can recall a small number of cited decisions, failures, recoveries, procedures, and outcomes before planning an implementation session. It combines SQLite FTS5, `sqlite-vec` cosine KNN when the local extra is installed, and graph/stage/role filters. A portable exact-cosine backend remains available as fallback. Reviewer sessions remain unprimed by default.

Every recall is traced with its candidates, selected IDs, score signals, latency,
and context budget. Evaluation may report exactly which memories were consumed;
repeated success promotes `validated` memory to `trusted`, while repeated failure
deprecates it and removes it from default recall. Project and repository filters
keep local experience from leaking across unrelated workspaces.

Credential-shaped content is rejected before persistence; detection errors expose
only the marker type and never echo the suspected secret.

The database is a rebuildable projection; accepted artifacts and append-only trails remain the source of truth. Embedding providers are adapters, so the dependency-free core sends no data to a model vendor. See [SQLite vector memory](docs/vector-memory.md) for the lifecycle, CLI, integration contract, and research basis.

Recall defaults are starting points, not production claims. `specjam memory calibrate` evaluates labelled positive and abstention cases and recommends `top_k` and `min_score` using precision, recall, ranking quality, abstention, and context cost.

## Closed learning loop

SpecJam treats execution history and reusable memory as different things:

```text
experience -> evaluation -> reflection candidates -> promotion policy -> memory
     ^                                                               |
     +--------- retrieval -> session -> execution --------------------+
```

Raw experience remains in append-only trails. A reflection candidate reaches
episodic memory only after an accepted evaluation with evidence and a minimum
confidence. Semantic memory has a stricter threshold because it generalizes an
episode into reusable guidance. Reflection produces candidates but cannot write
memory directly; `LearningLoop` owns the deterministic single-writer gate.

An accepted increment follows a validated lifecycle:

```text
RUNNING -> EVALUATING -> REFLECTING -> ACCEPTED -> LEARNED -> CLOSED
```

Every status change may be written to an append-only `SessionTrailStore` with
its run, increment, prior status, timestamp, evidence, and learning result.
Rejected evaluations become `BLOCKED`; inconclusive evaluations become
`WAITING`. Illegal transitions fail before an external harness is called.

This keeps the SQLite database rebuildable and prevents failed executions,
unsupported conclusions, and low-confidence model output from becoming future
context merely because they were generated.

## Versioned skill providers

Graph nodes may invoke workspace or external skills through `provider/name@version` references. `SkillResolver` records the resolved version and a SHA-256 content hash, so a run can explain exactly which capability was loaded. The default workspace configuration includes a provider contract for [Ligeiro Mindware](https://github.com/nandoligeiro/ligeiro-mindware); network and Git access remain adapter concerns outside the dependency-free core.

## Postmortem as a governed loop

The Postmortem graph now separates `triage → evidence → root-cause → actions → synthesis → follow-up`. Evidence collection and reviewers are read-only; only the synthesis session writes the shared postmortem. This preserves the distinction between evidence, hypothesis, cause, and corrective action.

Post-mortem is the default privileged producer of semantic memory. Its accepted,
evidenced reflections may be promoted when confidence is at least `0.90`;
ordinary Delivery and Discovery executions produce episodic memory by default.
The allowlist and thresholds are explicit `PromotionPolicy` configuration, not
model discretion.

## RWSA contract

The skill representation follows the routing-aware decomposition described in [Workflow-to-Skill: Skill Creation via Routing-Workflow-Semantics-Attachments Decomposition](https://arxiv.org/abs/2606.06893):

```text
Skill = Routing + (Workflow + Semantics + Attachments)
```

`src/specjam/rws.py` validates this contract without requiring a YAML or AI dependency. The bundled authoring skills render the contract into the portable `SKILL.md` format described by [Agent Skills](https://agentskills.io/specification).

## Development

```bash
PYTHONPATH=src uv run --no-sync python -m unittest discover -s tests -v
uv run specjam graph validate src/specjam/payload/workspace/graphs/delivery-graph.json
uv build --no-sources
```

The release workflow builds both wheel and source distribution on a `v*` tag and publishes them through PyPI Trusted Publishing. Configure the `pypi` GitHub environment and the matching PyPI trusted publisher before pushing a release tag.

```bash
uv version 0.4.0
uv build --no-sources
uv publish
```

The project intentionally keeps the engine dependency-free. Packaging helpers may use the Python standard library only; third-party model, agent, tracker, and cloud integrations are extension points.

## Status

Version 0.4.0 adds normalized Codex, Claude Code, and Devin execution adapters,
automatic evidence-based diagnosis, immutable trajectory replay, and comparative
benchmarking on top of the governed v0.3 meta-harness. Organization-specific
credentials, domain packs, evaluators, and tracker adapters stay outside the core.

## License

Licensed under the [Apache License 2.0](LICENSE).
