Metadata-Version: 2.4
Name: opencode-semantic-memory
Version: 0.18.1
Summary: Persistent semantic memory system for OpenCode sessions
Author-email: Gregory Havenga <ghavenga@gitlab.com>
License-Expression: MIT
Keywords: ai,assistant,mcp,memory,opencode
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Requires-Dist: croniter>=2.0.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: lancedb>=0.4.0
Requires-Dist: markdown-it-py>=3.0.0
Requires-Dist: mcp<2,>=1.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pylance>=0.20.0
Requires-Dist: sentence-transformers>=2.2.0
Requires-Dist: starlette>=0.36.0
Requires-Dist: tomli>=2.0.0
Requires-Dist: torch>=2.0.0
Requires-Dist: tree-sitter-javascript>=0.21.0
Requires-Dist: tree-sitter-python>=0.21.0
Requires-Dist: tree-sitter-ruby>=0.21.0
Requires-Dist: tree-sitter>=0.21.0
Requires-Dist: uvicorn>=0.27.0
Requires-Dist: watchdog>=3.0.0
Provides-Extra: dev
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pandas>=2.0.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
Requires-Dist: pytest-timeout>=2.1.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Description-Content-Type: text/markdown

# GHMEM — GitLab Harness & Enhanced Memory

_(unofficial; a.k.a. Greg H's Memory)_

Persistent semantic memory and an agentic harness for AI coding assistants. Works with [OpenCode](https://opencode.ai), [Claude Code](https://code.claude.com), and other MCP-capable agents.

## Overview

GHMEM is an MCP (Model Context Protocol) server that gives AI coding sessions long-term memory plus a workflow harness (a priority queue, resource watchers, and background tasks). It automatically learns from your workflow, stores decisions and procedures, and provides relevant context when needed.

The project began as an OpenCode-only memory plugin and has since grown well beyond that, hence the rebrand. It now supports multiple agents against one shared daemon, and the "memory" scope has widened into memory plus harness. GHMEM is the brand; the technical identifiers below (the `opencode-memory` repo, the `opencode_memory` Python package, the `opencode-semantic-memory` PyPI distribution, the `OPENCODE_MEMORY_*` environment variables, and the `~/.local/share/opencode-memory` data directory) keep their existing names for compatibility, so nothing about an existing install changes.

**Key benefits:**
- Remember decisions, blockers, procedures, and facts across sessions
- Semantic search finds relevant memories even with different wording
- Works across agents (OpenCode, Claude Code, and other MCP clients) sharing one memory
- Session coordination prevents conflicts when running multiple agent instances
- Project-scoped directives for per-project instructions
- Zero manual effort - learns passively from your workflow

## Features

- **Hybrid retrieval**: Combines full-text search (FTS5) with vector similarity search
- **Instant storage**: Memories stored immediately, embeddings computed async
- **Session coordination**: Tracks active sessions, supports item claiming to prevent conflicts
- **Contextual directives**: Global + project-specific instructions loaded at boot
- **Entity tracking**: Links memories to MRs, issues, epics (!123, #456, &789)
- **GitLab enrichment**: Fetches metadata for entities from GitLab API
- **Memory age**: All outputs show age (`now`, `5m`, `2h`, `3d`, `2w`, `1mo`, `2y`) to identify potentially outdated information
- **Proactive context injection**: Automatically injects relevant memories before each LLM call (OpenCode only, via plugin)
- **Agent-agnostic LLM support**: Works with OpenCode, Claude CLI, GitLab Duo, Ollama, or custom commands
- **GitLab bulk ingest (optional)**: Index an entire GitLab project's issues and MRs into a separate database for weak semantic matching in `memory_recall`, with incremental daily sync to pick up new and updated issues/MRs
- **Queue defer/postpone**: Defer queue items you're waiting on and have them automatically resurface when a linked reminder fires
- **Secret-guard**: Reversible redaction of credentials from ingested content. Secrets are masked with placeholders in transit (so they never reach the model or memory) and restored only on a safe round-trip (writing back to the same file), eliminating false-positive corruption; at true exposure sinks (memory, comments, foreign files) they are never restored. Includes pattern auto-discovery, an exclude-paths allowlist, and a `forget` tool to undo a false positive. Set `reversible: false` (or `SECRET_GUARD_REVERSIBLE=0`) for the legacy one-way mode. See [docs/SECRET_GUARD.md](docs/SECRET_GUARD.md) and the [hybrid design](docs/SECRET_GUARD_HYBRID_DESIGN.md).
- **Security self-audit (optional)**: A conservative, read-only pass (`SecurityAudit` job, default off; `[security_audit] enabled = true`) that scans recent memory content and the enabled-autonomy config for risk signals — secrets stored in memory content, prompt-injection phrasing from untrusted ingestion, and autonomy-surface config drift — and surfaces findings as `blocker`/`idea` memories visible at session start. It never mutates the inspected content. See the [threat model](docs/threat-model.md).
- **Tiered Docker sandboxing (optional)**: Run LLM-backed background tasks in a hardened container with per-task least-privilege GitLab tokens and a scrubbed LLM credential (default off; `mode = "host"`)
- **Migration system**: Versioned schema migrations with structured blockers/remediation and data-safety checks
- **Effectiveness judging**: Optional LLM material-influence judging of recalled memories to track which memories actually help
- **Optional Rust runtime**: Drop-in Rust backend sharing the same stores and HTTP/MCP API; selectable at install time
- **Version visibility**: `/health` and `/stats` report the running server version so you can confirm an upgrade took effect

## Prerequisites

- **Python 3.11+** (verify with `python3 --version` - must show 3.11 or higher)
- **git** (for cloning the repository)
- **venv** module (usually included with Python, or install via `python3-venv` package)

On Ubuntu/Debian:
```bash
sudo apt install python3.11 python3.11-venv git
```

On macOS (with Homebrew):
```bash
brew install python@3.11 git
```

## Installation

### From PyPI

```bash
pip install opencode-semantic-memory
```

### Quick Install Script

Works on both Linux and macOS, sets up the daemon as a background service:

```bash
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash
```

This will:
- Clone the repository to `~/.local/share/opencode-memory-install`
- Create a virtual environment and install dependencies
- Download the embedding model (~90MB, one-time, runs locally)
- Set up a background service:
  - **Linux**: systemd user service
  - **macOS**: launchd LaunchAgent
- Configure OpenCode integration
- Optionally bootstrap core directives (requires confirmation due to LLM costs)

**Re-running the installer is safe and non-interactive.** Your choices (runtime,
startup mode, LLM provider, permissions) are saved to
`~/.config/opencode-memory/config.toml` and reused on subsequent runs — re-runs
just update the install and fix anything missing, without re-asking. To change
your configuration, force the prompts with `--reconfigure`:

```bash
# from a local clone
bash scripts/setup.sh --reconfigure

# piped
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash -s -- --reconfigure
```

### From Source

```bash
git clone https://gitlab.com/ghavenga/opencode-memory.git
cd opencode-memory
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

# Download the embedding model (required, ~90MB, runs locally)
python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2')"
```

**Important:** After installing from source, you must manually set up the service (see Quick Start below). The service files in the repo assume the quick install path - you'll need to update the paths to match your clone location.

### Cost Information

**Default background processing is free** - The memory daemon uses:
- Local embedding model (sentence-transformers, no API calls)
- Pattern-based extraction (regex, no LLM)
- Session summaries (stores conversations as-is, no LLM processing)

**LLM-based knowledge extraction is OFF by default** - There is an optional
feature that uses `opencode run` to analyze old conversations and extract
procedures, decisions, and facts. This:
- Runs every 6 hours, processing up to 50 conversations per cycle
- Makes LLM API calls for each conversation (significant cost potential)
- Is **disabled by default** to avoid unexpected charges

To enable LLM extraction (if you want it), add to `~/.config/opencode-memory/config.toml`:
```toml
[ingestion]
llm_extraction = true
```

### LLM Provider Configuration

The memory system supports multiple LLM providers for knowledge extraction and session analysis. By default, it uses OpenCode, but you can configure alternatives:

```toml
# ~/.config/opencode-memory/config.toml

# Option 1: OpenCode (default) - uses your configured OpenCode LLM
[llm]
provider = "opencode"

# Option 2: Anthropic Claude CLI (requires claude-cli package)
[llm]
provider = "claude"
model = "claude-sonnet-4-20250514"  # optional

# Option 3: GitLab Duo CLI (requires glab with Duo enabled)
[llm]
provider = "duo"

# Option 4: Ollama (local models)
[llm]
provider = "ollama"
model = "qwen2.5:14b"  # or any model you have pulled

# Option 5: Custom command
[llm]
provider = "custom"
command = "my-llm-cli"
args = ["--prompt"]  # prompt is appended after these args
```

**Note:** Proactive context injection (automatic memory loading) works with OpenCode via the plugin system and with Claude Code via its SessionStart and UserPromptSubmit hooks (see the "Using with Claude Code" section). Other MCP agents can use all memory tools manually.

## Updating

To update an existing installation, re-run the quick install script. It is idempotent and will pull the latest code, refresh dependencies, and restart the background services:

```bash
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash
```

The update is **non-destructive** for your data:

- Your memory database at `~/.local/share/opencode-memory/memory.db` is not touched
- Your vector store at `~/.local/share/opencode-memory/vectors/` is not touched
- Your existing `~/.config/opencode-memory/config.toml` is preserved (the script only writes a default if one does not already exist)

The script's `rm -rf` operations are scoped to the install directory (`~/.local/share/opencode-memory-install`) and only trigger when the git checkout or virtual environment is in a broken state. Any uncommitted local changes inside the install directory will be discarded (`git reset --hard` + `git clean -fd`) before the pull.

If you want to be extra cautious before a major update, snapshot the data directory first:

```bash
# Consistent SQLite backup (handles WAL files correctly)
TS=$(date +%Y%m%d-%H%M%S)
BACKUP_DIR="$HOME/.local/share/opencode-memory-backups/$TS"
mkdir -p "$BACKUP_DIR"
sqlite3 "$HOME/.local/share/opencode-memory/memory.db" ".backup '$BACKUP_DIR/memory.db'"
cp -R "$HOME/.local/share/opencode-memory/vectors" "$BACKUP_DIR/vectors"
cp "$HOME/.config/opencode-memory/config.toml" "$BACKUP_DIR/config.toml"
```

After the update, verify the service is healthy:

```bash
curl -s http://127.0.0.1:9824/health
# {"status":"ok","server":"opencode-memory","daemon":{"running":true}}
```

### Opt-in auto-updates

The memory system can keep itself up to date. This is **off by default**; enable it in `~/.config/opencode-memory/config.toml`:

```toml
[update]
enabled = true          # run the periodic check (daily by default)
channel = "stable"      # "stable" tracks release tags; "edge" tracks master
auto_patch = true       # auto-apply x.y.Z bugfix releases
auto_minor = false      # x.Y.0: notify and wait for confirmation
auto_major = false      # X.0.0: notify and wait for confirmation
# pin = "0.13.0"        # never auto-apply beyond this version
keep_backups = 3        # data snapshots kept before each apply
```

How it works:

- A background job checks the configured channel on `check_interval_seconds`.
- **Patch** releases can apply automatically. **Minor** and **major** releases are held back and surfaced in your boot context; apply them from a session with `workflow(action="update_apply")` (or by re-running the installer).
- Before applying, it records the current version and snapshots your data (`memory.db`, `vectors/`, `config.toml`) so a rollback can always restore both code and data.
- Applying restarts the background services. Because the MCP daemon is stateless the client reconnects transparently, **unless the update changed the tool interface** — in that case your boot context will tell you to restart opencode so the new tools load.
- On the **edge** channel there is no semver tier; `auto_patch` governs whether new `master` commits auto-apply, since riding the tip is the point of edge.

Check status or roll back at any time from a session:

```
workflow(action="update_status")   # current version, channel, pending update, restart hint
workflow(action="update_apply")    # apply an available (or held-back) update now
workflow(action="rollback")        # revert to the previous version
```

Or from the shell:

```bash
# Roll back to the previous version (code only; data untouched)
bash ~/.local/share/opencode-memory-install/scripts/rollback.sh

# Roll back to a specific release, restoring the newest data snapshot too
bash ~/.local/share/opencode-memory-install/scripts/rollback.sh --to v0.12.0 --restore-data
```

## Quick Start

### 1. Start the Memory Services

The recommended setup uses 3 background services (see [ARCHITECTURE.md](ARCHITECTURE.md)):

**Using the setup script (recommended):**
```bash
# The setup script handles everything including service installation
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash
```

**Manual start (development/testing):**
```bash
source .venv/bin/activate

# Terminal 1: HTTP/MCP server
python -m opencode_memory.http_server

# Terminal 2: File watcher daemon
python -m opencode_memory.daemon

# Terminal 3: Background worker
python -m opencode_memory.jobs.worker
```

**Install services manually (Linux systemd):**
```bash
mkdir -p ~/.config/systemd/user
INSTALL_DIR="$PWD"  # Run from your clone directory

# Copy service files
for svc in opencode-memory opencode-memory-daemon opencode-memory-worker; do
    sed "s|%h|$HOME|g" "$INSTALL_DIR/${svc}.service" | \
    sed "s|\.local/share/opencode-memory-install|${INSTALL_DIR#$HOME/}|g" \
    > ~/.config/systemd/user/${svc}.service
done

# Enable and start
systemctl --user daemon-reload
systemctl --user enable --now opencode-memory opencode-memory-daemon opencode-memory-worker

# Check status
systemctl --user status opencode-memory*
```

**Install services manually (macOS launchd):**

See the setup script for plist templates, or run the setup script which handles this automatically.

### 2. Configure OpenCode

Add to `~/.config/opencode/opencode.json`:

```json
{
  "mcp": {
    "memory": {
      "type": "remote",
      "url": "http://localhost:9824/mcp"
    }
  }
}
```

The memory server can also be run as a local stdio process via
`scripts/memory-mcp.sh`. This launches a shim that serves the GitLab
`workflow` tool in-process (inheriting `GITLAB_TOKEN` from your shell) and
proxies every other `memory_*` tool to the shared HTTP daemon:

```json
{
  "mcp": {
    "memory": {
      "type": "local",
      "command": ["/path/to/opencode-memory-install/scripts/memory-mcp.sh"]
    }
  }
}
```

Use the local (`memory-mcp.sh`) form if you want the `workflow` tools; the
remote (`url`) form exposes the memory tools but not `workflow`, since
`workflow` needs your session's GitLab token which the shared daemon does not
have.

### Reducing context: lazy-mcp

Since v0.12.0 the MCP surface is already consolidated: the shim advertises
only five tools up front (`remember`, `recall`, `memory`, `workflow`,
`memory_shutdown`), and the ~60 other operations are reached through
`memory(action=...)`, which forwards to the daemon. So on any current release
the per-session memory tool overhead is already small; make sure you are
updated before reaching for anything more.

If you still want to squeeze that surface further, or you run several MCP
servers and want to lazy-load all of them together, front them with
[lazy-mcp](https://gitlab.com/gitlab-org/ai/lazy-mcp), a proxy that replaces a
server's schemas with four meta-tools (`list_servers`, `list_commands`,
`describe_commands`, `invoke_command`) and discovers the real tools on demand.
For memory alone the win is marginal now (five schemas down to four shared
across all proxied servers); the real value is aggregating a larger server
(for example a GitLab MCP with dozens of tools) in the same proxy.

Point lazy-mcp at the **local shim** so `workflow` keeps working (it needs your
session's `GITLAB_TOKEN`, which the shim inherits and the shared daemon does
not have). In `~/.config/lazy-mcp/servers.json`:

```json
{
  "servers": [
    {
      "name": "memory",
      "description": "opencode-memory: persistent semantic memory + GitLab workflow tools",
      "command": ["/path/to/opencode-memory-install/scripts/memory-mcp.sh"]
    }
  ]
}
```

Then replace the `memory` entry in `opencode.json` with a single lazy-mcp
entry, and set the plugin to `{ "tools": false }` so it injects context without
adding its own tool schema either:

```json
{
  "mcp": {
    "lazy-mcp": {
      "type": "local",
      "command": ["npx", "-y", "lazy-mcp@latest", "--config", "~/.config/lazy-mcp/servers.json"]
    }
  },
  "plugin": [["/path/to/opencode-memory-install/plugin", { "tools": false }]]
}
```

Notes:

- Front the **local shim** (`scripts/memory-mcp.sh`), not the daemon `url`.
  The shim advertises the consolidated set (`remember`, `recall`, `memory`,
  `workflow`, `memory_shutdown`) and serves `workflow` in-process with your
  `GITLAB_TOKEN`. Fronting the daemon by `url`
  (`"url": "http://localhost:9824/mcp"`) instead exposes the raw, un-shrunk
  tool list, `workflow` fails there with a missing-token error, and
  `memory_shutdown` is not exposed.
- Proactive context injection and boot context are unaffected: they are driven
  by the plugin's hooks, not by tool calls, so lazy discovery does not delay
  them.
- lazy-mcp can aggregate multiple servers (e.g. a GitLab MCP alongside memory)
  in the same `servers.json`, so this composes with other lazy-loaded servers.

### 3. Enable Proactive Context Injection (OpenCode Only)

The proactive context plugin automatically injects relevant memories before each LLM call, so the AI has historical context without needing explicit `memory_recall` calls.

**Requirements:**
- OpenCode (not supported with other LLM providers)
- Node.js 20+ (for building the plugin)

**Setup:**

The setup script automatically builds and configures the plugin if Node.js 20+ is available. To verify or configure manually:

```json
// ~/.config/opencode/opencode.json
{
  "mcp": { ... },
  "plugin": ["/path/to/opencode-memory-install/plugin"]
}
```

**How it works:**
1. User sends a message → Plugin extracts entities (!MR, #issue, @user)
2. Plugin calls memory server for relevant context
3. Context is injected into the system prompt before LLM call
4. AI sees relevant history automatically

**Thinking-aware injection:** The plugin also analyzes the assistant's responses to inject context mid-conversation. If the AI is "thinking" about a topic it has prior knowledge of, relevant memories are injected before the next LLM call.

**Relevance filtering:** Items with semantic similarity below 0.35 are filtered out to reduce noise. This threshold was tuned based on effectiveness analysis showing low-relevance items had 0% actual usefulness.

**Note:** If you're using a different LLM provider (Claude, Duo, Ollama), you'll need to call memory tools explicitly - proactive injection only works with the OpenCode plugin.

For more details, see the [plugin documentation](plugin/README.md).

### 4. Boot Context (Automatic)

When using the proactive context plugin (step 3), boot context is **automatically injected** at session start. This includes:
- Active parallel sessions (collision warning)
- Standing instructions (boot gates)
- Unresolved blockers
- Due reminders
- Memory usage instructions

**No manual configuration needed** - the plugin handles everything.

**Legacy AGENTS.md:** If you have an existing `~/.config/opencode/AGENTS.md` with `memory_get_boot_context()` instructions, you can remove it - boot context is now auto-injected. The setup script will offer to clean this up.

**Without the plugin:** If you're not using the proactive context plugin (e.g., a raw `claude` CLI prompt or Ollama), you'll need to call `memory_get_boot_context()` manually at session start. Note this is different from **Claude Code**, which does get automatic injection through its own hooks (see [Using with Claude Code](#using-with-claude-code) below).

## Using with Claude Code

opencode-memory works with [Claude Code](https://code.claude.com) as well as OpenCode, and both can run against the same daemon at the same time. There is no either/or: one install can configure both agents.

Claude Code gets the two halves of the system through its own mechanisms:

- **Memory tools** via an MCP server registration, which exposes every `memory_*` tool.
- **Automatic context injection** via two hook scripts (`SessionStart` and `UserPromptSubmit`) that mirror what the OpenCode plugin does. They live in `claude-code/hooks/` and call the same daemon endpoints.

| OpenCode plugin hook | Claude Code hook | Purpose |
|---|---|---|
| `experimental.chat.system.transform` | `SessionStart` | Inject boot context at session start |
| `chat.message` (proactive) | `UserPromptSubmit` | Extract entities from the prompt, inject relevant memories |

The setup script offers Claude Code configuration when it finds the `claude` CLI. To pick agents explicitly (comma-separated, either or both), set `OPENCODE_MEMORY_AGENTS`:

```bash
# Configure both agents
OPENCODE_MEMORY_AGENTS=opencode,claude bash scripts/setup.sh --reconfigure

# Add Claude Code to an existing install
OPENCODE_MEMORY_AGENTS=claude bash scripts/setup.sh --reconfigure
```

Claude Code needs `ANTHROPIC_API_KEY` in your environment or an interactive `claude` `/login`; the installer never handles credentials. Full manual steps and environment variables are documented in [claude-code/README.md](claude-code/README.md).

## Configuration

Create `~/.config/opencode-memory/config.toml`:

```toml
[identity]
user = "your-gitlab-username"  # Optional, auto-detected from git
instance = "gitlab.com"

[boot]
identity = true
active_sessions = true
hot_items = true
unresolved_blockers = true
recent_decisions = false
max_hot_items = 5

[ingestion]
watch_paths = ["~/.local/share/opencode/opencode.db"]
db_poll_interval = 30
llm_extraction = false
working_directory = "/path/to/your/projects"

[storage]
path = "~/.local/share/opencode-memory"
```

## MCP Tools

> **Note:** Tools are registered with the `memory_` prefix when accessed through the "memory" MCP server.
> For example, `recall` becomes `memory_recall` in OpenCode sessions.

### Core Memory Tools

| Tool | Description |
|------|-------------|
| `memory_recall(query, limit?, project?, category?, compact?)` | Semantic search across memories |
| `memory_remember(content, category, what?, why?, learned?, entities?, project?)` | Store a memory |
| `memory_get_context(entity_ref)` | Get all memories for !MR, #issue, or &epic |
| `memory_get_boot_context()` | Load startup context (identity, blockers, directives) |
| `memory_get_linked_memories(memory_id, link_types?)` | Get memories linked to a specific memory |

### Session Coordination

| Tool | Description |
|------|-------------|
| `memory_session_start(session_id, working_on?)` | Register session |
| `memory_session_end(session_id, summary?)` | End session with summary |
| `memory_session_heartbeat(session_id)` | Keep session alive |
| `memory_get_active_sessions()` | List active sessions |
| `memory_claim_item(session_id, item_ref)` | Claim exclusive ownership |
| `memory_release_item(session_id, item_ref)` | Release claimed item |

### Memory Management

| Tool | Description |
|------|-------------|
| `memory_resolve_blocker(memory_id)` | Mark blocker as resolved |
| `memory_unresolve_blocker(memory_id)` | Reopen a blocker |
| `memory_archive_memory(memory_id, reason?)` | Archive outdated memory (soft delete) |
| `memory_delete_memory(memory_id, also_delete_vector?)` | Permanently delete a memory |
| `memory_edit_memory(memory_id, content?, what?, why?, learned?)` | Edit memory content or metadata |
| `memory_bulk_archive(memory_ids?, category?, older_than_days?, reason)` | Archive multiple memories |
| `memory_consolidate_memory(days_stale?, project?)` | Find stale/duplicate memories |
| `memory_search_history(query, category?, limit?)` | Search with category filter |

### Backup & Transfer

| Tool | Description |
|------|-------------|
| `memory_export_memories(output_path?, project?, categories?, since_days?)` | Export to JSON |
| `memory_import_memories(input_path, dry_run?, skip_duplicates?)` | Import from JSON |

### Reminders

| Tool | Description |
|------|-------------|
| `memory_set_reminder(reminder_at, content?, memory_id?, recurrence?)` | Set time-triggered reminder |
| `memory_clear_reminder(memory_id)` | Dismiss a reminder |
| `memory_advance_reminder(memory_id)` | Move recurring reminder to next occurrence |
| `memory_mark_procedure_reviewed(memory_id)` | Mark procedure as still accurate |
| `memory_set_review_interval(memory_id, interval_days)` | Set procedure review frequency |

### Knowledge Graph

| Tool | Description |
|------|-------------|
| `memory_graph_aware(entity_ref, max_depth?)` | Check what knowledge exists for an entity |
| `memory_graph_explore(entity_ref, types?, limit?)` | Discover related entities |
| `memory_graph_retrieve(entity_refs, include_memories?)` | Get full content for entities |
| `memory_graph_status(project?)` | Knowledge graph statistics |

### Code Indexing

| Tool | Description |
|------|-------------|
| `memory_index_codebase(path?, incremental?)` | Index code structure (classes, functions) |
| `memory_initialize_workspace(path?, max_depth?)` | Scan directories for git repos |
| `memory_extract_concepts(batch_size?)` | Extract semantic concepts via LLM |
| `memory_search_code(query, symbol_types?)` | Search indexed code symbols |

### GitLab Spider

| Tool | Description |
|------|-------------|
| `memory_spider_search(entity_ref, project?, max_depth?)` | Crawl GitLab entity into knowledge graph |
| `memory_spider_status()` | Check spider queue status |
| `memory_spider_query(query?, author?, entity_type?)` | Search crawled GitLab content |

### Priority Queue

Work item queue for tracking todos, MRs to review, and authored MRs. Supports P0–P5 priorities with atomic claiming to prevent parallel session conflicts.

Items can also carry an optional `due_date`. As a deadline nears, an item's **effective** priority escalates toward P0 so milestone-driven work surfaces itself — without rewriting the stored base priority. Escalation only ever raises urgency, and pushing a due date out relaxes the item back to its base priority.

Escalation is **lead-time-proportional**: it tracks how far the item has travelled along its *runway* (from `created_at` to `due_date`), not an absolute number of days. The effective priority climbs incrementally from base toward P0 across the whole runway, clamping to P0 at/after the due date. So a task planned far in the future starts climbing early and gradually (a long horizon implies commensurate effort), while a short-runway task climbs quickly — both escalate over their own span. The curve is configurable (`start_fraction`, `gamma` shape, `max_runway_days` cap). Ordering uses effective priority, then soonest due date, then base priority, then age.

| Tool | Description |
|------|-------------|
| `memory_queue_get_next(project?, source?)` | Get next unclaimed item by effective priority |
| `memory_queue_claim(item_id, session_id)` | Claim item for your session |
| `memory_queue_release(item_id, session_id)` | Release claimed item back to pending |
| `memory_queue_complete(item_id, session_id, outcome_notes?)` | Mark item as completed |
| `memory_queue_skip(item_id, reason?, session_id?)` | Skip item (no longer relevant) |
| `memory_queue_list(status?, priority?, source?, project?)` | List queue items with filters (shows effective priority + due date) |
| `memory_queue_stats()` | Queue counts by status, priority, source, and due-date pressure |
| `memory_queue_reprioritize(item_id, new_priority)` | Change item base priority |
| `memory_queue_set_due_date(item_id, due_date?)` | Set or clear a due date (drives escalation) |
| `memory_queue_requeue(archived_id)` | Move archived item back to queue |
| `memory_queue_populate(items, refresh_id?)` | Bulk add/update queue items (accepts `due_date` per item) |

### Analytics

| Tool | Description |
|------|-------------|
| `memory_effectiveness_report(days?)` | Memory usage patterns, injection effectiveness, and suggestions |
| `memory_injection_metrics_trend(days?)` | Daily injection metrics over time for tracking improvements |
| `memory_background_report(days?)` | Background process activity |
| `memory_tool_metrics(action?, since_hours?)` | MCP tool call timing metrics |
| `memory_vector_index(action?)` | Manage vector search index |

### Utilities

| Tool | Description |
|------|-------------|
| `memory_ingest_file(file_path)` | Manually ingest a markdown file |
| `memory_enrich_entity(entity_ref, project?)` | Fetch GitLab metadata |
| `memory_bootstrap_memory(path?)` | Scan project files for initial facts |
| `memory_log_session(summary, learnings?, entities?)` | Log session summary |
| `memory_memory_status()` | Check system health and queue status |
| `memory_get_proactive_context(query, entities?, project?)` | Get context for proactive injection (plugin use) |

## GitLab Bulk Ingest (Optional)

Bulk ingest indexes **all issues and merge requests** from a GitLab project (default: `gitlab-org/gitlab`) into a **separate** database, so `memory_recall` can surface related upstream issues/MRs alongside your personal memories.

It is fully isolated from your main memory: a dedicated SQLite DB and vector store that can be disabled or deleted at any time without affecting your memories.

### How it works

1. **Fetch** - pulls issue/MR metadata (title, description, labels, state, dates) via the GitLab API, rate-limited and resumable.
2. **Keywords** - extracts the top ~50 keywords per entity (stop words, URLs, @mentions, and paths stripped). No LLM required.
3. **Index** - embeds title + description + labels with the same local `all-MiniLM-L6-v2` model used for memories.
4. **Search** - results merge into `memory_recall`, scored by vector similarity (weighted `0.8` vs personal memories) plus a small keyword-overlap boost (`0.1 * overlapping_keywords`).

All processing is local. The only network calls are to the GitLab API (fetching metadata) and a one-time embedding-model download.

### Data isolation

| | Main memory | Bulk ingest |
|---|---|---|
| SQLite | `~/.local/share/opencode-memory/memory.db` | `~/.local/share/opencode-memory/gitlab-org.db` |
| Vectors | `~/.local/share/opencode-memory/vectors/` | `~/.local/share/opencode-memory/gitlab-org.lance/` |

Bulk-ingested entities are never written into your main memory DB, knowledge graph, or consolidation/archiving flows. They are a read-time overlay only.

### Activation

```bash
# 1. Create the config file (~/.config/opencode-memory/bulk_ingest.toml)
opencode-memory bulk-ingest config --init

# 2. Fetch all issues + MRs (resumable; safe to re-run if interrupted)
#    For gitlab-org/gitlab this is ~580K entities and takes ~2 hours.
GITLAB_TOKEN=<your-token> opencode-memory bulk-ingest fetch

# 3. Build the vector index (embeds locally; ~3 hours for 580K on CPU)
opencode-memory bulk-ingest index --create-index

# 4. Check progress / counts at any time
opencode-memory bulk-ingest status
```

Once indexed, results appear automatically in `memory_recall` with `match_type: "bulk_ingest"` and a `bulk_ingest:<project>#<iid>` source.

### Keeping it up to date

To pick up newly created and recently updated issues/MRs, run an incremental sync. It fetches everything changed since the latest `updated_at` already in storage, so it catches both new entities and edits to existing ones:

```bash
# One-shot incremental sync
GITLAB_TOKEN=<your-token> opencode-memory bulk-ingest sync

# Explicit window
opencode-memory bulk-ingest sync --since 2026-06-01T00:00:00Z
```

For hands-off operation, the `BulkIngestDaemon` runs the incremental sync on a schedule (default every 24 hours, configurable via `sync_interval_hours`), optionally re-indexing new entities after each pass. Schedule it however you prefer (systemd timer, cron, or the daemon's own loop). The sync is poll-based rather than webhook-driven, so a daily cadence is the typical setup.

### CLI commands

| Command | Description |
|---------|-------------|
| `bulk-ingest config --init` | Create the default config file |
| `bulk-ingest fetch` | Fetch issue/MR metadata (resumable) |
| `bulk-ingest index --create-index` | Embed entities and build the IVF-PQ search index |
| `bulk-ingest index --reindex` | Re-embed everything (default skips already-indexed) |
| `bulk-ingest sync` | Incremental fetch of entities updated since last run |
| `bulk-ingest status` | Show entity counts, tiers, and DB size |
| `bulk-ingest enable` / `disable` | Toggle inclusion in `memory_recall` |

### Configuration

Key settings in `~/.config/opencode-memory/bulk_ingest.toml`:

```toml
[gitlab]
project = "gitlab-org/gitlab"   # any project path
api_rate_limit = 100            # requests per minute

[query]
enabled = true                  # include results in memory_recall
relevance_weight = 0.8          # discount vs personal memories

[summarizer]
enabled = false                 # LLM summaries are OPTIONAL and off by default
provider = "none"               # title + description is enough for search
```

> **Note:** LLM summarization is optional and disabled by default. Avoid the `opencode` summarizer provider - it has MCP access and would write entries into your main memory DB.

### Disabling

```bash
opencode-memory bulk-ingest disable        # stop including in recall (keeps data)
# or remove the data entirely:
rm -rf ~/.local/share/opencode-memory/gitlab-org.db ~/.local/share/opencode-memory/gitlab-org.lance
```

## Memory Categories

| Category | Use For |
|----------|---------|
| `boot_gate` | STOP trigger shown every session (≤400 chars, points to directive) |
| `directive` | Full standing instructions (loaded via recall) |
| `procedure` | How-to knowledge, workflows |
| `decision` | Architectural choices, design decisions |
| `blocker` | Obstacles preventing progress |
| `fact` | Project-specific information |
| `plan` | Sequence of steps with end state |
| `goal` | Sustained target/threshold (ongoing, no end state) |
| `idea` | Future possibilities, deferred considerations |
| `event` | Significant occurrences |
| `conversation` | Full conversation content (auto-generated) |

## Storage

All data is stored locally:

- **SQLite**: `~/.local/share/opencode-memory/memory.db` - Memories, entities, sessions, FTS index
- **LanceDB**: `~/.local/share/opencode-memory/vectors/` - Vector embeddings for semantic search

The daemon automatically:
- Cleans up old resolved blockers (>90 days)
- Archives old conversations (>180 days)
- Compacts LanceDB versions (keeps last 10)

## Environment Variables

| Variable | Description |
|----------|-------------|
| `GITLAB_TOKEN` | Enable GitLab entity enrichment |
| `HF_HUB_OFFLINE=1` | Prevent model downloads (use cached) |
| `TRANSFORMERS_OFFLINE=1` | Prevent model downloads |
| `OPENCODE_MEMORY_HOST` | HTTP server bind address (default: 127.0.0.1) |
| `OPENCODE_MEMORY_PORT` | HTTP server port (default: 9824) |
| `OPENCODE_MEMORY_API_KEY` | Optional API key for authentication |
| `OPENCODE_MEMORY_RATE_LIMIT` | Requests per minute per client (default: 60) |
| `SKIP_BOOTSTRAP` | Set to `1` to skip directive bootstrapping prompt in setup script |

## Development

```bash
# Activate environment
source .venv/bin/activate

# Run tests
python -m pytest tests/ -v

# Lint
ruff check src/

# Type check
mypy src/

# Check memory stats
python -m opencode_memory.cli stats
```

## Architecture

opencode-memory uses a **3-process architecture** to separate concerns:

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                           opencode-memory                                   │
├─────────────────────────────────────────────────────────────────────────────┤
│  ┌─────────────────────┐  ┌─────────────────────┐  ┌─────────────────────┐  │
│  │   HTTP/MCP Server   │  │       Daemon        │  │  Background Worker  │  │
│  │  (opencode-memory)  │  │ (opencode-memory-   │  │ (opencode-memory-   │  │
│  │                     │  │       daemon)       │  │       worker)       │  │
│  ├─────────────────────┤  ├─────────────────────┤  ├─────────────────────┤  │
│  │ • MCP tool handlers │  │ • File watching     │  │ • Embedding compute │  │
│  │ • Request/response  │  │ • OpenCode DB poll  │  │ • GitLab enrichment │  │
│  │ • Health endpoints  │  │ • Queue work items  │  │ • Memory linking    │  │
│  │                     │  │                     │  │ • Cleanup/archival  │  │
│  │                     │  │                     │  │ • LLM extraction    │  │
│  └─────────┬───────────┘  └─────────┬───────────┘  └─────────┬───────────┘  │
│            │                        │                        │              │
│            └────────────────────────┼────────────────────────┘              │
│                          ┌──────────▼──────────┐                            │
│                          │   Shared Storage    │                            │
│                          ├─────────────────────┤                            │
│                          │ • SQLite + FTS5     │                            │
│                          │ • LanceDB (vectors) │                            │
│                          │ • Work queues       │                            │
│                          └─────────────────────┘                            │
└─────────────────────────────────────────────────────────────────────────────┘
```

| Process | Purpose | Module |
|---------|---------|--------|
| **HTTP Server** | MCP tools, health endpoints | `opencode_memory.http_server` |
| **Daemon** | File watching, DB polling | `opencode_memory.daemon` |
| **Worker** | All heavy background processing | `opencode_memory.jobs.worker` |

This separation ensures:
- API requests are never blocked by background processing
- Worker can crash/restart without affecting the API
- Multiple workers can run in parallel for scaling

See [ARCHITECTURE.md](ARCHITECTURE.md) for detailed documentation.

### Startup mode: always-on vs lazy

The HTTP server keeps the embedding model resident (~2 GB) so semantic search is
instant and shared across sessions. The installer lets you choose when the
services run:

| Mode | Boot behaviour | First-session cost | Idle RAM |
|------|----------------|--------------------|----------|
| **Always-on** (default) | Started at login | None | ~2 GB resident |
| **Lazy / on-demand** | Not started at boot | A few seconds (model load) | 0 until first use |

In **lazy mode** the services start automatically the first time an opencode
session connects (the `memory-mcp.sh` wrapper sees the marker at
`~/.config/opencode-memory/lazy` and runs `memory-ctl.sh start`), then stay
running until explicitly stopped. This suits low-RAM machines.

Choose lazy at install time interactively, or non-interactively:

```bash
OPENCODE_MEMORY_STARTUP=lazy bash scripts/setup.sh
```

Start/stop/inspect the stack at any time:

```bash
scripts/memory-ctl.sh status   # services + current mode
scripts/memory-ctl.sh start    # start all three (no-op if already healthy)
scripts/memory-ctl.sh stop     # stop all three, freeing the model's RAM
```

From inside a session you can free the RAM on demand with the **`memory_shutdown`**
MCP tool, which stops all three services. In lazy mode the next session restarts
them automatically; in always-on mode restart with `memory-ctl.sh start`.

## Operations

### Log Rotation

When running as a systemd service, logs are managed by journald. To configure log rotation:

```bash
# Check current log size
journalctl --user -u opencode-memory --disk-usage

# Set max journal size (add to ~/.config/systemd/user.conf or override)
# Or create ~/.config/systemd/journald.conf.d/opencode-memory.conf:
cat > ~/.config/systemd/journald.conf.d/opencode-memory.conf << 'EOF'
[Journal]
SystemMaxUse=100M
MaxRetentionSec=7d
EOF

# Or manually vacuum old logs
journalctl --user --vacuum-time=7d
journalctl --user --vacuum-size=100M
```

For file-based logging, configure rotation in the systemd service:

```ini
[Service]
StandardOutput=append:/var/log/opencode-memory/server.log
StandardError=append:/var/log/opencode-memory/error.log
```

Then use logrotate (`/etc/logrotate.d/opencode-memory`):

```
/var/log/opencode-memory/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 $USER $USER
}
```

### Monitoring

The HTTP server exposes several endpoints:

| Endpoint | Description |
|----------|-------------|
| `/health` | Basic health check (returns 200 if healthy, 503 if degraded) |
| `/stats` | Detailed statistics (memories, cache, queue, links) |
| `/metrics` | Prometheus-format metrics |

Example monitoring with curl:
```bash
# Health check (for load balancers/monitoring)
curl -s http://localhost:9824/health | jq .

# Full stats
curl -s http://localhost:9824/stats | jq .

# Prometheus metrics
curl -s http://localhost:9824/metrics
```

### CLI Tools

```bash
# Show comprehensive statistics
python -m opencode_memory.cli stats

# Ingest markdown files
python -m opencode_memory.cli ingest /path/to/notes --recursive

# Archive old memories
python -m opencode_memory.cli cleanup --dry-run

# Enrich entities with GitLab metadata
python -m opencode_memory.cli enrich --limit 100
```

## Troubleshooting

### Check service status

**Linux (systemd):**
```bash
# Check all services
systemctl --user status opencode-memory*

# View logs for specific service
journalctl --user -u opencode-memory -f         # Server logs
journalctl --user -u opencode-memory-daemon -f  # Daemon logs
journalctl --user -u opencode-memory-worker -f  # Worker logs

# Restart all services
systemctl --user restart opencode-memory opencode-memory-daemon opencode-memory-worker
```

**macOS (launchd):**
```bash
# Check all services
launchctl list | grep opencode

# View logs
tail -f ~/.local/state/opencode-memory/server.log
tail -f ~/.local/state/opencode-memory/daemon.log
tail -f ~/.local/state/opencode-memory/worker.log

# Restart all services
for svc in com.opencode.memory com.opencode.memory.daemon com.opencode.memory.worker; do
    launchctl unload ~/Library/LaunchAgents/${svc}.plist 2>/dev/null
    launchctl load ~/Library/LaunchAgents/${svc}.plist
done
```

### Check memory system health
Use `memory_status` tool or:
```bash
# HTTP health endpoint
curl http://localhost:9824/health

# Detailed stats
curl http://localhost:9824/stats
```

### Prometheus metrics
The HTTP server exposes metrics at `/metrics`:
```bash
curl http://localhost:9824/metrics
```

### Backup and restore
```bash
# Export all memories
memory_export_memories(output_path="/backup/memories.json")

# Import on new machine
memory_import_memories(input_path="/backup/memories.json", dry_run=True)  # Preview
memory_import_memories(input_path="/backup/memories.json")  # Actually import
```

### Reset memory database

**Linux:**
```bash
systemctl --user stop opencode-memory opencode-memory-daemon opencode-memory-worker
rm -rf ~/.local/share/opencode-memory/
systemctl --user start opencode-memory opencode-memory-daemon opencode-memory-worker
```

**macOS:**
```bash
for svc in com.opencode.memory com.opencode.memory.daemon com.opencode.memory.worker; do
    launchctl unload ~/Library/LaunchAgents/${svc}.plist 2>/dev/null
done
rm -rf ~/.local/share/opencode-memory/
for svc in com.opencode.memory com.opencode.memory.daemon com.opencode.memory.worker; do
    launchctl load ~/Library/LaunchAgents/${svc}.plist
done
```

### Upgrade from older versions

If you're upgrading from a version that used `opencode-memory-enrich.service`, the setup script will automatically migrate to the new 3-process architecture. Run the setup script again:

```bash
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash
```

This will:
1. Stop and disable the old `opencode-memory-enrich` service
2. Install the new `opencode-memory-daemon` and `opencode-memory-worker` services
3. Start all services

## License

MIT
