# Changelog

All notable changes to this project are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [Unreleased]

Resolves issues from [ProjectPlan/BaseCML/Issues.md](ProjectPlan/BaseCML/Issues.md). See below for issue ID mapping.

Implementations from [ProjectPlan/BaseCML/CML_Audit_2026-03-06.md](ProjectPlan/BaseCML/CML_Audit_2026-03-06.md) (AUD-01–AUD-12) and [ProjectPlan/BaseCML/Packages_Audit_2026-03-06.md](ProjectPlan/BaseCML/Packages_Audit_2026-03-06.md) (PKG-01–PKG-09).

### Added

- **Modeling preflight validator + manifest v2 (MOD-R2, MOD-R5, MOD-R7)** - Training now records `manifest_schema_version = 2` with `configured_tasks`, `preflight_validation`, `task_training_status`, and `build_metadata` (python/dependency/git provenance). Artifact validation runs before success exit.
- **Task capability API for runtime gating (MOD-R6)** - `ModelPackRuntime` now exposes `supports_task(task)` and `capability_report()` so retrieval/extraction/reconsolidation modules can gate task usage by capability instead of broad modelpack availability.
- **Dashboard Facts Explorer page** — New `/dashboard/#facts` page for browsing, filtering, and invalidating semantic facts. Includes category and tenant filters, current-only toggle, paginated table with confidence gauges, and inline invalidation. Backend: `GET /api/v1/dashboard/facts` and `POST /api/v1/dashboard/facts/{id}/invalidate` (in `fact_routes.py`). Frontend: `facts.js` page module, API methods `getFacts()` and `invalidateFact()`.
- **Dashboard Models Status endpoint** — New `GET /api/v1/dashboard/models/status` returns loaded modelpack families, task models, load errors, and models directory path. Backend: `src/api/dashboard/models_routes.py`. Frontend: Components page shows a "Custom Models (ModelPack)" card with KPI counts, family/task tag lists, and load error details.
- **Retrieval supersedes_id** — `RetrievalResultItem` and `DashboardMemoryDetail` now include `supersedes_id: UUID | None` for semantic lineage. Retrieval test page shows supersession badges and links. Memory detail page shows a visual lineage chain (previous/current version nodes) in the Provenance section.
- **py-cml RetrievalResultItem sync** — Added `supersedes_id: UUID | None = None` to `packages/py-cml/src/cml/models/responses.py` `RetrievalResultItem` to match the server-side schema.
- **Local unified write extractor** — `src/extraction/local_unified_extractor.py` provides `LocalUnifiedWriteExtractor`, a model-based alternative to the LLM unified extractor. Composes `fact_extraction_structured`, `write_importance_regression`, `pii_span_detection`, and `memory_type` router model. Used as fallback in `HippocampalStore` when LLM is disabled.
- **Modelpack write path wiring** — `LocalUnifiedWriteExtractor` now predicts `context_tag`, `confidence_bin`, and `decay_profile` from the router model, providing content-derived context tags, confidence, and decay rate when `FEATURES__USE_LLM_ENABLED=false`. `MemoryOrchestrator.create()` and `create_lite()` instantiate and wire `LocalUnifiedWriteExtractor` into `HippocampalStore` when the LLM unified extractor is not used. Both `encode_chunk` and `encode_batch` consume these fields from the local extractor result. Label-to-value mappings: confidence_bin (low→0.35, medium→0.65, high→0.9), decay_profile (very_fast→0.35, fast→0.2, medium→0.1, slow→0.05, very_slow→0.02).
- **Shadow mode logging** — `src/utils/shadow_logger.py` provides `ShadowModeLogger` for parallel heuristic/model execution and comparison. `compare()` runs both paths, logs latency deltas and decision disagreements. Configurable `sample_rate` for production use.
- **Conflict evaluation framework** — `src/evaluation/conflict_eval.py` provides `ConflictDetectorEvaluator` for offline comparison of heuristic vs. model-based conflict detection. Loads JSONL corpus, computes precision/recall/F1 for both paths, generates shadow comparison reports.
- **Semantic lineage APIs** — `SemanticFactStore.get_fact_lineage(tenant_id, key=..., fact_id=...)` returns the full supersession chain (oldest to newest). `get_superseded_chain(tenant_id, fact_id)` returns all facts superseded by a given fact. Lineage metadata captured during consolidation and reconsolidation; surfaced via `supersedes_id` in retrieval API responses.
- **Multilingual data preparation** — `packages/models/scripts/prepare.py` supports multilingual synthetic data generation across 15 languages (English, Chinese, Spanish, Arabic, Hindi, Portuguese, French, Japanese, Russian, German, Korean, Turkish, Indonesian, Vietnamese, Italian). New `packages/models/scripts/multilingual_prompts.py` defines `SUPPORTED_LANGUAGES`, `pick_language()`, and language-specific system/user prompt builders. Config: `[multilingual]` section in `model_pipeline.toml` with `enabled = true` and `english_weight = 0.30`. CLI: `--no-multilingual` for English-only. Output parquet includes a `language` column.
- **10 task-specific models** — Custom model pipeline (`packages/models/`) trains lightweight task-specific models replacing heuristic logic: `retrieval_constraint_relevance_pair`, `memory_rerank_pair`, `novelty_pair`, `fact_extraction_structured`, `schema_match_pair`, `reconsolidation_candidate_pair`, `write_importance_regression`, `pii_span_detection`, `consolidation_gist_quality`, `forgetting_action_policy`. All paths preserve deterministic fallbacks. Runtime adapter: `src/utils/modelpack.py`.
- **Content length validation (F-04)** — `WriteMemoryRequest.content` enforces `min_length=1` and `max_length=100_000`. Requests exceeding 100,000 characters return 400. See `ProjectPlan/UsageDocumentation.md` § POST /memory/write.
- **API versioning documentation (S-09)** — New `docs/api-versioning.md` describing v1 stability, deprecation policy, and version transition strategy.
- **Retrieval metadata (E-01)** — Optional `retrieval_meta` field in `ReadMemoryResponse` for sources completed/timed out and elapsed ms (future use).
- **Embedding retries (F-10)** — Tenacity retries on `OpenAIEmbeddings.embed` and `embed_batch` for transient network errors (3 attempts, exponential backoff).
- **LLM memory type in Unified Extractor** — When `FEATURES__USE_LLM_MEMORY_TYPE` is true (default), the Unified Write Path Extractor outputs `memory_type` per chunk in the same LLM call used for constraints, facts, salience, importance, and PII.
- **Extractor plugin registry (A-01)** — New `src/extraction/registry.py` with decorator-based `ExtractorRegistry` for registering custom entity, relation, constraint, and fact extractors without modifying core source. Thread-safe singleton `extractor_registry`.
- **Per-tenant feature flag overrides (A-02)** — New `src/core/tenant_flags.py` with Redis-backed `TenantFeatureOverrides`. Enables per-tenant A/B testing and gradual rollout of LLM features via `get_tenant_overrides()`, `set_tenant_overrides()`, and `apply_overrides()`.
- **OpenTelemetry tracing stubs (A-07)** — New `src/utils/tracing.py` with `trace_span()` and `async_trace_span()` context managers. No-ops when OpenTelemetry SDK is not installed; full distributed tracing when `opentelemetry-sdk` is present. `configure_tracing()` for startup bootstrap.
- **Streaming read endpoint (E-02)** — New `POST /memory/read/stream` SSE endpoint for progressive rendering. Returns each memory as a `data:` event, with `event: done` (count + elapsed ms) or `event: error` on completion.
- **ModelPack capability reporting (PKG-03)** — `ModelPackRuntime` now exposes `available_families` and `available_tasks` for partial-load transparency. INFO-level logging lists loaded vs pending families when manifest references models not yet trained. `CML_MODELS_DIR` environment variable overrides default models path.
- **Packages audit tests** — New `tests/unit/test_packages_audit.py` with 29 unit tests covering ModelPack capability reporting, retry delay wiring, export/import fidelity, training strict mode, legacy wrapper cleanup, CML_MODELS_DIR, eval CLI paths, client parity, shared endpoints, per-task model loading, and packaging structure.
- **Ranked memories in read/turn API (AUD-04)** — `MemoryPacket` has `ranked_memories`; `MemoryPacketBuilder.build()` preserves reranker order; API responses use `packet.ranked_memories` for the `memories` field so API order matches relevance.
- **Associative graph expansion (AUD-07)** — Retriever derives graph seeds from top constraint/fact results (entities, subjects, scopes) and runs a post-retrieval GRAPH step when the plan had no GRAPH step, improving semantic-disconnect recall.
- **Batch graph edge writes (AUD-09)** — `Neo4jGraphStore.merge_edges_batch()` merges multiple edges via `UNWIND`; `NeocorticalStore.store_relations_batch()` uses it. Graph sync can run as a background task to avoid blocking the write response.
- **Trigram indexes for semantic fact search (AUD-10)** — Migration `002_add_trigram_indexes` adds `pg_trgm` and GIN indexes on `semantic_facts.key`, `subject`, and `value::text` for efficient `ILIKE` search.
- **CML audit tests** — New `tests/unit/test_audit_fixes.py` coverage for derived graph seeds, post-retrieval graph expansion, batch edge writes, background graph sync, typed candidate selection, type-aware conflict detection, and constraint-type-aware revision strategy.

### Changed

- **Strict-by-default modeling contract (MOD-R1, MOD-R3, MOD-R4)** - `TrainConfig.strict` defaults to `True`; `cml-models train` and `cml-models pipeline` accept `--strict` / `--allow-skips`; strict mode hard-fails on deferred/unsupported objectives, selected disabled tasks, zero-row task splits, and artifact mismatches.
- **Deferred task defaults in model pipeline (MOD-R3, MOD-R4)** - `write_importance_regression`, `fact_extraction_structured`, and `pii_span_detection` stay disabled in default config until score supervision/token trainer contracts are available.
- **Dashboard visual overhaul** — Comprehensive UI refresh across all dashboard pages. KPI cards now have accent-colored left borders and hover lift effects. Tables have smooth row hover transitions. Charts use gradient fills, animations, and percentage tooltips. Sidebar navigation uses inline SVGs with active-state left-border highlight. All pages include standardized `.page-desc` introductory paragraphs. Empty states have animated icons. Cards and panels have hover shadow effects.
- **Dashboard graph display** — Neovis.js configuration updated to render relationship edge labels (`predicate`), improved DOM flushing before initialization, and refined physics settings (spring length, gravitational constant, stabilization).
- **Dashboard session navigation** — Fixed race condition in sessions page where `navigateTo('memories')` was called before `sessionStorage.setItem('cml_filter_session_id', ...)`, causing the memories page to miss the session filter.
- **Dashboard charts** — All chart types (doughnut, bar, line) enhanced: doughnut charts animate rotation/scale with percentage tooltips; bar charts have rounded corners and hover highlight; line charts use linear gradient fills with styled data points.
- **Dashboard sidebar icons** — Replaced HTML entity icons with inline SVG icons for visual consistency across all navigation items.
- **Dashboard module structure (S-03)** — Split `dashboard_routes.py` (2,765 lines) into `src/api/dashboard/` package: `overview_routes`, `memory_routes`, `events_routes`, `graph_routes`, `config_routes`, `jobs_routes`, `fact_routes`, `models_routes`, `_shared`. Import via `from .dashboard import dashboard_router`.
- **Orchestrator write() decomposition (S-01)** — Refactored 337-line `write()` into phases: `_phase_ingest`, `_phase_unified_extraction`, `_phase_deactivate_constraints`, `_phase_encode_and_store`, `_phase_write_time_facts`, `_phase_write_constraints`, `_build_write_response`.
- **DatabaseManager initialization (F-09)** — Added `async DatabaseManager.create()` for deterministic cleanup on init failure. App lifespan uses `await DatabaseManager.create()`; `get_instance()` retained for Celery/legacy.
- **Exception handling (S-07)** — `orchestrator.update()` raises `MemoryNotFoundError` (404) and `MemoryAccessDenied` (403). FastAPI exception handlers map these to JSON responses.
- **Logging consistency (S-06)** — Standardized on structlog (`get_logger`) across orchestrator, storage, retrieval, reconsolidation, and forgetting; replaced stdlib `logging`.
- **Forget performance (F-02, F-05)** — Added `get_by_ids_batch()` to reduce N+1 queries; time-based forgetting now uses cursor pagination (no 500-record cap).
- **Graph sync parallelization (F-07)** — `_sync_to_graph` parallelizes entity and relation merges via `asyncio.gather`.
- **Rate limiter (F-08)** — In-memory fallback uses `TTLCache` (maxsize=10000, ttl=120s) instead of unbounded dict.
- **Dashboard config write-protection (F-12)** — Write-protected fields (`auth.api_key`, `auth.admin_api_key`, `database.neo4j_password`, etc.) cannot be modified via dashboard; returns 403.
- **Tenant override audit (F-11)** — Admin key tenant override logs `tenant_override_used` with admin_key_hash, original_tenant, target_tenant.
- **CSRF header (F-13)** — Dashboard POST/PUT/DELETE/PATCH require `X-Requested-With: XMLHttpRequest`.
- **Abstract store types (S-05)** — ReconsolidationService, ForgettingWorker, ConsolidationWorker, migrator, sampler use `MemoryStoreBase`; removed `cast(PostgresMemoryStore)` in create_lite.
- **Session read deprecation (E-03)** — `POST /session/{session_id}/read` marked deprecated; use `/memory/read` instead.
- **Coverage gate (A-04)** — `fail_under = 50` in pyproject.toml (target 70% over time).
- **Memory type override warning (F-01)** — Invalid `memory_type` override now logs `invalid_memory_type_override` instead of silent `pass`.
- **API key build caching (F-06)** — `_build_api_keys()` decorated with `@lru_cache`; call `_build_api_keys.cache_clear()` when mutating settings in tests.
- **Silent error handling (AI-01)** — Request counter middleware logs `request_counter_increment_failed` on exception instead of bare `pass`; DatabaseManager init failure logs with `exc_info=True`.
- **Magic numbers (AI-03)** — Named constants: `_PG_POOL_SIZE`, `_PG_MAX_OVERFLOW` in connection; `_REQUEST_COUNT_TTL` in middleware; `_SESSION_CONTEXT_LIMIT` in orchestrator; `_CONFLICT_TOP_K` in reconsolidation.
- **WritePathConfig (S-02)** — Frozen `WritePathConfig` dataclass resolves all write-path feature flags once at `write()` entry. Replaces repeated `get_settings()` calls and compound flag checks in `_phase_unified_extraction`, `_phase_deactivate_constraints`, `_phase_write_time_facts`, `_phase_write_constraints`.
- **Consolidated deferred imports (S-04)** — `HippocampalStore.encode_chunk()` loads `get_settings()` once at method start instead of 4 separate deferred imports.
- **Coverage threshold (A-04)** — `fail_under` bumped from 50% to 60% (target 70%).
- **Ruff, Black, and mypy (IA-04)** — Resolved Ruff issues (TC006, F841, B010); Black formatting; mypy passes on `src`.
- **Minimal .env** — `.env` can be trimmed to required and optional-but-recommended variables; see `.env.example`.
- **Config and docs aligned with codebase** — Synced `scripts/init_structure.py`; CONTRIBUTING.md; CI runs ruff on py-cml.
- **Packaging (PKG-01)** — Server-only dependencies (fastapi, uvicorn, neo4j, redis, celery, etc.) moved to `[project.optional-dependencies.server]`. Base package requires only SDK essentials (httpx, pydantic, openai, tiktoken, structlog, etc.). Wheel build includes both `packages/py-cml/src/cml` and `src`.
- **Legacy model scripts (PKG-08)** — `packages/models/scripts/prepare.py` and `train.py` no longer mutate `sys.path`; they fail with a clear install message when `cml.modeling.prepare` or `cml.modeling.train` is not importable.
- **Eval CLI path handling (PKG-08)** — Eval commands require explicit paths when run outside the repo; `_require_path()` raises a clear error when repo root cannot be detected and required path defaults are `None`.
- **Turn path constraint-first formatting (AUD-03)** — `/memory/turn` now uses `MemoryPacketBuilder.to_llm_context()` for context assembly so constraints appear first, matching `/memory/read` behavior.
- **Planner constraint fallback (AUD-05)** — When `constraint_categories` is empty or missing, the retriever skips constraint retrieval instead of expanding to all five cognitive categories, avoiding irrelevant constraints on generic queries.
- **Constraint metadata and provenance (AUD-06)** — `_fact_to_record()` populates `record.metadata` with `constraints` (type, evidence_ids), `valid_from`, `valid_to`, `supersedes_id`. `get_facts_by_category` and `get_facts_by_categories` use `ORDER BY confidence DESC, updated_at DESC`.
- **Constraint key identity (AUD-02)** — `constraint_fact_key()` now includes a normalized description hash so distinct constraints of the same type and scope get different keys and can coexist; only same-description updates supersede.
- **Type-preserving consolidation (AUD-08)** — Summarizer `_fallback_gist_type()` preserves specific constraint subtypes (goal, state, value, causal, policy); worker `_fallback_gist` keeps dominant subtype instead of collapsing to "policy".
- **Batched evidence mapping (AUD-11)** — Write-time facts and constraints use a chunk-to-record-ID map so each chunk’s facts point to the correct stored memory ID instead of `stored[0].id`.
- **Typed constraint reconsolidation (AUD-12)** — Reconsolidation service prioritizes same-constraint-type candidates; conflict detector applies type-aware pre-filtering (different type/scope → NONE, same type+subject → CORRECTION); belief revision uses stable vs volatile type for strategy (CORRECTION vs TIME_SLICE).
- **Non-LLM write path semantic fidelity (AUD-01)** — Hippocampal store uses local extractor `memory_type` when LLM is disabled; local extractor `facts` are written as semantic facts; `WriteTimeFactExtractor` processes `STATEMENT` chunks when local classification is present.

### Tests

- **Modeling contract and probes (MOD-R8)** - Added unit coverage for modeling validation and strict-path behavior (`packages/py-cml/tests/unit/test_modeling_validation.py`) plus probe tests (`tests/unit/test_models_artifact_probe.py`, `tests/unit/test_constraint_retrieval_probe.py`, `tests/unit/test_package_surface_probe.py`). `scripts/models_artifact_probe.py` now supports `--fail-on-mismatch`.
- **CML and Packages audits** — `test_audit_fixes.py` covers CML Audit (graph seeds, batch edges, typed reconsolidation); `test_packages_audit.py` covers Packages Audit (ModelPack, retry, import fidelity, training strict, paths, client parity). `test_deep_research_improvements.py`: STATEMENT chunks processed for write-time facts when local classification present.
- **Custom model pipeline** — Unit tests for constraint layer (`test_constraint_layer.py`), hippocampal write gate and redactor (`test_hippocampal_write_gate_redactor.py`), reconsolidation labile conflict belief (`test_reconsolidation_labile_conflict_belief.py`), consolidation worker guardrails (`test_consolidation_worker_guardrails.py`), NER normalization (`test_utils_ner_normalization.py`), prepare.py incremental merging (`test_models_prepare_incremental.py`).
- **Dashboard and retrieval** — Tests for retrieval classifier/planner/reranker pipeline, query classifier constraints, orchestrator seamless provider, forgetting executor/scorer/policy, config/summarizer internals, HF summarizer utility.
- **py-cml embedded client** — Unit tests for embedded client (`test_embedded_client.py`).
- **Content length validation** — Unit tests for `WriteMemoryRequest` (max/min length, at-limit); E2E test for write exceeding 100k returns 4xx.
- **Dashboard CSRF** — Unit tests include `X-Requested-With: XMLHttpRequest` for dashboard POST; added `test_dashboard_post_without_csrf_header_returns_403`.
- **Exception handlers** — Unit tests for `update()` 404 (MemoryNotFoundError) and 403 (MemoryAccessDenied) via mocked orchestrator.
- **DatabaseManager** — `test_timestamp_ingestion` uses `await DatabaseManager.create()`.
- **init_structure** — `scripts/init_structure.py` updated with dashboard package structure (replaces `dashboard_routes.py`).
- **py-cml SDK** — Added unit tests for WriteRequest content validation; transport CSRF header tests; ReadResponse retrieval_meta parse test.

### Dependencies

- **cachetools** — For TTLCache in rate limiter in-memory fallback (F-08).
- **tenacity** — For embedding API retries (F-10).

### Issues Resolved (from Issues.md)

| ID | Category | Resolution |
|----|----------|------------|
| F-04 | Security | Content length validation (1–100k chars) |
| F-09 | Reliability | DatabaseManager async `create()` with deterministic cleanup |
| F-02, F-05 | Performance | `get_by_ids_batch()`; cursor pagination for time-based forget |
| F-11 | Security | Tenant override audit logging |
| F-12 | Security | Dashboard config write-protected fields (403) |
| F-13 | Security | CSRF header for dashboard state-changing requests |
| F-07 | Performance | Graph sync parallelized via `asyncio.gather` |
| F-08 | Performance | TTLCache for rate limiter (bounded memory) |
| F-10 | Reliability | Tenacity retry on embedding API calls |
| S-03 | Structure | Dashboard split into `api/dashboard/` package |
| S-01 | Readability | Orchestrator `write()` phase decomposition |
| S-06 | Consistency | Structlog throughout (replaced stdlib logging) |
| S-07 | Consistency | MemoryNotFoundError/MemoryAccessDenied + FastAPI handlers |
| S-05 | Modularity | Abstract `MemoryStoreBase` in consumers |
| S-09 | Documentation | `docs/api-versioning.md` |
| E-01 | Responsiveness | `retrieval_meta` in ReadMemoryResponse |
| E-03 | Intuitiveness | `session_read` deprecated |
| A-04 | Maintainability | Coverage `fail_under = 50` |
| F-01 | Correctness | Memory type override warning (no silent pass) |
| F-06 | Performance | `@lru_cache` on `_build_api_keys` |
| AI-01 | Error handling | Log instead of silent `except: pass` |
| AI-03 | Readability | Named constants for magic numbers |
| S-02 | Readability | WritePathConfig eliminates repeated flag checks |
| S-04 | Readability | Consolidated deferred imports in encode_chunk |
| A-01 | Extensibility | ExtractorRegistry plugin pattern |
| A-02 | Scalability | Per-tenant feature flag overrides (Redis) |
| A-07 | Observability | OpenTelemetry tracing stubs |
| E-02 | Responsiveness | SSE streaming read endpoint |
| IA-04 | Lint | Ruff/Black/mypy fixes |

### Audit resolutions (CML_Audit_2026-03-06, Packages_Audit_2026-03-06)

| ID | Audit | Resolution |
|----|--------|------------|
| AUD-01 | CML | Non-LLM write path: local memory_type and facts honored; STATEMENT chunks processed when local classification present |
| AUD-02 | CML | constraint_fact_key includes description hash for distinct keys per constraint |
| AUD-03 | CML | /memory/turn uses MemoryPacketBuilder.to_llm_context (constraint-first) |
| AUD-04 | CML | ranked_memories preserved; API memories use reranker order |
| AUD-05 | CML | No broad constraint fallback; skip constraints when categories empty |
| AUD-06 | CML | _fact_to_record metadata; ORDER BY in fact_store category fetches |
| AUD-07 | CML | Post-retrieval graph expansion from derived seeds |
| AUD-08 | CML | Type-preserving consolidation (fallback_gist_type, worker) |
| AUD-09 | CML | merge_edges_batch; optional background graph sync |
| AUD-10 | CML | Migration 002: pg_trgm GIN indexes on semantic_facts |
| AUD-11 | CML | Chunk-to-evidence ID mapping for batched writes |
| AUD-12 | CML | Typed reconsolidation (candidates, conflict pre-filter, revision strategy) |
| PKG-01 | Packages | Server deps in optional server extra; base SDK slimmed |
| PKG-02 | Packages | CML_MODELS_DIR; EmbeddedCognitiveMemoryLayer models_dir param |
| PKG-03 | Packages | available_families, available_tasks; pending-families INFO logging |
| PKG-04 | Packages | config.max_retry_delay wired into retry backoff |
| PKG-05 | Packages | embedded_utils import preserves type, timestamp, confidence, etc. |
| PKG-06 | Packages | Token-classification tasks disabled; --strict; README planned tasks |
| PKG-08 | Packages | No sys.path in legacy scripts; eval CLI _require_path |
| PKG-09 | Packages | _endpoints.py shared helpers; client write/read use shared builders |

### Breaking

- **Environment variable consolidation (client)** — Client and examples now use a single canonical env var per concept; fallbacks to alternate names were removed. Set **`CML_BASE_URL`** (no longer `MEMORY_API_URL`), **`CML_API_KEY`** (no longer `MEMORY_API_KEY` or `AUTH__API_KEY` for client), and **`CML_TIMEOUT`** where a timeout is needed (no longer `MEMORY_API_TIMEOUT`). py-cml reads only `CML_BASE_URL` and `CML_API_KEY` from the environment. For local dev, set `CML_API_KEY` to the same value as the server's `AUTH__API_KEY`. See `.env.example` migration note.

- **LLM and embedding config consolidation** — Removed `LLMSettings` and `LLM__*` env vars entirely. Use only `LLM_INTERNAL__*` for internal memory tasks and `LLM_EVAL__*` for evaluation (QA, judge); if `LLM_EVAL__*` is unset, `LLM_INTERNAL__*` is used. Renamed `EMBEDDING__*` to `EMBEDDING_INTERNAL__*`; when embedding config is unset, defaults to `nomic-ai/nomic-embed-text-v2-moe` (768 dims, 512 max seq). Added `FEATURES__USE_LLM_ENABLED` (default `false`) as master switch; when false, all internal LLM usage is disabled and heuristics are used. No backward compatibility: migrate `LLM__*` → `LLM_INTERNAL__*` and `EMBEDDING__*` → `EMBEDDING_INTERNAL__*` in `.env`. See `.env.example` and plan `llm_embedding_config_cleanup_32cb4f39.plan.md`.

### Fixed

- **Evaluation progress tracking by turn** — The Phase A ingestion progress bar in `eval_locomo_plus.py` now tracks progress by individual conversation turns rather than whole samples. Previously, the progress bar could appear stuck at 0% during slow ingestion phases or when individual turns triggered retry backoffs (e.g., due to database deadlocks), because an entire sample had to complete before the bar updated. Tracking by turns provides accurate, real-time feedback as data is inserted into the databases.

- **Dashboard Knowledge Graph Depth control** — The Depth slider on the graph page (`/dashboard/#graph`) now affects both the default overview and entity Explore views. Overview query uses a `$depth` parameter instead of a hardcoded 2-hop path; initial load and tenant change pass the current slider value; changing the slider re-renders the current view (overview or explore) with the new depth. State `lastGraphParams` tracks the last rendered mode so depth changes update the same view.

- **Evaluation progress display in non-TTY contexts** — Progress bars (Phase A ingestion, Phase A-B consolidation, Phase B QA, Phase C judge) now display when `eval_locomo_plus.py` is run as a subprocess from `run_full_eval.py`, e.g. in Cursor/IDE terminals or on Windows where stderr is not a TTY. tqdm calls now use `disable=False` in `eval_locomo_plus.py` and `llm_as_judge.py`; `run_full_eval.py` invokes the eval script with Python `-u` (unbuffered) for responsive output.

## [1.3.3] - 2026-02-22

### Documentation

- **Documentation consolidation** — README restructured to focus on architecture, research, basic usage, and eval highlights; consolidated and deduplicated documentation across the repo.

### Added

- **Dashboard Knowledge Graph: neovis.js (offline)** — Replaced vis-network CDN with [neovis.js](https://github.com/neo4j-contrib/neovis.js) for the Knowledge Graph page. The graph connects directly from the browser to Neo4j (bolt/WebSocket) for visualization. Requires `npm run build` in `src/dashboard/` for local dev; Docker builds the bundle at image build time. New endpoint `GET /api/v1/dashboard/graph/neo4j-config` returns Neo4j connection config for the browser (admin-only). New env var `DATABASE__NEO4J_BROWSER_URL` for when Neo4j is not reachable at `DATABASE__NEO4J_URL` from the browser (e.g. Docker: set to `bolt://localhost:7687`). Cypher updated for Neo4j 5 (`COUNT {}` instead of deprecated `size()`). See `.env.example`, `ProjectPlan/UsageDocumentation.md`.

- **Neo4j graph: Unified Extractor prompt engineering and graph sync** — Fixes incorrect relation names, wrong entity text (e.g. system prompts), and entity_type issues in the knowledge graph. Unified Extractor prompts redesigned with schema-first, few-shot, and explicit exclusion rules (cross-model techniques: Reintech, PromptPort, PromptNER). Entities now use typed objects `{text, normalized, type}` with allowed types (PERSON, LOCATION, ORGANIZATION, etc.); relations use `{subject, predicate, object}` with snake_case predicates. Exclusion rule instructs the LLM to omit system prompts, role instructions, and non-conversational content. Relation schema bug fixed (`subject`/`predicate`/`object` instead of `source`/`target`/`type`). Hippocampal store uses unified entities and relations for graph sync when the unified path is enabled. Fallbacks removed (no regex JSON recovery, no batch-to-per-chunk fallback). New: `tests/unit/test_unified_write_extractor.py` (12 tests), `scripts/verify_neo4j_graph.py` (diagnostic script for Neo4j). See plan: `neo4j_graph_troubleshooting_b3afb682.plan.md`.

### Changed

- **LLM feature flags gating** — When `FEATURES__USE_LLM_*` flags are on (default), only the LLM path updates salience, importance, constraints, PII, and facts; rule-based logic for those fields is skipped. WriteGate accepts optional `unified_result` and uses `unified_result.importance`/`unified_result.salience`/`unified_result.pii_spans` when the respective flags are on. Orchestrator constraint supersession gates on `use_llm_constraint_extractor`. `encode_chunk` runs unified extraction before the gate when the LLM path is enabled. `encode_batch` Phase 1 passes `unified_result` to the gate; Phase 1 regex redaction skipped when PII comes from LLM. See UsageDocumentation § Write Path LLM Gating.

- **Chunker replaced with semchunk** — Replaced `SemanticChunker`, `RuleBasedChunker`, and `ChonkieChunkerAdapter` with a single `SemchunkChunker` using [semchunk](https://github.com/isaacus-dev/semchunk) and a Hugging Face tokenizer. New config: `CHUNKER__TOKENIZER` (default: google/flan-t5-base), `CHUNKER__CHUNK_SIZE` (default: 500), `CHUNKER__OVERLAP_PERCENT` (default: 0.15). Removed `chonkie[semantic]`; added `semchunk` and `transformers`. Removed feature flags: `use_fast_chunker`, `use_chonkie_for_large_text`, `chunker_large_text_threshold_chars`, `chunker_require_llm`. See `.env.example`, README, evaluation/README.

### Added

- **Rule-based extractor LLM replacement** — Implements [SLMReplacementReport.md](ProjectPlan/CustomModel/SLMReplacementReport.md): 8 LLM-based replacements for rule-based extractors with unified write-path extraction (one LLM call per chunk returns constraints, facts, salience, importance, PII spans). New feature flags (default true): `FEATURES__USE_LLM_CONSTRAINT_EXTRACTOR`, `FEATURES__USE_LLM_WRITE_TIME_FACTS`, `FEATURES__CHUNKER_REQUIRE_LLM`, `FEATURES__USE_LLM_SALIENCE_REFINEMENT`, `FEATURES__USE_LLM_PII_REDACTION`, `FEATURES__USE_LLM_WRITE_GATE_IMPORTANCE`, `FEATURES__USE_LLM_CONFLICT_DETECTION_ONLY`. Uses `LLM_INTERNAL__*` when set, else `LLM__*`. New tests: `test_llm_constraint_extractor`, `test_llm_write_time_facts`, `test_chunker_require_llm`, `test_llm_query_classifier_only`, `test_llm_salience_refinement`, `test_llm_pii_redaction`, `test_llm_write_gate_importance`, `test_llm_conflict_detection_only`, and `test_unified_write_path` integration. See `.env.example`, UsageDocumentation § Configuration Reference.

- **BaseCML constraint-first write/read and supersession (ImplementationPlan)** — Implements constraint-first semantics and fixes semantic disconnect, constraint dilution, and stale constraints:
  - **Constraint write path**: Episodic CONSTRAINT records now use `ConstraintExtractor.constraint_fact_key(constraint)` as their `key` (in `encode_chunk` and `encode_batch`) so supersession can match by key. `PostgresMemoryStore.deactivate_constraints_by_key(tenant_id, constraint_key)` sets `status=SILENT` for existing episodic CONSTRAINT records with that key. Orchestrator deactivates by key *before* `encode_batch` for all extracted fact keys so the new constraint is written as ACTIVE and previous ones become SILENT.
  - **Constraint read path**: `RetrievalStep` has optional `constraint_categories: list[str] | None`; the CONSTRAINTS step is built with `constraint_categories=analysis.constraint_dimensions`. `HybridRetriever._retrieve_constraints()` uses `step.constraint_categories` to filter semantic-fact lookup to those categories when present; otherwise uses all cognitive categories (GOAL, VALUE, STATE, CAUSAL, POLICY).
  - **Context assembly**: In `MemoryPacketBuilder._format_markdown`, "Recent Context" includes only episodes with `relevance_score > EPISODE_RELEVANCE_THRESHOLD` (0.4) to avoid diluting constraints. Constraint lines are prefixed with `[!IMPORTANT]` in the markdown.
  - **Consolidation**: After gist extraction, the worker re-runs `ConstraintExtractor` on each gist text and persists extracted constraints as semantic facts via `migrator.semantic.store_fact()` with the same key format, so constraints are not lost in summarization.
  - **Config**: `RerankerSettings` (recency_weight, relevance_weight, confidence_weight, active_constraint_bonus) added under `RetrievalSettings`; `MemoryRetriever` builds `RerankerConfig` from `get_settings().retrieval.reranker`.
  - **Verification**: New unit tests — retrieval plan for "Can I afford dinner?" includes CONSTRAINTS step with `constraint_categories`; packet markdown filters low-relevance episodes and shows `[!IMPORTANT]` for constraints; `search()` calls `increment_access_counts` once with result IDs. Integration test `test_constraint_supersession_first_silent_second_active` asserts first constraint SILENT and second ACTIVE after deactivate + second write. Optional script `scripts/verify_constraint_flow.py` ingests constraints and prints packet markdown for manual check of "Active Constraints (Must Follow)".

- **Dashboard reconsolidate and eval Phase A–B pipeline** — New `POST /api/v1/dashboard/reconsolidate` endpoint to release all labile state for a tenant (no belief revision), with job tracking in `dashboard_jobs`. `LabileStateTracker.release_all_for_tenant(tenant_id)` added for Redis and in-memory backends. Dashboard Management page includes a Reconsolidation panel (tenant selector, "Release labile" button, result and job history). Evaluation script `eval_locomo_plus.py` runs consolidation then reconsolidation for each eval tenant between Phase A (ingestion) and Phase B (QA), unless `--skip-consolidation` is set; the API key must have dashboard/admin permission for this step. Documentation updated in evaluation/README.md, README.md, ProjectPlan/UsageDocumentation.md, packages/py-cml/README.md, and `dashboard_jobs.job_type` comment (values: consolidate, forget, reconsolidate).

- **Cognitive Constraint Layer (LoCoMo-Plus Level-2 optimisation)** — Full implementation of latent constraint extraction, storage, and retrieval based on the constraint-layer deep-research analysis. Addresses issues ISS-01 through ISS-09 across 5 phases:
  - **Phase 1 – Evaluation harness correctness (ISS-04, ISS-05)**: `eval_locomo_plus.py` now parses LoCoMo `DATE:` lines into UTC timestamps and passes them to the CML write API (`timestamp` field). Metadata enriched with `speaker`, `date_str`, `session_idx`. The `COGNITIVE_PROMPT` replaced with a neutral continuation prompt (no memory-aware task disclosure).
  - **Phase 2 – Constraint extraction & storage (ISS-01, ISS-06, ISS-03)**: New `ChunkType.CONSTRAINT` in the working-memory model. `RuleBasedChunker` and `SemanticChunker` detect constraint cue phrases (goals, values, policies, causal reasoning) with salience 0.85+. New `ConstraintExtractor` (`src/extraction/constraint_extractor.py`) with `ConstraintObject` schema and rule-based pattern groups (goal/value/state/causal/policy). Write gate maps `ChunkType.CONSTRAINT` → `MemoryType.CONSTRAINT` with importance boost. `HippocampalStore` runs constraint extraction on every chunk, attaches structured constraints to `metadata["constraints"]`, and overrides memory type when high-confidence. Orchestrator stores constraints as semantic facts gated by `FEATURES__CONSTRAINT_EXTRACTION_ENABLED` (default: true). New cognitive `FactCategory` values (`GOAL`, `STATE`, `VALUE`, `CAUSAL`, `POLICY`) and matching `DEFAULT_FACT_SCHEMAS`.
  - **Phase 3 – Constraint-aware retrieval routing (ISS-02, ISS-07)**: `QueryAnalysis` extended with `constraint_dimensions` and `is_decision_query`. Classifier detects `CONSTRAINT_CHECK` intent via fast patterns ("should I", "can I", "is it ok", "recommend", etc.) and enriches constraint dimensions. New `RetrievalSource.CONSTRAINTS` with highest-priority retrieval step. `HybridRetriever._retrieve_constraints()` uses two-pronged approach: vector search filtered to `MemoryType.CONSTRAINT` + semantic fact lookup across cognitive categories. Packet builder shows up to 6 constraints with provenance in "Active Constraints (Must Follow)" section. Reranker uses type-dependent recency weights (stable=0.05, semi-stable=0.15, volatile=0.2) and caps pairwise diversity at min(n, 50).
  - **Phase 4 – Supersession & consolidation fixes (ISS-08, ISS-03)**: Consolidation sampler includes `MemoryType.CONSTRAINT` with 90-day window (vs 7-day for episodes). Summarizer prompt updated to output cognitive gist types. Schema aligner maps cognitive types to `FactCategory` and generates integration keys. `ConstraintExtractor.detect_supersession()` for same-type+scope constraint replacement.
  - **Phase 5 – Observability & silent-failure fixes (ISS-09)**: Silent `except: pass` blocks in orchestrator replaced with structured `logger.warning()` calls with context. Eval script `--verbose` flag emits per-sample retrieval type counts. `_cml_read` supports `return_full` for diagnostics.
  - **API**: `ReadMemoryResponse` includes `constraints` field (list of `MemoryItem`). Read endpoint populates constraints from `packet.constraints`.
  - **Package exports**: `src/extraction/__init__.py` exports `ConstraintExtractor` and `ConstraintObject`. `WriteTimeFactExtractor` now also processes `ChunkType.CONSTRAINT` chunks.
  - **Config**: New feature flag `FEATURES__CONSTRAINT_EXTRACTION_ENABLED` (default true).

- **Deep-research implementation (BaseCML plan, all 6 phases)** — Performance, correctness, and observability improvements:
  - **Phase 1 – Correctness**: Stable fact keys (SHA256) in consolidation migrator; content-based hippocampal keys so different facts don’t overwrite; write-time fact extraction (`WriteTimeFactExtractor`) for preference/identity/location/occupation when `FEATURES__WRITE_TIME_FACTS_ENABLED` is true.
  - **Phase 2 – Hot path**: Hippocampal `encode_batch()` refactored to gate+redact → single `embed_batch()` → bounded concurrency upsert; optional `AsyncStoragePipeline` (Redis queue) for turn writes; `CachedEmbeddings` wrapper when Redis + `FEATURES__CACHED_EMBEDDINGS_ENABLED`; feature flags `batch_embeddings_enabled`, `store_async`, `cached_embeddings_enabled`.
  - **Phase 3 – Retrieval**: Per-step and total retrieval timeouts (`RETRIEVAL__DEFAULT_STEP_TIMEOUT_MS`, `RETRIEVAL__TOTAL_TIMEOUT_MS`); `QueryAnalysis.user_timezone` and timezone-aware "today"/"yesterday" in planner; cross-group `skip_if_found`; step duration/result metrics.
  - **Phase 4 – Background**: `bulk_dependency_counts()` in Postgres for forgetting (one query vs O(n²)); neocortical `multi_hop_query()` uses `asyncio.gather` for entity lookups.
  - **Phase 5 – Stability**: Sensory buffer stores token IDs (tiktoken), batch decode in `get_text()`; `BoundedStateMap` (LRU+TTL) for working/sensory in-memory state; feature flags `bounded_state_enabled`, `db_dependency_counts`.
  - **Phase 6 – Observability**: Optional HNSW ef_search tuning per query; metrics `RETRIEVAL_STEP_DURATION`, `RETRIEVAL_STEP_RESULT_COUNT`, `RETRIEVAL_TIMEOUT_COUNT`, `FACT_HIT_RATE`.
  - **Config**: `FeatureFlags` and `RetrievalSettings` in `src/core/config.py`; all toggles and retrieval timeouts documented in UsageDocumentation and `.env.example`. New unit tests in `tests/unit/test_deep_research_improvements.py` (38 tests); total server tests 301.

- **user_timezone on read and turn API** — `ReadMemoryRequest` and `ProcessTurnRequest` now accept optional `user_timezone` (IANA string, e.g. `America/New_York`). The retrieval planner uses it for timezone-aware "today"/"yesterday" filters. The Python SDK (`packages/py-cml`) supports `user_timezone` on `read()`, `read_safe()`, and `turn()` (sync, async, and embedded). Embedded `read()` now passes `memory_types`, `since`, and `until` to the orchestrator (previously accepted but not passed). See ProjectPlan/UsageDocumentation.md and packages/py-cml CHANGELOG.

- **Optional LLM/embedding tests** — Integration tests that depend on a real LLM or embedding model now skip (instead of fail) when the service is unavailable.

- **Optional LLM_INTERNAL for internal tasks** — New optional `LLM_INTERNAL__*` env vars for a dedicated model for chunking, entity/relation extraction, consolidation, and forgetting. When set, these tasks use the internal model instead of the primary `LLM__*` model; otherwise they fall back to the primary LLM. Helps accelerate bulk ingestion (e.g. evaluation) by using a smaller/faster model for extraction. See `.env.example`, `ProjectPlan/UsageDocumentation.md`, and `packages/py-cml/docs/configuration.md`.

- **Batch entity and relation extraction** — `RelationExtractor.extract_batch()` processes multiple (chunk, entities) pairs in one LLM call. Hippocampal `encode_batch()` uses batch extraction for Phase 3/4 entity and relation extraction instead of per-chunk calls, reducing latency for bulk writes.

- **Evaluation concurrent ingestion** — `eval_locomo_plus.py` Phase A ingestion supports `--ingestion-workers N` (default 10). When N > 1, samples are ingested concurrently via `ThreadPoolExecutor`; when N == 1, behaviour matches the previous sequential flow with inter-sample delay. See `evaluation/README.md` and `ProjectPlan/LocomoEval/RunEvaluation.md`. Server: `tests/integration/test_forgetting_llm_compression.py` — `llm_client` fixture skips when no LLM is configured; `_ensure_llm_reachable()` skips on any exception (e.g. rate limit 429, connection error); both tests are marked `@pytest.mark.requires_llm`. py-cml: embedded lite-mode write/read tests in `packages/py-cml/tests/embedded/test_lite_mode.py` skip on `ImportError`/`OSError`/`RuntimeError` or when the exception message indicates model/embed/rate-limit issues. See `tests/README.md` § Optional LLM/embedding tests.

### Changed

- **Evaluation: Locomo-Plus only** — Removed legacy `evaluation/locomo/` (cloned LoCoMo repo), `eval_locomo.py`, and `compare_results.py`. All evaluation now uses `eval_locomo_plus.py` with unified LoCoMo + Locomo-Plus (LLM-as-judge). `locomo10.json` moved to `evaluation/locomo_plus/data/`. `run_full_eval.py` runs eval_locomo_plus and prints the performance table. See evaluation/README.md and ProjectPlan/LocomoEval/RunEvaluation.md.
- **Tests and config from .env** — All test and Docker configuration (including `EMBEDDING__DIMENSIONS`) is read from the project root `.env`. Removed the `EMBEDDING__DIMENSIONS` override from `docker/docker-compose.yml` for the `app` service so both `app` and `api` use `.env` and migrations/tests share the same vector dimension. Server test fixtures (`mock_embeddings` in `tests/conftest.py`, `test_retrieval_flow.py`) use `get_settings().embedding.dimensions` instead of hardcoded values.
- **Pytest** — Marker `requires_llm` added in `pyproject.toml` under `[tool.pytest.ini_options]` for integration tests that need a reachable LLM; run with `pytest -m requires_llm` to execute only those tests when the LLM is available.

### Documentation

- **Cognitive Constraint Layer docs** — README: new "Cognitive Constraint Layer (Level-2 Memory)" section documenting constraint extraction pipeline, retrieval routing, consolidation, supersession, and API fields. Updated neuroscience mapping tables (Hippocampal Store: constraint extraction row; Neocortical Store: cognitive fact categories; Retrieval: constraint retrieval). Feature flags table: added `FEATURES__CONSTRAINT_EXTRACTION_ENABLED`. Project structure: extraction folder description updated. UsageDocumentation: `FEATURES__CONSTRAINT_EXTRACTION_ENABLED` in Configuration Reference; `constraints` field in read response formats; `constraint` memory type expanded with cognitive extraction note. `.env.example`: new `FEATURES__CONSTRAINT_EXTRACTION_ENABLED` flag. evaluation/README: new "Level-2 Cognitive Memory" section; `--verbose` flag documented. py-cml CHANGELOG: `constraints` field on `ReadResponse`. py-cml README: constraint mention in server compatibility. py-cml api-reference: `ReadResponse` model updated with `constraints` field and `CONSTRAINT` memory type; models section expanded.
- **Deep-research implementation docs** — README: "Performance & reliability" section (feature flags table), updated neuroscience mapping and project structure, test count 529. UsageDocumentation: Configuration Reference extended with Feature Flags (`FEATURES__*`) and Retrieval settings (`RETRIEVAL__*`), plus retrieval Prometheus metrics. UsageDocumentation: POST /memory/read and POST /memory/turn request bodies now document optional `user_timezone`. README: optional `user_timezone` noted for retrieve and seamless turn with link to UsageDocumentation. `.env.example`: optional `FEATURES__*` and `RETRIEVAL__*` with comments. tests/README: total 301 tests, note on `test_deep_research_improvements.py`. ImplementationPlan: implementation status callout (all 6 phases complete) and links to config docs.
- **tests/README.md** — Expanded Configuration with “Environment and .env” (Auth, Database, Embedding vars), embedding dimensions and Docker note, py-cml tests (host), and “Optional LLM/embedding tests (skip when unavailable)”.
- **.env.example** — Clarified `EMBEDDING__DIMENSIONS` (server, migrations, and tests read from .env; Docker does not override). Added `EMBEDDING__DIMENSIONS` to the embedded section. Updated Part F (development & testing) to state that server and py-cml tests read all variables from this file.
- **README.md** — Run Tests section now states that tests read config (including `EMBEDDING__DIMENSIONS`) from `.env` and points to `tests/README.md`.
- **ProjectPlan/UsageDocumentation.md** — `EMBEDDING__DIMENSIONS` description updated to note that server and tests read from `.env` and Docker app/api use `.env` without override.
- **evaluation/README.md** — Noted that server and tests read `EMBEDDING__DIMENSIONS` from `.env` and Docker `app`/`api` do not override it.
- **packages/py-cml/CONTRIBUTING.md** — Integration/E2E section: tests load config from repo root `.env`, copy `.env.example` to `.env`, and optional LLM/embedding tests skip when the service is unavailable.

### Added (dashboard and features)

- **Dashboard expansion** — Major enhancement of the admin dashboard with 6 new pages and multiple new features:
  - **Tenants page** — Lists all tenants with memory/fact/event counts, active memory counts, last activity timestamps, and quick-link buttons to filter Overview/Memories/Events by tenant.
  - **Sessions page** — Shows active sessions from Redis (with TTL badges and metadata) and memory counts per `source_session_id` from the database. Click a session to filter Memory Explorer.
  - **Knowledge Graph page** — Interactive neovis.js visualization of entities and relations from Neo4j. Connects directly from the browser to Neo4j. Search entities by name, explore neighborhoods with configurable depth (1-5 hops), view node/edge details. Bundled for offline use.
  - **API Usage page** — Current rate-limit buckets with utilization bars, hourly request volume chart (Chart.js). KPI cards for active keys, avg utilization, configured RPM, and 24h request count.
  - **Configuration page** — Read-only config snapshot showing all settings grouped by section (Application, Database, Embedding, LLM, Auth) with secrets masked. Editable settings can be changed inline at runtime (stored in Redis).
  - **Retrieval Test page** — Interactive query tool for debugging memory retrieval. Input tenant + query with optional filters; returns scored memories with relevance bars, type badges, and metadata.
  - **Enhanced Overview** — Reconsolidation queue status KPI and 24h request sparkline chart.
  - **Enhanced Management** — Reconsolidation/labile status per tenant; job history table persisted in PostgreSQL (`dashboard_jobs` table via Alembic migration `002`).
  - **Enhanced Memory Explorer** — Bulk actions (archive/silence/delete selected memories via checkboxes), select-all, JSON export button.
  - **New backend endpoints** — `/sessions`, `/ratelimits`, `/request-stats`, `/graph/stats`, `/graph/explore`, `/graph/search`, `/config` (GET + PUT), `/labile`, `/retrieval`, `/jobs`, `/memories/bulk-action`, `/export/memories`.
  - **Request counting middleware** — `RequestLoggingMiddleware` now increments hourly counters in Redis for the API Usage page.
  - **Alembic migration** — `002_dashboard_jobs.py` adds the `dashboard_jobs` table for consolidation/forgetting job history.

- **X-Eval-Mode header on write** — Optional request header `X-Eval-Mode: true` on `POST /memory/write` and `POST /session/{session_id}/write`. When set, the response includes `eval_outcome` (`"stored"` or `"skipped"`) and `eval_reason` (write-gate reason) so evaluation scripts can aggregate gating statistics. See Usage documentation and `ProjectPlan/LocomoEval/RunEvaluation.md`.
- **LoCoMo evaluation: gating and timing** — `evaluation/scripts/eval_locomo.py` supports `--eval-mode` (sends X-Eval-Mode on writes, writes `locomo10_gating_stats.json`) and `--log-timing` (records per-question CML read and Ollama latency and token usage, writes `locomo10_qa_cml_timing.json`). See evaluation README and RunEvaluation.md §4.2.
- **DELETE /memory/all** — New admin-only endpoint; SDK `delete_all(confirm=True)` is now supported by the server.
- **Startup validation** — `validate_embedding_dimensions()` is called in the app lifespan to catch embedding-dimension vs DB schema mismatch at startup.

- **Temporal fidelity**
  - Optional `timestamp` field in `WriteMemoryRequest` and `ProcessTurnRequest` allows specifying event time for memories
  - Enables historical replay for benchmarks (e.g., Locomo evaluation with session dates)
  - Defaults to current time if not provided (backward compatible)
  - Threads through: API → Orchestrator → Short-term → Working → Chunker → Hippocampal → Storage

- **Documentation**
  - `CONTRIBUTING.md` — Guidelines for contributors and development setup.
  - `SECURITY.md` — Security policy and vulnerability reporting.
  - `.env.example` — Example environment configuration for database, API, LLM, and embedding settings.

- **Public API**
  - Meaningful exports in package `__init__.py` files:
    - `src.core`: `get_settings`, `Settings`, `MemoryContext`, `MemoryType`, `MemoryStatus`, `MemorySource`, `MemoryRecord`, `MemoryRecordCreate`, `MemoryPacket`.
    - `src.api`: `create_app`, `AuthContext`, `get_auth_context`, `require_write_permission`, `require_admin_permission`.
    - `src.storage`: `DatabaseManager`, `Base`, `EventLogModel`, `MemoryRecordModel`, `SemanticFactModel`, `PostgresMemoryStore`.
    - `src.utils`: `EmbeddingClient`, `OpenAIEmbeddings`, `LLMClient`, `get_llm_client`.
    - `src.memory`: `MemoryOrchestrator`, `ShortTermMemory`, `HippocampalStore`, `NeocorticalStore`.
    - `src.retrieval`: `MemoryRetriever`.
    - `src.extraction`: `EntityExtractor`, `FactExtractor`, `LLMFactExtractor`.
    - `src.consolidation`: `ConsolidationReport`, `ConsolidationWorker`.
    - `src.forgetting`: `ForgettingReport`, `ForgettingWorker`, `ForgettingScheduler`.
    - `src.reconsolidation`: `LabileMemory`, `LabileSession`, `LabileStateTracker`, `ReconsolidationResult`, `ReconsolidationService`.

- **Sensory buffer**
  - `SensoryBuffer._tokenize()` now uses tiktoken (cl100k_base) when available for accurate token counting, with a whitespace-split fallback when tiktoken is not installed.

### Changed

- **CORS** — Default origins no longer use a placeholder. Tiered behavior: use `cors_origins` if set; in debug mode use `["*"]`; otherwise use `["http://localhost:3000", "http://localhost:8080"]`.
- **DatabaseManager.close()** — Safe shutdown: only disposes/closes connections that are not `None`, avoiding `AttributeError` on partial initialization.
- **Orchestrator read API** — Read now supports `memory_types`, `since`, and `until`; these are forwarded from the API through the orchestrator into the retriever and applied to retrieval steps.
- **Packaging** — Root package name renamed from `cognitive-memory-layer` to `cml-server` to avoid pip collision with the SDK package.
- **WriteDecision** — `STORE_SYNC` and `STORE_ASYNC` are collapsed into a single `STORE` (aliases kept for compatibility); async write path is not implemented.
- **RetrievalSource** — Removed unused `LEXICAL` enum value (no lexical retrieval implementation).
- **Read format "list"** — When `format` is `"list"`, the response now returns a flat `memories` list with empty `facts`, `preferences`, and `episodes` (differentiated from `"packet"`).

### Fixed

- **Read API `session_read` constraints** — The `/session/{session_id}/read` endpoint now correctly maps and includes constraint memories in the `ReadMemoryResponse` object (previously they were silently dropped from the response).
- **Silent database errors in `encode_batch`** — The phase 4 `upsert` loop in `HippocampalStore.encode_batch` now catches and logs (`logger.error`) underlying database exceptions rather than silently swallowing them and losing data.
- **Unsafe async initialization** — `DatabaseManager.__init__` exception cleanup fallback now avoids calling `asyncio.run()` in a synchronous constructor when no event loop is running, logging the error safely instead of crashing.
- **Retrieval fact typing** — Semantic-fact text search results from the neocortical store are now correctly typed as `semantic_fact` (and appear under `packet.facts`) instead of falling back to episodic.
- **API contract: write** — Request `metadata` is now merged into memory records (user keys override system defaults). Optional `memory_type` is respected as an override for the write gate classification.
- **API contract: read** — Request fields `memory_types`, `since`, and `until` are now applied: they are forwarded from the API through the orchestrator into the retriever and used to filter vector/search steps.
- **Rate limiting** — Rate-limit key is now derived from the `X-API-Key` header (hashed) instead of the spoofable `X-Tenant-Id`; fallback to client IP when no API key is present.
- **CORS** — When `origins` includes `"*"`, `allow_credentials` is set to `False` to comply with the CORS spec.
- **Session scoping** — `get_session_context(session_id)` now filters by `source_session_id`, returning only memories for that session when `session_id` is provided.
- **Stats by type** — `PostgresMemoryStore.count()` now supports `type` (and `since`/`until`) filters so orchestrator `get_stats()` returns correct `by_type` counts.
- **Dashboard routes** — Removed duplicate `/dashboard` route; the catch-all `/dashboard/{rest_of_path:path}` serves the root.
- Empty `src/api/dependencies.py` — Implemented with re-exports of auth dependencies.
- Redundant empty `src/memory/hippocampal/encoder.py` — Removed; encoding is handled by `HippocampalStore` in `store.py`.

---

*For earlier history and design docs, see the [ProjectPlan/](ProjectPlan/) directory (e.g. [ActiveCML/Issues.md](ProjectPlan/ActiveCML/Issues.md), [CreatePackage/CreatePackageStatus.md](ProjectPlan/CreatePackage/CreatePackageStatus.md)).*
