Metadata-Version: 2.4
Name: yait-byheart
Version: 0.1.0
Summary: Conversation memory that knows the dialogue by heart: verbatim, zero infrastructure, cache-warm.
Author-email: yait <hello@yait.io>
License: MIT
Project-URL: Homepage, https://github.com/yaitio/byheart
Keywords: memory,llm,conversation,context,prompt-cache
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# byheart

Conversation memory that knows the dialogue **by heart**.

Most memory layers take notes about the conversation: they extract facts,
rewrite them, and discard the rest. byheart remembers the conversation
verbatim — what reaches the model is the dialogue's own words, selected, never
paraphrased. Nothing is lost at ingest, because nothing is extracted at ingest.

```
~400 lines of code      stdlib only: math, re, datetime
0 model calls           at ingest and at composition
0 infrastructure        no server, no vector store, no embeddings
```

## Install

```bash
pip install yait-byheart
```

## Use

```python
from yait_byheart import Memory

mem = Memory(budget=20_000)          # characters, ≈ 5 000 tokens

mem.append("user", "we decided on 55 euro a month", when="8 May, 2023")
mem.append("assistant", "Noted — 55/month, annual at a discount.")
# ... hundreds of turns later ...

result = mem.compose("what did we decide about pricing?")
result["messages"]      # ready for any provider, in the universal format
result["stable_upto"]   # cache breakpoint: everything up to here repeats
                        # verbatim between turns — mark it and pay a tenth
```

`compose` returns a plain messages array, so it drops into any framework and
requires none. Persistence is `mem.to_dict()` / `Memory.from_dict(...)` —
plain JSON-safe data; where it lives is your business.

## How it works — four rules

1. **Below the budget, send everything.** A full history on a warm prompt
   cache is cheaper than any clever assembly.
2. **Above it, select whole topic stretches by relevance.** Everything
   competes, the newest turns included — recency holds its place by being on
   topic, not by reservation. Grouping turns into stretches is worth +0.10
   evidence recall over single turns; the boundaries come from a character
   counter, because an LLM segmenter measured no better (±0.5 pp, 0.3 σ) and
   cost twenty model calls per conversation.
3. **Keep the selection still.** The selection changes only when the subject
   genuinely moves, so the prompt prefix stays byte-identical between turns
   and the provider's cache holds. Measured live: **−90.8 %** of the per-turn
   bill, with every cache miss falling exactly on a rebuild. This is the rule
   competitors cannot copy — a selection recomputed per query has no stable
   prefix to cache, by construction.
4. **Outside knowledge arrives last, and only when it beats the conversation
   itself.** A comparison, not a threshold: nothing to tune, and it holds at
   92 % junk rejected / 89 % needed admitted on the free retriever.

Plus one repair that is not a rule: relative dates are resolved by arithmetic
at ingest — *"I ran the race last Saturday"* becomes *"… last Saturday
[20 May 2023]"*, annotation beside the words, never instead of them. Worth
+11 points on time-anchored questions.

## Measured

Everything below is reproducible from `eval/`; the raw traces, including every
model answer and every judge verdict, are in `eval/results/`.

| | |
|---|---|
| LoCoMo J-score (Mem0's own rubric and prompt) | **87.3 %** — Mem0 reports 92.5 |
| vs. plain RAG, conversation questions | **+13.3 pp** (2.7 σ) |
| vs. plain RAG, all mixed questions | **+6.1 pp** (2.1 σ) |
| unanswerable questions, honest refusals | **99 / 100** |
| per-turn cost, warm cache, live calls | **740 effective tokens** (plain RAG: 2 320) |

The five points behind Mem0 are the price of extraction never happening. The
threefold running-cost advantage is the price extraction keeps charging them:
two model calls per ingested exchange, and a per-query context no cache can
hold.

## Limits, honestly

- **English-leaning retrieval.** BM25 with a light English stemmer. It
  degrades gracefully on paraphrase (measured), but across languages lexical
  match fails entirely — the Russian word for "memory" shares no character
  with it. `retriever`
  accepts anything with `fit`/`rank` if you need dense retrieval — a
  cosine/BM25 hybrid was built and measured here, and netted zero on a
  monolingual corpus, which is why none ships.
- **The cache economics assume an active conversation.** The provider cache
  refreshes on read but expires in minutes of silence; a cold return pays full
  price once.
- **Benchmarked on one model pair** (Haiku answering, Sonnet judging), 385 of
  1 536 LoCoMo questions, stepped with pre-registered stopping rules.
- **Not thread-safe.** One conversation, one `Memory`, one thread.

## Repository

```
yait_byheart/  the package — memory.py, _time.py, ~400 lines of code
tests/         47 tests, including the live-usage path no bench covers
eval/          every benchmark, the measurement protocol, raw result traces
```
