Metadata-Version: 2.4
Name: glyphcache
Version: 1.0.2
Summary: A deterministic exact and Hyper-Glyph-inspired semantic cache for LLM prompts
Project-URL: Homepage, https://github.com/Arkay92/GlyphCache
Project-URL: Repository, https://github.com/Arkay92/GlyphCache
Project-URL: Issues, https://github.com/Arkay92/GlyphCache/issues
Project-URL: Changelog, https://github.com/Arkay92/GlyphCache/blob/main/CHANGELOG.md
Author: Robert McMenemy
License-Expression: MIT
License-File: LICENSE
Keywords: ai,caching,hyperdimensional-computing,llm,prompt-cache,semantic-cache,sqlite
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: numpy>=1.26
Provides-Extra: compression
Requires-Dist: zstandard; extra == 'compression'
Provides-Extra: dev
Requires-Dist: build; extra == 'dev'
Requires-Dist: hypothesis; extra == 'dev'
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-asyncio; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: twine; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs; extra == 'docs'
Requires-Dist: mkdocs-material; extra == 'docs'
Requires-Dist: mkdocstrings[python]; extra == 'docs'
Provides-Extra: embeddings
Requires-Dist: sentence-transformers>=3; extra == 'embeddings'
Description-Content-Type: text/markdown

# GlyphCache

<p align="center">
  Deterministic exact and Hyper-Glyph-inspired semantic caching for LLM responses.
</p>

<p align="center">
  <img width="256" height="256" alt="GlyphCache Logo" src="https://github.com/Arkay92/GlyphCache/blob/main/glyphcache.png" />
</p>

<p align="center">
  <a href="https://github.com/Arkay92/GlyphCache/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/Arkay92/GlyphCache/actions/workflows/ci.yml/badge.svg" /></a>
  <a href="https://github.com/Arkay92/GlyphCache/actions/workflows/publish.yml"><img alt="Publish" src="https://github.com/Arkay92/GlyphCache/actions/workflows/publish.yml/badge.svg" /></a>
  <a href="https://pypi.org/project/glyphcache/"><img alt="PyPI" src="https://img.shields.io/pypi/v/glyphcache.svg" /></a>
  <img alt="Python" src="https://img.shields.io/pypi/pyversions/glyphcache.svg" />
  <img alt="License" src="https://img.shields.io/pypi/l/glyphcache.svg" />
  <img alt="Downloads" src="https://img.shields.io/pypi/dm/glyphcache.svg" />
</p>

> **Package name:** `glyphcache`  
> **Project name:** GlyphCache  
> **Install:** `pip install glyphcache`  
> **Import:** `from glyphcache import PromptCache`

**GlyphCache** is a lightweight, offline-first LLM response cache combining
deterministic exact matching with an opt-in
[Hyper-Glyph](https://github.com/Arkay92/Hyper-Glyph)-inspired semantic
signature index.

It requires no embedding API, vector database, external cache server, or model
download.

GlyphCache combines:

- **Authoritative exact matching** over the complete normalized LLM request.
- **Provider-neutral requests** with messages, tools, schemas, model parameters,
  tenant, namespace, and application cache version.
- **Deterministic Hash-HDC signatures** derived directly from cryptographic hash
  output rather than model embeddings.
- **Immutable semantic prototypes** with compact sparse XOR residual encoding.
- **LSH candidate filtering** that keeps full similarity comparisons bounded.
- **Conservative semantic guards** for numbers, money, dates, URLs, email
  addresses, UUIDs, paths, quoted text, and identifiers.
- **Memory and SQLite backends** with TTL and LRU/size eviction.
- **Synchronous and asynchronous APIs** with in-process single-flight handling.
- **SQLite generation leases** for cross-process duplicate suppression.
- **Cacheability and redaction hooks** for application-specific safety.
- **Statistics and event callbacks** without raw prompt content by default.
- **A small CLI** for initialization, inspection, maintenance, health checks, and
  benchmarks.
- **A typed Python API** designed for extension through backend, serializer, and
  encoder protocols.

---

## Before / After

Without GlyphCache, every request reaches the model:

```python
response = call_model(request)
```

With GlyphCache, compatible cached responses bypass the model call:

```python
from glyphcache import Message, PromptCache, PromptRequest, SQLiteBackend

cache = PromptCache(SQLiteBackend("glyphcache.db"))

request = PromptRequest(
    provider="my-provider",
    model="my-model",
    messages=(
        Message(role="system", content="Answer clearly and concisely."),
        Message(role="user", content="Explain hyperdimensional computing."),
    ),
    params={"temperature": 0, "max_tokens": 500},
)

response = cache.get_or_set(
    request,
    producer=lambda: call_model(request),
    ttl_seconds=86_400,
)
```

The core job is simple:

```text
Complete request
  -> deterministic exact key
  -> compatible cached response

Exact miss and semantic mode enabled
  -> strict semantic scope
  -> compact deterministic signature
  -> bounded candidates
  -> safety guards and acceptance policy
  -> optional semantic response
```

The original prompt is never reconstructed from the semantic representation.
The exact request hash remains separate and authoritative.

---

## Why Not Just an Embedding Cache?

Embedding caches typically require an embedding API or local model, a vector
index, and operational choices around model versions and distance thresholds.
GlyphCache uses a different representation: normalized prompt features are
bundled into a deterministic bit-packed signature, grouped around immutable
prototypes, and filtered through explicit safety controls.

That makes GlyphCache useful when you want a small, inspectable, offline cache
layer. It does not claim to understand language better than neural embeddings,
and semantic reuse is never treated as proof that two requests are equivalent.

---

## Architecture

```text
                     +----------------------+
                     |    PromptRequest     |
                     +----------+-----------+
                                |
                     +----------v-----------+
                     | Canonical normalizer |
                     +------+---------+-----+
                            |         |
                   exact key|         |semantic scope
                            |         |
                 +----------v--+   +--v----------------+
                 | Exact lookup |   | Hash-HDC encoder |
                 +-------+------+   +--------+---------+
                         |                   |
                      hit|             bit signature
                         |                   |
                         |          +--------v---------+
                         |          | Prototype codec  |
                         |          +--------+---------+
                         |                   |
                         |          +--------v---------+
                         |          | LSH candidates   |
                         |          +--------+---------+
                         |                   |
                         |          +--------v---------+
                         |          | Safety guards    |
                         |          +--------+---------+
                         |                   |
                         +---------+---------+
                                   |
                            Cached response
```

The exact key includes provider, model, messages, tool definitions,
structured-output schema, generation parameters, namespace, tenant, and
application cache version. Semantic comparisons remain inside a stricter scope
that also fixes system/developer prompts, encoder version, and safety-policy
version.

---

## Install

```bash
pip install glyphcache
```

For optional Zstandard compression support:

```bash
pip install "glyphcache[compression]"
```

For documentation dependencies:

```bash
pip install "glyphcache[docs]"
```

For development:

```bash
pip install -e ".[dev,compression,docs]"
pytest
python -m build
```

---

## Quick Start

### Exact Cache

```python
from glyphcache import MemoryBackend, Message, PromptCache, PromptRequest

cache = PromptCache(MemoryBackend())
request = PromptRequest(
    provider="example",
    model="example-model",
    messages=(Message("user", "What is HDC?"),),
    params={"temperature": 0},
)

hit = cache.get(request)
if hit is None:
    response = call_model(request)
    cache.set(request, response)
else:
    response = hit.value
```

Exact matching is enabled by default and always checked first.

### Semantic Cache

```python
from glyphcache import CacheConfig, SemanticConfig

config = CacheConfig(
    semantic=SemanticConfig(
        enabled=True,
        similarity_threshold=0.94,
        ambiguity_margin=0.025,
        require_anchor_match=True,
    )
)

cache = PromptCache(MemoryBackend(), config)
```

Semantic caching is opt-in. By default, it only considers single-user-turn,
low-temperature requests without tools or tool messages.

### Async Cache

```python
response = await cache.aget_or_set(
    request,
    producer=lambda: call_model_async(request),
    ttl_seconds=86_400,
)
```

### Multi-Tenant Isolation

```python
request = PromptRequest(
    provider="example",
    model="example-model",
    messages=(Message("user", "Summarize my account."),),
    namespace="support",
    tenant_id="customer-42",
)
```

Tenant and namespace participate in exact keys and semantic scopes, so entries
cannot cross those boundaries.

### Cacheability Hooks

```python
cache = PromptCache(
    MemoryBackend(),
    should_cache_request=lambda request: request.namespace != "sensitive",
    should_cache_response=lambda request, response: not contains_secret(response),
    redact_metadata=lambda request, metadata: {"trace_id": metadata.get("trace_id")},
)
```

GlyphCache cannot automatically recognize every secret, permission boundary,
one-time code, medical or legal response, or side effect. Applications must use
these hooks where appropriate.

---

## CLI

Initialize a SQLite cache:

```bash
glyphcache init ./glyphcache.db
```

Inspect cache totals:

```bash
glyphcache stats ./glyphcache.db
```

Inspect an entry with the response redacted:

```bash
glyphcache inspect ./glyphcache.db --entry ENTRY_ID
```

Explicitly show stored response bytes:

```bash
glyphcache inspect ./glyphcache.db --entry ENTRY_ID --show-response
```

Purge expired entries or a tenant namespace:

```bash
glyphcache purge ./glyphcache.db --expired
glyphcache purge ./glyphcache.db --namespace support --tenant customer-42
```

Run maintenance and health checks:

```bash
glyphcache optimize ./glyphcache.db
glyphcache vacuum ./glyphcache.db
glyphcache doctor ./glyphcache.db
```

Run a local storage benchmark:

```bash
glyphcache benchmark --entries 100000 --database ./benchmark.db
```

---

## Main Features

### 1. Deterministic Exact Keys

Canonicalization normalizes NFKC Unicode, CRLF/CR line endings, insignificant
trailing line whitespace, and mapping insertion order. It rejects NaN and
infinity. Output-affecting fields still change the key.

```python
from glyphcache.canonical import exact_cache_key

key = exact_cache_key(request)
```

### 2. Hash-HDC Signatures

The default semantic encoder derives bipolar feature vectors directly from
SHAKE-256 output. It is role-aware, message-position-aware, token-block-aware,
deterministic across processes, and requires no downloaded model.

A default 4,096-bit signature occupies 512 raw bytes before prototype-residual
encoding.

Hash-HDC is positioned as deterministic lexical and near-duplicate matching,
not general semantic understanding. For genuine semantic input, install
`glyphcache[embeddings]` and use `EmbeddingHDCEncoder`, or pass a custom
`PromptEncoder`.

### 3. Prototype-Residual Encoding

Each semantic scope keeps a bounded set of immutable signatures as prototypes.
When a signature is close to a prototype, GlyphCache stores the differing bit
indices as delta-varints. Dense residuals fall back to the raw signature.

```text
signature:  101101001110...
prototype:  101100001110...
xor:        000001000000...
residual:   [5]
```

### 4. Conservative Safety

A semantic hit must satisfy:

- The same strict semantic scope.
- The configured anchor policy.
- A non-expired and non-rejected entry.
- The minimum similarity threshold.
- The ambiguity margin over the second-best candidate.

Hard negatives such as `Refund order 123` versus `Refund order 124`, Python
versions, CVE identifiers, invoice IDs, and currency amounts are expected to
miss under the default guard policy.

### 5. SQLite Persistence

The standard-library SQLite backend enables WAL mode, foreign keys, a bounded
busy timeout, short transactions, LRU/size eviction, integrity checks,
prototype storage, LSH rows, and expiring generation leases.

SQLite is intended for local disk, not arbitrary shared network filesystems.

### 6. Single-Flight Protection

Concurrent misses for the same exact key share one producer call inside a
process. SQLite-backed synchronous producers also use expiring cross-process
leases and never hold a write transaction while the model call runs.

### 7. Statistics and Events

```python
stats = cache.statistics()

print(stats.exact_hits)
print(stats.semantic_hits)
print(stats.misses)
print(stats.stored_bytes)
```

Events contain IDs, timing, match type, and rejection reason rather than raw
prompt content by default.

### 8. Prepared and Bulk APIs

```python
prepared = cache.prepare(request)
hit = cache.get_prepared(prepared)
cache.set_prepared(prepared, response)

cache.set_many(
    [
        (request_a, response_a),
        (request_b, response_b),
    ]
)
```

Prepared requests remove repeated canonicalization and hashing from unchanged
hot paths. Memory values remain native Python objects; SQLite values are
serialized persistently and access accounting is flushed in batches.

---

## Configuration

```python
from glyphcache import CacheConfig, SemanticConfig

config = CacheConfig(
    default_ttl_seconds=86_400,
    max_entries=100_000,
    max_bytes=1_073_741_824,
    store_request_body=False,
    compress_values=True,
    busy_timeout_ms=5_000,
    lease_seconds=120,
    semantic=SemanticConfig(
        enabled=False,
        dimension=4096,
        token_block_size=16,
        max_features=4096,
        similarity_threshold=0.94,
        ambiguity_margin=0.025,
        prototype_threshold=0.78,
        max_candidates=128,
        max_prototypes_per_scope=32,
        lsh_bands=8,
        lsh_bits_per_band=16,
        require_anchor_match=True,
        single_turn_only=True,
        allow_tools=False,
        maximum_temperature=0.2,
        seed=42,
    ),
)
```

Key settings:

- **`default_ttl_seconds`** controls expiry when a write does not supply a TTL.
- **`max_entries`** and **`max_bytes`** bound backend storage.
- **`similarity_threshold`** is the minimum accepted semantic similarity.
- **`ambiguity_margin`** rejects a best match that is too close to the runner-up.
- **`prototype_threshold`** controls assignment to an existing prototype.
- **`max_candidates`** provides a hard bound on full similarity comparisons.
- **`require_anchor_match`** protects numeric and identifier-sensitive prompts.
- **`single_turn_only`** keeps multi-turn semantic reuse disabled by default.
- **`allow_tools`** keeps tool-bearing requests excluded unless explicitly enabled.
- **`seed`** makes signature and LSH generation deterministic.

The default threshold is a starting point, not a universal constant. Measure
precision and false-hit rate on representative application data.

---

## Benchmarking

The repository includes:

```bash
python benchmarks/benchmark_exact.py --entries 10000
python benchmarks/benchmark_semantic.py
python benchmarks/benchmark_storage.py
python benchmarks/benchmark_competitors.py --entries 1000 --probes 5000 --runs 5
```

The original 1.0.0 comparison and the improved 1.0.1 report are stored in:

- [`benchmarks/results/competitor_benchmark.md`](benchmarks/results/competitor_benchmark.md)
- [`benchmarks/results/competitor_benchmark_1.0.1.md`](benchmarks/results/competitor_benchmark_1.0.1.md)
- [`benchmarks/results/competitor_benchmark_1.0.2.md`](benchmarks/results/competitor_benchmark_1.0.2.md)
- [`benchmarks/results/semantic_quality.md`](benchmarks/results/semantic_quality.md)

The competitor benchmark defaults to five runs and reports medians, sample
variance, bulk insertion throughput, SQLite stage timings, and tail operations
over one millisecond. For reusable semantic presets, use
`SemanticConfig.profile("strict")`, `"balanced"`, or `"recall"`. Prepared
semantic requests cache their encoding with `cache.prepare(request,
semantic=True)`.

Semantic evaluation should report precision, recall, F1, false-hit rate,
candidate counts, signature time, lookup percentiles, prototype compression
rate, and raw-signature fallback rate. Performance reports should identify the
hardware, Python version, operating system, concurrency, database warmth, and
dataset.

The test suite currently covers exact caching, TTL, semantic retrieval, hard
negative guards, signature and codec behavior, SQLite WAL/integrity, eviction,
hooks, CLI operations, and sync/async single-flight handling.

---

## Project Structure

```text
src/glyphcache/
  __init__.py             # Stable public API
  __about__.py            # Runtime package version
  cache.py                # PromptCache lookup, write, and single-flight flow
  canonical.py            # Canonical requests, exact keys, semantic scopes
  cli.py                  # Command-line interface
  config.py               # CacheConfig and SemanticConfig
  decorators.py           # Function caching decorator
  exceptions.py           # Package exceptions
  guards.py               # Numeric and identifier anchor extraction
  models.py               # Typed public and internal data models
  policies.py             # Semantic eligibility policy
  statistics.py           # Runtime statistics collector
  backends/
    base.py               # Backend protocol
    memory.py             # In-process backend
    sqlite.py             # SQLite backend and schema
  codec/
    prototypes.py         # Raw and prototype-XOR signature codec
    signature.py          # Hamming similarity
    varint.py             # Varint and delta-index encoding
  encoders/
    base.py               # Encoder protocol
    hash_hdc.py           # Deterministic Hash-HDC encoder
  indexes/
    lsh.py                # Deterministic LSH bands
  serializers/
    base.py               # Serializer protocol
    json.py               # JSON, text, and bytes serializer
  py.typed                # Typing marker
tests/
  test_*.py               # Unit, safety, storage, and concurrency tests
docs/
  index.md                # Documentation home
  concepts.md             # Exact and semantic concepts
  exact-cache.md          # Exact key behavior
  semantic-cache.md       # Semantic pipeline
  safety.md               # Risks and safety policy
  configuration.md        # Configuration reference
  backends.md             # Backend notes
  cli.md                  # CLI reference
  benchmarks.md           # Benchmark methodology
examples/
  basic_cache.py          # Exact in-memory example
  async_cache.py          # Async single-flight example
  semantic_cache.py       # Semantic cache example
  multitenant_cache.py    # Tenant isolation example
  custom_serializer.py    # Serializer extension example
benchmarks/
  benchmark_exact.py      # Exact-cache throughput and latency
  benchmark_semantic.py   # Paraphrase and hard-negative evaluation
  benchmark_storage.py    # SQLite byte accounting
  benchmark_competitors.py # LangChain, GPTCache, and DiskCache comparison
glyphcache.png            # Project logo
pyproject.toml            # Package metadata and dependencies
CHANGELOG.md              # Release history
CONTRIBUTING.md           # Contribution guide
RELEASE.md                # Release checklist
LICENSE                   # MIT license
```

---

## Development

```bash
# Install development, compression, and documentation extras
pip install -e ".[dev,compression,docs]"

# Run tests with the release coverage threshold
pytest --cov=glyphcache --cov-report=term-missing --cov-fail-under=90

# Run linting and formatting checks
ruff check .
ruff format --check .

# Type-check package code
mypy src/glyphcache

# Build and validate distributions
python -m build
twine check dist/*
```

---

## Security and Limitations

GlyphCache stores response payloads in plaintext unless the application places
its SQLite database on encrypted storage. It does not implement encryption,
distributed consistency, a hosted cache server, GPU indexing, automatic provider
monkey-patching, cross-model reuse, or automatic tool-execution caching.

Semantic caching can return an incorrect response when application-specific
differences are not captured by scope or anchors. Keep it disabled for risky
request classes and favor false misses over false hits.

---

## License

MIT

---

## Contributing

Contributions are welcome. Open an issue or pull request with the request shape,
backend, configuration, expected cache behavior, hard-negative examples, and
benchmark or correctness evidence used to evaluate the change.

---

## Citation

If you use GlyphCache in research, please cite:

```bibtex
@software{GlyphCache2026,
  title={GlyphCache: Deterministic Exact and Hyper-Glyph-Inspired Semantic Caching for LLM Responses},
  author={Robert McMenemy},
  year={2026},
  version={1.0.2},
  url={https://github.com/Arkay92/GlyphCache},
}
```
