Metadata-Version: 2.5
Name: citadeldb-haystack
Version: 2.0.0
Summary: Haystack document store backed by Citadel: encrypted at rest, with deletes that destroy the key
Project-URL: Homepage, https://citadeldb.dev
Project-URL: Repository, https://github.com/yp3y5akh0v/citadel
Author: Yuriy Peysakhov
License-Expression: Apache-2.0
Keywords: document-store,encryption,haystack,rag,vector-store
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: citadeldb<3,>=2.0
Requires-Dist: haystack-ai<4,>=2.9
Provides-Extra: test
Requires-Dist: pytest-asyncio>=0.23; extra == 'test'
Requires-Dist: pytest>=8; extra == 'test'
Description-Content-Type: text/markdown

# citadeldb-haystack

A [Haystack](https://github.com/deepset-ai/haystack) `DocumentStore` backed by
[Citadel](https://citadeldb.dev). Encrypted at rest, embedded in your process, and deletes
that destroy the key, not just the row.

Passes deepset's own `DocumentStoreBaseTests` conformance suite.

```
pip install citadeldb-haystack
```

```python
from haystack import Document
from haystack.utils import Secret
from citadeldb_haystack import CitadelDocumentStore

store = CitadelDocumentStore("corpus.cdl", Secret.from_env_var("CITADEL_KEY"))
# CITADEL_KEY must be set: an env-var secret is what lets a pipeline serialize.

store.write_documents([Document(id="d1", content="...", meta={"chapter": "intro"})])
store.filter_documents({"field": "meta.chapter", "operator": "==", "value": "intro"})
```

`dim` defaults to 768 and must match your embedding model.

## The passphrase never lands in a pipeline file

The passphrase is a Haystack `Secret`. Pipelines are serialized to disk, and a literal
token refuses to serialize, so a passphrase cannot be written into a pipeline by accident:

```python
CitadelDocumentStore("literal.cdl", "literal-passphrase").to_dict()
# ValueError: Cannot serialize token-based secret.

CitadelDocumentStore("corpus.cdl", Secret.from_env_var("CITADEL_KEY")).to_dict()
# {... "key": {"type": "env_var", "env_vars": ["CITADEL_KEY"], ...}}
```

Use `Secret.from_env_var` for any store that goes into a saved pipeline.

## Deletes destroy the key

Every document is sealed under its own key. Deleting destroys that key and then removes the
row, so any ciphertext surviving elsewhere stays unreadable.

```python
store.delete_documents(["d1"])
store.delete_all()             # returns the number erased
```

`DuplicatePolicy.NONE` falls back to `FAIL`, as `InMemoryDocumentStore` does, so an
accidental re-write is reported rather than silently replacing a document whose key would
then be destroyed.

## Retrieval

```python
query_embedding = [0.0] * 768        # from your Haystack text embedder, `dim` wide

store.embedding_retrieval(query_embedding, top_k=5,
                          filters={"field": "meta.chapter", "operator": "==", "value": "intro"})
```

Filtering uses Haystack's own evaluator, so the whole filter language, date comparisons
included, matches `InMemoryDocumentStore` operator for operator. `top_k` is `top_k`: a
filter matching only distant documents still returns them, however many others outrank
them.

A top-level `AND` of string equality conditions is pushed into the scan, including nested
paths like `meta.person.name`. Everything else is evaluated afterwards, so the two agree:
nothing is pushed under `OR` or `NOT`, and numbers are not pushed either, because `==` here
is Python's (`1 == 1.0`) where the stored comparison is JSON-type exact.

## Notes

Documents Haystack did not embed are stored, filterable and countable, but take no part in
`embedding_retrieval`, as in `InMemoryDocumentStore`, which retrieves only documents that
have embeddings.

Citadel is embedded and one process owns the file. A path already open on this thread,
under the same passphrase, is shared, so this can sit on the same database as another
Citadel adapter; construct them on the same thread.

## License

Apache-2.0
