Metadata-Version: 2.5
Name: fieldseal
Version: 0.1.2
Summary: EXPERIMENTAL, NOT INDEPENDENTLY REVIEWED, NOT FOR PRODUCTION DATA: the format may change before 1.0 and data written now may need re-encrypting. Portable field-level encryption at rest: the reference Python core.
Project-URL: Homepage, https://fieldseal.dev
Project-URL: Documentation, https://fieldseal.dev/docs/
Project-URL: Repository, https://github.com/fieldseal-dev/fieldseal-spec
Project-URL: Issues, https://github.com/fieldseal-dev/fieldseal-spec/issues
Project-URL: Specification, https://github.com/fieldseal-dev/fieldseal-spec/blob/main/docs/02-spec-v0.1.md
Author: Fieldseal contributors
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: blind-index,encryption,encryption-at-rest,field-level-encryption
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Security :: Cryptography
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: cryptography>=42
Provides-Extra: argon2
Requires-Dist: argon2-cffi>=23; extra == 'argon2'
Provides-Extra: dev
Requires-Dist: hypothesis>=6; extra == 'dev'
Requires-Dist: jsonschema>=4; extra == 'dev'
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Description-Content-Type: text/markdown

# fieldseal — Python core

> **Experimental release: not independently reviewed, not for production data.**
> The cryptographic design this package implements has not been reviewed by
> anyone outside the project. It is pre-1.0: the stored format may change
> before 1.0, and data written with it now may have to be re-encrypted if
> review changes a construction. Writing refuses until you explicitly arm
> provisional use (spec §4.8). This release is for evaluation and feedback;
> the terms it is published under are in [PRD §8](https://github.com/fieldseal-dev/fieldseal-spec/blob/main/docs/01-prd.md#8-scope-and-phasing).

The reference Python implementation of the Fieldseal specification, built to
[`docs/10-core-python.md`](https://github.com/fieldseal-dev/fieldseal-spec/blob/main/docs/10-core-python.md).

> **Not for production use, and the library will refuse.** Every registered
> cipher suite is *provisional* (spec §4.8): its constructions have not been
> independently reviewed, and Gate 0b of the Phase 0 exit gate
> ([`docs/01-prd.md`](https://github.com/fieldseal-dev/fieldseal-spec/blob/main/docs/01-prd.md) §8) is still open. `encrypt()` and
> `rotate()` raise `SUITE_PROVISIONAL` unless you explicitly arm provisional
> use — `FIELDSEAL_ARM_PROVISIONAL_SUITES=1` in the environment, or
> `arm_provisional_suites=True` on the constructor. Decryption is deliberately
> ungated.

## Status

| | |
|---|---|
| Vector suite | **178/178** pinned results pass on suite `0.7.0-provisional` (146 vectors; `envelope/` counted in both directions, some `blind-index/` vectors also end to end — see `harness_notes` in the report); **no family held out**; both §3.5 out-of-band checks pass |
| Gate, parity and totality tests | 131 pass (`tests/test_gates.py`, `tests/test_parity.py`) |
| Suites | `0xFF01` (AES-256-GCM). `0xFF02` is registered and refused at construction — it needs an XChaCha backend, blocked on gap G7 |
| Conformance report | `tests/run_vectors.py` writes the [`docs/14`](https://github.com/fieldseal-dev/fieldseal-spec/blob/main/docs/14-conformance-ci.md) §4 JSON to stdout, including `pinned_decisions` and `harness_notes`; the TypeScript core's report has the same shape and the same result ids, so the two diff cleanly |
| Milestone | **M1 met** for the families in the pinned suite. M2 (the independent TypeScript reproduction, [`docs/18`](https://github.com/fieldseal-dev/fieldseal-spec/blob/main/docs/18-m2-report.md)) is what makes these values trustworthy |

## Running

```sh
py -3 -m venv .venv
.venv/Scripts/python -m pip install -e ".[argon2,dev]"
.venv/Scripts/python -m pytest tests -q
.venv/Scripts/python tests/run_vectors.py > conformance-python.json   # report on stdout, prose on stderr
.venv/Scripts/python -m mypy --strict src
```

## What the vectors do not reach, and what this core pins

Spec §9 leaves the precedence among its error codes open (gap G5) and obliges a
Gate 0a implementation to pin an order and declare it. This core follows
[`docs/09`](https://github.com/fieldseal-dev/fieldseal-spec/blob/main/docs/09-core-architecture.md) §3.2 step for step and declares
every pin under `pinned_decisions` in its report, under the keys `docs/14` §4
reserves. The ones an operator will meet:

- **Read modes (spec §10.3).** `strict` raises `NOT_CIPHERTEXT` on non-envelope
  input; `permissive` and `readonly` return it as-is, warn at construction
  (`FieldsealWarning`) and count it in `Fieldseal.plaintext_reads`. `readonly`
  refuses `encrypt()` and `rotate()` with `MODE_VIOLATION` before reading
  anything. `rotate()` in `permissive` mode is literally decrypt-then-encrypt,
  so it *encrypts* unmigrated plaintext (D-13).
- **Recognition before policy (spec §3.4).** An unregistered suite, an
  unrecognized version byte or an implausible length is "not one of ours" —
  never `SUITE_NOT_ALLOWED`. Only a registered suite that the allow-list
  excludes is `SUITE_NOT_ALLOWED`. One exception: `fmt_ver = 0x02` at a
  plausible length raises `UNKNOWN_FORMAT_VERSION` in every mode (D-03).
- **Every currently-valid key version is tried (spec §8).**
  `KeyProvider.decryption_keys(header)` returns the candidates in preference
  order; the core verifies each one's commitment constant-time before any AEAD
  open. No candidate → `KEY_UNAVAILABLE`; none commits → `COMMITMENT_INVALID`;
  an open that fails after a verified commitment → `TAG_INVALID`.
  **`AAD_MISMATCH` is never raised**: under dual-layer binding a wrong context
  derives a wrong record key and is indistinguishable from key confusion (G5).
- **Blind indexes are bytes-in/bytes-out.** `nfc-casefold-v1` over bytes
  decodes strict UTF-8 first and refuses invalid input with `INVALID_ARGUMENT`
  rather than folding through replacement characters; `identity` and
  `digits-only-v1` never decode. An unknown IDF or normalizer is a
  `CONFIGURATION_ERROR`, never a default. The Unicode version is the
  interpreter's (`unicodedata.unidata_version`, reported in the report's
  `environment`) — CPython 3.14 folds with Unicode 16.0 where the TypeScript
  core vendors 17.0, which is a real cross-core risk for shared indexes until
  [`docs/09`](https://github.com/fieldseal-dev/fieldseal-spec/blob/main/docs/09-core-architecture.md) §7 pins a table (D-10).

`fieldseal.testing.encrypt_with_materials` runs the same API boundary as
`encrypt()` — mode, arming and length gates included — and replaces only the two
entropy draws (docs/08 §6).

## What is deliberately not proven yet

**Passing these vectors is weak evidence on its own.** The generator that
produced them is not an oracle ([`docs/08`](https://github.com/fieldseal-dev/fieldseal-spec/blob/main/docs/08-test-vector-spec.md)
§7); what makes an expected value trustworthy is two independently written
implementations agreeing on it. This core is one. The TypeScript core, written
from the specification without reading this source, is the other, and that is
M2. The behaviours listed above are *not* covered by any vector; they are
covered by `tests/test_parity.py` against this core's own pins, and the
`errors/` vector family that would make them a shared check does not exist yet.

This core was written without importing anything from `tools/vector-gen/`, and
takes HKDF from pyca/cryptography where the generator hand-rolls it from `hmac`.
That is a deliberate divergence from `docs/10` §7, which anticipated the
generator importing `fieldseal.testing`. Had it done so, M1 would have been
close to tautological — the same code checking itself. The cross-check is
narrower than "independent", though: the two share the `canonical_context`
layout by construction, so the independence is in HKDF only. See the
divergence note in [`docs/07`](https://github.com/fieldseal-dev/fieldseal-spec/blob/main/docs/07-implementation-plan.md) §7.

**`blind-index/argon2id.json` is pinned**, and has been since suite `0.6.0-provisional` (2026-08-31, `docs/07` §7), and this core runs it like any other family. It was held out while the primitive had no external known-answer source: RFC 9106 §5.3's vector supplies a nonzero secret (`K`) and associated data (`X`), both forbidden by spec §7.3 and unsuppliable from Python, so passing the project's own vectors would have proved only that two implementations copied one unverified assumption. That is answered — the generator checks argon2-cffi against libsodium's seven published `crypto_pwhash` answers on every run (libsodium cannot supply `K` or `X` either, which makes it the right source for the case §7.3 uses), and the TypeScript core reproduces the same values through `node:crypto`.

Promoting it also found what the hold-out had been hiding: eight of the family's nineteen vectors declared `idf: argon2id` with no `idf_params`, and both cores reject a missing cost as malformed rather than assuming the minimum (`docs/08` §4.4). Nothing had run them, so nothing had said so. The #108 review then found that this harness's rejection was an abort with no report where the TypeScript harness's was eight recorded failures; `run_blind_index` now has the per-vector boundary the other runners had, and a vector it cannot derive — malformed `idf_params`, or the `argon2` extra not installed — is a recorded failure with a reason. The same round added two vectors at a raised cost (`t = 4`), the only ones that can tell a harness deriving at the declared cost from one deriving at its default (`docs/07` §7, 2026-09-01).

## Honest limitations

The specification requires every implementation to state these, and
`docs/07` §4 requires every shipped artifact to carry them.

- **No protection against a compromised application process** (spec §2.2
  N1). The keys are in that process. Query logs, slow-query logs and
  replication logs are sensitive artifacts and must be protected like the
  ciphertext (§2.3).
- **Storage overhead is real** (§3.3). Every envelope carries 111 bytes of
  fixed overhead under `0xFF01`: a 9-byte value becomes 120 bytes binary.
  This core is bytes in, bytes out and never emits base64; a deployment that
  stores base64 pays a further 33% on every row (about 160 bytes for the same
  value) and must document it. Across a 20-column, 100M-row table the fixed
  overhead alone is roughly 220 GB, before index bloat.
- **The key service is a hard dependency in the read path** (§8.1).
  `EnvelopeKeyProvider` unwraps KMS-wrapped DEKs only in `warm()`; the value
  path reads the cache and nothing else, so a miss is `KEY_UNAVAILABLE`, and a
  KMS outage means `KEY_UNAVAILABLE` for everything not already cached. The
  `degradation` argument records the deployment's mode (`fail-closed` /
  `serve-cached`); on the value path both mean the same thing — serve only
  what the cache can decrypt (`docs/09` §8.2). No KMS client ships: the
  provider calls a `Wrapper` you supply.
- **The DEK cache is an in-memory plaintext key cache** (§5.5). It is
  exposed to memory dumps, core files and swap. Entries are held as
  `bytearray` and overwritten with zeros when evicted — by max-age, max-uses,
  capacity or `DekCache.clear()` — which narrows that window and does not
  close it: CPython `bytes` are immutable and freely copied, copies inside
  dependencies are out of reach, and there is no `mlock` (`docs/10` §5). The
  per-operation record key and the §7.3 Argon2id salt are `bytes` and are not
  erased at all. `CachePolicy.max_age` and `max_uses` are security
  parameters, not tuning knobs. In a prefork server, construct the client
  after the fork (gunicorn `post_fork`): cache contents that survive a fork
  are DEK copies in every child (`docs/09` §10).
- **Two of spec §8's three providers ship.** `StaticKeyProvider` is
  test-only, holds key material for the process lifetime, and does not yet
  emit the outside-test-configuration warning spec §8 asks for.
  `EnvelopeKeyProvider` is the production path above. `DerivedKeyProvider` is
  not yet ported (`docs/10` §3).
- **Error precedence is provisional.** Everything under "what this core pins"
  above may change at Gate 0b; the pins are declared so that a change is
  visible, not because they are settled.
- **Argon2id costs 10–100 ms per query term** (spec §7.3). That is wall-clock
  latency on the requesting thread and a product constraint, not a bug to
  fix. It is **not** a process-wide stall: this README previously said the
  GIL is held for most of it, and that was wrong. Measured 2026-09-09 on
  argon2-cffi 25.1.0 / argon2-cffi-bindings 26.1.0, CPython 3.14.6 — one
  hash at the §7.3 parameters takes 36.9 ms and two on separate threads take
  40.0 ms, where serialization would cost ~74 ms. A threaded deployment
  serves other requests through it.
