Metadata-Version: 2.4
Name: vectorguard-pyhsm
Version: 1.7.0
Summary: Production-grade software Key Management Service (KMS) — key lifecycle, authenticated encryption, digital signing, and tamper-evident audit logging.
Author: Pavon Dunbar
License-Expression: MIT
Project-URL: Homepage, https://github.com/pavondunbar/PyHSM
Project-URL: Repository, https://github.com/pavondunbar/PyHSM
Project-URL: Issues, https://github.com/pavondunbar/PyHSM/issues
Keywords: hsm,kms,cryptography,key-management,encryption,signing,audit
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security :: Cryptography
Classifier: Topic :: Software Development :: Libraries
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: cryptography<48.0.0,>=43.0.0
Requires-Dist: argon2-cffi<25.0.0,>=23.1.0
Provides-Extra: dev
Requires-Dist: pytest<9.0.0,>=8.0.0; extra == "dev"
Requires-Dist: pytest-cov<7.0.0,>=6.0.0; extra == "dev"
Dynamic: license-file

# PyHSM

A production-grade software Key Management Service (KMS) providing cryptographic key lifecycle management, authenticated encryption, digital signing, and tamper-evident audit logging.

Available as a **Python CLI and library** and a **production-hardened TypeScript/Node.js library**.

---

## Why PyHSM

Most applications that need key management face a difficult choice: implement it themselves (error-prone), pay for cloud KMS (vendor lock-in, data sovereignty concerns), or buy a hardware HSM ($20K+, complex). PyHSM is a third path — a well-engineered software KMS that you own, deploy anywhere, and extend freely.

**What makes it production-grade:**

- AES-256-GCM-SIV encryption (nonce-misuse resistant, TypeScript) / AES-256-GCM with hybrid nonce + AAD binding (Python)
- Argon2id key derivation (OWASP recommended, 64 MB memory-hard)
- HKDF key separation — independent encryption, MAC, and KEK subkeys derived from master
- AES-KWP (RFC 5649) per-key wrapping — keys are double-encrypted at rest in both layers
- Salt-bound KEK derivation — KEK uses a dedicated salt stored inside the encrypted envelope, derived through full Argon2id → HKDF path
- Encrypt-then-MAC keystore with HMAC-SHA256 tamper detection
- Pluggable storage backends — file, memory, or custom (database, cloud, etc.)
- Atomic file writes — keystore never corrupts on crash
- Key versioning — rotate without breaking old ciphertexts
- Per-key policies: expiry, operation limits, caller ACLs, rate limiting
- Per-caller ACL enforcement — `allowed_callers` policy with audit trail on denial
- Per-key concurrency — sharded locks allow parallel operations on different keys
- AAD-bound ciphertext — cryptographically binds ciphertext to key ID and version
- Hybrid nonce strategy — random + counter eliminates birthday-bound collisions
- Input size validation — rejects payloads over 64 MB on both encrypt and decrypt paths
- HMAC-chained append-only audit log with HMAC key derived from master password
- Caller ID tracking — every operation records the caller identity in the audit log
- Deterministic memory zeroization via `SecureBytes` / `SecureBuffer` (key material stored as mutable `bytearray`, not immutable strings)
- Process isolation via Unix domain socket IPC
- Shamir M-of-N master password unlock ceremony
- Startup Known-Answer Tests (KATs) before accepting any operations
- Prometheus metrics
- Backward-compatible ciphertext format versioning (v1 legacy, v2 AAD-bound)
- JWK (RFC 7517) key import/export for interoperability (supports P-256, P-384, P-521, secp256k1, Ed25519, RSA, AES)
- EC P-256, P-384, P-521, and secp256k1 signing with ECDSA (SHA-256, SHA-384, SHA-512)
- Ed25519 (EdDSA) signing for high-performance, compact signatures (Solana, Cosmos, SSH keys)
- secp256k1 support for Ethereum, Bitcoin, and EVM-compatible blockchain transaction signing
- Fully typed Python API (PEP 561 `py.typed` marker included)
- JSON-structured logging for production observability (SIEM-ready)
- Concurrency stress-tested (16 threads, data integrity proofs)
- 80%+ code coverage enforced in CI
- Reproducible builds via pinned dependency lockfile
- 214 tests across both layers

---

## Architecture

```
┌─────────────────────────────────────────────────────────────────────────┐
│  Your Application                                                        │
│                                                                          │
│  hsm.sign("eth-wallet", tx_hash)                                         │
│  hsm.encrypt("app-key", data)                                            │
│                                                                          │
│  ► Raw key material is NEVER returned to this layer                      │
└──────────────────────────────────┬───────────────────────────────────────┘
                                   │ API call (or IPC via Unix socket)
                                   ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  PyHSM Core                                                              │
│                                                                          │
│  ┌─────────────┐ ┌──────────────┐ ┌────────────┐ ┌──────────────────┐   │
│  │ Key Unwrap  │ │ Policy Check │ │ Crypto Op  │ │ Audit + Metrics  │   │
│  │ (AES-KWP)  │→│ ACL, Rate,   │→│ Sign/Enc/  │→│ HMAC-chained log │   │
│  │             │ │ Expiry, Ops  │ │ Dec/Verify │ │                  │   │
│  └─────────────┘ └──────────────┘ └────────────┘ └──────────────────┘   │
│                                          │                               │
│                                    Key zeroized                           │
│                                    from memory                            │
└──────────────────────────────────────┬───────────────────────────────────┘
                                       │
                                       ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  Persistent Storage                                                      │
│                                                                          │
│  keystore.enc                          keystore.enc.audit.jsonl           │
│  ┌───────────────────────────────┐     ┌──────────────────────────────┐  │
│  │ AES-256-GCM Outer Envelope    │     │ HMAC-chained append-only log │  │
│  │ + HMAC-SHA256 Tamper Seal     │     │ (tamper-evident)             │  │
│  │  ┌─────────────────────────┐  │     └──────────────────────────────┘  │
│  │  │ AES-KWP Per-Key Wrapping│  │                                       │
│  │  │  • eth-wallet (secp256k1)│  │                                       │
│  │  │  • sol-wallet (ed25519) │  │                                       │
│  │  │  • app-key (aes-256)    │  │                                       │
│  │  └─────────────────────────┘  │                                       │
│  └───────────────────────────────┘                                       │
└─────────────────────────────────────────────────────────────────────────┘
```

---

## Performance

Benchmarks measured on Python 3.13, macOS (Apple Silicon). Each operation includes
the full security pipeline: key unwrapping, policy enforcement, cryptographic
operation, audit logging, and keystore persistence.

| Operation | Ops/sec | Avg Latency | p50 | p99 |
|---|---|---|---|---|
| AES-256 encrypt | ~10 | 103 ms | 103 ms | 110 ms |
| AES-256 decrypt | ~10 | 104 ms | 104 ms | 105 ms |
| RSA-2048 sign | ~6 | 158 ms | 158 ms | 158 ms |
| RSA-2048 verify | ~10 | 104 ms | 104 ms | 104 ms |
| EC P-256 sign | ~10 | 104 ms | 104 ms | 105 ms |
| EC P-256 verify | ~10 | 104 ms | 104 ms | 105 ms |
| secp256k1 sign | ~9 | 105 ms | 105 ms | 106 ms |
| secp256k1 verify | ~10 | 105 ms | 105 ms | 135 ms |
| Ed25519 sign | ~10 | 104 ms | 105 ms | 106 ms |
| Ed25519 verify | ~9 | 106 ms | 104 ms | 165 ms |
| Key generate (AES-256) | ~10 | 104 ms | 104 ms | 107 ms |
| Key generate (secp256k1) | ~10 | 105 ms | 105 ms | 105 ms |
| Key generate (Ed25519) | ~10 | 104 ms | 104 ms | 105 ms |
| Key rotate (AES-256) | ~10 | 104 ms | 104 ms | 105 ms |

**Where the time goes:** ~103 ms is keystore persistence (encrypt + HMAC + atomic write).
The actual cryptographic operation is sub-millisecond for symmetric and EC operations.
RSA-2048 signing adds ~54 ms of computation on top of persistence.

**For higher throughput:** The TypeScript layer uses deferred persistence — operation
counts are flushed on the next structural mutation or session close, giving significantly
higher ops/sec for encrypt/decrypt/sign/verify workloads.

Run the benchmarks yourself:

```bash
python benchmarks/bench.py
```

---

## Table of Contents

- [Architecture](#architecture)
- [Performance](#performance)
- [Python Layer](#python-layer)
  - [Installation](#python-installation)
  - [CLI Usage](#cli-usage)
  - [Library Usage](#python-library-usage)
  - [Storage Backends](#storage-backends)
  - [Architecture](#python-architecture)
- [TypeScript Layer](#typescript-layer)
  - [Installation](#typescript-installation)
  - [Library Usage](#typescript-library-usage)
  - [Process Isolation Mode](#process-isolation-mode)
  - [Architecture](#typescript-architecture)
- [Shared: Shamir Secret Sharing](#shamirs-secret-sharing)
- [Blockchain Transaction Signing](#blockchain-transaction-signing-secp256k1--ed25519)
- [Security Model](#security-model)
- [Threat Model](#threat-model)
- [Performance Benchmarks](#performance)
- [Running Tests](#running-tests)
- [Operations Guide](#operations-guide)
- [FAQ](#faq)

---

## Python Layer

### Python Installation

```bash
# Install from PyPI
pip install vectorguard-pyhsm

# Or install from source (with pyproject.toml)
pip install .

# For development (includes pytest + pytest-cov)
pip install ".[dev]"

# For reproducible builds (CI and production deployments)
pip install -r requirements.lock
pip install -e .
```

### CLI Usage

All commands require `--store` (keystore path) and a master password (minimum 12 characters). The password is always entered interactively via a hidden prompt (never visible in the terminal or process list). For scripting and CI, you can set the `PYHSM_MASTER_PASSWORD` environment variable.

The `--store` flag can appear **before or after** the subcommand — put it wherever feels natural:

```bash
# These are equivalent:
vectorguard-pyhsm --store keystore.enc generate my-key
vectorguard-pyhsm generate my-key --store keystore.enc
```

If the keystore file does not yet exist, the CLI prints a notice to stderr so you can tell when you're accidentally pointing at the wrong path:

```
$ vectorguard-pyhsm --store /wrong/path.enc list
Master password:
Created new keystore: /wrong/path.enc
No keys stored.
```

```bash
# List keystore files in the current directory (no password required)
vectorguard-pyhsm stores
vectorguard-pyhsm stores /path/to/keystores

# Generate keys (password prompted interactively)
vectorguard-pyhsm --store keystore.enc generate my-aes-key --type aes-256
vectorguard-pyhsm --store keystore.enc generate my-rsa-key --type rsa-2048
vectorguard-pyhsm --store keystore.enc generate my-ec-key  --type ec-p256
vectorguard-pyhsm --store keystore.enc generate my-ec384   --type ec-p384
vectorguard-pyhsm --store keystore.enc generate my-ec521   --type ec-p521
vectorguard-pyhsm --store keystore.enc generate my-secp256k1 --type ec-secp256k1
vectorguard-pyhsm --store keystore.enc generate my-ed25519   --type ed25519

# Generate a key with a policy
vectorguard-pyhsm --store keystore.enc generate limited-key \
  --type aes-256 \
  --max-operations 500 \
  --expires-at 2027-01-01T00:00:00Z \
  --no-decrypt

# List keys in a keystore (shows type, current version, creation date)
vectorguard-pyhsm --store keystore.enc list

# Encrypt / Decrypt
vectorguard-pyhsm --store keystore.enc encrypt my-aes-key -d "secret message"
vectorguard-pyhsm --store keystore.enc decrypt my-aes-key -d <ciphertext-hex>

# Pipe via stdin
echo "secret message" | vectorguard-pyhsm --store keystore.enc encrypt my-aes-key

# Sign / Verify (uses stored public key for verify — private key never exposed)
vectorguard-pyhsm --store keystore.enc sign   my-ec-key -d "message to sign"
vectorguard-pyhsm --store keystore.enc verify my-ec-key "message to sign" <sig-hex>

# Export public key (PEM)
vectorguard-pyhsm --store keystore.enc pubkey my-rsa-key

# Rotate an AES key (archives current version, generates new one)
vectorguard-pyhsm --store keystore.enc rotate my-aes-key

# Destroy a key (zeroizes all versions, removes from store)
# Requires confirmation; use --yes/-y to skip the prompt
vectorguard-pyhsm --store keystore.enc delete my-aes-key
vectorguard-pyhsm --store keystore.enc delete my-aes-key --yes  # skip prompt

# Metrics
vectorguard-pyhsm --store keystore.enc metrics
vectorguard-pyhsm --store keystore.enc metrics --prometheus

# Audit log
vectorguard-pyhsm --store keystore.enc audit                       # dump all entries
vectorguard-pyhsm --store keystore.enc audit --verify              # verify HMAC chain
vectorguard-pyhsm --store keystore.enc audit --operation encrypt   # filter by operation
vectorguard-pyhsm --store keystore.enc audit --key-id my-aes-key   # filter by key
vectorguard-pyhsm --store keystore.enc audit --since 2025-01-01T00:00:00Z

# For scripting/CI, set password via environment variable:
export PYHSM_MASTER_PASSWORD="your-master-password"
vectorguard-pyhsm --store keystore.enc list
```

### Python Library Usage

```python
from hsm import PyHSM

# Master password is always required — minimum 12 characters
hsm = PyHSM(
    storage_path="keystore.enc",
    master_password="your-master-password",
    session_timeout_s=300,      # auto-lock after 5 min inactivity (0 = disabled)
    rate_limit_max_ops=100,     # max ops per key per window
    rate_limit_window_s=60,
)

# Or use as a context manager for automatic cleanup
with PyHSM(storage_path="keystore.enc", master_password="your-master-password") as hsm:
    hsm.generate_key("my-key")
    ct = hsm.encrypt("my-key", "secret")
    # Key material is automatically zeroized on exit

# Generate keys
hsm.generate_key("aes-key")                          # AES-256 by default
hsm.generate_key("rsa-key", "rsa-2048")
hsm.generate_key("ec-key",  "ec-p256")
hsm.generate_key("ec384",   "ec-p384")              # NIST P-384 (SHA-384)
hsm.generate_key("ec521",   "ec-p521")              # NIST P-521 (SHA-512)
hsm.generate_key("eth-key", "ec-secp256k1")         # Bitcoin/Ethereum (SHA-256)
hsm.generate_key("sol-key", "ed25519")              # Ed25519 (Solana, SSH, high-perf signing)

# Generate a key with a policy (including caller ACL)
hsm.generate_key("restricted", policy={
    "allow_encrypt": True,
    "allow_decrypt": False,     # encrypt-only
    "max_operations": 1000,
    "expires_at": "2027-01-01T00:00:00Z",
    "allowed_callers": ["service-a", "service-b"],  # caller ACL
})

# Encrypt / Decrypt (AES-256-GCM with AAD binding and hybrid nonce)
ciphertext = hsm.encrypt("aes-key", "secret message")  # returns hex string
plaintext  = hsm.decrypt("aes-key", ciphertext)        # returns bytes

# All operations support caller_id for audit tracking and ACL enforcement
ciphertext = hsm.encrypt("aes-key", "data", caller_id="my-service")
plaintext  = hsm.decrypt("aes-key", ciphertext, caller_id="my-service")

# Rotate a key (old ciphertexts remain decryptable via version prefix)
new_version = hsm.rotate_key("aes-key")

# Sign / Verify
signature = hsm.sign("ec-key", "message")
is_valid   = hsm.verify("ec-key", "message", signature)  # uses stored public key only

# Sign with P-384 (uses SHA-384 automatically) or P-521 (uses SHA-512)
sig384 = hsm.sign("ec384", "message", caller_id="signer-service")
is_valid = hsm.verify("ec384", "message", sig384, caller_id="verifier")

# Sign with secp256k1 (Ethereum/Bitcoin transaction signing)
sig_eth = hsm.sign("eth-key", tx_hash, caller_id="tx-service")
is_valid = hsm.verify("eth-key", tx_hash, sig_eth, caller_id="verifier")

# Sign with Ed25519 (high-performance, compact 64-byte signatures)
sig_ed = hsm.sign("sol-key", "message", caller_id="signing-service")
is_valid = hsm.verify("sol-key", "message", sig_ed, caller_id="verifier")

# Export public key (PEM)
pub_pem = hsm.get_public_key("rsa-key")

# Expiry enforcement (archives expired keys)
hsm.enforce_expiry()

# Metrics
metrics_dict = hsm.get_metrics()
prometheus   = hsm.get_prometheus_metrics()

# Audit log
audit = hsm.get_audit_log()
audit.verify()                                          # returns -1 if clean
entries = audit.export_jsonl(operation="encrypt")       # SIEM-ready list of dicts

# JWK export (RFC 7517) — interoperate with other KMS systems
# Requires allow_export=True in the key's policy (defaults to False for security)
hsm.generate_key("export-aes", policy={"allow_encrypt": True, "allow_decrypt": True, "allow_export": True})
jwk = hsm.export_jwk("export-aes")                    # {"kty": "oct", "k": "...", ...}

hsm.generate_key("export-ec", "ec-p256", policy={"allow_sign": True, "allow_export": True})
ec_jwk = hsm.export_jwk("export-ec")                  # {"kty": "EC", "crv": "P-256", ...}

# JWK import — bring keys from external systems
hsm.import_key_jwk("imported-key", {
    "kty": "oct",
    "k": "base64url-encoded-key-material",
    "alg": "A256GCM",
})

# Import an Ethereum/Bitcoin private key via JWK
hsm.import_key_jwk("eth-wallet", {
    "kty": "EC",
    "crv": "secp256k1",
    "x": "...",   # base64url public key x-coordinate
    "y": "...",   # base64url public key y-coordinate
    "d": "...",   # base64url private key scalar
})

# Import an Ed25519 key (e.g., from Solana or SSH)
hsm.import_key_jwk("ed-key", {
    "kty": "OKP",
    "crv": "Ed25519",
    "x": "...",   # base64url 32-byte public key
    "d": "...",   # base64url 32-byte private key seed
})

# Explicit close (zeroizes master password and key material from memory)
hsm.close_session()
```

### Storage Backends

PyHSM supports pluggable storage backends. The default is file-based with atomic writes, but you can implement custom backends for database, cloud storage, or any other persistence layer.

```python
from hsm import PyHSM, KeyStore
from hsm.backends import StorageBackend, FileBackend, MemoryBackend

# Default: file backend (backward-compatible)
hsm = PyHSM(storage_path="keystore.enc", master_password="pw")

# Explicit file backend
from hsm.backends import FileBackend
store = KeyStore(master_password="pw", backend=FileBackend("/secure/keystore.enc"))

# In-memory backend (for testing or ephemeral use)
from hsm.backends import MemoryBackend
store = KeyStore(master_password="pw", backend=MemoryBackend())

# Custom backend — implement the StorageBackend interface:
#   exists() -> bool
#   read() -> bytes
#   write(data: bytes) -> None
#   delete() -> None
```

The `StorageBackend` interface deals only with raw encrypted bytes — all encryption, HMAC verification, and key management logic stays in `KeyStore`. Backends never see plaintext key material.

### Python Environment Variables

| Variable | Default | Description |
|---|---|---|
| `PYHSM_MASTER_PASSWORD` | *(none)* | Master password for the CLI. When set, the CLI uses this instead of prompting interactively. Useful for scripting and CI. |
| `PYHSM_LOG_LEVEL` | `WARNING` | Structured log verbosity: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`. Set to `INFO` in production for operational visibility. |
| `PYHSM_ALLOW_PBKDF2_FALLBACK` | `0` (disabled) | Set to `1` to permit degraded PBKDF2 key derivation when `argon2-cffi` is unavailable. **Testing/migration only — never set in production.** |
| `PYHSM_AUDIT_HMAC_KEY` | *(derived from master password)* | Hex-encoded 32-byte key for audit HMAC chain. When not set, derived automatically from the master password via HKDF. |
| `PYHSM_AUDIT_WEBHOOK` | *(none)* | URL for non-blocking audit event POST. Webhook failures are logged (not silently dropped). |

### Structured Logging

PyHSM outputs JSON-structured logs via Python's stdlib `logging` module. Every log line is a single JSON object with consistent fields for machine parsing:

```json
{"timestamp": "2026-01-15T10:30:00.123456+00:00", "level": "INFO", "logger": "hsm.core", "message": "key generated", "event": "generate_key", "key_id": "my-key", "key_type": "aes-256"}
```

Configure from your application:

```python
import logging

# See all PyHSM operational events
logging.getLogger("hsm").setLevel(logging.INFO)

# Or via environment variable before import:
# export PYHSM_LOG_LEVEL=INFO
```

Logged events include: `session_open`, `session_close`, `self_test_pass`, `self_test_fail`, `generate_key`, `rotate_key`, `destroy_key`, `encrypt`, `decrypt`, `sign`, `access_denied`, `rate_limited`, `tamper_detected`, `kdf_migration`, `webhook_failure`.

### Python Architecture

```
hsm/
  core.py           — PyHSM class: key lifecycle, encrypt/decrypt, sign/verify,
                      per-key AES-KWP wrapping, AAD binding, hybrid nonce,
                      per-key sharded locks, caller_id ACL enforcement
  storage.py        — KeyStore: Argon2id (required) key derivation,
                      HKDF key separation (enc/mac/kek subkeys),
                      AES-256-GCM + HMAC-SHA256, cached KEK, pluggable StorageBackend,
                      bytearray key_data for deterministic zeroization, auto-migration
                      from PBKDF2 to Argon2id
  backends.py       — StorageBackend ABC, FileBackend (atomic writes), MemoryBackend
  logging.py        — JSON-structured logging via stdlib (machine-parseable, SIEM-ready)
  secure_memory.py  — SecureBytes: deterministic bytearray zeroization, context manager
  jwk.py            — JWK (RFC 7517) import/export: oct, EC (P-256/P-384/P-521/secp256k1), OKP (Ed25519), RSA
  shamir.py         — Shamir secret sharing over GF(256)
  audit.py          — HMAC-chained append-only audit log (HMAC key derived from master password)
  rate_limiter.py   — Sliding-window per-key rate limiter
  metrics.py        — Prometheus-format metrics collector
  self_test.py      — Startup Known-Answer Tests (KATs)
  __init__.py       — Public API exports
  py.typed          — PEP 561 marker for type checker support
cli.py              — Full-featured command-line interface
tests/
  test_pyhsm.py     — 112 pytest tests (unit + integration)
  test_concurrency.py — 8 concurrency stress tests (16 threads, data integrity proofs)
```

---

## TypeScript Layer

A production-hardened Node.js library in `./pyhsm-ts/` with additional features: Argon2id KDF, AES-256-GCM-SIV (nonce-misuse resistant), SecureBuffer zeroization, and process isolation mode.

Both layers now use Argon2id as the primary key derivation function (OWASP recommended, 64 MB memory-hard). The Python layer **requires** `argon2-cffi` and will refuse to start without it. A PBKDF2-SHA256 fallback (480,000 iterations) is available only when `PYHSM_ALLOW_PBKDF2_FALLBACK=1` is set (for testing/migration scenarios only — not for production).

### TypeScript Installation

```bash
cd pyhsm-ts
npm install
npm run build
```

Requires Node.js ≥ 18. All dependency versions are pinned exactly.

### TypeScript Library Usage

#### Synchronous constructor (PBKDF2 fallback)

```typescript
import { PyHSM } from "./pyhsm-ts";

const hsm = new PyHSM({
  storePath: "./keystore.enc",
  masterPassword: "your-master-password",
  sessionTimeoutMs: 300_000,
  backupDir: "./backups",
});
```

#### Async factory — Argon2id KDF (recommended for production)

```typescript
const hsm = await PyHSM.create({
  storePath: "./keystore.enc",
  masterPassword: "your-master-password",
});
```

The `create()` factory guarantees Argon2id (64 MB / 3 passes / 4 threads) is used for
all key derivation — including the first save on a new keystore. The synchronous
constructor falls back to PBKDF2-SHA256 at 480,000 iterations.

#### Custom storage backend

```typescript
import { PyHSM, MemoryBackend } from "./pyhsm-ts";

// In-memory backend for testing
const hsm = new PyHSM({
  storePath: "test",
  masterPassword: "pw",
  backend: new MemoryBackend(),
});

// Custom backend — implement the StorageBackend interface:
//   exists(): boolean
//   read(): Buffer
//   write(data: Buffer): void
//   delete(): void
```

#### Key operations

```typescript
// Generate
hsm.generateKey("my-key");

// Generate with specific key types
hsm.generateKey("eth-key", "ec-secp256k1");    // Bitcoin/Ethereum signing
hsm.generateKey("sol-key", "ed25519");          // Solana/SSH/high-performance signing

// Generate with policy
hsm.generateKey("restricted", {
  allowEncrypt: true,
  allowDecrypt: true,
  maxOperations: 1000,
  expiresAt: "2027-01-01T00:00:00Z",
  allowedCallers: ["service-a", "service-b"],
});

// Encrypt / Decrypt (AES-256-GCM-SIV, base64 output with version prefix)
const ct = hsm.encrypt("my-key", "secret message");
const pt = hsm.decrypt("my-key", ct);              // returns string

// Rotate (old ciphertexts remain decryptable)
hsm.rotateKey("my-key");

// Destroy (zeroizes all versions)
hsm.destroyKey("my-key");

// Backup and verify
const backupPath = hsm.createBackup();
hsm.verifyBackup(backupPath);   // HMAC + decrypt check without loading into live store

// Metrics
const metrics = hsm.getMetrics();
const prom    = hsm.getPrometheusMetrics();

// Audit log
const audit = hsm.getAuditLog();
const clean  = audit.verify();                     // -1 = no tampering
const events = audit.exportJsonl({ operation: "encrypt", since: "2025-01-01T00:00:00Z" });
const ndjson = audit.toNdjson({ onlyFailed: true }); // SIEM-ready NDJSON string

// Close (zeroizes all Buffers holding sensitive material)
hsm.closeSession();
```

#### JWK Import / Export (RFC 7517)

```typescript
// Export a key as standard JWK — interoperate with any system
const jwk = hsm.exportJwk("my-key");  // {"kty": "oct", "k": "...", "alg": "A256GCM"}

// Import a key from external JWK
hsm.importKeyJwk("external-key", {
  kty: "oct",
  k: "base64url-encoded-key-material",
  alg: "A256GCM",
});

// Import an EC key from another identity provider
hsm.importKeyJwk("idp-signing-key", {
  kty: "EC",
  crv: "P-256",
  x: "...",
  y: "...",
  d: "...",
});

// Import an Ethereum/Bitcoin private key (secp256k1)
hsm.importKeyJwk("eth-wallet", {
  kty: "EC",
  crv: "secp256k1",
  x: "...",
  y: "...",
  d: "...",
});

// Import an Ed25519 key (Solana, SSH, etc.)
hsm.importKeyJwk("sol-wallet", {
  kty: "OKP",
  crv: "Ed25519",
  x: "...",   // 32-byte public key (base64url)
  d: "...",   // 32-byte private key seed (base64url)
});
```

#### Key ID rules

Key IDs must match `^[a-zA-Z0-9][a-zA-Z0-9._-]{0,127}$`:

- 1–128 characters
- Must start with a letter or digit
- May contain letters, digits, `.`, `_`, `-`
- Rejects path traversal (`../`), prototype pollution (`__proto__`), spaces

### Process Isolation Mode

For maximum security, run the HSM in a separate process. A vulnerability in your
application cannot directly read key material in the HSM process's memory.

**Start the HSM process:**

```bash
export PYHSM_MASTER_PASSWORD="your-master-password"
export PYHSM_KEYSTORE_PATH="/secure/keystore.enc"
export PYHSM_SOCKET_PATH="/run/pyhsm/pyhsm.sock"
export PYHSM_CALLER_SECRET="shared-hmac-secret"
export PYHSM_BACKUP_DIR="/secure/backups"

npx tsx pyhsm-ts/process.ts
```

**Connect from your application:**

```typescript
import { PyHSMClient } from "./pyhsm-ts";

const client = new PyHSMClient("/run/pyhsm/pyhsm.sock", "my-service");

await client.generateKey("app-key");
const ct = await client.encrypt("app-key", "secret");
const pt = await client.decrypt("app-key", ct);

await client.rotateKey("app-key");
const path = await client.backup();
const ok   = await client.verifyBackup(path);
const h    = await client.health();
const m    = await client.metrics();
```

### TypeScript Environment Variables

| Variable | Default | Description |
|---|---|---|
| `PYHSM_MASTER_PASSWORD` | — | Master password (**required** unless using `PYHSM_SHARES`) |
| `PYHSM_SHARES` | — | Comma-separated Shamir share JSON objects |
| `PYHSM_KEYSTORE_PATH` | `./pyhsm-keystore.enc` | Encrypted keystore location |
| `PYHSM_AUDIT_LOG_PATH` | `<storePath>.audit.jsonl` | HMAC-chained audit log path |
| `PYHSM_AUDIT_HMAC_KEY` | *(auto-generated)* | Hex 32-byte audit HMAC key |
| `PYHSM_AUDIT_WEBHOOK` | — | URL for non-blocking audit event POST |
| `PYHSM_BACKUP_DIR` | — | Directory for encrypted backups |
| `PYHSM_SOCKET_PATH` | `/tmp/pyhsm.sock` | Unix domain socket path (IPC mode) |
| `PYHSM_CALLER_SECRET` | — | Shared secret for IPC caller HMAC auth |
| `PYHSM_SESSION_TIMEOUT_MS` | `300000` | Idle ms before auto-lock |
| `PYHSM_RATE_LIMIT` | `100` | Max operations per key per window |
| `PYHSM_RATE_WINDOW_MS` | `60000` | Rate limit window duration (ms) |
| `PYHSM_KEY_ID` | `pyhsm-master` | Default key ID for singleton helpers |

### TypeScript Architecture

```
pyhsm-ts/
  core.ts             — PyHSM class: key lifecycle, encrypt/decrypt, backup,
                        AES-KWP per-key wrapping, HKDF key separation, pluggable StorageBackend
  storage-backend.ts  — StorageBackend interface, FileBackend, MemoryBackend
  types.ts            — TypeScript interfaces, key ID validation, config with backend option
  jwk.ts              — JWK (RFC 7517) import/export: oct, EC (P-256/P-384/P-521/secp256k1), OKP (Ed25519), RSA
  shamir.ts           — Shamir secret sharing over GF(256)
  audit.ts            — HMAC-chained audit log, SIEM export
  rate-limiter.ts     — Sliding-window per-key rate limiter
  metrics.ts          — Prometheus metrics collector
  self-test.ts        — Startup Known-Answer Tests (KATs), FIPS mode
  secure-buffer.ts    — SecureBuffer: deterministic Buffer zeroization
  process.ts          — IPC server (process isolation via Unix socket)
  client.ts           — IPC client with HMAC caller auth
  index.ts            — Public API exports and singleton factory
  pyhsm.test.ts       — 94 tests (vitest)
  OPERATIONS.md       — Full operator guide (env vars, deployment, procedures)
  package.json        — Pinned exact dependency versions
  tsconfig.json       — Strict TypeScript configuration
```

---

## Shamir's Secret Sharing

Both layers implement Shamir secret sharing over GF(256) with the AES irreducible polynomial. This can be used to split a master password or any secret into N shares where K are required to reconstruct — and K-1 or fewer shares reveal zero information (information-theoretic security).

**Python:**

```bash
# Split a hex secret into 5 shares, 3 required
vectorguard-pyhsm split -k 3 -n 5 -s "deadbeefcafe..."

# Reconstruct from any 3
vectorguard-pyhsm reconstruct \
  --share '{"index":1,"data":"..."}' \
  --share '{"index":3,"data":"..."}' \
  --share '{"index":5,"data":"..."}'
```

**TypeScript:**

```typescript
import { splitMasterPassword, PyHSM } from "./pyhsm-ts";

// One-time: split the master password into 5 shares, 3 required to unlock
const shares = splitMasterPassword("my-master-password", 3, 5);
// Distribute shares[0..4] to five separate key custodians

// At startup: collect K shares from operators
const hsm = new PyHSM({
  storePath: "./keystore.enc",
  shares: [
    JSON.stringify(shares[0]),
    JSON.stringify(shares[2]),
    JSON.stringify(shares[4]),
  ],
});
```

Intermediate share buffers are zeroized from memory after reconstruction in both layers.

---

## Blockchain Transaction Signing (secp256k1 / Ed25519)

PyHSM can serve as a self-hosted signing infrastructure for Ethereum, Bitcoin, Solana, and other blockchain networks. The private key is imported once, encrypted at rest, and never exposed again — all signing happens through PyHSM.

### Step 1: Import an Existing Private Key (One-Time)

```python
from hsm import PyHSM
import base64

def b64url(b: bytes) -> str:
    return base64.urlsafe_b64encode(b).rstrip(b"=").decode()

# Raw private key (e.g., from MetaMask export or key generation)
raw_key_hex = "0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80"
privkey_bytes = bytes.fromhex(raw_key_hex.removeprefix("0x"))

# Derive public key coordinates (using eth_keys, coincurve, or similar)
from eth_keys import keys
pk = keys.PrivateKey(privkey_bytes)
x_bytes = pk.public_key.to_bytes()[:32]
y_bytes = pk.public_key.to_bytes()[32:]

# Import into PyHSM — key is now AES-KWP double-encrypted at rest
hsm = PyHSM(storage_path="/secure/keystore.enc", master_password="strong-pw")
hsm.import_key_jwk("eth-wallet", {
    "kty": "EC",
    "crv": "secp256k1",
    "x": b64url(x_bytes),
    "y": b64url(y_bytes),
    "d": b64url(privkey_bytes),
})
hsm.close_session()

# DELETE the raw private key from disk/memory — it now lives only in PyHSM
```

Or generate a fresh wallet key directly:

```python
hsm = PyHSM(storage_path="/secure/keystore.enc", master_password="strong-pw")
hsm.generate_key("eth-wallet", "ec-secp256k1")

# Derive the Ethereum address from the public key PEM
pub_pem = hsm.get_public_key("eth-wallet")
```

### Step 2: Sign Transactions Through PyHSM

```python
hsm = PyHSM(storage_path="/secure/keystore.enc", master_password="strong-pw")

# Your application builds and hashes the transaction
tx_hash_hex = "0x..."  # keccak256 of the unsigned RLP-encoded transaction

# Sign through PyHSM — private key is unwrapped, used, and immediately zeroized
signature_hex = hsm.sign("eth-wallet", bytes.fromhex(tx_hash_hex.removeprefix("0x")),
                         caller_id="tx-service")

# Parse the DER-encoded ECDSA signature into (r, s) for Ethereum
from cryptography.hazmat.primitives.asymmetric.utils import decode_dss_signature
r, s = decode_dss_signature(bytes.fromhex(signature_hex))

# Determine v (recovery ID) and broadcast the signed transaction
```

### Step 3: Ed25519 (Solana / Cosmos)

```python
hsm = PyHSM(storage_path="/secure/keystore.enc", master_password="strong-pw")
hsm.generate_key("sol-wallet", "ed25519")

# Sign a Solana transaction payload
signature_hex = hsm.sign("sol-wallet", transaction_bytes, caller_id="sol-service")

# Ed25519 signatures are 64 bytes — use directly with Solana SDK
signature_bytes = bytes.fromhex(signature_hex)
```

### TypeScript (Process Isolation)

For maximum security, run PyHSM in a separate process so even an application-layer
exploit cannot read key material:

```typescript
// Start the HSM process:
// PYHSM_MASTER_PASSWORD="..." npx tsx pyhsm-ts/process.ts

import { PyHSMClient } from "./pyhsm-ts";
const client = new PyHSMClient("/run/pyhsm/pyhsm.sock", "tx-service");

// Sign — key never enters the application process memory
const sig = await client.encrypt("eth-wallet", txHash);
```

### Security Gains for Blockchain Use

| Threat | Without PyHSM | With PyHSM |
|---|---|---|
| Key in `.env` or config | Plaintext on disk | AES-256-GCM + AES-KWP double-encrypted |
| Server compromise (memory dump) | Key exposed | Key in memory only during sign, then zeroized |
| Insider theft | Copy the key file silently | Requires master password + keystore (or M-of-N Shamir shares) |
| No signing audit trail | Attacker signs silently | HMAC-chained audit log records every operation with caller_id |
| Unlimited signing after theft | No constraints | Rate limiting + `max_operations` policy + `expires_at` |
| Single admin controls keys | One person holds everything | Shamir 3-of-5 unlock ceremony |

---

## Security Model

| Property | Mechanism |
|---|---|
| Keys encrypted at rest | AES-256-GCM + AAD binding (Python) / AES-256-GCM-SIV (TypeScript) |
| Per-key double encryption | AES-KWP RFC 5649 wrapping in both layers — keys encrypted inside the encrypted envelope |
| Keystore tamper detection | Encrypt-then-MAC with separated keys (HKDF-derived enc + mac subkeys) |
| Key derivation | Argon2id 64MB (required, Python + TypeScript) → HKDF-Expand. PBKDF2-SHA256 480k iter available only via explicit env var escape hatch for testing |
| Key separation | HKDF-Expand with distinct info strings (`pyhsm-enc-v1`, `pyhsm-mac-v1`, `pyhsm-kek-v1`) — encryption, MAC, and KEK keys are cryptographically independent |
| KEK derivation | Dedicated salt stored inside encrypted keystore → Argon2id → HKDF-Expand. KEK is cached in memory for session lifetime and zeroized on close |
| Memory zeroization | Key material stored as mutable `bytearray` (Python) / `Buffer` (TypeScript) with deterministic in-place zeroing. Immutable hex strings eliminated from memory path |
| Nonce safety | Hybrid nonce: random(4) + counter(4) + random(4) eliminates birthday-bound (Python); AES-256-GCM-SIV nonce-misuse resistant (TypeScript) |
| Ciphertext binding | AAD ties ciphertext to key_id + version — prevents cross-key confusion attacks |
| Ciphertext versioning | Format byte distinguishes v2 (AAD-bound) from v1 (legacy) for backward compatibility |
| Input validation | 64 MB maximum enforced on both encrypt (plaintext) and decrypt (ciphertext) paths |
| Atomic writes | `os.replace()` (Python) / `fs.renameSync` on temp file (TypeScript) |
| Audit integrity | Per-entry HMAC chain; audit HMAC key derived from master password via HKDF (Python) or stored independently (TypeScript) |
| Caller ID tracking | All operations accept optional `caller_id`; recorded in every audit entry |
| Caller ACL enforcement | Per-key `allowed_callers` policy; unauthorized callers denied with `accessDenied` audit entry |
| Constant-time comparisons | `hmac.compare_digest` (Python) / length-padded `timingSafeEqual` (TypeScript) |
| Crypto primitive verification | Known-Answer Tests against RFC vectors at startup |
| Session isolation | Auto-lock on inactivity; explicit `close_session()` / `closeSession()` |
| Concurrency | Per-key sharded locks (Python) — parallel operations on different keys; serialized save lock prevents write races |
| Process memory isolation | Optional: IPC mode runs HSM in a separate process (TypeScript) |
| M-of-N startup ceremony | Shamir split/reconstruct on master password |
| Pluggable storage | `StorageBackend` interface — swap file I/O for database, S3, etc. |
| Key interoperability | JWK (RFC 7517) import/export — supports P-256, P-384, P-521, secp256k1, Ed25519, RSA, AES |
| EC curve support | P-256 (SHA-256), P-384 (SHA-384), P-521 (SHA-512), secp256k1 (SHA-256) — NIST/SEC recommended hash pairing |
| EdDSA support | Ed25519 signing — high-performance 64-byte signatures (Solana, Cosmos, SSH keys) |
| Type safety | PEP 561 `py.typed` marker; `str | bytes` annotations on public API |
| Observability | JSON-structured logging via stdlib `logging`; configurable via `PYHSM_LOG_LEVEL` env var; webhook failures logged (not silently dropped) |

**Honest scope statement:** PyHSM is a software KMS. It does not carry FIPS 140-2/3 validation (which requires NIST laboratory certification of the specific binary). It does not provide the physical tamper evidence of a hardware HSM. Key material is protected by OS-level process boundaries, not a secure enclave or physically separate processor. For regulated environments that mandate certified hardware, use a certified HSM; PyHSM is appropriate where software key management is acceptable.

---

## Threat Model

See [THREAT_MODEL.md](THREAT_MODEL.md) for the full formal threat model, including:

- Assets protected and trust boundary diagrams
- Five threat actor profiles (T1–T5) with specific mitigations
- Cryptographic design decisions and rationale
- Assumptions and known limitations
- Comparison to hardware HSM threat coverage

---

## FAQ

See [docs/FAQ.md](docs/FAQ.md) for detailed answers to common architecture and security questions, including:

- Why build a software HSM instead of using Vault?
- How do you prevent key extraction?
- How is key rotation implemented?
- How are audit logs protected from tampering?
- What cryptographic guarantees do you provide?
- What threat model did you design against?
- Why should I use this instead of AWS KMS?

---

## Running Tests

**Python (pytest):**

```bash
# Run all tests with coverage
python -m pytest tests/ -v
# 120 tests (112 unit/integration + 8 concurrency stress tests)

# Coverage report (80% minimum threshold enforced in CI)
python -m pytest tests/ --cov=hsm --cov-report=term-missing --cov-fail-under=80
```

**Reproducible installs** use the pinned lockfile:

```bash
pip install -r requirements.lock
pip install -e .
```

**TypeScript (vitest):**

```bash
cd pyhsm-ts
npm test
# 94 tests
```

**CI** runs both suites on every push and pull request, across Python 3.11/3.12/3.13 and Node.js 20. Coverage is enforced at 80% minimum. See `.github/workflows/ci.yml`.

---

## Operations Guide

See [`pyhsm-ts/OPERATIONS.md`](pyhsm-ts/OPERATIONS.md) for the full operator guide, including:

- Deployment architectures (embedded vs. process-isolated)
- All environment variables with descriptions and defaults
- Shamir ceremony procedure
- Key rotation, backup, and backup verification procedures
- Audit log verification and SIEM export
- Prometheus metrics reference
- Security considerations

---

## License

MIT
