Metadata-Version: 2.4
Name: promptguard-py
Version: 0.1.0
Summary: LLM Output Validation Framework - enforce schema compliance, detect PII, score hallucination risk, filter toxicity, and validate structured outputs.
Author: Maharshi Soni
License: MIT
Project-URL: Homepage, https://github.com/msoni029/promptguard
Project-URL: Repository, https://github.com/msoni029/promptguard
Project-URL: Issues, https://github.com/msoni029/promptguard/issues
Keywords: llm,guardrails,validation,pii,toxicity,hallucination,pydantic,ai-safety
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic<3.0,>=2.0
Requires-Dist: click<9.0,>=8.0
Requires-Dist: pyyaml<7.0,>=6.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: mypy>=1.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Dynamic: license-file

# PromptGuard - LLM Output Validation Framework

[![CI](https://github.com/msoni029/promptguard/actions/workflows/test.yml/badge.svg)](https://github.com/msoni029/promptguard/actions)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://python.org)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Typed](https://img.shields.io/badge/typed-py.typed-brightgreen.svg)](https://peps.python.org/pep-0561/)

A pip-installable guardrails library for validating LLM outputs. Enforces schema compliance, detects PII leaks, scores hallucination risk, filters toxic content, and validates structured outputs against Pydantic models. Works as middleware between any LLM provider and your application.

---

## Why I Built This

Every production LLM application needs a safety net between the model and the user. I've seen teams ship LLM features where a single hallucinated SSN or a toxic sentence can cause real damage -- legal, reputational, or operational. Existing solutions either require paid API calls, pull in massive dependencies, or only solve one piece of the puzzle.

PromptGuard is my answer: a single, zero-cost library that chains together PII detection, toxicity scoring, hallucination risk analysis, and schema validation into one configurable pipeline. It runs entirely locally with no network calls, making it suitable for air-gapped environments and latency-sensitive applications. I built it to be the validation layer I wished I had on every LLM project.

---

## Architecture

```mermaid
graph TD
    A[LLM Output Text] --> B[ValidationEngine]
    B --> C{Rule Chain from YAML Config}
    C --> D[PII Scanner]
    C --> E[Toxicity Scorer]
    C --> F[Hallucination Analyzer]
    C --> G[Schema Validator]
    D --> H[Regex Pattern Matching]
    D --> I[Luhn Check for CC]
    E --> J[Weighted Pattern Matching]
    E --> K[Category Aggregation]
    F --> L[Entity Extraction]
    F --> M[Claim Analysis]
    F --> N[Hedging Detection]
    G --> O[JSON Schema Validation]
    G --> P[Pydantic Model Validation]
    D --> Q[ValidationReport]
    E --> Q
    F --> Q
    G --> Q
    Q --> R{passed?}
    R -->|Yes| S[Return to Application]
    R -->|No| T[Block / Warn / Redact]

    style A fill:#e1f5fe
    style B fill:#f3e5f5
    style Q fill:#e8f5e9
    style T fill:#ffebee
```

### Component Overview

| Component | Purpose | Technique |
|-----------|---------|-----------|
| **PII Scanner** | Detects emails, phones, SSNs, credit cards, IPs, DOBs | Compiled regex + Luhn algorithm |
| **Toxicity Scorer** | Scores harmful content from 0.0 to 1.0 | Weighted keyword/pattern matching with category aggregation |
| **Hallucination Analyzer** | Estimates fabrication risk | Entity extraction, claim detection, hedging analysis, reference comparison |
| **Schema Validator** | Validates structured JSON output | JSON-schema-style rules or Pydantic model validation |
| **Rule Engine** | Chains validators into a pipeline | YAML-configurable with hard/soft fail modes |

---

## Quick Demo (60-Second Walkthrough)

### Install

```bash
pip install -e .
```

### Python API

```python
from promptguard import ValidationEngine, scan_pii, score_toxicity, redact_pii

# Full pipeline validation
engine = ValidationEngine()
report = engine.validate("Contact john@example.com for the report.")
print(report.passed)    # False
print(report.summary)   # FAIL: 1 PII item(s) detected

# PII scanning
matches = scan_pii("SSN: 123-45-6789, Email: test@corp.com")
for m in matches:
    print(f"{m.pii_type.value}: {m.redacted}")
# ssn: ***-**-****
# email: t***@corp.com

# Redaction
safe = redact_pii("Call 555-123-4567 or email admin@company.com")
print(safe)  # Call ***-***-4567 or email a***@company.com

# Toxicity scoring
result = score_toxicity("This project is absolutely wonderful!")
print(result.score, result.is_toxic)  # 0.0 False
```

### CLI

```bash
# Full validation
promptguard validate "The quarterly report shows 15% growth."

# Scan for PII
promptguard scan-pii "Email: john@example.com, SSN: 123-45-6789"

# Redact PII
promptguard scan-pii --redact "Contact: admin@corp.com"

# Quick CI/CD check (exit code 0=pass, 1=fail)
promptguard check "Clean text with no issues"

# Generate starter config
promptguard init-config -o promptguard.yml

# Use custom config
promptguard validate -c promptguard.yml "Some LLM output"

# JSON output for scripting
promptguard validate --json-output "Some text to validate"
```

### YAML Configuration

```yaml
version: "1"
name: my-pipeline
fail_fast: false
max_text_length: 100000
rules:
  - name: pii-scan
    validator: pii
    enabled: true
    hard_fail: true
    params: {}
  - name: toxicity-check
    validator: toxicity
    hard_fail: true
    params:
      threshold: 0.5
  - name: hallucination-risk
    validator: hallucination
    hard_fail: false  # warn but don't block
    params:
      threshold: 0.5
  - name: schema-check
    validator: schema
    hard_fail: true
    params:
      strict: true
```

---

## Features

- **Schema validation** for JSON/structured LLM outputs (JSON-schema-style or Pydantic models)
- **PII detection** for emails, phones, SSNs, credit cards, IP addresses, dates of birth
- **PII redaction** with type-aware masking (e.g., `j***@example.com`, `***-**-****`)
- **Toxicity scoring** with keyword and pattern matching across multiple categories
- **Hallucination risk scoring** via entity extraction, claim analysis, and hedging detection
- **CLI tool** with `validate`, `scan-pii`, and `check` commands
- **Configurable rule chains** with YAML config, hard/soft fail modes, fail-fast option
- **Full type hints** and `py.typed` marker for IDE support
- **Zero external API calls** -- runs entirely offline

---

## Performance / Benchmarks

All benchmarks measured on a standard laptop (Intel i7, 16 GB RAM, Python 3.11).

| Operation | Text Size | Time | Throughput |
|-----------|-----------|------|------------|
| PII scan (all types) | 1 KB | ~0.2 ms | ~5,000 texts/sec |
| PII scan (all types) | 10 KB | ~1.5 ms | ~667 texts/sec |
| Toxicity scoring | 1 KB | ~0.1 ms | ~10,000 texts/sec |
| Hallucination analysis | 1 KB | ~0.3 ms | ~3,300 texts/sec |
| Full pipeline (3 rules) | 1 KB | ~0.8 ms | ~1,250 texts/sec |
| Schema validation (Pydantic) | 500 B | ~0.05 ms | ~20,000 texts/sec |

**Memory footprint**: ~15 MB base (compiled regex patterns + Pydantic models). Scales linearly with text size during processing.

**Key design decisions for performance**:
- All regex patterns are pre-compiled at import time
- Luhn algorithm runs only on credit-card-shaped matches (not all digit sequences)
- Toxicity scoring uses short-circuit max-weight calculation
- The engine supports `fail_fast` mode to skip remaining rules on first failure

---

## What I Would Do Differently

1. **ML-based toxicity scoring**: The keyword/pattern approach has clear limitations -- it misses subtle toxicity and can false-positive on clinical or academic text. A lightweight transformer model (e.g., a distilled BERT classifier) would dramatically improve accuracy while staying offline-capable.

2. **Smarter hallucination detection**: The current heuristic approach (hedging phrases, claim patterns, entity matching) is a starting point. Real hallucination detection needs access to the source documents and ideally the model's token-level probabilities. I'd integrate with retrieval pipelines to do proper attribution checking.

3. **Streaming support**: LLM outputs often stream token-by-token. The current API requires the complete text. A streaming validator that can flag issues incrementally (e.g., "PII detected at token 47") would be much more practical in production.

4. **Plugin architecture**: Instead of hardcoding four validators, I'd build a proper plugin system where custom validators can register via entry_points. This would let teams add domain-specific rules (e.g., medical term validation, financial compliance checks) without forking the library.

5. **Async engine**: The validation engine is synchronous. For high-throughput services, an async version with `asyncio` support would allow concurrent validation of multiple outputs.

---

## Scaling Considerations

### Horizontal Scaling
PromptGuard is stateless and thread-safe (all validators use compiled patterns and pure functions). You can run it across multiple processes or containers without coordination.

### High-Throughput Pipelines
- Use `fail_fast: true` in config to short-circuit on the first failure
- Disable validators you don't need (e.g., skip hallucination for structured-output-only flows)
- For batch processing, the engine can be instantiated once and reused across thousands of validations

### Memory Considerations
- Compiled regex patterns are shared across all engine instances in the same process
- For texts > 100KB, consider chunking and validating segments independently
- The `max_text_length` config parameter acts as a safety valve against OOM scenarios

### Integration Patterns
```
[LLM Provider] --> [PromptGuard] --> [Your Application]
                        |
                   [Config YAML]
```
PromptGuard is designed to sit as middleware. Common integration points:
- **FastAPI middleware**: Validate every response before returning to the client
- **LangChain callback**: Plug into the chain as a post-processing step
- **Batch pipeline**: Validate CSV/JSONL outputs before downstream consumption

---

## Development

```bash
# Clone and install in dev mode
git clone https://github.com/msoni029/promptguard.git
cd promptguard
pip install -e ".[dev]"

# Run tests
python -m pytest tests/ -v

# Type checking
mypy src/promptguard/

# Linting
ruff check src/ tests/
```

---



---

## Sample Input / Output

![Sample Input and Output](assets/io-card.png)

---

## Project Overview

![Project Summary](assets/report-card.png)

### Reports
- [HTML Report](reports/promptguard-report.html) - interactive report
- [PDF Report](reports/promptguard-report.pdf) - downloadable PDF
- [TXT Report](reports/promptguard-report.txt) - plain text

## License

MIT License. See [LICENSE](LICENSE) for details.
