Metadata-Version: 2.4
Name: pillar-gguf-scanner
Version: 0.1.1
Summary: Fast, programmatic scanning utilities for GGUF chat templates.
Author: Pillar Security
License: Apache-2.0
Project-URL: Homepage, https://pillar.security
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: gguf>=0.6
Requires-Dist: httpx>=0.25
Requires-Dist: rich>=13.0
Dynamic: license-file

# GGUF Template Scanner

High-level scanning utilities for GGUF model files. The library extracts embedded chat templates, runs heuristic checks for prompt-injection markers, and optionally consults Pillar's remote scanning API for deeper analysis.

## Background

This project addresses security threats identified in research by [Pillar Security on LLM backdoors at the inference level](https://www.pillar.security/blog/llm-backdoors-at-the-inference-level-the-threat-of-poisoned-templates). The research demonstrates how malicious chat templates embedded in GGUF model files can be exploited for prompt injection and backdoor attacks, enabling unauthorized control over model behavior.

## Features

- Parse GGUF headers and extract default or named chat templates with a small, dependency-light API.
- Run a shipped lightweight anomaly classifier for unknown templates using the existing trained model artifact.
- Run configurable heuristics (URLs, base64 payloads, normalize.js patterns, etc.) to flag suspicious templates.
- Invoke Pillar’s hosted scanning service when an API key is provided, returning unified findings.
- Stream GGUF headers from plain URLs or Hugging Face repositories using ranged requests.
- Async and sync functions for local, remote, and Hugging Face scans.

## Offline Template Classifier

The scanner ships with a fully offline template classifier designed for fast first-pass security review of GGUF chat
templates.

At runtime, the scanner analyzes each extracted template with a lightweight CPU classifier in milliseconds and returns an
offline triage verdict with confidence scores. Template hashes are still included in the scan evidence for integrity tracking
and downstream workflows, but the OSS package does not rely on a shipped verdict corpus.

This allows the core scanner to inspect the actual GGUF chat templates it discovers with **no network required at runtime**.
Remote calls to Pillar remain optional.

### Classifier training corpus

The shipped classifier was trained from a population-level analysis pipeline over GGUF chat templates discovered on Hugging
Face:

- approximately **3.3M GGUF files** crawled
- **2,951 unique templates** after SHA-256 deduplication
- labels produced by batch analysis over the deduplicated corpus

- **2,946 clean**
- **29 suspicious**
- **80 malicious**

### Classifier

The shipped offline artifact at
`src/pillar_gguf_scanner/data/template_classifier.json.gz` uses an ordinal,
two-stage design:

1. A shallow gradient-boosted tree model estimates whether a template is
   clean or risky.
2. A regularized logistic model estimates malicious versus suspicious for
   risky templates. Length, generic jailbreak language, and persona strength
   are excluded from this harm stage.
3. Platt calibration and family-grouped out-of-fold predictions set thresholds
   that target a 1% clean review rate and a 0.25% clean-to-malicious rate.

Both models consume structural and security-oriented features extracted from
Jinja2 chat templates, including:

- control-flow complexity
- message rewriting and patching behavior
- direct and aliased content-triggered hidden system instructions
- concealment and jailbreak language
- exfiltration indicators
- sandbox escape / RCE markers
- hardcoded URLs and script patterns
- conditional dependency-install actions across package, artifact, container, and system registries
- repository-history staging, sensitive-path collection, archive/encryption, and outbound transfer combinations
- Git remote replacement followed by mirror, all-branch, or tag pushes
- system-message manipulation signals

High-confidence semantic findings remain authoritative. The ordinal models
rank and classify templates that do not complete a deterministic attack chain:

- extracted template → classifier triage
- suspicious or malicious results can then be escalated to deeper analysis or human review

This model is designed for fast offline triage, not as the only source of truth for novel attacks.

The full methodology, data provenance, label definitions, hyperparameters,
calibration policy, evaluation protocol, artifact hash, and reproducibility
limitations are documented in [`model/MODEL_CARD.md`](model/MODEL_CARD.md).

### Reported validation metrics

The primary metrics use five-fold `StratifiedGroupKFold`. Complete collected
model families are held out together, and FARA, public research-backdoor,
installation, exfiltration, and near-miss variants remain within one fold.
Related variants cannot appear in both training and validation.

| Class | Precision | Recall | F1 | Support |
| --- | ---: | ---: | ---: | ---: |
| clean | 0.99 | 0.99 | 0.99 | 2,946 |
| suspicious | 0.38 | 0.28 | 0.32 | 29 |
| malicious | 0.93 | 0.89 | 0.91 | 80 |
| macro average | 0.77 | 0.72 | 0.74 | 3,055 |

Confusion matrix:

| Actual \\ Predicted | clean | suspicious | malicious |
| --- | ---: | ---: | ---: |
| clean | 2,928 | 13 | 5 |
| suspicious | 21 | 8 | 0 |
| malicious | 9 | 0 | 71 |

The observed clean review rate is 0.611%, and the clean-to-malicious rate is
0.170%. Suspicious remains data-limited; malicious behavior is the principal
blocking target.

### Caveats

- The **suspicious** class remains the weakest class because 29 examples cannot represent the full diversity of partial
  attack-chain behavior and overlap with legitimate strong system prompts or uncensored personas. Treat its score as
  exploratory rather than production-quality.
- The **malicious** class is materially stronger and is the primary purpose of the classifier: fast offline triage for novel
  or previously unseen templates.
- The malicious training set is still concentrated around several attack families, including namespace patching, sandbox
  escapes, jailbreak injection, script-tag injection, system-message hijacking, conditional supply-chain installation, and
  staged workspace or repository-history exfiltration.
- A genuinely novel attack that does not activate the current feature set can still evade the classifier. That is why the
  classifier complements, rather than replaces, longer-running analysis pipelines and human review.

## Installation

### With uv (recommended)

```bash
# install runtime dependencies
uv sync

# add testing extras
uv sync --group test
```

### Via pip

```bash
pip install pillar-gguf-scanner
```

## Quickstart

```python
from pillar_gguf_scanner import GGUFTemplateScanner, Verdict

scanner = GGUFTemplateScanner()

# Scan a local GGUF file
result = scanner.scan("models/my-model.gguf")

print(result.verdict)  # Verdict.CLEAN, Verdict.SUSPICIOUS, Verdict.MALICIOUS, or Verdict.ERROR
for finding in result.findings:
    print(f"{finding.rule_id}: {finding.message}")

if result.errors:
    for detail in result.errors:
        print(f"error[{detail.code}] -> {detail.message}")
        if detail.context:
            print(detail.context)
```

### Using the Pillar API

```python
from pillar_gguf_scanner import GGUFTemplateScanner

scanner = GGUFTemplateScanner(pillar_api_key="your-api-key")
result = scanner.scan("models/my-model.gguf", use_pillar=True)
```

Set `use_pillar=False` to opt out of remote calls on a per-scan basis. Remote requests use `httpx` clients supplied by the caller or managed internally. Attach `ScannerConfig(event_handler=...)` to receive structured telemetry such as `pillar_response`, `remote_fetch_failed`, and `heuristic_match` events.

## Scanning Different Sources

### Scanning Hugging Face Models

Scan GGUF files hosted on Hugging Face repositories using the `scan_huggingface()` method:

```python
from pillar_gguf_scanner import GGUFTemplateScanner

scanner = GGUFTemplateScanner()

# Synchronous scanning
result = scanner.scan_huggingface(
    repo_id="TheBloke/Llama-2-7B-GGUF",
    filename="llama-2-7b.Q4_K_M.gguf",
    revision="main",  # optional, defaults to "main"
    token="hf_xxx"    # optional, for private repos
)

print(f"Verdict: {result.verdict.value}")
for finding in result.findings:
    print(f"[{finding.severity.value}] {finding.rule_id}: {finding.message}")
```

Alternatively, use the unified `scan()` method with a `HuggingFaceRepoRef`:

```python
from pillar_gguf_scanner import GGUFTemplateScanner, HuggingFaceRepoRef

scanner = GGUFTemplateScanner()
ref = HuggingFaceRepoRef(
    repo_id="TheBloke/Llama-2-7B-GGUF",
    filename="llama-2-7b.Q4_K_M.gguf",
    revision="main"
)
result = scanner.scan(ref)
```

### Scanning from URLs

Scan GGUF files from direct download URLs:

```python
from pillar_gguf_scanner import GGUFTemplateScanner

scanner = GGUFTemplateScanner()

# Direct URL scanning
result = scanner.scan_url("https://example.com/model.gguf")

# Or use the unified scan() method
result = scanner.scan("https://example.com/model.gguf")
```

## Async Scans

All scanning methods have async variants for concurrent operations:

```python
import asyncio
from pillar_gguf_scanner import GGUFTemplateScanner

scanner = GGUFTemplateScanner()

async def scan_models():
    # Async Hugging Face scanning
    hf_result = await scanner.ascan_huggingface(
        repo_id="TheBloke/Llama-2-7B-GGUF",
        filename="llama-2-7b.Q4_K_M.gguf",
        token="hf_xxx"  # optional
    )

    # Async URL scanning
    url_result = await scanner.ascan_url("https://example.com/model.gguf")

    # Async local file scanning
    path_result = await scanner.ascan_path("models/local-model.gguf")

    return hf_result, url_result, path_result

results = asyncio.run(scan_models())
```

For efficient batch scanning with connection pooling, use `scanner_session()` or `ascanner_session()` context managers:

```python
from pillar_gguf_scanner import ascanner_session

async def batch_scan(repo_files):
    async with ascanner_session() as scanner:
        tasks = [
            scanner.ascan_huggingface(repo_id, filename)
            for repo_id, filename in repo_files
        ]
        return await asyncio.gather(*tasks)
```

The low-level helpers `fetch_chat_templates_from_url`, `afetch_chat_templates_from_url`, and `fetch_chat_templates_from_huggingface` are also available for integrating into existing pipelines.

## Common Patterns

### Quick Reference: Scanning Methods

```python
from pillar_gguf_scanner import GGUFTemplateScanner, HuggingFaceRepoRef

scanner = GGUFTemplateScanner()

# Local file
result = scanner.scan("path/to/model.gguf")
result = scanner.scan_path("path/to/model.gguf")  # explicit method

# Direct URL
result = scanner.scan("https://example.com/model.gguf")
result = scanner.scan_url("https://example.com/model.gguf")  # explicit method

# Hugging Face - Method 1: Direct method (recommended for clarity)
result = scanner.scan_huggingface("owner/repo", "model.gguf")

# Hugging Face - Method 2: Via unified scan() with HuggingFaceRepoRef
ref = HuggingFaceRepoRef(repo_id="owner/repo", filename="model.gguf")
result = scanner.scan(ref)
```

### Checking Scan Results

```python
from pillar_gguf_scanner import Verdict

result = scanner.scan("model.gguf")

# Check overall verdict
if result.verdict == Verdict.MALICIOUS:
    print("⚠️  Malicious template detected!")
elif result.verdict == Verdict.SUSPICIOUS:
    print("⚠️  Suspicious patterns found")
elif result.verdict == Verdict.CLEAN:
    print("✅ No threats detected")
elif result.verdict == Verdict.ERROR:
    print("❌ Scan failed")
    for error in result.errors:
        print(f"  {error.code}: {error.message}")

# Access findings
for finding in result.findings:
    print(f"[{finding.severity.value}] {finding.rule_id}: {finding.message}")
    if finding.snippet:
        print(f"  Snippet: {finding.snippet[:100]}...")

# Access classifier results
for prediction in result.classifier_results:
    print(prediction.template_name, prediction.verdict.value, prediction.confidence)

# Check if templates were found
if result.evidence.default_template:
    print(f"Default template length: {result.evidence.template_lengths['default']}")
    print(f"Template hash: {result.evidence.template_hashes['default']}")
```

### Batch Scanning with Connection Reuse

```python
from pillar_gguf_scanner import scanner_session

models = [
    ("TheBloke/Llama-2-7B-GGUF", "llama-2-7b.Q4_K_M.gguf"),
    ("TheBloke/Mistral-7B-GGUF", "mistral-7b.Q4_K_M.gguf"),
]

# Connection pooling for efficiency
with scanner_session() as scanner:
    for repo_id, filename in models:
        result = scanner.scan_huggingface(repo_id, filename)
        print(f"{repo_id}/{filename}: {result.verdict.value}")
```

## Customising Heuristics

Provide a `ScannerConfig` with your own rule set or severity overrides:

```python
from pillar_gguf_scanner import (
    DEFAULT_PATTERNS,
    GGUFTemplateScanner,
    PatternRule,
    ScannerConfig,
    Severity,
    merge_heuristics,
)

custom_rules = [
    PatternRule(
        rule_id="custom-warning",
        severity=Severity.MEDIUM,
        message="Template contains forbidden phrase",
        search_terms=("do not disclose",),
    ),
]

config = ScannerConfig(
    heuristic_rules=merge_heuristics(DEFAULT_PATTERNS, custom_rules),
    url_severity=Severity.HIGH,
)

scanner = GGUFTemplateScanner(config=config)
result = scanner.scan("model.gguf")
```

## CLI

`pillar-gguf-scanner` ships with a `pillar-gguf-scanner` executable.

```bash
# binary installed by uv or pip
uv run pillar-gguf-scanner path/to/model.gguf

# JSON output and remote scanning
pillar-gguf-scanner path/to/model.gguf --json --pillar-api-key "$PILLAR_API_KEY"
```

Run `pillar-gguf-scanner --help` to see all options, including severity overrides and Pillar toggles.

The core package scans local files, direct URLs, and specific Hugging Face GGUF files. Larger-scale Hugging Face crawling,
live monitoring, and registry-wide batch orchestration are intended to sit on top of the library rather than inside the core scanner.

```text
usage: pillar-gguf-scanner [-h] [--pillar-api-key PILLAR_API_KEY] [--no-pillar]
                           [--json] [--url-severity {info,low,medium,high,critical}]
                           [--base64-severity {info,low,medium,high,critical}]
                           [--hf-repo HF_REPO] [--hf-filename HF_FILENAME]
                           [--hf-revision HF_REVISION] [--hf-token HF_TOKEN]
                           [source]
```

## Development

* `uv sync --group test` – install dev + test dependencies
* `uv run pytest` – execute the test suite
* `uv run ruff check src tests` – lint maintained package and test code with Ruff (optional but recommended)
* `uv run mypy src` – run static type checks
* `uv run python -m build` – create distribution artifacts

Tests live in `tests/` and cover parsing, heuristics, and remote fetch logic. The suite requires the `test` dependency group.

## Troubleshooting

### API Key Issues

**Problem**: "Pillar API key not working" or authentication errors
**Solution**:
- Verify your API key is set correctly: `export PILLAR_API_KEY="your-key-here"`
- Check the key is passed to the scanner: `GGUFTemplateScanner(pillar_api_key=os.environ["PILLAR_API_KEY"])`
- Ensure you're using `use_pillar=True` when calling `scan()`
- Contact Pillar support if authentication continues to fail

### Timeout Errors

**Problem**: "Remote fetch timeout" or requests timing out
**Solution**:
- Increase the timeout in your config:
  ```python
  config = ScannerConfig(request_timeout=120.0)  # 2 minutes
  scanner = GGUFTemplateScanner(config=config)
  ```
- Check your network connection and firewall settings
- For large models, the initial header fetch may take longer

### False Positives

**Problem**: Legitimate templates flagged as suspicious
**Solution**:
- Adjust severity levels to reduce noise:
  ```python
  config = ScannerConfig(
      url_severity=Severity.LOW,      # URLs are common in templates
      base64_severity=Severity.INFO,  # Reduce base64 alerts
  )
  ```
- Review the specific findings and snippets to understand what triggered the detection
- Create custom rules that override defaults using `merge_heuristics()`

### Range Request Errors

**Problem**: "Server does not support range requests" when scanning URLs
**Solution**:
- The URL must support HTTP Range headers for efficient scanning
- Download the file locally and use `scan_path()` instead:
  ```python
  scanner.scan_path("/path/to/downloaded/model.gguf")
  ```

### GGUF Parse Errors

**Problem**: "Invalid GGUF magic" or "Buffer underrun" errors
**Solution**:
- Verify the file is actually a GGUF file: `file model.gguf` should show binary data
- Check the file isn't corrupted or truncated
- Ensure you have read permissions: `ls -l model.gguf`
- For remote URLs, verify the URL points directly to the .gguf file, not an HTML page

### Missing Chat Templates

**Problem**: GGUF file scans as CLEAN but you expected findings
**Solution**:
- Check if the model actually has chat templates:
  ```python
  result = scanner.scan("model.gguf")
  if not result.evidence.has_template:
      print("No chat template found in this model")
  ```
- Some GGUF files don't include chat templates in metadata
- View extracted templates: `print(result.evidence.default_template)`

### Getting Help

If you encounter issues not covered here:
1. Check the examples in `examples/` directory for working code
2. Enable debug logging to see detailed error information:
   ```python
   import logging
   logging.basicConfig(level=logging.DEBUG)
   ```
3. Open an issue on GitHub with:
   - Error message and full traceback
   - Scanner configuration and code snippet
   - GGUF file source (if publicly accessible)

## Contributing

1. Fork and clone the repository.
2. Install dependencies with `uv sync --group test`.
3. Create a feature branch and ensure `pytest` passes.
4. Open a pull request describing the change and relevant context.

Bug reports and feature suggestions are welcome through GitHub issues.

## License

Distributed under the terms of the Apache License 2.0. See `LICENSE` for full text.
