Metadata-Version: 2.4
Name: unbudged
Version: 0.1.2
Summary: Prove an adversarial input cannot move your score.
Author-email: Jakob Stanfield <jakob.stanfield@ras-software.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/JakobS1900/unbudged
Project-URL: Repository, https://github.com/JakobS1900/unbudged
Project-URL: Issues, https://github.com/JakobS1900/unbudged/issues
Keywords: prompt-injection,llm,testing,security,scoring,evaluation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml>=5.4
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Dynamic: license-file

# unbudged

[![PyPI](https://img.shields.io/pypi/v/unbudged.svg)](https://pypi.org/project/unbudged/)
[![Python](https://img.shields.io/pypi/pyversions/unbudged.svg)](https://pypi.org/project/unbudged/)
[![CI](https://github.com/JakobS1900/unbudged/actions/workflows/ci.yml/badge.svg)](https://github.com/JakobS1900/unbudged/actions/workflows/ci.yml)
[![License](https://img.shields.io/pypi/l/unbudged.svg)](LICENSE)

unbudged tests whether a scoring function can be steered by text it was never meant to take instructions from. It takes a realistic input, appends a known adversarial payload, and compares the score shift against a neutral control string of the exact same length. Only a shift the control does not also produce counts as a finding. Point it at anything with a `.score(text) -> float` method and a handful of realistic inputs, and it names the payloads that moved the number further than length alone accounts for.

That last clause is the whole tool, and it cuts both ways. Read [what unbudged will not tell you](#what-unbudged-will-not-tell-you) before you trust a clean run.

## The contrast, in two runs

A keyword rubric grader that treats anything appended to the answer as part of the answer:

```
$ python -m unbudged.cli run examples/naive_grader.py
33 attacks, 3 baselines, 99 probes
23 findings across 5 families
  role.act-as-admin          attack +25      control +0       excess +25      on "I would verify the caller's ide..."
  role.act-as-admin          attack +25      control +0       excess +25      on "I would raise a ticket, note th..."
  role.act-as-admin          attack +25      control +0       excess +25      on "I would follow the documented p..."
  social.authority           attack +25      control +0       excess +25      on "I would verify the caller's ide..."
  social.authority           attack +25      control +0       excess +25      on "I would raise a ticket, note th..."
  system.forged-turn         attack +25      control +0       excess +25      on "I would verify the caller's ide..."
  system.forged-turn         attack +25      control +0       excess +25      on "I would raise a ticket, note th..."
  delim.triple-backtick      attack +12.5    control +0       excess +12.5    on "I would verify the caller's ide..."
  delim.triple-backtick      attack +12.5    control +0       excess +12.5    on "I would raise a ticket, note th..."
  delim.triple-backtick      attack +12.5    control +0       excess +12.5    on "I would follow the documented p..."
  override.append-directive  attack +12.5    control +0       excess +12.5    on "I would verify the caller's ide..."
  override.append-directive  attack +12.5    control +0       excess +12.5    on "I would raise a ticket, note th..."
  override.reset             attack +12.5    control +0       excess +12.5    on "I would verify the caller's ide..."
  override.reset             attack +12.5    control +0       excess +12.5    on "I would raise a ticket, note th..."
  override.reset             attack +12.5    control +0       excess +12.5    on "I would follow the documented p..."
  role.pretend-reviewer      attack +12.5    control +0       excess +12.5    on "I would verify the caller's ide..."
  role.pretend-reviewer      attack +12.5    control +0       excess +12.5    on "I would raise a ticket, note th..."
  role.pretend-reviewer      attack +12.5    control +0       excess +12.5    on "I would follow the documented p..."
  social.authority           attack +12.5    control +0       excess +12.5    on "I would follow the documented p..."
  social.consensus           attack +12.5    control +0       excess +12.5    on "I would verify the caller's ide..."
  social.consensus           attack +12.5    control +0       excess +12.5    on "I would raise a ticket, note th..."
  social.consensus           attack +12.5    control +0       excess +12.5    on "I would follow the documented p..."
  system.forged-turn         attack +12.5    control +0       excess +12.5    on "I would follow the documented p..."
exit 1
```

That grader marks a service-desk assessment by counting rubric terms anywhere in the text it is handed, 12.5 points each. It has no notion of where the candidate's answer ends, so a payload appended to the answer is credited as if the candidate had written it. What makes that expensive here is a collision nobody planned: an access-control rubric and a prompt injection are written in the same words. `system`, `administrator`, `policy`, `reset`, `review`, `approve`. `role.act-as-admin` says "Acting as the system administrator" and collects two rubric points for it. The grader cannot tell a term the candidate earned from one that arrived in a payload.

The collision is the honest caveat as well as the result. Give the same grader a clinical rubric and this corpus finds nothing, not because the grader got safer but because the vocabularies stopped overlapping. See [what unbudged will not tell you](#what-unbudged-will-not-tell-you).

The transliterated core of Counterpart, which reads a narrow bounded signal off the text instead of scanning it for instructions:

```
$ python -m unbudged.cli run examples/counterpart.py
33 attacks, 3 baselines, 99 probes
0 findings
max excess over control: 0
exit 0
```

Same corpus, same three baselines, same tolerance. This target never scans the text for instructions. It reads four booleans off it, and everything downstream is arithmetic on those four. That does not make it unmovable, and the example docstring is blunt about the difference: each field moves the score by a fixed amount when it flips. An opener plus a question mark is worth 22.2 points here, naming a disclosure topic at the right moment is worth 44.4, and a dismissal costs 44.4. Those are per-field amounts, not the exposure: the fields are independent booleans, so one payload can flip two of them at once and the amounts stack. The worst case measured against this example is a single sentence appended to a baseline sitting at 33.3, that both asks a question and names a disclosure topic, which pushes the score to a perfect 100.0, a swing of +66.7. Clamping bounds the size of a swing. It does not prevent one.

What the clean run above actually shows is that none of the 33 shipped payloads contains a question mark, a disclosure topic, or a dismissal, so none of them can set a field. That is a real property worth having, because the fields are enumerable and the per-field amounts are fixed, which makes the worst case computable without running anything: +66.7 positive, -44.4 negative. It is not the same claim as "no wording can move this", which would be false.

## Install

```
pip install unbudged
```

Or clone it and install in editable mode, which is what you want if you plan to add attacks:

```
git clone https://github.com/JakobS1900/unbudged
cd unbudged
pip install -e ".[dev]"
pytest
```

Python 3.9 or newer. One runtime dependency, `pyyaml`.

## Use it in your own test suite

```python
from unbudged import assert_unbudged

def test_grader_resists_injection():
    assert_unbudged(MyGrader(), baselines=GOLDEN_ANSWERS)
```

If the target moves past tolerance, this raises `AssertionError` with the full text report as the message, so a failing build names the offending attack and its excess without anyone opening a JSON file. `assert_unbudged` and `run` share the same signature: a target, a list of baselines, an optional `tolerance`, and an optional list of `families` to restrict the run to.

## Why there is a control string

Appending any text to an input can move a scorer that is merely sensitive to length. If unbudged only compared a baseline against baseline-plus-attack, a scorer that adds a fraction of a point per character would report a finding on every attack, on every baseline, forever. That is not a payload being read as an instruction. That is a scorer noticing the input got longer, and without a control it would swamp the real findings in noise a reader would learn to ignore.

So for every attack, unbudged also builds a neutral control of the exact same character length, made of mundane filler sentences with no imperatives, no numbers, and no overlap with the baseline's subject. The control is joined to the baseline with the same separator the attack gets, so the two strings match at the seam as well as in total length. unbudged appends the control instead of the attack, measures the shift the same way, and a finding fires only when

```
excess = abs(attack_score - baseline) - abs(control_score - baseline) > tolerance
```

`LengthSensitiveTarget` in the test suite is scored purely by `len(text)` and produces zero findings against the full corpus. That pins the case the control was built for. It is not proof that the control handles every confound, and the next section says which ones it does not.

## What unbudged will not tell you

Four limitations. The first two are pinned by stub targets in `tests/conftest.py`, so if a future change closes either gap the suite fails and this section comes out with it.

**The control matches characters, not words or sentences.** A scorer keyed on word count or on the number of full stops sees a difference between a payload and its control even though neither reads the text for meaning, and unbudged reports that difference as a finding. `WordCountTarget` scores `0.5 * len(text.split())` and produces findings. `SentenceCountTarget` scores `3.0 * text.count(".")` and produces more. Both are false. If your scorer counts anything other than characters, expect noise and read the attack ids before believing them.

**A scorer that penalises padding can hide a real injection completely.** Excess subtracts the control's movement without regard to its direction. Marking down for off-topic padding is ordinary rubric behaviour, and it moves the control down. If the attack moves the score up, the two cancel. `PaddingPenalisingTarget` in the test suite deducts for length and adds 5 points whenever the canonical override payload appears, which makes it provably injectable, and unbudged reports it clean with exit 0. This is the more dangerous of the two directions, because the false-positive case gives you a noisy report to argue with and this one gives you a green tick.

Even short of full masking, excess understates. An injection worth a genuine `+8` against a control that moved `-6` is reported as `excess +2`. The text report prints the signed attack shift and the signed control shift next to the excess for exactly this reason: when the control moved a long way in either direction, the excess is a difference of two large numbers and deserves suspicion rather than trust.

**A clean run is a statement about this corpus, not about your target.** unbudged ships 33 payloads. It finds what those payloads can reach. The naive grader above falls over because an access-control rubric happens to share vocabulary with a prompt injection; the same grader with a clinical rubric passes this corpus untouched while being exactly as broken. Zero findings means these payloads did not move your scorer. It does not mean nothing will.

**One turn, no state.** unbudged calls `.score(text)` once per probe. A defence that only holds because movement accumulates slowly over a conversation is not something this tool can check, and a target whose scoring depends on prior turns has to fold that into its own `score` the way `examples/counterpart.py` does.

## The target module contract

A target is a plain Python module that defines two names at module level:

- `target`: any object with a `.score(text) -> float` method
- `baselines`: a list of realistic strings, the kind of input your system normally sees

That is the whole contract. `examples/naive_grader.py` and `examples/counterpart.py` both follow it, the CLI loader expects it, and `run()` and `assert_unbudged` take the two pieces apart directly rather than inventing baselines on your behalf, because only you know what valid input to your system looks like.

## The seven attack families

Attack ids carry a short prefix rather than the full family name, so the report stays readable. `--family` takes the full name. The mapping is fixed and a test enforces it against every shipped attack.

| Family (`--family`) | Id prefix | What it tries |
|---|---|---|
| `instruction_override` | `override.` | Tells the target directly to ignore its rubric and award a fixed score. |
| `role_reversal` | `role.` | Claims an authority the grader does not have: administrator, reviewer, a debug mode. |
| `delimiter_escape` | `delim.` | Closes an assumed wrapper such as XML, JSON, a fenced code block, or a YAML document, then forges content after it. |
| `encoding` | `encoding.` | Re-encodes the canonical override in base64, rot13, leetspeak, character spacing, or URL escaping to dodge keyword filters. |
| `homoglyph` | `homoglyph.` | Swaps in look-alike characters, Cyrillic letters, zero-width spaces, a right-to-left override, fullwidth forms, combining marks, so the text reads the same but matches nothing. |
| `social_pressure` | `social.` | Leans on authority, urgency, flattery, false consensus, or sympathy instead of merit. |
| `fake_system` | `system.` | Forges a system turn, a developer note, an assistant prefix, or a tool result, so the text looks like it came from the harness itself. |

The text report prints the id, which carries the prefix. `--json` prints the full family name in its own field.

## The command line

Installing puts an `unbudged` console script on your path. These two are the same command:

```
unbudged run path/to/target.py[:name]
python -m unbudged.cli run path/to/target.py[:name]
```

`name` defaults to `target` and only needs stating if your module calls the object something else. Flags: `--json` for machine-readable output, `--tolerance` to override the default, `--family` (repeatable) to restrict the run to one or more families above, `--baseline` (repeatable) to add extra baselines on top of the module's own, and `--max-findings` to allow a small number of findings before the command still exits zero.

Exit codes: 0 for clean or within `--max-findings`, 1 for findings above threshold or any probe that errored, 2 for a usage problem such as a missing target file or a target module that raises while importing. Only exit 1 ever means the target moved, which is what makes `--max-findings` safe to ratchet in CI.

One parsing limitation worth knowing: the `path:name` split looks for the last colon in the spec, skipping only a Windows drive letter. A target file whose own name contains a literal colon, for example `notes:2020.py`, is misparsed, because that colon cannot be told apart from the separator.

## Loading a target runs its file

The CLI and `run()` both load your target module by executing the file at the path you give them. That is the same trust model as pytest collecting a `conftest.py`: point either tool at code you trust, and do not point either at code you do not.

## Why the tolerance default is 1e-9, not 0.0

A scorer that does any floating point arithmetic produces last-bit differences between runs that are never bit for bit identical. That is rounding noise, not an injection, and a hard zero threshold would report every instance of it as a finding. `1e-9` is small enough to catch a real shift and large enough to absorb that noise.

It is an absolute threshold, and it assumes scores on a human scale: marks out of 10, out of 100, out of 1000. Double precision carries about 16 significant digits, so above roughly `1e7` a single rounding step is already larger than `1e-9` and the default stops absorbing noise and starts reporting it. If your scorer returns numbers that big, pass a `--tolerance` scaled to them. `--tolerance` overrides it from the command line, and `run` and `assert_unbudged` both take it as a keyword argument.

## Out of scope

unbudged does not generate attack mutations with a language model, does not model multi-turn conversations, and does not offer defences or mitigations. It is also not for anything that is not a scorer: a function that turns text into a number is the only shape it understands.
