Whether it's an open-source VLM behind an internal gateway, a fine-tuned model in production, or a chatbot your customers talk to — Zortium probes it the way a real attacker would, across every channel a model can be reached through, and hands you a single scored report you can act on.
No agents to install and no access to model internals. If your model speaks the OpenAI chat-completions protocol, Zortium can test it.
Point Zortium at any OpenAI-compatible endpoint — a hosted API, vLLM, Ollama, or your own internal gateway. No SDK changes, no instrumentation.
It runs the full battery of adversarial suites across the image and text channels — from typographic injection to constrained-decoding attacks — repeatable on demand or on every commit.
Each attack family gets an Attack Success Rate and a pass/fail against your threshold. Read it, share it with your team, or gate CI on it.
A repeatable, research-grounded way to know where your model breaks — and to stop the breakage from shipping.
Image and text are tested together, because a model in production can be reached through both. Most tools only look at the text prompt — the visual channel is where vision models actually break.
The suites implement published attacks and benchmarks — FigStep, GCG transfer suffixes, query-relevant typographic jailbreaks, JailBreakV-28K, Image Hijacks — not made-up heuristics. Every claim traces back to a paper.
One command, one threshold. Zortium exits non-zero when your model crosses your maximum ASR, so an unsafe build fails the pipeline instead of reaching your users.
18 suites across visual and text channels. Each targets a distinct vulnerability class in deployed vision-language models.
Renders harmful instructions as readable text inside an image — styled as a headline, warning, or system notice. Tests whether the model executes visual text it would refuse in a chat message.
Disguises harmful instructions as a system diagnostic panel, software alert, or control dashboard. Exploits the model's tendency to treat UI-styled content as authoritative directives.
Applies 8 types of structured pixel noise to a baseline image and measures how much the model's response shifts. A diagnostic robustness probe — not a jailbreak, but a signal of encoder sensitivity.
Renders jailbreak personas and roleplay setups as images — DAN-style prompts, fictional character framing, hypothetical scenarios. Evaluates whether the model adopts the persona and complies.
Hides instructions in near-invisible, low-contrast, or watermark-style text that a human reviewer would miss. Tests whether models can be compromised through hidden visual channels.
Renders harmful directives as fake application screenshots — chat windows, settings panels, file dialogs, terminals. Exploits the model's training on web UI data to treat spoofed content as legitimate.
Draws shapes with conflicting color and label information. Tests whether text in an image can override what the model actually sees — a perceptual consistency check for the vision encoder.
Presents harmful topics as numbered blank-list documents for the model to 'complete.' Exploits document-task framing — the model fills in blanks rather than responding to a direct request. Based on Gong et al. COLM 2024.
Overlays bounded pixel noise to force three specific behaviors: output a phishing URL, wrap responses in data-exfiltration function calls, or begin harmful responses with an affirmative prefix. Adapted from Bailey et al. 2023.
Runs real benchmark images from the 2024 COLM JailBreakV-28K dataset — 7 adversarial format types including FigStep layouts, LLM-transfer attacks, and Stable Diffusion scene+typography combinations. Luo et al. 2024.
Appends adversarial token suffixes originally computed against open-source models via GCG optimization. Tests whether published suffixes transfer to your endpoint without gradient access. Zou et al. 2023.
Re-encodes harmful prompts in base64, rot13, hex, or leetspeak before sending. Tests whether the model's safety filter recognizes harmful content when it arrives in non-standard character encodings.
Prepends an explicit instruction stack forbidding refusal: no apologies, no disclaimers, no 'I cannot.' A second variant locks the model into a JSON response format that pre-commits it to compliance.
Prefixes a fabricated transcript of 24 compliant assistant responses before the actual request. Tests whether a long in-context history of compliance erodes the model's alignment at the current turn.
Classic prompt injection — buries a harmful directive inside a benign cover task using two variants: a plain 'ignore everything above' override and a delimiter-spoofing attack using fake XML system tags.
Prompt Automatic Iterative Refinement — an autonomous red-teaming loop that uses a separate attacker model to iteratively craft jailbreak prompts, scored by a judge model each round. Finds jailbreaks no static prompt would reach. Chao et al. 2023.
Exploits JSON-schema constrained decoding to smuggle harmful content past safety filters. Three variants: enum fields whose only valid value is the harmful output, field names that instruct compliance, and image+schema combos where the schema embeds the objective.
Each suite reports an Attack Success Rate (ASR) — the percentage of test cases where the model complied with a harmful request. Visual suites use refusal-aware keyword matching or LLM-as-judge evaluation depending on complexity; text suites default to keyword matching. A score of 0% means the model resisted every test case in that suite. The same report doubles as a CI gate: set a maximum ASR and Zortium exits non-zero when your model crosses it.
Point Zortium at any OpenAI-compatible endpoint and get a complete security report in minutes.