# Promptbench

> The purpose of `prompt-bench` is to systematically evaluate and compare different prompt compositions across multiple LLM providers by running Cartesian products of templates, payloads, and skills. It enables developers and researchers to measure how variations in prompt structure affect output quality, consistency, and performance. The system supports recursive fusion, multi-provider execution, and structured artifact logging, making it ideal for benchmarking prompt engineering workflows. Additionally, the TUI enhances usability for interactive exploration and configuration.

**Remember:**
- Triple-Layer Recursive Fusion
- System Template (House)
- Influence Skill (Primary)
- Target Payload (Secondary)
- Composition Modes: Engine, Marker, Append
- Multi-Provider Support (LM Studio, Codex CLI, Gemini CLI)

## Docs
- [Oracle Actions Pack 2026 01 07](https://github.com/AcidicSoil/promptbench/blob/main/docs/oracle-actions-pack-2026-01-07.md): docs page.
- [Oracle Pack 2026 01 07](https://github.com/AcidicSoil/promptbench/blob/main/docs/oracle-pack-2026-01-07.md): docs page.
- [01 Contracts Interfaces Surface](https://github.com/AcidicSoil/promptbench/blob/main/docs/oracle-questions-2026-01-07/01-contracts-interfaces-surface.md): docs page.
- [02 Contracts Interfaces Integration](https://github.com/AcidicSoil/promptbench/blob/main/docs/oracle-questions-2026-01-07/02-contracts-interfaces-integration.md): docs page.
- [03 Invariants Domain](https://github.com/AcidicSoil/promptbench/blob/main/docs/oracle-questions-2026-01-07/03-invariants-domain.md): docs page.
- [04 Invariants Validation](https://github.com/AcidicSoil/promptbench/blob/main/docs/oracle-questions-2026-01-07/04-invariants-validation.md): docs page.
- [05 Caching State Layers](https://github.com/AcidicSoil/promptbench/blob/main/docs/oracle-questions-2026-01-07/05-caching-state-layers.md): docs page.
- [06 Caching State Consistency](https://github.com/AcidicSoil/promptbench/blob/main/docs/oracle-questions-2026-01-07/06-caching-state-consistency.md): docs page.
- [07 Background Jobs Discovery](https://github.com/AcidicSoil/promptbench/blob/main/docs/oracle-questions-2026-01-07/07-background-jobs-discovery.md): docs page.
- [08 Background Jobs Reliability](https://github.com/AcidicSoil/promptbench/blob/main/docs/oracle-questions-2026-01-07/08-background-jobs-reliability.md): docs page.

## Tutorials
- [Expected Output](https://github.com/AcidicSoil/promptbench/blob/main/examples/expected-output.md): worked example.
- [User Input](https://github.com/AcidicSoil/promptbench/blob/main/examples/user-input.md): worked example.
- [Example Prd](https://github.com/AcidicSoil/promptbench/blob/main/.taskmaster/templates/example_prd.txt): worked example.
- [Example Prd Rpg](https://github.com/AcidicSoil/promptbench/blob/main/.taskmaster/templates/example_prd_rpg.txt): worked example.
- [Composed System](https://github.com/AcidicSoil/promptbench/blob/main/benchmarks/demo/71d38850b64a/composed_system.txt): worked example.
- [Output](https://github.com/AcidicSoil/promptbench/blob/main/benchmarks/demo/71d38850b64a/output.txt): worked example.

## Concepts
- [Promptbench Mcp Server Design Implementation](https://github.com/AcidicSoil/promptbench/blob/main/tickets/mcp/promptbench_mcp_server_design_implementation.md): docs page.

## Optional
- [Agents](https://github.com/AcidicSoil/promptbench/blob/main/.taskmaster/AGENTS.md): docs page.
- [README](https://github.com/AcidicSoil/promptbench/blob/main/README.md): docs page.
- [Skill Context Router](https://github.com/AcidicSoil/promptbench/blob/main/scripts/skill_context_router.py): docs page.
- [When To Use This Skill](https://github.com/AcidicSoil/promptbench/blob/main/skills/when-to-use-this-skill.md): docs page.

## Payloads
- [Haiku](https://github.com/AcidicSoil/promptbench/blob/main/payloads/haiku.md): docs page.
- [Transcript Cleaner V2](https://github.com/AcidicSoil/promptbench/blob/main/payloads/transcript-cleaner_v2.md): docs page.

## Promptbench
- [Cli](https://github.com/AcidicSoil/promptbench/blob/main/promptbench/cli.py): docs page.
- [Init](https://github.com/AcidicSoil/promptbench/blob/main/promptbench/__init__.py): docs page.
- [Main](https://github.com/AcidicSoil/promptbench/blob/main/promptbench/__main__.py): docs page.
- [Base](https://github.com/AcidicSoil/promptbench/blob/main/promptbench/providers/base.py): docs page.
- [Codex Cli](https://github.com/AcidicSoil/promptbench/blob/main/promptbench/providers/codex_cli.py): docs page.
- [Compose](https://github.com/AcidicSoil/promptbench/blob/main/promptbench/core/compose.py): docs page.
- [Config](https://github.com/AcidicSoil/promptbench/blob/main/promptbench/core/config.py): docs page.
- [Constants](https://github.com/AcidicSoil/promptbench/blob/main/promptbench/core/constants.py): docs page.
- [Contract](https://github.com/AcidicSoil/promptbench/blob/main/promptbench/core/contract.py): docs page.
- [Discovery](https://github.com/AcidicSoil/promptbench/blob/main/promptbench/core/discovery.py): docs page.

## Templates
- [Creative Writer](https://github.com/AcidicSoil/promptbench/blob/main/templates/creative_writer.md): docs page.
- [Prompt Composition Engine](https://github.com/AcidicSoil/promptbench/blob/main/templates/prompt-composition-engine.md): docs page.
- [Prompt Composition Engine V3](https://github.com/AcidicSoil/promptbench/blob/main/templates/prompt-composition-engine_v3.md): docs page.

## Tests
- [Test Artifacts](https://github.com/AcidicSoil/promptbench/blob/main/tests/test_artifacts.py): docs page.
- [Test Artifacts Concurrency](https://github.com/AcidicSoil/promptbench/blob/main/tests/test_artifacts_concurrency.py): docs page.
- [Test Cli](https://github.com/AcidicSoil/promptbench/blob/main/tests/test_cli.py): docs page.
- [Test Cli Providers](https://github.com/AcidicSoil/promptbench/blob/main/tests/test_cli_providers.py): docs page.
- [Test Compose](https://github.com/AcidicSoil/promptbench/blob/main/tests/test_compose.py): docs page.
- [Test Config](https://github.com/AcidicSoil/promptbench/blob/main/tests/test_config.py): docs page.
- [Test Contract](https://github.com/AcidicSoil/promptbench/blob/main/tests/test_contract.py): docs page.
- [Test Core](https://github.com/AcidicSoil/promptbench/blob/main/tests/test_core.py): docs page.
- [Test Engine Composition](https://github.com/AcidicSoil/promptbench/blob/main/tests/test_engine_composition.py): docs page.
- [Test Eventing Executor](https://github.com/AcidicSoil/promptbench/blob/main/tests/test_eventing_executor.py): docs page.

## Tickets
- [Prompt Bench Pipeline Enhancement](https://github.com/AcidicSoil/promptbench/blob/main/tickets/Prompt-Bench Pipeline Enhancement.md): docs page.
- [Web Search For Promptbench Skills](https://github.com/AcidicSoil/promptbench/blob/main/tickets/Web-search for promptbench skills.md): docs page.
- [Expose Mcp Server For Promptbench](https://github.com/AcidicSoil/promptbench/blob/main/tickets/mcp/Expose MCP Server for Promptbench.md): docs page.
- [Expose Promptbench As Mcp](https://github.com/AcidicSoil/promptbench/blob/main/tickets/mcp/Expose promptbench as MCP.md): docs page.
- [Lm Studio Model Selection](https://github.com/AcidicSoil/promptbench/blob/main/tickets/lm-studio/LM-Studio-Model-Selection.md): docs page.
- [Lm Studio Sampling Params](https://github.com/AcidicSoil/promptbench/blob/main/tickets/lm-studio/LM Studio Sampling Params.md): docs page.
- [Lmstudioadapter Enhancement](https://github.com/AcidicSoil/promptbench/blob/main/tickets/lm-studio/LMStudioAdapter Enhancement.md): docs page.
- [Mcp Goal](https://github.com/AcidicSoil/promptbench/blob/main/tickets/mcp/mcp-goal.md): docs page.
- [Mcp Server Implementation](https://github.com/AcidicSoil/promptbench/blob/main/tickets/mcp/MCP server implementation.md): docs page.
- [Parent Ticket Summary](https://github.com/AcidicSoil/promptbench/blob/main/tickets/bugs/Parent Ticket Summary.md): docs page.
