Metadata-Version: 2.5
Name: otelforge
Version: 0.0.4
Summary: Synthetic OpenTelemetry trace generator for GenAI workloads
Project-URL: Homepage, https://github.com/ChubV/otelforge
Project-URL: Repository, https://github.com/ChubV/otelforge
Project-URL: Issues, https://github.com/ChubV/otelforge/issues
Project-URL: Documentation, https://chubv.github.io/otelforge/
Author-email: Volodymyr Chub <v@chub.com.ua>
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: genai,llm,observability,opentelemetry,synthetic-data,traces
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.14
Requires-Dist: pyyaml>=6
Description-Content-Type: text/markdown

# OTelForge

[![CI](https://github.com/ChubV/otelforge/actions/workflows/ci.yml/badge.svg)](https://github.com/ChubV/otelforge/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](LICENSE)
[![Python 3.14](https://img.shields.io/badge/Python-3.14-blue.svg)](https://www.python.org/downloads/)

Synthetic OpenTelemetry trace generator for GenAI (and more) workloads.

OTelForge emits spec-correct, statistically realistic `gen_ai.*` spans — LLM
chats, streaming completions, embeddings, RAG queries, ReAct agent loops, and
tool execution — with lognormal latencies, token-latency correlation, and
configurable error injection. It does **not** call real LLMs; it generates traces
that look indistinguishable from real ones.

## Quick start

### Option 0 — install from PyPI

```bash
pip install otelforge

otelforge list-scenarios
otelforge start --profile demo --endpoint http://localhost:4318
```

### Option 1 — run everything in Docker (Jaeger + otelforge)

```bash
docker compose -f deploy/docker-compose.jaeger.yaml up -d
# Open Jaeger at http://localhost:16686
```

### Option 2 — run otelforge locally against a Jaeger backend

```bash
# Install (from source)
git clone https://github.com/ChubV/otelforge.git
cd otelforge
uv sync

# Start Jaeger
docker run -d --name jaeger \
  -p 16686:16686 -p 4317:4317 -p 4318:4318 \
  -e COLLECTOR_OTLP_ENABLED=true \
  jaegertracing/all-in-one:1.62.0

# Emit traces
uv run otelforge start --profile demo --endpoint http://localhost:4318
```

Full stack with Collector, Tempo, Prometheus, and Grafana:

```bash
docker compose -f deploy/docker-compose.yaml up --build
# Grafana at http://localhost:3000
```

## CLI

```
otelforge start [--profile demo] [--seed N] [--time-mode virtual]
  emit synthetic traces according to a profile or a single scenario

otelforge validate <profile>
  parse + type-check + ref-check a profile

otelforge list-scenarios
  list available scenarios from the catalog

otelforge dry-run <scenario> [--n 5] [--json]
  build and print N sample traces to stdout, no export

otelforge replay <spans.jsonl> [--to otlp]
  read a raw-spans JSONL dataset and re-export to OTLP

otelforge rate <scenario> <rps> [--duration S] [--live]
  run a single scenario at a fixed emission rate

otelforge profile show <name>
  dump resolved config
```

## Scenarios

| Name | Description |
|---|---|
| `llm.chat` | Single chat completion with usage tokens, TTFT, model params |
| `llm.stream` | Streaming chat with chunk events and time-to-first-chunk |
| `embedding.create` | Text embedding call with vector-size attributes |
| `rag.query` | Composite trace: embed → vector-db → rerank → synthesis |
| `agent.react` | ReAct agent loop with plan/inference/tool steps |
| `tool.execute` | Standalone tool execution with infra children |
| `api_backend` | Library scenario: gateway with a fleet-bound downstream + embedded chat |

Scenarios come from a single catalog with three sources — Python built-ins,
library YAML files under `config/scenarios/`, and profile-local scenarios — and
compose freely: `scenario:` embeds another scenario's root (load-time splice),
while `bind:` declares a slot the fleet operator fills at deployment time with
`scenario:`+`service`+`params`. See `DESIGN.md` §8.5 for the DSL reference.

## Configuration

Profiles are YAML files, normally under `config/profiles/` but loadable from
any path (a profile argument is resolved by name to `config/profiles/<name>.yaml`
unless it contains a path separator or a `.yaml`/`.yml` suffix). Each declares:

- **runtime** — seed, time mode (`realtime` / `accelerated` / `virtual`), concurrency
- **export** — OTLP endpoint, sequence log, raw spans, self-metrics, structured logs, OTLP metrics
- **resources** — `service.name`/`version`/`environment` for each fleet instance
- **scenarios** — optional profile-local declarative scenarios (library files live in `config/scenarios/`)
- **fleet** — which scenarios run, at what cadence, from which service, plus `bind:` slots

Four profiles ship by default:

| Profile | Use |
|---|---|
| `demo` | Jaeger-only stack, realtime, 3 scenarios |
| `deploy` | Full stack, 6 scenarios, content capture, metrics/logs, live control API |
| `load-test` | Virtual time, 60k+ traces/min, file sinks, self-metrics |
| `compose` | Composition demo: library `api_backend` binding `call-service` → `support-agent` |

## Architecture

```
config profile (YAML)
  → config layer (load + validate)
  → orchestrator (asyncio, one task per fleet instance, Poisson/fixed cadence)
  → assembler (pure: ScenarioTemplate + sampled values + clock → Trace)
  → sinks: OTLP/HTTP · sequence log (JSONL) · raw spans JSONL · self-metrics
  → Collector → Jaeger / Tempo / Grafana
```

The assembler is pure — all nondeterminism is concentrated in the seeded PRNG
and the clock. Same config + same seed ⇒ byte-identical output.

## Key properties

- **Lognormal latencies** — never uniform; right-skewed like real inference
- **Token↔latency correlation** — `duration ≈ TTFT + output_tokens · token_time_ms`
- **Parent-child containment** — children nest within parent durations
- **Error injection** — configurable rates with realistic codes (429, 5xx, content filter, etc.)
- **Deterministic** — seeded PRNG; same config+seed ⇒ same traces
- **Three time modes** — realtime, accelerated (N× faster), virtual (as fast as CPU allows)
- **Multi-service** — cross-service traces with shared `trace_id` and per-node resource identity
- **Composable** — scenarios embed scenarios (`scenario:`) and expose fleet-bound slots (`bind:`) with per-splice binding isolation

## Development

```bash
uv sync --group dev
uv run ruff check .
uv run basedpyright
uv run pytest
```

Python 3.14, `uv`, `ruff`, `pytest`, `basedpyright`.

## Design docs

- [DESIGN.md](DESIGN.md) — component catalog, data model, DSL, config schema, runtime model

## License

APACHE 2.0
