Metadata-Version: 2.5
Name: infinity-observability-sdk
Version: 0.7.0
Summary: Unified observability SDK for Infinity Constellation services — tracing, structured logging, error tracking.
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: opentelemetry-api>=1.39.1
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.39.1
Requires-Dist: opentelemetry-sdk>=1.39.1
Requires-Dist: structlog>=25.1.0
Provides-Extra: django
Requires-Dist: opentelemetry-instrumentation-django>=0.48b0; extra == 'django'
Provides-Extra: logfire
Requires-Dist: logfire>=4.18.0; extra == 'logfire'
Provides-Extra: sentry
Requires-Dist: sentry-sdk>=2.0.0; extra == 'sentry'
Description-Content-Type: text/markdown

# Infinity Observability SDK

[![CI](https://github.com/infinity-constellation/infinity-observability-sdk/actions/workflows/ci.yml/badge.svg)](https://github.com/infinity-constellation/infinity-observability-sdk/actions/workflows/ci.yml)
[![PyPI version](https://img.shields.io/pypi/v/infinity-observability-sdk)](https://pypi.org/project/infinity-observability-sdk/)
[![Python](https://img.shields.io/pypi/pyversions/infinity-observability-sdk)](https://pypi.org/project/infinity-observability-sdk/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![basedpyright](https://img.shields.io/badge/type_checker-basedpyright-blue)](https://github.com/DetachHead/basedpyright)

Unified observability for Infinity Constellation services — OTel tracing, structured logging, and optional error tracking in a single `configure_observability()` call.

## Installation

```bash
pip install infinity-observability-sdk
```

Optional extras:

```bash
pip install infinity-observability-sdk[sentry]    # Sentry error tracking
pip install infinity-observability-sdk[logfire]   # pydantic-ai / httpx auto-instrumentation
pip install infinity-observability-sdk[django]    # Django OTel auto-instrumentation
```

## Quick start

```python
from infinity_observability_sdk import (
    ObservabilityConfig,
    configure_observability,
    get_logger,
)

configure_observability(ObservabilityConfig(
    business_unit_name="my_bu",
    service_name="my_service",
))

logger = get_logger(__name__)
logger.info("service.started", port=8000)
```

All environment variables are optional — the endpoint is auto-detected
(in-cluster collector DNS, else `http://localhost:4318`) when unset:

| Variable | Description |
|---|---|
| `INFINITY_OBSERVABILITY_ENDPOINT` | OTLP collector endpoint (e.g. `https://collector.example.com`). Auto-detected when unset. |
| `INFINITY_OBSERVABILITY_API_KEY` | API key for collector authentication. Required only when the endpoint is set explicitly. |
| `ENVIRONMENT` | Deployment environment — controls log format. `development` → human-readable console output; anything else → JSON. Defaults to `production`. |

## Configuration

```python
ObservabilityConfig(
    business_unit_name="my_bu",       # required
    service_name="my_service",        # required

    # Integrations
    instrument_pydantic_ai=True,      # default: True
    instrument_httpx=True,            # default: True
    sentry_dsn=None,                  # optional — enables Sentry if set
    sentry_environment=None,          # falls back to ENVIRONMENT env var
    sentry_traces_sample_rate=0.0,    # 0.0–1.0, default: 0.0

    # Metrics
    enable_metrics=True,              # default: True — installs a MeterProvider
    metric_export_interval_millis=60000,

    # PII — opt-in, off by default (SOC2)
    sentry_send_default_pii=False,    # request/user PII in Sentry events
    httpx_capture_all=False,          # full httpx headers/bodies on spans

    # Logging
    log_level="INFO",                 # DEBUG | INFO | WARNING | ERROR | CRITICAL

    # Advanced
    additional_resource_attributes={},  # extra OTel resource attributes
)
```

`configure_observability()` is idempotent — safe to call multiple times; subsequent calls are no-ops.

## Metrics

`configure_observability()` installs an OTel `MeterProvider` with an OTLP HTTP
metrics exporter using the same endpoint and API key as traces. Without it every
instrument resolves against the no-op global meter and measurements are dropped.

```python
from infinity_observability_sdk import AGENT_RUNS_STARTED_TOTAL, get_meter

runs_started = get_meter(__name__).create_counter(AGENT_RUNS_STARTED_TOTAL)
runs_started.add(1, {"backend": "devin"})
```

Use the exported constants rather than string literals so services cannot typo a
metric name apart:

| Constant | Metric |
|---|---|
| `AGENT_RUNS_STARTED_TOTAL` | `agent.runs.started_total` |
| `AGENT_RUNS_COMPLETED_TOTAL` | `agent.runs.completed_total` |
| `AGENT_RUNS_FAILED_TOTAL` | `agent.runs.failed_total` |
| `AGENT_RUN_WALL_TIME_SECONDS` | `agent.runs.wall_time_seconds` |
| `AGENT_RUN_COST_USD` | `agent.runs.cost_usd` |
| `AGENT_RUN_TOKENS_TOTAL` | `agent.runs.tokens_total` |
| `EVAL_CASES_PASSED_TOTAL` | `eval.cases_passed_total` |
| `EVAL_CASES_FAILED_TOTAL` | `eval.cases_failed_total` |
| `EVAL_SUITE_DURATION_SECONDS` | `eval.suite_duration_seconds` |
| `CONTEXT_PACK_FREEZE_DURATION_SECONDS` | `context.pack.freeze_duration_seconds` |
| `CONTEXT_PACK_FREEZE_BYTES` | `context.pack.freeze_bytes` |

`METRIC_NAMES` holds the full set. `configure_metrics()` can also be called
standalone — pass `metric_readers=[InMemoryMetricReader()]` to collect
measurements in tests instead of exporting them.

OTel allows a meter provider to be installed only once. If something else got
there first, `configure_metrics()` does not pretend to have won: it reuses an
already-installed OTel SDK `MeterProvider` (logging a warning, since the export
configuration is then the other caller's) and raises `RuntimeError` for any
other provider type. Pass `enable_metrics=False` when a service installs its own
provider on purpose.

## PII and redaction

PII capture is **opt-in**: `sentry_send_default_pii` and `httpx_capture_all`
both default to `False`. Enable them per service only where the data is needed
and permitted.

Everything the SDK emits is scrubbed by the redactor before it leaves the
process — log values (including tracebacks) and the span attributes set through
`agent_context()` / `agent_run_context()`:

- credential shapes anywhere inside a string: `devinkey_*`, `gh[pousr]_*`,
  `github_pat_*`, `AKIA*`/`ASIA*`, `xox[baprs]-*`, `sk-*`, JWTs,
  `?token=` / `?access_token=` query parameters, `Authorization:` headers;
- any key whose name reads as a credential (`password`, `api_key`,
  `authorization`, `client_secret`, `access_token`, …) has its whole value
  replaced. Markers name a credential outright, so token *counting* keys
  (`max_tokens`, `prompt_tokens`, `tokens_total`) are left alone; only an exact
  `token` / `key` / `auth` key is redacted on name.

Key names are matched after normalisation — lowercased, with `-`, `.` and spaces
folded to `_` — so header and attribute-path spellings (`X-Api-Key`, `api.key`,
`request.headers.authorization`) match the same marker as `api_key`. A key that
*ends* in a bare `token` / `key` / `auth` is redacted when it reads as a header
or a path: an `x` / `http` / `header(s)` / `request` / `response` leading segment
(`X_Auth`, `X_Token`, `X_Key`, `http_auth`) or any dotted/hyphenated spelling
(`headers.x_auth`, `request.auth`). Nested mappings are matched key by key, so
`headers={"X_Auth": …}` is scrubbed inside the object.

Inside a mapping that *holds* headers — a key ending in `header(s)`, e.g.
`headers`, `request_headers`, `http.headers` — the prefix requirement is
dropped, because header names are vendor- and caller-defined and cannot be
enumerated: `headers={"Internal_Auth": …}` and `{"Vendor_Token": …}` are
redacted. Header context propagates to nested mappings and does not leak to
siblings, so a `usage={…}` mapping alongside it is still judged by the ordinary
rules. Token metric fields survive in either context.

The header rule stops short of "ends in `key`": routing and identity fields
(`sort_key`, `cache_key`, `partition_key`, `idempotency_key`) keep their values,
as do the token *counting* keys (`max_tokens`, `prompt_tokens`, `token_count`,
`token_kind`). Credential-named `*_key` variants (`private_key`, `signing_key`,
`ssh_key`, `license_key`, `encryption_key`, …) are markers in their own right.

Cookies are treated more bluntly: a key that *is* a cookie container — `Cookie`,
`Set-Cookie`, `cookies`, `set_cookies`, or a path ending in one
(`request.headers.set-cookie`) — has **every** value beneath it replaced, and a
serialised `Cookie: …` / `Set-Cookie: …` line inside a string is truncated at the
value. Cookie *names* (`session`, `sid`, `csrftoken`, `JSESSIONID`) carry no
signal about secrecy, so none of them can be allowlisted; mapping keys and list
arity are kept so *which* cookies were present is still visible. Keys that merely
*describe* cookies keep their values (`cookie_count`, `cookie_size_bytes`,
`third_party_cookie_ratio`). Inside a cookie container a `max_tokens`-style field
would also be redacted — keep metrics out of cookie mappings.

Headers passed as a sequence of pairs (`headers=[("Cookie", …), ("Authorization",
…)]`, as `httpx`, WSGI and `urllib3` hand them over) are matched by pair name,
not scrubbed as anonymous values, so they behave exactly like the mapping form.
Raw byte pairs (`httpx.Headers.raw`, `[(b"cookie", b"session=…")]`) are decoded
for matching and come back as text, and so are byte *mapping* keys
(`dict(httpx.Headers.raw)`) — `str(b"cookie")` would hand the matchers
`"b'cookie'"`, which no rule can recognise. Elsewhere a `bytes` value is only
decoded to *inspect* it: it is replaced when it holds a secret shape and
otherwise returned as the original object, so binary payloads are not mangled
(on spans bytes are decoded regardless, since OTel attributes cannot carry them).

Nested mappings keep their structure in logs (a JSON object with scrubbed
leaves); on spans they are stringified, because OTel attributes admit no
mappings and would otherwise drop the value entirely.

Matches are replaced with `[REDACTED]` (exported as `REDACTED`). The helpers
`redact_text()` and `redact_attributes()` are exported for services that build
their own payloads. Redaction is best-effort defence in depth — do not
deliberately log secrets and rely on it.

## Logging contract

`get_logger()` is the **only** application-facing log API. The stdlib `logging`
bridge exists so third-party libraries (Django, uvicorn, httpx) land in the same
pipeline — not so application code can bypass `get_logger()`. Do not log via
`logfire.*`: that emits span events and never reaches stdout.

```python
from infinity_observability_sdk import get_logger

logger = get_logger(__name__)
logger.info("run.started", run_id=run.id, backend="devin")
```

Records go to **stdout, one JSON object per line** — the SDK ships no log
transport of its own; Vector tails stdout and forwards to VictoriaLogs. The
shape below is a versioned contract (`LOG_SCHEMA_VERSION`); the tests in
`tests/test_log_contract.py` fail if a processor change alters it.

`LOG_SCHEMA_VERSION` is deliberately **not** emitted on every record — it keeps
records lean, and consumers pin the SDK version instead. Bump it when the key
set or a value format changes incompatibly.

### Record schema

| Key | Always present | Format | Notes |
|---|---|---|---|
| `timestamp` | yes | ISO-8601 UTC, e.g. `2026-08-01T05:00:48.496049Z` | always UTC |
| `level` | yes | lowercase (`info`, `warning`, `error`, …) | |
| `event` | yes | short dotted string, e.g. `run.started` | first positional arg |
| `service` | yes | `<business_unit>.<service>` | set by the SDK |
| `logger` | yes | logger name, e.g. `gravity_api.modules.reports` | |
| `trace_id` | in-span only | 32 lowercase hex chars (`032x`) | Grafana derived field |
| `span_id` | in-span only | 16 lowercase hex chars (`016x`) | Grafana derived field |
| `exception` | on `.exception()` / `exc_info` | one string field holding the whole traceback | |
| `run_id`, `task_id`, `actor_id` | inside `agent_run_context()` | string | EvidenceBundle join keys |
| *your keys* | — | JSON scalars | snake_case |

Rules:

- Keys are `snake_case`; event names are dotted and lowercase.
- The keys above are **reserved** (`RESERVED_LOG_KEYS`) — the processor chain
  owns them and overwrites any application-supplied value of the same name.
- Tracebacks are collapsed into the single `exception` string, so a record never
  spans more than one line and Vector needs no multiline stitching.
- The trace/span ID hex widths are fixed; Grafana derived fields and LogsQL
  queries match on them.
- `ConsoleRenderer` is selected **only** when `ENVIRONMENT` is exactly
  `development` (case-insensitive). Every other value — unset, `staging`,
  `production`, or a typo — renders JSON.
- The traceback is rendered to text by the SDK in *every* environment, including
  development, so the redactor sees it. Development therefore prints plain
  tracebacks rather than rich/better-exceptions ones.

## Agent runs

Wrap an agent run to emit the standard `agent.run` span and bind run-scoped log
context:

```python
from infinity_observability_sdk import agent_run_context, get_logger

logger = get_logger(__name__)

with agent_run_context(
    run.id,
    task_id=task.id,
    actor_id=actor.id,
    backend=run.backend_type,
    systems_context_hash=snapshot.systems_context_hash,
    context_hash=pack.content_hash,
):
    logger.info("run.launched")   # carries run_id / task_id / actor_id
```

Span attributes: `agent.run_id`, `agent.task_id`, `agent.actor_id`,
`agent.backend`, `agent.systems_context_hash`, `agent.context_hash`,
`agent.prompt_template_version`, `env`. Optional identifiers are omitted rather
than emitted as `None`; extra keyword arguments become span attributes verbatim.
Pass `span_name=` for a more specific name from the span vocabulary (e.g.
`agent.run.launch`).

`run_id` / `task_id` / `actor_id` are also bound into the structlog contextvars
for the duration of the block, so every log line written inside the run — via
`get_logger()` or the stdlib bridge — carries them as join keys independent of
`trace_id`. Previous values are restored on exit.

## Agent context (legacy)

The AI-wattage span API, frozen for back-compat. New code should use
`agent_run_context()`.

Wrap AI agent runs to emit a structured span with cost and identity metadata:

```python
from infinity_observability_sdk import agent_context

with agent_context(
    agent_employee_equivalent="data_engineer",
    hourly_rate=75.0,
    agent_name="my_agent",
    task_description="summarise quarterly report",
    task_instance_identifier="run-abc-123",
    approximate_person_hours=2.0,
    business_unit_name="my_bu",
    service_name="my_service",
) as span:
    # your agent logic here
    ...
```

Extra keyword arguments are forwarded as span attributes.

## Django integration

Call `instrument_django()` **before** the ASGI/WSGI app is loaded — typically at the top of `core/asgi.py`:

```python
from infinity_observability_sdk import configure_observability, ObservabilityConfig, instrument_django

configure_observability(ObservabilityConfig(
    business_unit_name="my_bu",
    service_name="my_api",
))
instrument_django()
```

Requires `infinity-observability-sdk[django]`.

## Standalone logging

For services that only need structured logging without full OTel tracing:

```python
from infinity_observability_sdk import configure_logging, get_logger

configure_logging(service_name="my_bu.my_service", log_level="INFO")
logger = get_logger(__name__)
```

The emitted records follow the same [logging contract](#logging-contract).

## Development

```bash
uv sync --dev   # install all dependencies
make test       # run tests
make lint       # ruff lint
make ci         # full CI suite (lint, format check, tests, build, package check)
```

## Release cycle

Releases are **fully automated** via [python-semantic-release](https://python-semantic-release.readthedocs.io/) on every merge to `main`. No manual version bumps or GitHub Releases are needed.

### How it works

1. **Every merge to `main`** triggers the release workflow, which first runs the full CI suite (lint, format, tests, build).
2. `python-semantic-release` analyses commit messages since the last release using [Conventional Commits](https://www.conventionalcommits.org/) to determine whether a release is warranted and what the version bump should be.
3. If a release is warranted, it creates and pushes a `vX.Y.Z` tag. **Nothing is committed back to `main`** — the org ruleset requires every commit on `main` to arrive through a pull request, so the tag is the only thing the release job writes.
4. The same job builds the package (the version comes from the tag via `hatch-vcs`), publishes to PyPI via OIDC (no stored secrets), and creates the GitHub Release with semantic-release-generated notes.

Because the version lives in the git tag, `pyproject.toml` has no `version` field and `CHANGELOG.md` is frozen at v0.5.2 — release notes live on the [Releases](https://github.com/infinity-constellation/infinity-observability-sdk/releases) page.

### Version bump rules

| Commit type | Example | Bump |
|---|---|---|
| `fix:`, `perf:` | `fix(sdk): handle missing env var` | patch |
| `feat:` | `feat(sdk): add configure_metrics()` | minor |
| `feat!:` or `BREAKING CHANGE:` footer | `feat(sdk)!: remove logfire hard dep` | major |
| `docs:`, `chore:`, `refactor:`, etc. | `docs: update README` | none |

> While the version is `0.x`, breaking changes produce a **minor** bump rather than jumping to `1.0.0`.

### Commit message format

```
<type>(<scope>): <short summary>

[optional body]

[optional footer]
```

All commit messages merged to `main` should follow this format. PR titles are used as the squash-merge commit message and must follow the same convention.
