Metadata-Version: 2.4
Name: annotations4all
Version: 0.2.0
Summary: Schema-first Python library for LLM-assisted span annotation
Keywords: ner,llm,nlp,span-annotation,tagger
Author: Nicole Dresselhaus
Author-email: Nicole Dresselhaus <nicole.dresselhaus@hu-berlin.de>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing
Classifier: Topic :: Text Processing :: Linguistic
Requires-Dist: fuzzysearch>=0.7.3,<1.0.0
Requires-Dist: openai>=2.31.0
Requires-Dist: typing-extensions>=4.0.0,<5.0.0
Requires-Dist: ollama>=0.4.7,<1.0.0 ; extra == 'ollama'
Requires-Python: >=3.11, <4.0
Project-URL: Repository, https://scm.cms.hu-berlin.de/annotations4all/annotations4all
Project-URL: Issues, https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/issues
Provides-Extra: ollama
Description-Content-Type: text/markdown

# annotations4all

**Languages:** [Deutsch](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/README.de.md) · [English](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/README.en.md)

`annotations4all` is a schema-first Python library for LLM-assisted span annotation (commonly used for named-entity recognition). It generates prompts from a user-defined tag schema, parses annotated LLM responses in `<<TAG>>…</TAG>>` format, and returns matches as offset-based spans.

> Status: `0.2.0` is intended as an alpha release. The stable v0.2 surface consists of prompt taggers, merge helpers, parsers, and client helpers. Experimental legacy clients are marked as such. DOI of this version: `10.5281/zenodo.22790107`.

![annotations* cosmos — libraries, data flow and evaluation](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/raw/main/docs/diagrams/annotations-kosmos.en.png)

Overview of the `annotations-*` library family — raw data and gold annotations,
annotating, running experiments, evaluating, reporting:

## Documentation

The [tutorial](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/docs/tutorial.md) documents the v0.2 API surface and shows examples against local, OpenAI-compatible installations (e.g. a llama.cpp server or Ollama's OpenAI-compatible endpoint).

## Installation

After the PyPI release:

```bash
python -m pip install annotations4all
```

For development, from the repository:

```bash
python -m venv .venv
. .venv/bin/activate
python -m pip install -e .
python -m pytest
```

## Schema-first quick start

The following example shows the preferred v0.2 entry point: define the tag schema first, then generate prompt messages and map the model response back to spans.

```python
from annotations4all import ConfigurableTagger

text = "Max Mustermann lives in Berlin."
tagger = ConfigurableTagger(
    tags=[("PER", "Person names"), ("LOC", "Places")],
    context="Short modern example sentence.",
    language="en",
)

messages = tagger.get_prompt(text)
for message in messages:
    print(message["role"])
    print(message["content"])

# Response of an LLM, e.g. from an OpenAI-compatible endpoint:
response = "<<PER>>Max Mustermann</PER>> lives in <<LOC>>Berlin</LOC>>."
spans = tagger.parse_response(response, text)
print(spans)
```

`tags` describes the tag schema of the concrete workflow. `context` holds material- or task-specific annotation hints, not arbitrary runtime data. Common NER tags such as `PER`, `LOC`, and `ORG` are well suited for quick starts, but the library does not enforce a fixed ontology.

## Minimal example without an LLM call

The following example shows the smallest stable core: an already annotated response is mapped back to character positions in the original text.

```python
from annotations4all import parse_region_response, parse_region_response_detailed

text = "Max Mustermann lives in Berlin."
response = "<<PER>>Max Mustermann</PER>> lives in <<LOC>>Berlin</LOC>>."

spans = parse_region_response(response, text)
detailed = parse_region_response_detailed(response, text)
print(spans)
print(detailed.warnings)
# [{'label': 'PER', 'meta': None, 'start': 0, 'end': 14}, {'label': 'LOC', 'meta': None, 'start': 24, 'end': 30}]
```

## Writing custom taggers

For advanced use cases, custom taggers can be implemented. The base class `ChatTagger` remains importable for this purpose but is not part of the highlighted package-root API of v0.2.

```python
from annotations4all.taggers.base import ChatTagger, Message
from annotations4all.utils.response_parser import parse_region_response


class MyTagger(ChatTagger):
    def name(self) -> str:
        return "my-tagger"

    def get_prompt(self, text: str) -> list[Message]:
        return [
            {
                "role": "system",
                "content": "Annotate persons as <<PER>>…</PER>> and places as <<LOC>>…</LOC>>.",
            },
            {"role": "user", "content": text},
        ]

    def parse_response(self, response: str, text: str, logfile=None):
        return parse_region_response(response, text, logfile=logfile)
```

The model response must reproduce the original text as exactly as possible and mark spans with opening and closing tags:

```text
<<PER>>Max Mustermann</PER>> lives in <<LOC>>Berlin</LOC>>.
```

## Tag schemas

The library does not enforce a fixed ontology. The tag schema belongs to the respective research or annotation workflow. Common quick-start tags include:

- `PER`: persons
- `LOC`: places
- `ORG`: organizations
- project-specific tags such as `DOM`, `DAT`, `KG`, etc.

Tags can optionally carry metadata, e.g. `<<LOC:city>>Berlin</LOC>>`. The parser returns this metadata as `meta` when present. The returned objects are spans with at least `label`, `start`, and `end`.

## Hierarchical tagging in one pass

Nested answers are parsed directly — the parser is a stack decoder, not a flat one:

```python
from annotations4all import ConfigurableTagger

text = "Ernst August von Testingen"
tagger = ConfigurableTagger(
    tags=[("PER", "person names (outer span)"), ("FN", "given name"), ("LN", "family name")],
    context="PER encloses the whole name mention; FN and LN are nested inside it.",
    language="en",
)
response = "<<PER>><<FN>>Ernst August</FN>> <<LN>>von Testingen</LN>></PER>>"
print(tagger.parse_response(response, text))
# [{'label': 'PER', 'meta': None, 'start': 0, 'end': 26},
#  {'label': 'FN', 'meta': None, 'start': 0, 'end': 12},
#  {'label': 'LN', 'meta': None, 'start': 13, 'end': 26}]
```

- The prompt decides whether the model nests: the packaged templates allow nesting explicitly and forbid crossing tags, and the intended structure belongs in `context` as well.
- Children are always searched **inside** their parent span, and repeated text is anchored via the tag position in the response (no silent first-match drift). A child that cannot be placed inside its parent is reported as `nested-out-of-parent`: `nested_policy="strict"` (default) drops it, `"repair"` keeps it at the anchor that was found.
- A closing tag written one `>` short (`</TAG>` instead of `</TAG>>`) is accepted and reported as `close-tag-single-bracket`; `tolerate_single_bracket_close=False` rejects it.
- Overlap is allowed as long as no span encloses the other — two flat spans such as `<<GIV>>…</GIV>> <<RCV>>…</RCV>>` are both parsed.
- **Crossing** tags (`<<GIV>>…<<RCV>>…</GIV>>…</RCV>>`) cannot be represented in the `<<TAG>>` format: the parser keeps the span that closes first, drops the other one and reports `unclosed-nested-tag` / `unmatched-close-tag`. Use the per-label mode for those (see below) — it keeps both spans and reports the `crossing` conflict.
- For long outer spans raise `max_match_length` (default 100); otherwise they are truncated (`span-too-long`).

## Nested fallback: per-label mode + merge

For nested or hierarchical schemas a single, nested model answer can be too unreliable (see the section above). The fallback annotates **one label (or one label group) per model call** and merges the flat runs deterministically afterwards:

```python
from annotations4all import OpenAICompatClient, PerLabelTagger

text = "Anna Müller lives in Berlin."
tagger = PerLabelTagger(
    tags=[("PER", "Person names"), ("LOC", "Places")],
    context="Short modern example sentence.",
    language="en",
)
client = OpenAICompatClient(api_url="https://example.invalid/v1", api_key="<token>")

result = tagger.annotate(text, client, model="my-model", stream=False)
print(result.flat_spans())
# [{'label': 'PER', 'meta': None, 'start': 0, 'end': 11}, {'label': 'LOC', 'meta': None, 'start': 21, 'end': 27}]
print(result.spans)  # nested: parent spans with `children`
print(result.conflicts)  # conflict list
```

The merge is also usable on its own — a pure function on flat spans (no client, no prompt):

```python
from annotations4all import merge_span_sets

result = merge_span_sets(
    {
        "PER": [{"label": "PER", "meta": None, "start": 0, "end": 11}],
        "LOC": [{"label": "LOC", "meta": None, "start": 21, "end": 27}],
    }
)
```

Merge rules:

- **Anchor rule:** the hierarchy is derived from containment only — a span's parent is the smallest span that strictly encloses it, never a similar span found elsewhere in the document. Bounds are half-open (`[start, end)`), so touching spans are adjacent, not overlapping.
- **Conflicts are reported, never silently repaired:** `duplicate` (same label, same bounds, twice), `intra-label-overlap` (same label, partial overlap) and `crossing` (different labels, partial overlap).
- **Order-independent:** permuting runs or spans yields identical output.

Cost: **one model call per run** — with one label per run that is ≈ the number of labels. For large hierarchical schemas the mode is therefore only sensible for the outer level; recursive prompting per detected element is the alternative. In addition, `parse_region_response` and `parse_region_response_detailed` accept `bounds=(start, end)` to anchor candidates only inside an already known parent span (no global fuzzy search).

## Backends and clients

For v0.2, only a narrow, explicit OpenAI-compatible chat-completions interface is officially supported. The target server must offer an endpoint such as `/v1/chat/completions` and be compatible with the request/response shape of the README examples.

```python
from annotations4all import ConfigurableTagger, OpenAICompatClient

text = "Max Mustermann lives in Berlin."
tagger = ConfigurableTagger(
    tags=[("PER", "Person names"), ("LOC", "Places")],
    context="Short modern example sentence.",
    language="en",
)
client = OpenAICompatClient(
    api_url="https://example.invalid/v1",
    api_key="<token>",
)

chunks = client.chat.create(
    model="my-model",
    messages=tagger.get_prompt(text),
    stream=True,
    temperature=0,
)
response = "".join(chunk.answer for chunk in chunks)
spans = tagger.parse_response(response, text)
print(spans)
```

`OpenAICompatClient` reads API keys either from `api_key=` or from an environment variable. The default is `OPENAI_API_KEY`; `api_key_env=` selects a different name. Local servers that do not require authentication automatically receive a dummy key, because the underlying OpenAI SDK still expects a value.

Provider-specific request fields (e.g. reasoning options) are passed through generically via `extra_body=` — the library does not interpret the payload, the endpoint defines the schema:

```python
client.chat.create(
    model="my-model",
    messages=tagger.get_prompt(text),
    extra_body={"reasoning": {"enabled": False}},
)
```

The v0.2 compatibility promise is deliberately narrow: standard content responses and the streaming form used in the tests/examples are the target. The semantics of provider-specific fields are explicitly not a stability promise — the generic `extra_body` passthrough itself is.

## Tests

```bash
python -m pytest
```

The tests check prompt invariants, parser golden cases, a small fuzz baseline, client helper structures, the merge rules (order independence, conflict classes) and the per-label mode. A live smoke test of the per-label mode against an OpenAI-compatible endpoint lives in `tests/integration/test_endpoint_smoke.py` (env-driven; it is skipped without configuration).

## Citation

If you use this software in academic work, please cite it as follows:

> Dresselhaus, Nicole. (2026). *annotations4all* (Version 0.2.0) [Software]. Humboldt-Universität zu Berlin. <https://scm.cms.hu-berlin.de/annotations4all/annotations4all>

DOI: `10.5281/zenodo.22790107`

The concept DOI `10.5281/zenodo.22011370` always resolves to the latest version (0.1.1: `10.5281/zenodo.22011371`).

Machine-readable metadata is available in [`CITATION.cff`](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/CITATION.cff).

## License

This software is licensed under the MIT License. See [`LICENSE`](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/LICENSE) for details.
