Metadata-Version: 2.4
Name: synthline
Version: 2.0.0
Summary: Constraint-driven synthetic data generation for text classification
Author: Abdelkarim El-Hajjami
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/abdelkarim-elhajjami/Synthline
Project-URL: Repository, https://github.com/abdelkarim-elhajjami/Synthline
Project-URL: Issues, https://github.com/abdelkarim-elhajjami/Synthline/issues
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: openai>=1.60.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: sentence-transformers>=2.2.0
Requires-Dist: torch>=2.0.0
Requires-Dist: transformers>=4.40.0
Requires-Dist: scikit-learn>=1.0.0
Requires-Dist: numpy>=1.21.0
Requires-Dist: PyYAML>=6.0.0
Requires-Dist: pydantic>=2.0.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: httpx>=0.24.0; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Provides-Extra: server
Requires-Dist: fastapi>=0.100.0; extra == "server"
Requires-Dist: uvicorn[standard]>=0.12.0; extra == "server"
Requires-Dist: websockets>=11.0; extra == "server"
Requires-Dist: pydantic-settings>=2.0.0; extra == "server"
Requires-Dist: python-multipart>=0.0.9; extra == "server"
Requires-Dist: httpx>=0.24.0; extra == "server"
Dynamic: license-file

<p align="center">
  <img src="https://raw.githubusercontent.com/abdelkarim-elhajjami/Synthline/v2.0.0/docs/header.svg" alt="Synthline" width="900"/>
</p>

<p align="center">
  <a href="https://github.com/abdelkarim-elhajjami/Synthline/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-Apache%202.0-00386E" alt="License"></a>
  <a href="https://pypi.org/project/synthline/"><img src="https://img.shields.io/pypi/v/synthline?color=00386E" alt="PyPI"></a>
  <a href="https://pypi.org/project/synthline/"><img src="https://img.shields.io/pypi/pyversions/synthline?color=00386E" alt="Python"></a>
</p>

Synthline is a constraint-driven approach to synthetic data generation for text classification.
A domain expert encodes the rules of a domain once, in a feature model (FM).
For each classification task, the user chooses the values to cover, and Synthline generates labeled texts with an LLM.

What Synthline guarantees, and what it does not:

- **Requests are structurally valid by construction.** The resolver derives the valid configurations from the FM and the user's coverage selection. Only valid configurations, each paired with a label, become generation requests and reach the LLM. The guarantee holds for the rules encoded in the FM and applies to the requests, not to the texts.
- **Alignment is verified, not guaranteed.** The Alignment Verifier (AV), optional, verifies with an NLI model that each text expresses its request. Rejected texts are regenerated from the same request, within a retry budget.
- **Diversity is encouraged, not controlled.** Multi-sample prompting requests several texts in one call.
- **PACE is optional.** It rewrites each prompt before generation with an actor-critic loop and keeps the candidate with the highest score Score(B) = α·Align(B) + (1 − α)·Div(B) on temporary texts.

A web interface is available on [Hugging Face Spaces](https://huggingface.co/spaces/abdelkarim-elhajjami/synthline).

---

## The process

Synthline separates the specification of what the generated data must satisfy from the generation itself.

**Phase 1: Domain Constraint Specification.**
A domain expert encodes the domain constraints into an FM in the FeatureIDE format: the attributes of the target data type, their values, and the rules that govern which combinations are admissible (groups and cross-tree constraints).
A contextual glossary, optional, maps feature values to their definitions.
The specification can be reused across the classification tasks of the domain.

**Phase 2: Constrained Generation.**
For each task, the user supplies the labels and their definitions, the coverage selection, and the target number of texts.
The pipeline has five steps:

1. **Resolve configurations.** The resolver expands the coverage selection, which may hold several values per group, into the valid configurations of the FM. OR groups are expanded in one of two modes, split or combine.
2. **Instantiate prompts.** Promptline pairs the label with each valid configuration into a generation request c = (y, v) and writes its initial prompt: the label and its definition, the configuration values, and their glossary definitions.
3. **Optimize prompts** (PACE, optional). PACE returns an optimized prompt for each request.
4. **Generate texts.** The Generator schedules the LLM calls needed to reach the target number of texts N_y of the label, with n texts per call (samples per prompt).
5. **Verify alignment** (AV, optional). The AV accepts a text if its entailment score reaches the threshold τ; rejected texts are regenerated from the same request, up to the retry budget r = 3.

<p align="center">
  <img src="https://raw.githubusercontent.com/abdelkarim-elhajjami/Synthline/v2.0.0/docs/process.svg" alt="The two-phase Synthline process" width="800"/>
</p>

<p align="center">
  <img src="https://raw.githubusercontent.com/abdelkarim-elhajjami/Synthline/v2.0.0/docs/pipeline.svg" alt="Constrained Generation: inputs, steps, and the artifacts passed between them" width="800"/>
</p>

---

## Installation

```bash
pip install synthline
```

From source:

```bash
git clone https://github.com/abdelkarim-elhajjami/Synthline.git
cd Synthline
pip install -e .
```

## Quick Start

The coverage selection lists, for each FM feature, the values to cover; several values per group are allowed.
The label and its definition enter the prompts; `samples_per_prompt` is the number n of texts requested in one call, and `target_texts` the target number of texts N_y for the label.

```python
from synthline import Synthline

sl = Synthline(
    fm="path/to/fm.xml",
    llm="openrouter/meta-llama/llama-3.3-70b-instruct",
    glossary="path/to/glossary.yaml",  # optional contextual glossary
)

# 1. Resolve configurations and instantiate the initial prompts (no LLM call)
prompts = sl.instantiate_prompts(
    label="Security",
    label_definition="Degree to which a product protects information and data.",
    samples_per_prompt=50,
    coverage_selection={
        "RequirementType": ["Quality"],
        "Domain": ["Healthcare Information System"],
        "AbstractionLevel": ["HighLevel", "DetailedLevel"],
        "DescriptionType": ["ProseNL"],
        "Context": ["Usage", "ITSystem"],
        "Language": ["EN"],
    },
)

# 2. Generate texts
output = await sl.generate_texts(prompts=prompts, target_texts=1000)

# 3. Export
output.save("output/")       # data.csv, metadata.json
df = output.to_dataframe()   # pandas DataFrame
```

### Optimize prompts with PACE (optional)

`alpha` is the weight α of alignment in the PACE objective; `iterations` the PACE iterations; `actor_critic_calls` the Actor and Critic calls per iteration; `candidates` the candidates per iteration.

```python
optimized = await sl.optimize_prompts(prompts, alpha=0.5, iterations=1, actor_critic_calls=4, candidates=2)
output = await sl.generate_texts(prompts=optimized, target_texts=1000)
```

### Verify alignment with the AV (optional)

`verify_threshold` is the threshold τ; the retry budget r is 3.

```python
output = await sl.generate_texts(prompts=prompts, target_texts=1000, verify=True, verify_threshold=0.6)
```

All SDK methods are async except `instantiate_prompts` (no LLM call).

---

## CLI

```bash
# Check a feature model
synthline check --fm fm.xml

# Resolve configurations and instantiate the initial prompts
synthline instantiate-prompts --fm fm.xml --label Security --label-definition "..." --coverage-selection coverage.yaml

# Optimize prompts with PACE
synthline optimize-prompts --fm fm.xml --llm openrouter/... --label Security --coverage-selection coverage.yaml --output optimized/

# Generate texts and verify alignment
synthline generate-texts --fm fm.xml --llm openrouter/... --target-texts 1000 --verify --output out/

# Generate texts from a file of run parameters
synthline generate-texts --parameters run.yaml --output out/
```

In `run.yaml`, each key is an option name written with underscores, such as `coverage_selection` or `target_texts`; an option given on the command line overrides the value in the file.

---

## LLM Providers

| Provider    | Prefix            | Environment variable |
| ----------- | ----------------- | -------------------- |
| OpenAI      | `openai/...`      | `OPENAI_API_KEY`     |
| OpenRouter  | `openrouter/...`  | `OPENROUTER_API_KEY` |
| Ollama      | `ollama/...`      | `OLLAMA_BASE_URL` (local) |

Keys can also be passed directly via `api_keys={"openrouter": "sk-or-..."}`.
For Ollama, set `OLLAMA_BASE_URL` to the server root, such as
`http://localhost:11434`; Synthline automatically uses its OpenAI-compatible `/v1` API.

> **Reasoning models are not supported.** Synthline is designed for predictable,
> high-throughput synthetic data sampling. Reasoning models are generally slower and costlier,
> and many reject the `temperature` and `top_p` controls Synthline uses. Select a standard chat
> or instruct model; known reasoning families and explicit reasoning options fail immediately
> with an actionable error.

> **Required:** The selected LLM must support **strict structured outputs with JSON Schema**
> (`response_format.type = "json_schema"`). JSON mode alone is not sufficient. Synthline does
> not fall back to plaintext or suppress provider/schema errors. Choosing a compatible model is
> the user's responsibility.

Alignment verification also fails loudly if its NLI model cannot run. Infrastructure failures
are never treated as rejected texts.

Compatibility is determined by the **model + provider + endpoint** combination. The same model
can support structured outputs through one serving stack and not through another.

The web interface lists only OpenRouter models that advertise strict structured outputs and every
standard sampling parameter Synthline sends, and removes known reasoning-model families.
Generation requests require OpenRouter to route only through providers that support the
requested parameters. OpenAI and Ollama catalogs do not expose an equally reliable per-model
capability flag through the endpoints Synthline uses, so check those models against the provider
documentation before selecting them:

- [OpenRouter structured outputs](https://openrouter.ai/docs/guides/features/structured-outputs)
- [OpenAI structured outputs](https://platform.openai.com/docs/guides/structured-outputs)
- [Ollama structured outputs](https://docs.ollama.com/capabilities/structured-outputs)
- [Mistral structured outputs](https://docs.mistral.ai/studio-api/conversations/structured-output)

---

## Web interface

The web interface is available on [Hugging Face Spaces](https://huggingface.co/spaces/abdelkarim-elhajjami/synthline) or self-hosted with Docker.

```bash
git clone https://github.com/abdelkarim-elhajjami/Synthline.git && cd Synthline && ./dev.sh
```

---

## Project Structure

```
synthline/          SDK package (pip install synthline)
  core/             FM parser, resolver, Promptline, PACE, Generator, Alignment Scorer, Alignment Verifier
  utils/            Logger, parsing, progress tracking
  client.py         The Synthline client — instantiate_prompts(), optimize_prompts(), generate_texts(), verify_alignment()
  types.py          PromptSet, Dataset
  cli.py            CLI entry point
server/             FastAPI + WebSocket server for the web interface
tests/              Unit and integration tests
web/                Next.js web interface
docs/               Header and diagrams of the process
```

## Citation

Publications refer to the version of the code they used:

| Version | Publication |
| ------- | ----------- |
| [`v0.1.0`](https://github.com/abdelkarim-elhajjami/synthline/tree/v0.1.0) | El-Hajjami & Salinesi, RCIS 2026 ([doi](https://doi.org/10.1007/978-3-032-26836-5_20)) |
| [`v0.0.0`](https://github.com/abdelkarim-elhajjami/synthline/tree/v0.0.0) | El-Hajjami & Salinesi, RCIS 2025 ([doi](https://doi.org/10.1007/978-3-031-92474-3_13)) |

```bibtex
@inproceedings{elhajjami2026rcis,
  author    = {El-Hajjami, Abdelkarim and Salinesi, Camille},
  title     = {Multi-sample Prompting and Actor-Critic Prompt Optimization for Diverse Synthetic Data Generation},
  booktitle = {Research Challenges in Information Science (RCIS 2026)},
  series    = {Lecture Notes in Business Information Processing},
  volume    = {585},
  pages     = {322--337},
  publisher = {Springer},
  year      = {2026},
  doi       = {10.1007/978-3-032-26836-5_20}
}

@inproceedings{elhajjami2025rcis,
  author    = {El-Hajjami, Abdelkarim and Salinesi, Camille},
  title     = {Synthline: A Product Line Approach for Synthetic Requirements Engineering Data Generation Using Large Language Models},
  booktitle = {Research Challenges in Information Science (RCIS 2025)},
  series    = {Lecture Notes in Business Information Processing},
  volume    = {547},
  pages     = {208--225},
  publisher = {Springer},
  year      = {2025},
  doi       = {10.1007/978-3-031-92474-3_13}
}
```

## License

[Apache License 2.0](https://github.com/abdelkarim-elhajjami/Synthline/blob/main/LICENSE)
