Metadata-Version: 2.4
Name: haspi
Version: 1.0.0
Summary: Inverse soft-Q learning (IQ-Learn) reward extraction for hate-speech analysis on the One Million Posts Corpus
Author-email: Tobias Kietreiber <tobias.kietreiber@ustp.at>
License-Expression: MIT
Project-URL: Homepage, https://github.com/fhstp/haspi
Project-URL: Documentation, https://haspi.readthedocs.io
Project-URL: Repository, https://github.com/fhstp/haspi
Project-URL: Issues, https://github.com/fhstp/haspi/issues
Keywords: hate-speech,content-moderation,nlp,german,imitation-learning,iq-learn,reward-learning,explainability
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Natural Language :: German
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jax<0.8,>=0.7
Requires-Dist: flax<0.13,>=0.12
Requires-Dist: optax<0.3,>=0.2
Requires-Dist: transformers<4.40,>=4.38
Requires-Dist: safetensors
Requires-Dist: numpy
Requires-Dist: scikit-learn
Requires-Dist: tqdm
Provides-Extra: cuda12
Requires-Dist: jax[cuda12]<0.8,>=0.7; extra == "cuda12"
Provides-Extra: gemma
Requires-Dist: torch; extra == "gemma"
Provides-Extra: leolm
Requires-Dist: torch; extra == "leolm"
Requires-Dist: accelerate; extra == "leolm"
Requires-Dist: sentencepiece; extra == "leolm"
Provides-Extra: finetune
Requires-Dist: torch; extra == "finetune"
Requires-Dist: accelerate; extra == "finetune"
Requires-Dist: sentencepiece; extra == "finetune"
Requires-Dist: peft<0.14,>=0.10; extra == "finetune"
Provides-Extra: demo
Requires-Dist: gradio<6,>=4; extra == "demo"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=7; extra == "docs"
Requires-Dist: myst-parser; extra == "docs"
Requires-Dist: furo; extra == "docs"
Dynamic: license-file

<div align="center">
<img src="https://raw.githubusercontent.com/fhstp/HaSPI/main/images/Logo_with_background.svg" alt="logo"></img>
</div>

# HaSPI

**haspi** scores German comments for hate speech and moderation removal, and tells you
**which words drove each verdict** — an explainable reward built on the
[One Million Posts Corpus](https://ofai.github.io/million-post-corpus/) (DerStandard,
OFAI). Rooted in inverse soft-Q learning (IQ-Learn); JAX / flax-nnx with an optional
PyTorch encoder. 

This package is aimed at forum moderators who want to test our method as well as
researchers who want to reproduce or enhance our results.

```console
$ haspi-explain
> Du bist ein widerlicher Idiot und gehörst abgeschoben.
  → HATE   (score +1.06, threshold -0.02)
     toward HATE:     'bist'+0.39  'widerlicher'+0.22  'gehörst'+0.22  'Du'+0.17
     toward non-hate: 'abgeschoben.'-0.10
> Danke für den sehr informativen Artikel.
  → non-hate   (score -0.35, threshold -0.02)
     toward HATE:     'Danke'+0.15  'informativen'+0.11
     toward non-hate: 'den'-0.10  'für'-0.08  'sehr'-0.05
```

The current method — the **sequence-χ² reward** — runs a comment through a frozen German
language model (LeoLM-7b), mean-pools its hidden states, and applies a small linear χ²
reward head. Because the whole pipeline is linear, the score decomposes **exactly** into
per-word contributions, and scoring is one model pass plus a dot product.

| held-out AUROC | |
|---|---|
| hate-vs-neutral (10-fold CV) | **0.76** |
| moderation removal (OMP online/offline) | 0.75 frozen → **0.77** [LoRA fine-tuned](#fine-tune-the-encoder) |
| *external cross-check* — moderation removal on **RP-Mod** (a different corpus, same splits as Assenmacher et al.) | **0.796** (their fine-tuned BERT: 0.791) |
| *external cross-check* — **HateXplain** (English; hate+offensive vs normal) | **0.857** |

The first two rows are reproducible from this repo with the commands below. The last two are
**additional experiments on other corpora**: this package is about the One Million Posts
Corpus, so their data and scripts are not shipped here. HateXplain also ships *human
rationales*, which lets it check the per-word attributions themselves — those land near a
fine-tuned BERT's attention/LIME explanations while being faithful by construction. Both are
written up in the [external benchmarks guide](https://haspi.readthedocs.io/en/latest/guide/benchmarks.html).

**Full documentation & the method diagram:** https://haspi.readthedocs.io

## Install

```bash
pip install "haspi[leolm]"          # the current method: + torch, accelerate, sentencepiece
pip install "haspi[leolm,cuda12]"   # …with CUDA-12 JAX for GPU
```

Python ≥ 3.11 (set by `jax`/`flax`). The reward head is CPU/JAX; running the language model (feature extraction
and the explainer) uses a GPU when one is available and otherwise falls back to CPU — which
works but is much slower for the 7B model, so a GPU is recommended. Note the pinned
`transformers>=4.38,<4.40` — Hugging Face v5 dropped Flax support; the 4.39.x line is the
last that ships it.

The fitted reward files are distributed via [GitHub Releases](https://github.com/fhstp/haspi/releases),
not committed to the repo. Fetch them into `models/` before running the explainers or the demo
(or [fit your own](#fit-your-own-reward)):

```bash
gh release download --repo fhstp/haspi --pattern '*.pkl' --dir models/
```

## Try the demo

`haspi-demo` launches a small web app — the project schematic made real. Type or **sample** a German
comment and see the verdict, a green→red risk meter, and the comment with each word coloured by its
exact contribution (risky words highlighted). It handles hate and moderation (with an optional
article/thread context), and a seeded "🎲 Aus Korpus" button pulls a reproducible real corpus comment.

```bash
pip install "haspi[demo,leolm]"
haspi-demo                          # → http://<host>:8000   (needs a fitted reward; GPU recommended, CPU works)
```

## Classify and explain

The main tools are the two explainers. They load a fitted reward
(`models/leolm_chi2_reward.pkl`) and the language model, then score whatever you give them.

```bash
haspi-explain                 # interactive: type a comment, get a verdict + word drivers
haspi-explain-corpus --n 8    # sample labelled corpus posts, show prediction vs. truth
```

Useful flags: `--topk N` (words shown per direction), `--level {word,token}` (word-level by
default; `token` shows the raw sub-word pieces), `--reward PATH` (a different reward file).

### Reading the output

- **`score` vs `threshold`** — the verdict. Score above the calibrated threshold → `HATE`,
  below → `non-hate`. The threshold was chosen once on the training set.
- **`toward HATE` / `toward non-hate`** — the words that pushed the score up or down, each
  with its exact contribution. These are **real, additive attributions**: they sum to the
  decision (relative to an average comment), not a heuristic highlight. A neutral comment's
  words roughly cancel; a slur or aggressive phrasing spikes.

> **Note.** The attribution is faithful to the decision, but the model reasons at the
> sentence level — trust the **overall verdict** and the **ranking of content words**, not
> any single function word (you'll sometimes see `und`, `.`, etc. carry a little weight).

### In Python

```python
from haspi.sequence import LeoLMScorer, format_attribution

scorer = LeoLMScorer("models/leolm_chi2_reward.pkl")
res = scorer.score("Du bist ein Idiot.")
# res = {"score": ..., "label": "HATE"/"non-hate", "threshold": ..., "tokens": [(word, contribution), ...], "n": ...}

toward_hate, toward_non_hate = format_attribution(res, topk=5)
```

## Fit your own reward

Two steps — extract frozen features once (GPU), then fit the linear head (CPU/JAX). Needs
`data/labels.npz` and `data/corpus.sqlite3` from `haspi-prepare-corpus`.

```bash
haspi-extract-features --model LeoLM/leo-hessianai-7b --out data/leolm7b_feats.npz
haspi-fit-reward       --features data/leolm7b_feats.npz --out models/leolm_chi2_reward.pkl
```

`haspi-fit-reward` standardises the features, reduces them with whitened PCA-512, fits the
χ² head, **folds** the standardise→PCA→head chain into a single vector (printing
`fold max|Δ| ≈ 1e-6` as a reproduction check), calibrates the decision threshold, and writes
a small reward file. A different encoder is fine — swap `--model` (any Llama/Mistral/Qwen2
architecture works with the pinned `transformers`).

To fit from cached features without any torch:

```python
import numpy as np
from haspi.data import hate_mask
from haspi.sequence import fit_reward

labels = dict(np.load("data/labels.npz"))
feats = np.load("data/leolm7b_feats.npz")["mean"]
payload, max_delta, auc = fit_reward(feats, hate_mask(labels))   # payload → pickle it
```

## Fine-tune the encoder

The reward above keeps the language model frozen. Unfreezing it is worth **+0.02** on the
OMP moderation task (0.747 → 0.766) — the best number we have there — via rank-16 LoRA
adapters trained through the same χ² objective:

```bash
pip install 'haspi[finetune]'
haspi-prepare-moderation --text-out data/moderation/text_512.npz --maxtok 512
haspi-finetune-reward --epochs 2 --batch 8
```

**Explainability survives.** The fine-tuned head is `r = w·φ_mean + b`, the same folded
form as the frozen reward, so it writes an ordinary reward file that every existing tool
reads — just point them at the adapters:

```bash
haspi-explain --reward models/leolm_lora_moderation_reward.pkl \
              --adapter models/leolm_lora_moderation.pt
```

Two ablations ship alongside, and both say the same thing — *the reward objective is not
the bottleneck*: swapping χ² for supervised cross-entropy (`--loss bce`) gains 0.006, and
fully fine-tuning a small encoder (`haspi-finetune-gbert`, pure JAX, no extras) reaches
only 0.67–0.69 on hate — below the **frozen** LeoLM's 0.76. The encoder matters far more
than whether it is trained. See the [fine-tuning guide](https://haspi.readthedocs.io/en/latest/guide/finetune.html).

## RL environment & corpus buffers

`haspi.environment` ships the token-generation MDP as a small, dependency-free JAX
environment plus a helper that fills a replay `Buffer` with a **filterable** sample from the
corpus — hate / non-hate, or online / offline moderation.

```python
from haspi.environment import TextEnv, sample_corpus_buffer

# a filtered demonstration buffer (hate posts; or task="moderation", select="offline")
buf = sample_corpus_buffer("data/corpus.sqlite3", task="hate", select="hate")

# the same MDP as a reset/step environment (reward is 0 unless you pass reward_fn)
env = TextEnv(vocab_size=50266, maxlength=128, eos_id=50265)
state, obs = env.reset()
state, obs, reward, done, info = env.step(state, action=1234)
```

```bash
haspi-sample-buffer --task moderation --select offline --n 2000 --out data/offline_buffer.npz
```

`TextEnv.reset`/`step` are pure and jittable; rolling a post through the env yields the same
transitions as the offline builder. The reward is left to an IQ-Learn head (pluggable
`reward_fn`). Full details in the [environment guide](https://haspi.readthedocs.io).

## Context-aware moderation

For the online/offline moderation task (a post kept vs. removed by moderators), a comment can be
scored with its **thread and article context** — the parent comment and the article's title and
topic path — prepended before encoding.

```bash
haspi-moderation-context --variants comment article topic
```

```console
pooling=comment
variant       random split  article-disjoint
--------------------------------------------
comment              0.722             0.726
article              0.757             0.750
topic                0.697             0.653
```

Two things make this honest. First, the context conditions the encoder but the reward **pools and
attributes only the comment tokens** — the article/thread words shape the comment's representation
(causal attention) yet never enter the pooled feature, so the reward can't just memorise *which
article* a post is under. Second, evaluation is on an **article-disjoint** split (whole articles
assigned to train or test, never both). The proof that this matters is model-free: predicting a
post's label purely from the training-set removal rate of its **article ID** (no text, no model)
scores **0.70** AUROC on a random split and exactly **0.50** (chance) when articles are disjoint — the
leakage is real and there to be exploited. Because we pool comment tokens only, the reward barely
touches it: the article-context reward moves just 0.757 → 0.750 from random to disjoint (a whole-blob
pooling of context+comment, `--pool full`, drops about twice as far: 0.756 → 0.736), and the small article-context gain over comment-only
(0.726 → 0.750) **survives** the honest split. The rule still stands — **evaluate context-aware
moderation on article-/thread-disjoint splits** — because the inflation is large in
article-concentrated samples (the article-ID control reaches 0.98 on a random split there).

`haspi-moderation-context --save-reward models/moderation_reward.pkl` writes a moderation reward that
`haspi-explain --reward models/moderation_reward.pkl` scores and explains as `remove`/`keep`. (Our text-only
reward is comparable to the prior text-only baseline on this corpus, 0.728. The rigorous external
comparison is the **RP-Mod** cross-check noted above, where the frozen reward matches a fine-tuned
German BERT — a separate experiment on a different corpus, not reproducible from this repo.)

## How it works (in one paragraph)

A frozen German decoder LM encodes the comment; its final hidden states are mean-pooled to
one vector *φ*. A linear χ² reward head — the one-step reduction of two-distribution
IQ-Learn — scores it, trained so an *expert* class (non-hate) and an *anchor* class (hate)
are separated. Standardisation and PCA-whitening condition the head; the whole linear chain
then folds into a single weight vector *g* with `r = g·φ + const`. Since *φ* is a mean over
token states, each token's contribution is exactly `orient · (1/L) · g·(hₜ − μ)` and the
parts sum to the score — that's where the per-word explanations come from. The diagram and
the full derivation are in the [documentation](https://haspi.readthedocs.io).

---

## Superseded: two-distribution IQ-Learn

The original method — a token-level IQ-Learn reward trained end to end on frozen GPT-2 — is
kept for reproducibility. It plateaus at ~0.55 AUROC on this corpus (the reward tracks post
length more than hate semantics), which is what motivated the sequence-χ² reformulation. Its
CLIs print a superseded notice; `haspi.CURRENT_METHOD` / `haspi.METHODS` record the full
method-evolution story.

```bash
haspi-prepare-corpus --num-sequences 50000 --maxlength 128
haspi-train --task hate --avenue B --lm-mode frozen --qhead factored \
    --weight-decay 1.0 --epochs 25 --save-path models/hate_frozen.pkl
haspi-classify --model models/hate_frozen.pkl        # REPL with per-token IQ-Learn rewards
haspi-evaluate --model models/hate_frozen.pkl --sweep
```

`haspi-train --cv` runs the 10-fold cross-validation protocol. See the
[architecture guide](https://haspi.readthedocs.io) for the agent internals, tuning notes
(`gamma ≤ 0.95`, `--target-entropy` near the policy's natural entropy, the `factored`
Q-head), and the library API.

## Tests & docs

```bash
pytest                                  # fast unit suite, no model downloads
pytest -m slow                          # end-to-end on real models (needs the download / GPU)
pip install -e ".[docs]" && sphinx-build -W -b html docs docs/_build/html   # build the docs
```
