Metadata-Version: 2.4
Name: morseformer
Version: 0.6.3
Summary: Open-source transformer-based Morse/CW decoder with a built-in ham-specialised language model
Project-URL: Homepage, https://github.com/sderhy/morseformer
Project-URL: Repository, https://github.com/sderhy/morseformer
Project-URL: Issues, https://github.com/sderhy/morseformer/issues
Author-email: sderhy <sderhy@gmail.com>
License: Apache-2.0
License-File: LICENSE
Keywords: amateur-radio,ctc,cw,dsp,ham-radio,morse,speech-to-text,transformer
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Communications :: Ham Radio
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.25
Requires-Dist: numpy>=1.24
Requires-Dist: scipy>=1.10
Requires-Dist: torch>=2.1
Requires-Dist: torchaudio>=2.1
Provides-Extra: dev
Requires-Dist: matplotlib>=3.7; extra == 'dev'
Requires-Dist: pytest>=7; extra == 'dev'
Requires-Dist: ruff>=0.1; extra == 'dev'
Provides-Extra: live
Requires-Dist: pasimple>=0.0.3; extra == 'live'
Provides-Extra: training
Requires-Dist: hydra-core; extra == 'training'
Requires-Dist: tqdm; extra == 'training'
Requires-Dist: wandb; extra == 'training'
Description-Content-Type: text/markdown

# morseformer

> Open-source transformer-based Morse / CW decoder. Fully local. Apache 2.0.

[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](#)
[![Release: v0.6.2](https://img.shields.io/badge/release-v0.6.2-brightgreen.svg)](#release-v062)
[![Model on HuggingFace](https://img.shields.io/badge/🤗%20Hub-sderhy/morseformer-yellow)](https://huggingface.co/sderhy/morseformer)

Conformer + RNN-T Morse decoder with a real-time streaming CLI, trained on a reproducible synthetic-HF pipeline. The **v0.6.2 release** reverts the recommended acoustic from `rnnt_phase5_8.pt` (v0.6.0/v0.6.1) to **`rnnt_phase5_5.pt`** after the first reproducible bench (`eval/bench_lcwo.py`, 6 real-audio clips: LCWO De Gaulle FR + Pacino EN at 24/20 WPM + Aznavour FR + websdr 5-letter + a synthetic contest exchange) showed Phase 5.5 outperforms 5.7 (−22 % mean CER) and 5.8 (−24 %). The streaming-offline decoder shipped in v0.6.0 stays — it's responsible for most of v0.6.0's visible gain. Phase 5.8 trained on extra English literary text without a counter-test against the bootstrap source, and the bench reveals that the curriculum hurt prose + random-group accuracy more than it helped. v0.6.2 is a code-only revert: `rnnt_phase5_5.pt` was already on the HF Hub, so no new weights are uploaded.

## Why

Existing open-source CW decoders (`fldigi`, `cwdecoder`, `MRP40`) rely on hand-tuned DSP and threshold-based segmentation; they struggle in weak-signal conditions, QRM, QSB, and with non-ideal operator timing. The commercial reference, `CW Skimmer` (VE3NEA), is closed-source and built on ~2009-era Kalman filtering. **As of April 2026, there is no published transformer-based CW decoder, and no open-source CW decoder with an integrated language model.** `morseformer` fills that gap.

## Quick start

```bash
# 1. Create and activate a virtual environment (Python 3.10+).
python3 -m venv venv
source venv/bin/activate          # Windows: venv\Scripts\activate

# 2a. Decode an audio file (offline). First run fetches the model from HuggingFace.
pip install morseformer
morseformer decode my_recording.wav

# 2b. Or decode a live receiver in real time (PulseAudio input).
pip install "morseformer[live]"
morseformer live
```

That's it. **No GPU needed** — the decoder runs on CPU by default.

> **No Nvidia card? Save ~4 GB.** PyTorch ships a 2-3 GB GPU build by default. On a CPU-only machine, install the lightweight CPU build *before* `morseformer`:
> ```bash
> pip install --index-url https://download.pytorch.org/whl/cpu torch torchaudio
> pip install morseformer            # or "morseformer[live]"
> ```

### More options

```bash
# LM shallow fusion (λ=0.7) for prose / ragchew audio.
morseformer decode my_recording.wav --preset prose

# Live presets — looser for fast exchanges, tighter for very noisy bands.
morseformer live --preset contest
morseformer live --preset conservative

# Inspect / download checkpoints.
morseformer models list
morseformer models list --advanced
morseformer models download rnnt_phase5_7
```

The four shipped presets — `live` (default), `prose`, `contest`, `conservative` — bundle the model + thresholds + optional LM behind one flag. `morseformer --help` lists every subcommand.

Developing from a checkout? Same CLI, plus the existing scripts:

```bash
git clone git@github.com:sderhy/morseformer.git
cd morseformer
pip install -e ".[dev,live]"
pytest -q

# CLI works against local release/ and checkpoints/ trees, no Hub fetch.
morseformer decode my_recording.wav

# Or call the underlying scripts directly for fine-grained control.
python -m scripts.decode_audio my_recording.wav \
    --ckpt    release/rnnt_phase5_7.pt \
    --lm-ckpt release/lm_phase5_2.pt \
    --fusion-weight 0.7 \
    --confidence-threshold 0.6 \
    --digit-threshold 0.90
```

Example output on a clean synthetic `CQ DE F4HYY K` @ 20 WPM / +20 dB SNR:

```
CTC  : 'CQ DE F4HYY K'
RNN-T: 'CQ DE F4HYY K'
```

## Release v0.6.3

Packaging-only refresh — no model change, same `rnnt_phase5_5.pt` and
`lm_phase5_2.pt` on the HF Hub.

- `morseformer/__init__.py`: `__version__` was stuck at `0.6.0` while
  `pyproject.toml` had moved to `0.6.2`; both now read `0.6.3`.
- `.github/workflows/ci.yml`: minimal CI (Python 3.10 / 3.11 / 3.12,
  CPU PyTorch wheels, `ruff check` + `pytest`).
- `[tool.ruff.lint]`: ignore `N812 / N806 / N802` (PyTorch idioms like
  `import torch.nn.functional as F` and tensor-shape vars `B/T/K/D`) and
  `B905` (`zip(strict=)` is opt-in safety, not a defect), plus a tree-wide
  `ruff --fix` pass.

PyPI was last updated at `0.6.1`, so `pip install -U morseformer` also picks
up everything that landed in `0.6.2` — see the section below for the
acoustic revert from `rnnt_phase5_8` back to `rnnt_phase5_5` (–24 % relative
mean CER on the LCWO bench).

## Release v0.6.2

The first reproducible real-audio bench (`eval/bench_lcwo.py`, 6 clips, identical decoder) ran on 2026-05-09 against four acoustic checkpoints — the bootstrap source `rnnt_phase5_5` plus its three descendants `rnnt_phase5_7` (v0.5.3), `rnnt_phase5_8` (v0.6.0/v0.6.1), and the new `rnnt_phase5_9`. The result inverts the live-test gut feel that drove v0.5.3 and v0.6.0 ship decisions:

| Acoustic            | de-gaulle FR | pacino-24 EN | pacino-20 EN | jme-voyais FR | websdr 5-letter | contest-synth | mean CER |
|---------------------|--------------|--------------|--------------|---------------|-----------------|---------------|----------|
| **rnnt_phase5_5**   | **1.97 %**   | **1.58 %**   | **1.68 %**   | 1.30 %        | **8.00 %**      | **2.57 %**    | **2.85 %** |
| rnnt_phase5_7       | 1.97 %       | 2.13 %       | 2.41 %       | **0.99 %**    | 10.22 %         | 5.14 %        | 3.81 %   |
| rnnt_phase5_8       | 2.30 %       | 1.86 %       | 2.35 %       | 1.03 %        | 10.44 %         | 5.61 %        | 3.93 %   |
| rnnt_phase5_9       | 3.29 %       | 2.16 %       | 2.77 %       | 1.17 %        | 10.00 %         | 4.21 %        | 3.93 %   |

`rnnt_phase5_5` wins 5/6 clips (jme-voyais 22 wpm goes to 5.7 by 0.31 pp), and beats the v0.6.0 release by **24 % relative mean CER**. Most striking: on the synthetic contest clip (30 contest_dense exchanges at 24 wpm), `rnnt_phase5_5` posts 2.57 % CER even though it had **zero contest_dense in its training mix** — vs 5.14 % for `rnnt_phase5_7` (30 % contest_dense in training) and 5.61 % for `rnnt_phase5_8` (13 %). The contest_dense curriculum did not improve contest performance; it just degraded everything else.

`rnnt_phase5_9` was a fresh 15 k-step fine-tune from `phase5_5/last` on a new 20 % `letter_groups` mix (FAV22-style strict 5-char cipher groups, see `morseformer.data.text.sample_letter_groups`) targeting the websdr 8 % CER gap. It regressed on every clip including its own target (8.00 → 10.00 % on websdr). Hypothesis: the mix amputated `prose` / `prose_fr` budget enough to trigger catastrophic forgetting; 15 k steps on a divergent mix is too long for a fine-tune. Code kept under `--curriculum phase5_9` for reproducibility.

### What v0.6.2 changes

- `morseformer/cli/registry.py`: `RECOMMENDED_ACOUSTIC = "rnnt_phase5_5"`, `rnnt_phase5_8` demoted to `recommended=False`.
- `morseformer/cli/presets.py`: all four presets (`live`, `prose`, `contest`, `conservative`) switch to `rnnt_phase5_5`.
- No model upload — `rnnt_phase5_5.pt` has been on the HF Hub since v0.5.1.
- The streaming offline decoder, the digit-threshold gate, the run-on-prosign cosmetic (`BK → =`), and the rest of v0.6.x's structural improvements stay.

After installing the upgrade, `morseformer decode my.wav` automatically downloads / uses `rnnt_phase5_5.pt` instead of `rnnt_phase5_8.pt`.

### Caveats and open questions

- **Bench is synthetic-leaning.** 4 of 6 clips are LCWO synthesis; 1 is a real websdr; 1 is a `morse_synth`-rendered contest exchange that does **not** apply the `operator_run_on_pairs` augmentation that 5.7 and 5.8 trained on. The latter slightly biases against 5_7/5_8 on contest material — but the prose + random-group dominance of 5_5 holds independently.
- **No live-radio bench yet.** v0.6.2 is shipped on the strength of the LCWO + websdr corpus alone. Live IC-7300 / SDR sessions remain the qualitative source-of-truth; we expect 5_5 to match its v0.5.1 / v0.5.2 live record.
- **Direction for the next phase.** Letter_groups densification is not a dead end — Phase 5.9 just over-rotated. A more conservative retry (5-8 % `letter_groups`, full prose proportions preserved, ≤ 8 k steps) is the natural Phase 5.10. Bench gates every ship from now on.

## Release v0.6.0

The v0.5.4 LCWO live test (3 hand-keyed-style samples: De Gaulle FR oratory + Al Pacino EN dialogue at 24 / 20 WPM, ~25 min total) confirmed that morseformer produces "fully readable" transcripts on out-of-domain literary prose — but flagged two systematic issues. The dominant visible failure was **window-boundary word cuts**: `MARTYRISÉ → MARTYRISÉ E`, `L'APPUI → L'AP PUI`, `INSTITUTION → INSTITUTIO E`. This was a *structural* problem, not acoustic — `decode_audio` cut the audio into non-overlapping 6 s chunks and stitched the per-chunk hypotheses together, so any word landing on a boundary lost characters. The secondary issue was acoustic / textual: long-form English literary narrative was outside the training distribution (the synthetic mix is QSO-heavy + multilingual cross-section) so words like `WANDERED → WANDERE D`, `BREEZE → BREEZE. E B BKB`.

v0.6.0 ships both fixes.

### Streaming offline decoder (structural)

`decode_audio` now uses a **central-zone-commit** sliding window — the same logic the live streaming decoder has used since v0.2 — for the RNN-T head. Adjacent commit zones tile the audio without gap or overlap, so words never fall in the middle of a "fresh" boundary. Diff on the v0.5.4 `test_8k.wav` Daffodils transcript:

```
v0.5.4 chunked:    I WANDERE D LONELY  L A CLOU E THAT FLOA S ON HIGH …
                   GM ALDEN DAV DAFI RODILS, SESIDE THE LAD N, BENEATS T EHE TLES,
                   L F F FLUN TERING AND DN E DPCING IN THE BREEZE.

v0.6.0 streaming:  IWANDERED LONELY AS ALOUD THAT FLOATS ON HIGH …
                   GOLDEN DAV DAFFODILS, BESIDE THE LAC, BENEATS THE TLES,
                   LF F FLUTTERING AND D DACING IN THE BREEZE.
```

Old behaviour available via `--legacy-chunking` for diagnostic. CTC head still goes through the chunked path (it is a research-only secondary hypothesis).

### Phase 5.8 — English-literary curriculum (acoustic)

`rnnt_phase5_8.pt` is a 20 k-step fine-tune from `phase5_5/last` (NOT from `phase5_7/last` — see Phase 5.7 carryover note) on a new text mix `PHASE_5_8_MIX`:

- New `prose_en` slice at 25 % weight, sampled exclusively from the four EN Gutenberg books (Moby Dick = ~42 % of EN, plus Pride & Prejudice + Sherlock Holmes + Frankenstein, ~3 M chars total)
- `contest_dense` cut from 30 % to 13 % to free room for `prose_en` while keeping the 5NN gain from Phase 5.7
- `prose_fr` preserved at 10 % so French diacritic gradient is not lost
- `operator_run_on_pairs` halved (`UR 0.50→0.25, SK 0.85→0.50, KN 0.70→0.30, BK 0.70→0.30`) to reduce phantom-prosign emissions on continuous prose, which the user report flagged as a Phase 5.7 cost
- Same channel / jitter / dash:dot / gap-inflation / word-gap-inflation / FP-prior as Phase 5.5

20 k steps, peak LR 5e-5, EMA 0.9999, bf16, batch 12, ~2 h on RTX 3060 Laptop. Bootstrap chain: `phase3_5/best → phase5_3/best → phase5_4/last → phase5_5/last → phase5_8/last`. As with every release since v0.5.1, the dormant `ValidationConfig.matching()` clean-CW val signal saturates fast (< step 1 k) — `last.pt` is the released checkpoint, not `best_rnnt.pt`.

### What's measurable on synthetic test_8k.wav

Phase 5.8 vs Phase 5.7 on the Daffodils synthetic test (5.4-era ground truth). The structural streaming patch contributes most of the visible improvement; Phase 5.8 adds incremental literary-EN signal on top:

| Token | v0.5.4 chunked | v0.5.3 streaming | **v0.6.0 streaming** |
|---|---|---|---|
| "WANDERED" | `WANDERE D` | `IWANDERED` | `IWANDERED` ✓ |
| "FLOATS" | `FLOA S` | `FLOATS` | `FLOATS` ✓ |
| "FLUTTERING" | `FLUN TERING` | `FLUTERING` | `FLUTTERING` ✓ |
| "DANCING" | `DN E DPCING` | `DNCINNG` | `DACING` (closer) |
| "A CLOUD" | `L A CLOU E` | `ALLOUD` | `ALOUD` (closer) |

Live evaluation on the user's LCWO samples (De Gaulle + Pacino) is the real target and was not re-run for the v0.6.0 release notes; the repo's auto-aligned Alice corpus (`data/real/aligned/all_alice.jsonl`) and synthetic FR-mix benches did not regress.

### Limitations of v0.6.0

- Phase 5.8 is incremental on the synthetic distribution (synthetic val is saturated since v0.5.x). The motivating gain — long-form EN literary prose — has not been formally CER-benched yet; the qualitative diff above is on a single in-distribution audio.
- `prose` preset's LM (`lm_phase5_2`, matched to `PHASE_3_4_MIX`) was reported by the v0.5.4 LCWO test as actively *hurting* on FR oratory (De Gaulle: `L'APPUI → LAEC LAGAP PUI` vs `L'AP PUI` in `live` preset). v0.6.0 does not change the LM. Use `live` preset for FR oratory; reserve `prose` for English ragchew / prose where the LM training corpus is closer.
- Spurious prosigns at chunk boundaries (`K`, `+`, `BK`) are reduced but not eliminated — Phase 5.8 halved the run-on-pair probabilities, which is a partial fix.
- `=` (BT, the typographic break prosign) is rendered as the `BK` digraph by the model when it detects a run-on B+K. Mapping `BK → =` at output is left as a future post-processing cosmetic fix; the model still emits the underlying tokens correctly.

### Recommended decode for v0.6.0

Default invocation just works:

```bash
pip install "morseformer[live]"             # or just `pip install morseformer` for offline only
morseformer decode my_recording.wav        # uses rnnt_phase5_8 + streaming
morseformer live                            # same model, live streaming
```

For French oratory or any non-English literary audio, prefer `live` preset over `prose` until the LM is matched (Phase 4.2 territory in the roadmap).

## Release v0.5.4

Packaging release — **no model change** (`rnnt_phase5_7.pt` unchanged from v0.5.3). v0.5.4 ships a unified `morseformer` console CLI that replaces the `python -m scripts.decode_*` invocations as the primary user surface:

- **`pip install morseformer`** now exposes `morseformer` as a console script with three subcommands: `decode <wav>`, `live`, and `models {list,download}`.
- **Four named presets** bundle model + thresholds + optional LM behind one flag: `live` (default; v0.5.3 streaming defaults), `prose` (LM shallow fusion λ=0.7), `contest` (looser thresholds for fast exchanges), `conservative` (tightened thresholds for very noisy bands).
- **Auto-download from HuggingFace** on first use — no manual `hf download` step. Model registry resolves from `release/` → `checkpoints/<phase>/last.pt` → HF cache, in that order, so dev checkouts and pip-installed environments share the same CLI.
- `morseformer models list` shows the recommended pair (`rnnt_phase5_7` + `lm_phase5_2`); `--advanced` lists every legacy 46-vocab checkpoint on the Hub.
- Underlying `scripts/decode_audio.py` and `scripts/decode_live.py` remain available with the same flags for fine-grained control.
- Streaming-fusion is still offline-only: when the `prose` preset is used with `morseformer live`, the LM is silently dropped with a one-line notice (the streaming-fusion path regresses CER, see Phase 4.1+).

Examples:

```bash
pip install "morseformer[live]"
morseformer decode my.wav                  # acoustic-only, default preset
morseformer decode my.wav --preset prose   # offline + LM fusion
morseformer live                           # real-time streaming
morseformer live --preset contest
morseformer models list --advanced
```

## Release v0.5.3

Phase 5.7 amateur-idiom curriculum on top of v0.5.2. **New acoustic checkpoint `rnnt_phase5_7.pt`** (4.13 M params, 49-vocab); LM and inference defaults unchanged.

The v0.5.2 release was live-validated as a "qualitative-parity-with-cwget" milestone, but the user transcript surfaced three letter-level residuals consistently visible on the IC-7300 stream: `5NN` (the spoken contest cut-number for `599`) decoded as `5MN` or `ENN`; the `0`-prefixed callsign `MI0MO` rendered as `MI4MO`; standard run-on prosigns (`UR`, `SK`, `KN`, `BK`) split into their constituent letters because the synthetic curriculum had only ever shown them with normal inter-letter gaps. Phase 5.7 is a 15 k-step fine-tune from `phase5_5/last` on a curriculum that targets these directly:

- **`PHASE_5_7_MIX`** — new text mix with 5NN cut-numbers and a `_CONTEST_DENSE_TEMPLATES` slice (densely-packed contest exchanges around `5NN <NR>`).
- **`OperatorConfig.run_on_pairs`** — a per-utterance probability of collapsing the inter-letter gap on declared digraphs to a single inter-element gap. Defaults: `UR 0.50, SK 0.85, KN 0.70, BK 0.70`. Old presets keep `run_on_pairs=()` so legacy curricula are byte-identical.
- **Bootstrap** from `phase5_5/last` (the v0.5.2 acoustic). Same channel, jitter, dash:dot, gap-inflation, word-gap-inflation, FP-prior, and 30 % real-audio mix as Phase 5.5. Strict ablation around the curriculum delta. LR 5e-5, EMA 0.9999, bf16, batch 12, ~2 h on RTX 3060.

### What's new in v0.5.3

Live-test on IC-7300 (2026-05-06, FAV22 5-letter groups + a 40 m QSO):

| Behaviour | v0.5.2 | **v0.5.3** |
|---|---|---|
| `5NN` cut-number explicit emission count, FAV22 + QSO transcript | 1× | **4–5×** |
| `MI0MO` callsign on weak signal | `MI4MO` | **`MI0MO`** ✅ |
| Run-on prosigns (`UR`, `SK`, `KN`, `BK`) | split into letters | emitted as digraphs |
| User verdict | "qualitative parity with cwget" | "plutôt bien, pas trop d'hallucination" |

### Limitations of v0.5.3

- **`BK` over-emission on extreme pile-ups.** On a noisy multi-station pile-up sample the model emitted 23 spurious `BK` tokens — the run-on prior is too eager when several weak signals overlap. A future Phase 5.8 candidate would lower `run_on_pairs` (`BK 0.70 → 0.30`, `KN 0.70 → 0.30`, `UR 0.50 → 0.25`, `SK 0.85 → 0.50`) bootstrapped from `phase5_5/last`, **not** from `phase5_7/last`.
- `UR` recall regressed slightly on legitimate (non-run-on) sequences and a few callsign trails went `D → B`. Net trade is positive on the live test but the regression is real.
- The `best_rnnt.pt` checkpoint at step ~11 k saturates the dormant `ValidationConfig.matching()` clean-CW val signal at 0 % CER; **`last.pt` (step 15 000) is the released checkpoint**, same as v0.5.1 / v0.5.2.
- Pile-up `noisy-pile-up.wav` (F6GCI / IU0TEZ / 5I9HFM overlapping at the same frequency) remains unrecoverable. This is a multi-source separation problem, not an acoustic one — out of scope for the current architecture.

### Recommended decode for v0.5.3

Same CLI as v0.5.2, just point the `--ckpt` at `rnnt_phase5_7.pt`:

```bash
# Live streaming (default — 0.6 threshold + 0.90 digit gate on)
python -m scripts.decode_live --ckpt checkpoints/phase5_7/rnnt_phase5_7.pt

# Offline with fusion
python -m scripts.decode_audio my_recording.wav \
    --ckpt    checkpoints/phase5_7/rnnt_phase5_7.pt \
    --lm-ckpt checkpoints/lm_phase5_2/lm_phase5_2.pt \
    --fusion-weight 0.7 \
    --confidence-threshold 0.6 \
    --digit-threshold 0.90
```

## Release v0.5.2

Inference-only fix — **no retraining**, same `rnnt_phase5_5.pt` checkpoint as v0.5.1. v0.5.1 was live-validated on an IC-7300 with the verdict "bien meilleur jusqu'à présent", but the transcript surfaced one consistent failure mode: pseudo-numerals (`061511813`, `5'9734`, `5NR45`) emitted with high confidence on band noise or transitional weak signal. The regular `--confidence-threshold 0.6` did not absorb them because the acoustic head was *confident* in its wrong digit hypothesis (94 % of the chars emitted on pure-noise samples were digits).

- **Class-conditional confidence threshold.** Digit tokens (vocab indices 28..37 = 0-9) gate at a stricter threshold than letters / punctuation. New `--digit-threshold` flag on `decode_live`, `decode_audio`, and `eval_false_positive`. `decode_live` ships `0.90` as the default; pass `0.0` to recover v0.5.1 behaviour.
- The fix is implemented inside `RnntModel.greedy_rnnt_decode` and propagated through `StreamingDecoder` — it stacks cleanly with the regular threshold and with shallow fusion.
- A failed retraining attempt (Phase 5.6 with a pseudo-Morse empty-sample mode at `empty_sample_probability=0.40`) is kept in the codebase for reference (commit fa2b461) but is **not** the v0.5.2 release path.

### What's new in v0.5.2

| Bench | v0.5.1 stock | **v0.5.2 (digit_thr=0.90)** |
|---|---|---|
| **Digit share of FP characters on pure noise** | 62 % | **~0 %** |
| **FP-bench mean chars/sample, no thresh** | 1.97 | 1.88 |
| **FP-bench mean chars/sample, +thresh 0.6** | 0.01 | **0.00** |
| **Alice ebook2cw n=120, greedy** | 16.39 % CER | **15.93 % CER** (−0.5 pp) |
| **`test_manu.wav` Wordsworth, hand-keyed** | 23.49 % CER | **23.26 % CER** (−0.2 pp) |
| **Synthetic FR mix overall (240 samples)** | 1.12 % | 1.12 % (=) |
| **`bench_word_gap` 6× inter-word silence** | 27 / 30 word_acc, 1.5 % CER | 26 / 30, 1.9 % CER |
| **Legitimate digit recall (10 ham phrases)** | 87 % | 84 % (−3 % rel) |

### Limitations of v0.5.2

- A fixed scalar threshold on digits is a coarse fix. A future release could replace it with class-conditional softmax temperature or a context-aware gate (digit-allowed if recently inside a callsign / RST exchange).
- The −1 word_acc on `bench_word_gap` 6× and the −3 % relative legitimate digit recall are the cost; both are within noise on real-air audio.
- LM fusion path (`greedy_rnnt_decode_with_lm`) does not yet propagate `digit_threshold`. Fusion + digit gating would need an extra plumbing pass; not a concern for the streaming default.

### Recommended decode for v0.5.2

Streaming live (default — digit gate on):

```bash
python -m scripts.decode_live --ckpt checkpoints/phase5_5/rnnt_phase5_5.pt
# implies --confidence-threshold 0.6 --digit-threshold 0.90
```

Offline with fusion:

```bash
python -m scripts.decode_audio my_recording.wav \
    --ckpt    checkpoints/phase5_5/rnnt_phase5_5.pt \
    --lm-ckpt checkpoints/lm_phase5_2/lm_phase5_2.pt \
    --fusion-weight 0.7 \
    --confidence-threshold 0.6 \
    --digit-threshold 0.90
```

## Release v0.5.1

The v0.5.0 live test surfaced one residual failure mode: when a slow / hesitant operator left several seconds of silence between two words, the model sometimes fused them or emitted spurious characters during the long silence. The synthetic training distribution had only ever shown the canonical 7-dit Farnsworth gap (~0.42 s at 20 WPM), so anything beyond that was out-of-domain. v0.5.1 closes that hole with a **single-knob curriculum extension** on top of Phase 5.3 / 5.4:

- **Phase 5.5 — long inter-word silences.** New `OperatorConfig` knob `word_gap_inflation`, a per-utterance multiplier on every Farnsworth inter-word gap, sampled from `U(1.0, 8.0)`. At 20 WPM this stretches the canonical 7-dit space up to ~56 dits (~3.4 s of silence between two words). Fine-tune of 15 k steps from `phase5_4/last` (LR 5e-5, bf16). All other knobs (jitter, dash:dot ratio, gap inflation, channel, text mix, empty-sample prior) kept identical to Phase 5.3 / 5.4 — strict ablation.

Eight artifacts on [🤗 sderhy/morseformer](https://huggingface.co/sderhy/morseformer):

| file | params | vocab | description | recommended |
|---|---|---|---|---|
| `rnnt_phase5_5.pt` | 4.13 M | **49** | **v0.5.1 acoustic — Phase 5.5 (long inter-word silences) on top of v0.5.0.** | ✅ |
| `lm_phase5_2.pt` | 4.76 M | 49 | v0.4.1 LM, unchanged. Use at λ_lm = 0.7 for fusion. | ✅ |
| `rnnt_phase5_4.pt` | 4.13 M | 49 | v0.5.0 acoustic — Phase 5.3 + 5.4. Kept for diff. | |
| `rnnt_phase3_5.pt` | 4.13 M | 49 | v0.4.0 / v0.4.1 acoustic — synthetic-only. Kept for diff. | |
| `lm_phase4_0.pt` | 4.76 M | 46 | Legacy v0.1-era LM, ham-radio-only mix. | |
| `rnnt_phase3_3.pt` | 4.13 M | 46 | v0.3 acoustic. | |
| `rnnt_phase3_2.pt` | 4.13 M | 46 | v0.2 acoustic. | |
| `rnnt_phase3_0.pt` | 4.13 M | 46 | v0.1 acoustic. | |

### What's new in v0.5.1

| Bench | v0.5.0 (`phase5_4/last`) | **v0.5.1 (`phase5_5/last`)** |
|---|---|---|
| **`bench_word_gap` 25 WPM, SNR 20 dB, 30 short pairs, 6× inter-word silence** | 8 / 30 word_acc, 17.4 % CER | **27 / 30 word_acc, 1.5 % CER** |
| **`bench_word_gap` same, 4× inter-word silence** | 20 / 30 word_acc, 7.7 % CER | **28 / 30 word_acc, 0.8 % CER** |
| **Alice ebook2cw prose, n=120 score≥0.5, greedy** | 18.82 % CER | **16.39 % CER** (−2.4 pp) |
| **`test_manu.wav` Wordsworth, hand-keyed, threshold 0.6** | 26.5 % CER | **23.5 % CER** (−3.0 pp) |
| **Synthetic FR-mix overall CER (240 samples, clean → −5 dB)** | 0.68 % | 1.12 % (regression confined to −5 dB; clean → 0 dB stay at 0 %) |
| **Synthetic FR é / à / ' precision** | 100 / 100 / 100 % | 100 / 100 / 100 % (recall 100 / 100 / 98.3 %) |
| **FP bench, threshold 0.6** | 0.00 chars / sample | 0.01 chars / sample (within noise) |

### Limitations of v0.5.1

- The single regression is on synthetic French at SNR = −5 dB (4.08 % → 6.71 %). Clean-to-0-dB FR is still 0 % CER, accent tokens preserved.
- Same `decode_live` streaming caveat as v0.4.1+: LM fusion is offline only.
- Same recommended decode as v0.5.0 (offline fusion `--fusion-weight 0.7`, streaming greedy with `--confidence-threshold 0.6`), just point at `rnnt_phase5_5.pt` instead of `rnnt_phase5_4.pt`.

## Release v0.5.0

The v0.4.1 acoustic was synthetic-only and tripped on real human keying — the live test surfaced systematic stutter (`F = ..-.` decoded as `A + V`, repeated emissions like `WWWVVUT`) on hand-keyed audio. v0.5.0 fixes the acoustic model itself with **two new training stages on top of v0.4.0's Phase 3.5**:

- **Phase 5.3 — wider operator-timing envelope.** New `OperatorConfig` knobs `dash_dot_ratio` (sampled per utterance from `U(2.5, 4.5)`, was a fixed `3.0`) and `gap_inflation` (`U(0.8, 1.6)` multiplicative on inter-element gaps, was identity). Element-jitter widened to `U(0, 0.30)` and gap-jitter to `U(0, 0.50)` (was `(0, 0.15)` / `(0, 0.25)`). Fine-tune of 15 k steps from `phase3_5/best`.

- **Phase 5.4 — real-audio mix.** Bootstrap from `phase5_3/best`, fine-tune 12 k steps with **30 % of every batch drawn from real human-keyed audio** (42 ground-truth-aligned 6 s chunks of contest exchanges + random sequences + Wordsworth's *Daffodils*, aligned via `scripts/align_ebook_cw.py` against a hand-written transcript). The synthetic 70 % keeps Phase 5.3's wider-envelope curriculum so the model does not regress on the synthetic distribution.

### What's new in v0.5.0

| Bench | v0.4.1 (`phase3_5/best` + fusion) | **v0.5.0 (`phase5_4/last` + fusion)** |
|---|---|---|
| **Alice ebook2cw prose, n=120 score≥0.7, greedy** | 36.44 % CER | **19.16 % CER** (−47 % rel.) |
| **Alice ebook2cw prose, n=120 score≥0.7, fusion λ=0.7** | 32.30 % CER | **16.59 % CER** (−49 % rel.) |
| **Synthetic FR-mix, n=8 (samples drawn from PHASE_3_4_MIX)** | 50.0 % CER | **12.5 % CER** (−75 % rel.) |
| **FP bench, threshold 0.6 + fusion** | 0.17 chars / noise sample | **0.00 / 150** (zero false positives) |
| **Hand-keyed `test.wav` Daffodils section** | `NNNNFFFOLS, BENEATS THE TLES` (letter-soup) | `NNFODILS, BENEATH T HE TREES, FLUT TERING AND DPCING IN TTHE BREEZE` (readable) |

The improvement is *not* localised to the user audio that was mixed in: the Alice (−47 %) and synthetic FR-mix (−75 %) gains are out-of-domain wins, confirming the wider operator envelope plus a small real-audio prior lifted the model out of a synthetic-only basin.

### Limitations of v0.5.0

- The real-audio mix is **42 chunks total** (~4 minutes of audio from a single recording session). The model has clearly generalised beyond it (Alice / synth FR gains) but a larger and more diverse real-audio corpus is the natural next step.
- LM fusion is still **offline only**. `decode_live.py` keeps the v0.4.1 streaming greedy path; plumbing fusion through the central-zone-commit logic is parked for a follow-up.
- **Caveat**: the `test.wav` improvement is partly in-domain — that audio was in the training mix. Out-of-domain validation is via Alice and the synthetic FR mix, both of which the model never saw.
- ILME / density-ratio fusion remains catastrophic on this distribution; do **not** pass `--ilm-weight > 0` (kept in the codebase for research only).

### Recommended decode for v0.5.0

Offline (any length, with fusion):

```bash
python -m scripts.decode_audio my_recording.wav \
    --ckpt    checkpoints/phase5_4/rnnt_phase5_4.pt \
    --lm-ckpt checkpoints/lm_phase5_2/lm_phase5_2.pt \
    --fusion-weight 0.7 \
    --confidence-threshold 0.6
```

Live streaming (no fusion yet — gating only):

```bash
python -m scripts.decode_live --ckpt checkpoints/phase5_4/rnnt_phase5_4.pt
```

## Release v0.4.1

Inference-time additions on top of the v0.4.0 acoustic model — **no retraining of the RNN-T**, just a config change in `decode_live.py` and a freshly retrained LM that fixes the long-standing fusion null result. Six artifacts on [🤗 sderhy/morseformer](https://huggingface.co/sderhy/morseformer):

| file | params | vocab | description | recommended |
|---|---|---|---|---|
| `rnnt_phase3_5.pt` | 4.13 M | **49** | **Acoustic model — unchanged from v0.4.0.** | ✅ |
| `lm_phase5_2.pt` | 4.76 M | **49** | **New LM matched to the Phase 3.5 text mix** (multilingual prose + ham radio). val_ppl 5.626. Pair with the RNN-T at `--fusion-weight 0.7`. | ✅ |
| `lm_phase4_0.pt` | 4.76 M | 46 | Legacy ham-radio-only LM (v0.1 era, kept for research). | |
| `rnnt_phase3_3.pt` | 4.13 M | 46 | v0.3 acoustic model. | |
| `rnnt_phase3_2.pt` | 4.13 M | 46 | v0.2 acoustic model. | |
| `rnnt_phase3_0.pt` | 4.13 M | 46 | v0.1 acoustic model. | |

### What's new in v0.4.1

1. **Streaming false-positive default**: `decode_live.py --confidence-threshold` is now `0.6` by default (was `0.0`). On the 150-sample noise-only bench this kills 90 % of noise-driven false positives — `1.50 → 0.17` chars per noise sample, with no measurable accuracy loss down to −5 dB SNR. Set `--confidence-threshold 0.0` to recover the old behaviour exactly.

2. **LM rescoring works now**. The v0.1 LM (`lm_phase4_0.pt`) was trained on a 100 % ham-radio mix (callsigns / Q-codes / QSO templates) and was *hostile* to prose decoding — Phase 4.1 measured it as null on synthetic, and a fresh autonomous bench on real ebook2cw prose audio confirmed it actively hurts (−9 % CER at λ = 0.2 on the 46-vocab LM). The new `lm_phase5_2.pt` is retrained on `PHASE_3_4_MIX` (the same text mix the Phase 3.5 acoustic model saw) at the 49-vocab tokenizer, val_ppl `5.626`. With λ = 0.7 shallow fusion through `scripts/decode_audio.py` it cuts CER on real Alice-in-Wonderland CW audio by **−11.4 %** (n = 120, score-filtered). ILME / density-ratio fusion remains catastrophic on this distribution and is *not* recommended (λ_lm = λ_ilm = 0.3 → CER +83 %).

3. **Threshold + fusion stack cleanly**. Confidence gating is intentionally applied to the *acoustic* head pre-LM, so the LM cannot "rescue" a low-confidence noise emission. The grid bench:

```
  thr  | λ_lm |  FP mean (chars/noise sample)
  -----+------+--------------------------------
   0.0 | 0.0  |  1.50  (baseline)
   0.0 | 0.7  |  1.01  (fusion alone, −33 %)
   0.6 | 0.0  |  0.17  (Phase 5.0 default — shipped here)
   0.6 | 0.7  |  0.17  (preserved, no regression)
```

4. **Autonomous validation pipeline**. The fusion verdict was reached without a live test on the user's IC-7300: `data/real/aligned/all_alice.jsonl` (9482 ground-truthed ebook2cw chunks produced by `scripts/align_ebook_cw.py`) is now the project's autonomous prose bench. `scripts/eval_fusion_realaudio.py` runs λ sweeps on it.

5. **`scripts/decode_audio.py` learns LM fusion**: new `--lm-ckpt`, `--fusion-weight`, and `--confidence-threshold` flags. Backward-compatible: omit the flags to reproduce v0.4.0 behaviour exactly.

6. **`scripts/train_lm.py` learns text mixes**: new `--mix {default | phase_3_2 | phase_3_3 | phase_3_4 | phase_3_5 | phase_3_6 | phase_4_0}` and `--vocab-size` flags. Use `--mix phase_3_5 --vocab-size 49` to reproduce `lm_phase5_2.pt`.

### Recommended decode for v0.4.1

Offline (any length, with fusion):

```bash
python -m scripts.decode_audio my_recording.wav \
    --ckpt    checkpoints/phase3_5/rnnt_phase3_5.pt \
    --lm-ckpt checkpoints/lm_phase5_2/lm_phase5_2.pt \
    --fusion-weight 0.7 \
    --confidence-threshold 0.6
```

Live streaming (no fusion yet — gating only):

```bash
python -m scripts.decode_live --ckpt checkpoints/phase3_5/rnnt_phase3_5.pt
# threshold 0.6 is the default; pass --confidence-threshold 0.7 for stricter
# noise rejection or 0.0 to disable.
```

### Limitations of v0.4.1

- **Fusion is offline only.** `decode_live.py` keeps the v0.4.0 streaming greedy path. Plumbing the LM through the central-zone-commit logic is a separate piece of work and was not validated in this release. Use `decode_audio.py` post-recording if you want the fusion gain on live captures.
- The Alice bench is ebook2cw (synthetic CW on real prose). The fusion gain on truly real-radio audio is expected to be at least as good — the LM addresses a text-distribution mismatch that exists regardless of channel — but is not directly measured here.
- ILME (`--ilm-weight > 0` in `eval_fusion`) actively hurts on this distribution. The fusion code keeps it for research / completeness but it is **not** the recommended path.

## Release v0.4.0

49-vocab tokenizer (adds **É / À / apostrophe**), Phase 3.4 French-rich prose curriculum, Phase 3.5 widened operator-jitter for live keying robustness. Five artifacts on [🤗 sderhy/morseformer](https://huggingface.co/sderhy/morseformer):

| file | params | vocab | description |
|---|---|---|---|
| `rnnt_phase3_5.pt` | 4.13 M | **49** | **v0.4 acoustic model** — Phase 3.4 (FR prose + extended É/À/' vocab) + Phase 3.5 (widened jitter). Recommended. |
| `rnnt_phase3_3.pt` | 4.13 M | 46 | v0.3 acoustic model — multilingual ASCII-normalised prose. |
| `rnnt_phase3_2.pt` | 4.13 M | 46 | v0.2 acoustic model — anti-hallucination curriculum. |
| `rnnt_phase3_0.pt` | 4.13 M | 46 | v0.1 acoustic model — AWGN-only training. |
| `lm_phase4_0.pt`   | 4.76 M | 46 | Character-level LM (research, unchanged from v0.1). |

Headline numbers on the French accent-rich bench (300 FR-prose samples sampled around É / À / apostrophe positions, evaluated through `scripts/eval_phase_3_4_french.py`):

```
  Metric                              v0.3 (46-vocab)   v0.4 (Phase 3.5)
  ----------------------------------------------------------------------
  Overall French CER                  n/a (no tokens)        6.46 %
  CER at clean / +20 / +10 / +5 dB    n/a                    0.00 %
  CER at 0 dB                         n/a                    4.99 %
  É  precision / recall               0 % / 0 %        100.0 % / 91.6 %
  À  precision / recall               0 % / 0 %         97.8 % / 95.7 %
  '  precision / recall               0 % / 0 %         98.4 % / 98.4 %
```

Live-validated on 2026-04-29 with an IC-7300 streaming decoder on FAV22 clair + Nerval *El Desdichado* + Poe *Annabel Lee*. Decoded `L'HEURE`, `L'AUTOMNE`, `MA SEULE ÉTOILE EST MORTE`, `À LA TOUR ABOLIE` with proper diacritics. No false-positive É / À in English passages. Residuals: occasional `WA → À` confusion at fast keying (the `.--.-` ambiguity) and recall on É below precision.

Realistic-channel SNR ladder (Phase 3.1 channel) is at parity with v0.3 — the curriculum changes only affect text mix and operator jitter. See [MODEL_CARD.md](MODEL_CARD.md) for the full v0.3 ladder retained as reference.

## Release v0.3.0

Phase 3.3 multilingual prose curriculum + streaming-decoder space-bug fix. Four artifacts on [🤗 sderhy/morseformer](https://huggingface.co/sderhy/morseformer):

| file | params | description |
|---|---|---|
| `rnnt_phase3_3.pt` | 4.13 M | **v0.3 acoustic model** — Phase 3.2 channel + 12 % multilingual prose (FR / DE / ES / EN) drawn from Project Gutenberg. Recommended. |
| `rnnt_phase3_2.pt` | 4.13 M | v0.2 acoustic model — anti-hallucination curriculum, no multilingual data. |
| `rnnt_phase3_0.pt` | 4.13 M | v0.1 acoustic model — AWGN-only training. |
| `lm_phase4_0.pt`   | 4.76 M | Character-level LM (research, unchanged from v0.1). |

Headline gains over v0.2 on the realistic-channel SNR ladder (1200 samples, 40 / WPM × 5 WPM × 6 SNR; full Phase 3.1 channel: QSB / QRN / QRM / carrier jitter / drift):

```
  SNR (dB) | v0.2 (3.2) CER | v0.3 (3.3) CER |  Δ
  ---------+----------------+----------------+--------
     +20.0 |   0.0017       |   0.0000       |  −0.17 pp
     +10.0 |   0.0000       |   0.0000       |   0.00 pp
      +5.0 |   0.0000       |   0.0000       |   0.00 pp
       0.0 |   0.0030       |   0.0030       |   0.00 pp
      −5.0 |   0.0542       |   0.0502       |  −0.40 pp
     −10.0 |   0.4442       |   0.3965       |  −4.77 pp
  ---------+----------------+----------------+--------
  overall  |   0.0838       |   0.0749       |  −0.89 pp
```

AWGN guard ladder (no QSB/QRN/QRM, pure AWGN — the regression bench from v0.2):

```
  SNR (dB) | v0.2 CER | v0.3 CER |  Δ
  ---------+----------+----------+----------
       0.0 |   0.0229 |   0.0099 |  −1.30 pp
      −5.0 |   0.3348 |   0.2943 |  −4.05 pp
     −10.0 |   0.8919 |   0.8826 |  −0.92 pp
  ---------+----------+----------+----------
  overall  |   0.2083 |   0.1978 |  −1.05 pp
```

The v0.2-introduced AWGN regression at low SNR is partly recovered in v0.3 — the multilingual prose appears to make the anti-hallucination prior less brittle on noise-shaped inputs.

False-positive bench (noise-only audio, mean characters emitted per 6 s sample, target = 0):

```
  metric                     v0.1     v0.2     v0.3
  --------------------------------------------------
  mean chars on noise       11.17     1.05     1.01
  fraction "letter-soup"   98.7 %    0.0 %    0.0 %
```

Streaming-decoder bug fix included in v0.3: `tokenizer.decode()` previously called `.strip()` on every output, dropping any inter-word space landing at a window boundary. Multi-word prose like `LE PONT MIRABEAU COULE LA SEINE` came back as `LEPONTMIRABEAUCOULELASEINE`. Fixed in `tokenizer.decode(strip=False)` for streaming callers (see commit `ecba5f9`); v0.1 and v0.2 also benefit if you rebuild from source.

Live-validated on a real IC-7300 with `scripts/decode_live.py` on a mixed-language stream of CQ macros + Verlaine *Chanson d'automne* + Poe *Annabel Lee* + Apollinaire *Le Pont Mirabeau* (2026-04-27) — see [MODEL_CARD.md](MODEL_CARD.md) for the full verdict and the failure modes earmarked for Phase 3.4.

## Release v0.2.0

Phase 3.2 anti-hallucination curriculum + real-time streaming CLI. Artifacts on [🤗 sderhy/morseformer](https://huggingface.co/sderhy/morseformer):

| file | params | description |
|---|---|---|
| `rnnt_phase3_2.pt` | 4.13 M | v0.2 acoustic model — Phase 3.1 realistic channel + 30 % random-text + 20 % 3-mode empty-audio curriculum. |
| `rnnt_phase3_0.pt` | 4.13 M | v0.1 acoustic model, kept for reference / reproducibility. |
| `lm_phase4_0.pt`   | 4.76 M | Character-level LM (research, unchanged from v0.1). |

Headline gains over v0.1 on the realistic-channel SNR ladder (Phase 3.1 channel: QSB / QRN / QRM / carrier jitter / drift, 300 samples):

```
  SNR (dB) |  v0.1 CER  |  v0.2 CER  |  Δ
  ---------+------------+------------+--------
     +20.0 |   0.2421   |   0.0067   | −24 pp
     +10.0 |   0.3816   |   0.0000   | −38 pp
      +5.0 |   0.3361   |   0.0000   | −34 pp
       0.0 |   0.5000   |   0.0040   | −50 pp
      −5.0 |   0.7871   |   0.0446   | −74 pp
     −10.0 |   1.3316   |   0.4703   | −86 pp
  ---------+------------+------------+--------
  overall  |   0.5964   |   0.0876   | −51 pp
```

False-positive bench (noise-only audio, mean characters emitted per 6 s sample, target = 0):

```
  metric                        v0.1     v0.2
  ----------------------------------------------
  mean chars on noise         11.17     1.05
  fraction "letter-soup" >5    98.7 %    0.0 %
```

See [MODEL_CARD.md](MODEL_CARD.md) for the full eval, including the AWGN-only regression at low SNR (a known side-effect of the strong anti-hallucination prior — see *Limitations*).

## Release v0.1.0

The release consists of three artifacts, all on [🤗 sderhy/morseformer](https://huggingface.co/sderhy/morseformer):

| file                        | params | purpose                                             |
|-----------------------------|--------|-----------------------------------------------------|
| `rnnt_phase3_0.pt`          | 4.13 M | **main acoustic model** — Conformer + RNN-T/CTC     |
| `lm_phase4_0.pt`            | 4.76 M | character-level language model (optional, research) |
| `MODEL_CARD.md`             | —      | architecture, training data, benchmarks, limits     |

The acoustic model is the primary release. The LM and fusion decoder are bundled for reproducibility — on the current synthetic distribution they do **not** improve CER (see [Phase 4.1 result](#phase-41--lm-fusion-does-not-help-on-synthetic-data) below), but shipping them makes the code auditable and leaves the door open for future work on real-text LM corpora.

## Benchmarks

**Phase 3.0 SNR-ladder** (1200 in-distribution validation samples: 40 per WPM × 5 WPM bins × 6 SNR; synthetic AWGN + 500 Hz RX filter; same operator-jitter distribution as training):

```
  SNR (dB) |  CER    |  WER    |  notes
  ---------+---------+---------+---------------------------------
     +20.0 |  0.0000 |  0.0000 |
     +10.0 |  0.0000 |  0.0000 |
      +5.0 |  0.0000 |  0.0000 |
       0.0 |  0.0000 |  0.0000 |
      −5.0 |  0.0280 |  0.0737 |
     −10.0 |  0.7620 |  0.9460 |  ← single bin dominates aggregate CER
  ---------+---------+---------+
  overall  |  0.1317 |  0.1718 |
```

RNN-T head vs CTC head on the same encoder: RNN-T wins by ~4 pp at −10 dB (the only bin where the two disagree). On SNR ≥ 0 dB the heads are tied at 0 %. See [`scripts/eval_fusion.py`](scripts/eval_fusion.py) to reproduce.

**Speed-up vs the rule-based baseline** (`python -m eval.cli --decoder rule_based`, same seed, same bench):

```
  SNR (dB) |  baseline CER  |  Phase 3.0 CER  |  speed-up
  ---------+----------------+-----------------+----------
       0.0 |      2.5355    |      0.0000     |    ∞
      −5.0 |      5.8327    |      0.0280     |  208 ×
     −10.0 |     19.4111    |      0.7620     |   25 ×
```

## Architecture

A compact 5-stage pipeline, fully local, CPU-real-time at inference:

```
audio ──▶ [1] DSP front-end (complex BPF at carrier)
      ──▶ [2] Conformer encoder (d=144, L=8, RoPE, 4× subsample)
      ──▶ [3] Dual heads: CTC (framewise) + RNN-T (prediction + joint)
      ──▶ [4] Optional LM fusion (shallow or ILME density-ratio)
      ──▶ text
```

- **Encoder**: 8-layer Conformer with RoPE attention, depth-wise conv module with LayerNorm, 4× time sub-sampling. ~3.9 M params. Shared between the CTC and RNN-T heads.
- **CTC head**: single linear on encoder output → per-frame vocab logits.
- **RNN-T head**: 128-dim LSTM prediction network + 256-dim joint network, blank at index 0. ~0.2 M params.
- **LM** (v0.1 release includes, but fusion gain is null — see Phase 4.1): decoder-only GPT (RMSNorm + SwiGLU + RoPE + tied embeddings + causal SDPA), d=256, L=6. ~4.8 M params. Trained to PPL 3.75 on the synthetic morseformer text distribution.

Vocabulary: 46 tokens (blank + 26 letters + 10 digits + 9 punctuation / Morse prosigns).

## Training data

All training audio in v0.1 is synthetic:

- **Text corpus**: a mix of random-callsign strings weighted by real ITU prefix-activity distributions (RBN-derived), contest QSO exchanges (CQ WW / ARRL-DX / SS formats), common Q-codes and abbreviations, RST reports, and ragchew fragments. See `morseformer/data/text.py`.
- **Waveform renderer**: parametric operator model (WPM, element / gap jitter), Morse-code keying with sine carrier, HF-channel simulator (AWGN, RX bandpass) — see `morse_synth/`.
- **Curriculum**: Phase 2.1 conditions — AWGN SNR ∈ U(0, 30) dB, 500 Hz RX filter at 600 Hz carrier, element jitter ∈ U(0, 0.05), gap jitter ∈ U(0, 0.10), WPM ∈ U(16, 28), fixed 6 s utterances at 8 kHz. Phase 3.0 trained 80 k steps on this.

No real-audio data is used in v0.1. Real-audio finetuning is on the Phase 3.1+ roadmap.

## Training history

<details>
<summary><b>Phase 0</b> — evaluation harness + rule-based DSP baseline</summary>

CER ranges from 0 at high SNR to ≈ 25 at −15 dB (threshold decoders hallucinate on noise). Used as the scoreboard the neural pipeline must beat. See `eval/` for the bench.
</details>

<details>
<summary><b>Phase 1</b> — HF-channel simulator + operator model</summary>

AWGN, QRN, QSB, carrier drift, RX filter. Feeds every subsequent phase. See `morse_synth/`.
</details>

<details>
<summary><b>Phase 2.0</b> — clean-audio acoustic baseline (CTC only)</summary>

Conformer + RoPE, 3.9 M params. 14 k steps on clean 6 s clips. **1.17 % CER** on balanced clean validation.
</details>

<details>
<summary><b>Phase 2.1</b> — noisy curriculum (CTC only)</summary>

Same arch, 50 k steps on AWGN ∈ U(0, 30) dB + RX filter + mild jitter. Beat the rule-based baseline by 33× at 0 dB, 7× at −5 dB, 16× at −10 dB. Encoder weights bootstrap Phase 3.
</details>

<details>
<summary><b>Phase 2.2</b> — wider-jitter ablation (archived)</summary>

Traded high-SNR precision for less hallucination at −10 / −15 dB. Kept as ablation; does not supersede 2.1.
</details>

### Phase 3.0 — RNN-T head + multi-task CTC / RNN-T training (v0.1 release)

Added a PredictionNetwork (single-layer LSTM, d_pred = 128) and a JointNetwork (d_joint = 256) on top of the Phase 2.1 Conformer encoder, trained multi-task with `ctc_weight = 0.3` and `rnnt_weight = 1.0` for 80 k steps. Encoder weights bootstrapped from `phase2_1/best_cer.pt`; CTC head re-used. Same curriculum as Phase 2.1, same AMP (bf16) config, EMA 0.9999. ~10 h wall time on an RTX 3060.

Result: **13.15 % overall CER**, with the RNN-T head beating the CTC head by 4 pp at the single −10 dB bin (where the RNN-T's sequence-level prior actually matters). At SNR ≥ 0 dB the two heads are tied at 0 %.

A d=144×L=16 scaling attempt (Phase 3-scale, 8 M params) tied with Phase 3.0 to within noise (13.16 % vs 13.15 %) — **capacity is not the bottleneck, the −10 dB AWGN bin is intrinsic to the synthetic distribution**.

### Phase 4.0 — character-level LM

A 4.76 M-param decoder-only transformer (RMSNorm, SwiGLU, RoPE, tied embeddings, causal SDPA), trained on the same synthetic text distribution as the acoustic model. Plateaued at val PPL 3.75 (step ~2.5 k); killed at step 8.5 k once it was clear the synthetic corpus was saturated. Released for research; not required for decoding.

### Phase 3.1 — realistic-channel fine-tune (intermediate, not released)

Phase 3.0 fine-tuned for 40 k steps on the full realistic synthetic channel: carrier-frequency jitter (±50 Hz), QSB (slow fading 0.05–1 Hz), QRN (atmospheric impulses), carrier drift, 25 % chance of a secondary CW signal at ±50–300 Hz offset (QRM), and a 5 % branch of empty-audio samples labelled with the empty string. Closed the realistic-channel CER from 59.64 % to 52.85 % (−7 pp), zero regression on the AWGN guard.

Live-tested on a real IC-7300 receiver: synthetic gain only partly transferred. The model still produced "letter-soup" (long runs of `E I S T` short-symbol sequences) on quiet bands and weak signals — **the actual release-blocker**. Phase 3.1 was not published.

### Phase 3.2 — anti-hallucination curriculum (v0.2 release)

Same architecture, fine-tuned 80 k steps from Phase 3.1 on a curriculum that targets the live failure mode directly: **30 % random A-Z / 0-9 / punctuation sequences** (no linguistic prior — breaks the model's tendency to fall back on plausible-English letter combinations on weak signal) and **20 % "no decodable signal" samples in three sub-modes** (pure AWGN, AWGN + atmospheric impulses, distant weak CW labelled empty — teaches "real signal but below the floor is still no signal"). Channel impairments unchanged from Phase 3.1.

Result: realistic-channel CER **52.85 % → 8.76 %** (−51 pp). Letter-soup hallucination on pure noise: **98.7 % → 0.0 %**. Live-validated on a real receiver across English / French poetry and contest fragments.

Trade-off: AWGN-only at −5 / −10 dB regresses (4 % → 37 %, 80 % → 91 %) because the anti-hallucination prior is now strong enough that the model emits blank when a signal is real but very faint. This is desirable for real-band audio (which always has propagation impairments — the realistic bench is meaningful) but visible on synthetic AWGN-only ladders. The v0.1 checkpoint stays available on HF for applications where the AWGN-only profile is preferred.

A real-time streaming decoder (`scripts/decode_live.py` v1) ships with v0.2: 6 s sliding window, 2 s hop, central-zone commit, ~4 s end-to-end latency. Replaces the v0.1 chunked decoder which stuttered at chunk boundaries (`CCCCQ`) and cut callsigns mid-word (`F4HY|Y`).

### Phase 4.1 — LM fusion does not help on synthetic data

Both shallow fusion (Gulati et al. 2020) and ILME / density-ratio fusion (Meng et al. 2021, arXiv:2011.01991) were implemented on top of the RNN-T greedy decoder and swept over the SNR ladder:

```
  λ_lm  |  λ_ilm  |  overall CER  |  CER at −10 dB
  ------+---------+---------------+----------------
  0.00  |  0.00   |    13.17 %    |    76.20 %     (baseline, n = 1200)
  0.20  |  0.00   |    13.26 %    |    76.70 %     (shallow)
  0.30  |  0.00   |    13.09 %    |    75.63 %     (shallow, best λ)
  0.20  |  0.20   |    13.65 %    |    78.37 %     (ILME, n = 150)
  0.50  |  0.50   |    21.19 %    |   116.52 %     (ILME, aggressive)
```

The λ curve is flat inside the ±0.3 pp noise floor at n = 1200 — **fusion gives no measurable gain** on this distribution. ILME is actively harmful at −10 dB.

Root cause: the LM and the RNN-T prediction network see **exactly the same synthetic text corpus**. Standard ASR results where external-LM fusion wins rely on a domain shift between training text and eval text, which we do not have here. The fusion code ships as a documented capability; expect it to start helping once v0.2 adds a ham-realistic LM corpus (QSO logs, contest exchanges, RBN spots) distinct from the acoustic model's training text.

## Roadmap

- [x] **Phase 0** — evaluation harness + rule-based baseline
- [x] **Phase 1** — operator model + HF-channel simulator
- [x] **Phase 2.0** — clean-audio CTC baseline (1.17 % CER)
- [x] **Phase 2.1** — noisy CTC curriculum (beats rule-based 7-33× on SNR ladder)
- [x] **Phase 2.2** — wider-jitter ablation (archived)
- [x] **Phase 3.0** — RNN-T head, multi-task training — **v0.1 release**
- [x] **Phase 3-scale** — d×L ablation; capacity is not the bottleneck
- [x] **Phase 4.0** — character-level LM (PPL 3.75)
- [x] **Phase 4.1** — LM fusion (shallow + ILME); neutral on synthetic data
- [x] **Phase 3.1** — realistic synthetic channel (QRM, QRN, selective fading) — fine-tune; live-validated, not released
- [x] **Phase 3.2** — anti-hallucination curriculum (random text + 3-mode empty audio) — **v0.2 release**
- [x] **Phase 6** — real-time streaming CLI (`scripts/decode_live.py` v1, sliding window + central-zone commit) — shipped with v0.2
- [x] **Phase 3.3** — multilingual prose corpus (FR / DE / ES / EN, ASCII-normalised) — **v0.3 release**; closes the English-prior bias on French live audio
- [ ] **Phase 3.4** — extend the 46-token vocabulary to 49 tokens with the French-specific Morse codes for `é` (`..-..`), `à` (`.--.-`), and apostrophe (`.----.`); retrain from scratch. Addresses the live-test failure where `L'AUTOMNE` is decoded as `L1AUTOMNE`.
- [ ] **Phase 4.2** — ham-realistic LM corpus (RBN spots, contest logs, real QSO transcripts) — prerequisite for fusion gains
- [ ] **Phase 5** — real-audio finetuning at scale (W1AW aligned transcripts + RBN/SDR pairings + user IC-7300 captures)
- [ ] **Phase 7** — callsign-aware beam search with ITU prefix priors

## License

Apache 2.0 — see [LICENSE](LICENSE). The released model weights are distributed under the same license.

## Acknowledgements

- **Sébastien Derhy** — design, engineering, and on-air validation of morseformer
- **Mauri Niininen (AG1LE)** — pioneering ML-based CW decoding work
- **Alex Shovkoplyas (VE3NEA)** — CW Skimmer, the commercial reference
- **Andrej Karpathy** — `nanoGPT`, the aesthetic reference for the language model
- **Project Gutenberg** — public-domain literary texts in English, French, German, and Spanish used to build the Phase 3.3 multilingual prose corpus
- The amateur-radio community — decades of publicly available CW recordings and transcripts

---

*73 de morseformer*
