Metadata-Version: 2.5
Name: pantogloss
Version: 0.19.0
Summary: TensorFlow/Keras many-to-English machine translation
Project-URL: Homepage, https://github.com/chrismattmann/pantogloss
Project-URL: Repository, https://github.com/chrismattmann/pantogloss
Project-URL: Model repository, https://huggingface.co/chrismattmann/pantogloss-500-en
Project-URL: Compact model repository, https://huggingface.co/chrismattmann/pantogloss-500-en-compact
Project-URL: FP16 model repository, https://huggingface.co/chrismattmann/pantogloss-500-en-fp16
Project-URL: INT8 model repository, https://huggingface.co/chrismattmann/pantogloss-500-en-int8
Project-URL: Issues, https://github.com/chrismattmann/pantogloss/issues
Author: Chris A. Mattmann
License: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: keras,machine translation,multilingual,tensorflow
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: huggingface-hub<2,>=0.26
Requires-Dist: marisa-trie<2,>=1.2
Requires-Dist: nlcodec<0.6,>=0.5
Requires-Dist: numpy<3,>=1.26
Requires-Dist: sacremoses<0.3,>=0.2
Requires-Dist: tensorflow<2.19,>=2.18; sys_platform == 'darwin' and platform_machine == 'arm64'
Requires-Dist: tensorflow<2.22,>=2.18; sys_platform != 'darwin' or platform_machine != 'arm64'
Provides-Extra: conversion
Requires-Dist: ruamel-yaml>=0.17; extra == 'conversion'
Requires-Dist: torch<3,>=2.2; extra == 'conversion'
Provides-Extra: cuda
Requires-Dist: tensorflow[and-cuda]<2.22,>=2.18; (sys_platform == 'linux') and extra == 'cuda'
Provides-Extra: evaluation
Requires-Dist: sacrebleu<3,>=2.4; extra == 'evaluation'
Provides-Extra: metal
Requires-Dist: tensorflow-metal<1.3,>=1.2; (sys_platform == 'darwin' and platform_machine == 'arm64') and extra == 'metal'
Requires-Dist: tensorflow<2.19,>=2.18; (sys_platform == 'darwin' and platform_machine == 'arm64') and extra == 'metal'
Provides-Extra: server
Requires-Dist: fastapi<1,>=0.115; extra == 'server'
Requires-Dist: uvicorn<1,>=0.34; extra == 'server'
Provides-Extra: test
Requires-Dist: fastapi<1,>=0.115; extra == 'test'
Requires-Dist: httpx<1,>=0.27; extra == 'test'
Requires-Dist: pypdf<7,>=6.16; extra == 'test'
Requires-Dist: pytest-cov<8,>=5; extra == 'test'
Requires-Dist: pytest<10,>=8; extra == 'test'
Requires-Dist: sacrebleu<3,>=2.4; extra == 'test'
Provides-Extra: tika
Requires-Dist: pypdf<7,>=6.16; extra == 'tika'
Requires-Dist: tika<4,>=3.3; extra == 'tika'
Description-Content-Type: text/markdown

# Pantogloss

[![Tests](https://github.com/chrismattmann/pantogloss/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/chrismattmann/pantogloss/actions/workflows/test.yml)
[![PyPI](https://img.shields.io/pypi/v/pantogloss.svg)](https://pypi.org/project/pantogloss/)
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](LICENSE)

Pantogloss is a TensorFlow/Keras library that translates text from many
languages into English. It provides a Python API, command-line interface, and
optional persistent local server with a browser UI.

The default public model is
[`pantogloss-500-en-compact`](https://huggingface.co/chrismattmann/pantogloss-500-en-compact),
a 538 MiB model validated on NVIDIA CUDA and Apple Metal. Model weights are
downloaded separately from Hugging Face and are never bundled in the Python
wheel. Public models normally require no Hugging Face login.

## What it can do

- Translate individual strings, batches, files, and bounded input streams.
- Run on CPU, NVIDIA CUDA, or Apple Silicon Metal.
- Keep a model resident behind a local HTTP API for fast repeated requests.
- Provide a simple Any language → English browser translator.
- Offer fast greedy and higher-quality beam-search decoding presets.
- Return plain strings or structured results with timing and runtime metadata.
- Attach caller-supplied language quality guidance and conservative warnings.
- Preserve input order and isolate individual failures in long-running streams.

Pantogloss translates text; it does not detect the source language or parse
document formats. See [Project scope](#project-scope).

## Installation

Pantogloss supports Python 3.10–3.12. Use the extra for your accelerator:

```bash
# CPU
python -m pip install pantogloss

# NVIDIA GPU on Linux
python -m pip install "pantogloss[cuda]"

# Apple Silicon Metal
python -m pip install "pantogloss[metal]"

# Persistent API and browser UI; combine with cuda or metal when needed
python -m pip install "pantogloss[server,metal]"
```

The first use downloads the selected model. Later loads use the local
Hugging Face cache.

## Python API

Load one translator and reuse it:

```python
from pantogloss import Translator

translator = Translator.from_pretrained(device="auto")
print(translator.translate("Comment allez-vous ?"))
```

Translate a batch or request structured results:

```python
translations = translator.translate([
    "Hola señor",
    "Wie geht es Ihnen?",
])

result = translator.translate_detailed("Bonjour le monde.")
print(result.text)
print(result.elapsed_seconds, result.execution_device)
```

For a large iterable, `translate_iter()` and `translate_iter_detailed()` process
bounded batches without retaining the complete input or output collection.

```python
with open("source.txt", encoding="utf-8") as source:
    for translation in translator.translate_iter(source, batch_size=16):
        print(translation)
```

Use the quality preset when latency is less important than beam-search quality:

```python
print(translator.translate("Comment allez-vous ?", preset="quality"))
```

## Command line

```bash
pantogloss translate "Comment allez-vous ?"
pantogloss translate --preset quality "Comment allez-vous ?"
pantogloss translate --input source.txt --output english.txt --batch-size 16
pantogloss models
pantogloss info --device gpu --json
pantogloss doctor
```

Translation output stays clean on stdout. Use `--verbose`, `--report`, or
`--tensorflow-logs` when diagnostics are wanted. Use `--offline` to require an
already cached model.

## Persistent server and browser UI

Each standalone CLI invocation reloads the model. For repeated requests, keep
one warmed translator in memory:

```bash
python -m pip install "pantogloss[server,metal]"  # or server,cuda
pantogloss serve --device gpu
```

Open `http://127.0.0.1:8765/` for the two-pane browser translator. The same
process exposes:

- `GET /health`
- `GET /info`
- `GET /metrics`
- `POST /translate`

```bash
curl http://127.0.0.1:8765/translate \
  -H 'Content-Type: application/json' \
  -d '{"text":"Comment allez-vous ?"}'
```

The server binds to loopback, warms the model before reporting readiness, and
serializes inference by default. Queuing is bounded, shutdown drains active
TensorFlow work, and optional request logs contain timing/count metadata rather
than source or translated text. On Apple Silicon, the server defaults to the
bounded-memory eager decoder because TensorFlow Metal retains memory during
compiled decoding; `--metal-compiled-decode` is an explicit faster but
memory-growing opt-in. See the
[Server and Web UI wiki page](https://github.com/chrismattmann/pantogloss/wiki/Server-and-Web-UI)
for deployment, authentication, limits, API schemas, and benchmarks.

## Models

| Name | Role | Approximate artifact size |
| --- | --- | ---: |
| `pantogloss-500-en-compact` | Recommended default | 538 MiB |
| `pantogloss-500-en-fp16` | Accelerator-oriented FP16 reference | 1.02 GiB |
| `pantogloss-500-en` | Historical FP32 numerical reference | 2.03 GiB |
| `pantogloss-500-en-int8` | Smaller experimental alternative | 539 MiB |

Select a model explicitly when needed:

```python
translator = Translator.from_pretrained("pantogloss-500-en-fp16", device="gpu")
```

The compact model retains the best tested balance of size, quality, memory, and
speed. The INT8 alternative is not the default because it is substantially
slower. Detailed evidence and backend caveats live in the wiki and checked-in
experiment reports.

## Documentation

- [Installation and Quickstart](https://github.com/chrismattmann/pantogloss/wiki/Installation-and-Quickstart)
- [Server and Web UI](https://github.com/chrismattmann/pantogloss/wiki/Server-and-Web-UI)
- [Decoding Presets](https://github.com/chrismattmann/pantogloss/wiki/Decoding-Presets)
- [Platform Validation](https://github.com/chrismattmann/pantogloss/wiki/Platform-Validation)
- [Language Quality Catalog](https://github.com/chrismattmann/pantogloss/wiki/Language-Quality-Catalog)
- [Quality Evaluation](https://github.com/chrismattmann/pantogloss/wiki/Quality-Evaluation)
- [Development and Testing](https://github.com/chrismattmann/pantogloss/wiki/Development-and-Testing)
- [Tika Document Translation](https://github.com/chrismattmann/pantogloss/wiki/Tika-Document-Translation)

The repository also retains reproducible model-conversion, parity, compression,
and evaluation artifacts under [`docs`](docs) and [`evaluation`](evaluation).

## Project scope

Pantogloss translates text and ordered text segments. It intentionally does not
identify languages, detect file types, or parse documents. `DocumentTranslator`
is a neutral text-segmentation helper. `pantogloss-tika` remains a compatibility
example and is not a direction for new core dependencies. See the
[architecture boundary](docs/architecture.md).

Translation quality varies by language, domain, and input. Pantogloss does not
provide calibrated confidence and should not be relied on without review for
medical, legal, safety-critical, or other high-stakes decisions.

## Provenance and license

The original model was described by Thamme Gowda, Zhao Zhang, Chris A. Mattmann,
and Jonathan May in *Many-to-English Machine Translation Tools, Data, and
Pretrained Models*, ACL-IJCNLP 2021 System Demonstrations,
[DOI 10.18653/v1/2021.acl-demo.37](https://doi.org/10.18653/v1/2021.acl-demo.37).

Pantogloss and its converted models are licensed under the
[Apache License, Version 2.0](LICENSE). See [NOTICE](NOTICE) for attribution.
