Metadata-Version: 2.2
Name: termflow-ir
Version: 0.1.2
Summary: English text analysis for information retrieval
Keywords: text-analysis,information-retrieval,tokenization,stemming,search
Author: Mustafa Abualsaud
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: C++
Classifier: Topic :: Text Processing :: Linguistic
Project-URL: Homepage, https://github.com/gathera/termflow
Project-URL: Documentation, https://github.com/gathera/termflow/blob/main/docs/installation.md
Project-URL: Repository, https://github.com/gathera/termflow
Project-URL: Issues, https://github.com/gathera/termflow/issues
Project-URL: Releases, https://github.com/gathera/termflow/releases
Requires-Python: >=3.10
Description-Content-Type: text/markdown

# termflow

English text analysis for information retrieval workloads in C++ and Python.

[![PyPI](https://img.shields.io/pypi/v/termflow-ir.svg)](https://pypi.org/project/termflow-ir/)
[![Python Versions](https://img.shields.io/pypi/pyversions/termflow-ir.svg)](https://pypi.org/project/termflow-ir/)
[![Python Package](https://github.com/gathera/termflow/actions/workflows/python-package.yml/badge.svg)](https://github.com/gathera/termflow/actions/workflows/python-package.yml)

`termflow` is a library-first analysis stack for search, indexing, tagging, and query normalization. It provides a built-in English analyzer, term extraction helpers, and a lightweight query rewrite layer without trying to be a full search engine.

## Why termflow

- C++20 core library with optional Python bindings
- English analyzer with configurable stemming, stop words, possessive handling, and ASCII folding
- Term extraction API for finalized search/index terms
- Query parser and rewrite support for canonicalization, equivalents, and expansions
- Installable Python wheels for Linux and macOS
- CMake install flow for downstream C++ consumers

## Install

Python package:

```bash
pip install termflow-ir
```

Python import:

```python
import termflow
```

CLI quick check:

```bash
termflow analyze "The Running Cars"
```

For C++ installation and `find_package(termflow)` usage, see [docs/installation.md](docs/installation.md).

## Quick Start

Python:

```python
import termflow

analyzer = termflow.EnglishAnalyzer()
terms = analyzer.analyze_terms("The Running Cars")
normalized = analyzer.normalize("Running Café")

print(terms)  # ['run', 'car']
print(normalized)  # 'running café'
```

C++:

```cpp
#include <iostream>
#include "termflow/analysis/english_analyzer.hpp"

int main() {
  termflow::EnglishAnalyzer analyzer;
  const auto terms = analyzer.analyze_terms("The Running Cars");

  for (const auto& term : terms) {
    std::cout << term << "\n";
  }
}
```

## Features

| Area | What it includes |
| --- | --- |
| Analysis | `EnglishAnalyzer`, token analysis, normalization, stemming, stop words, ASCII folding |
| Term extraction | `TermExtractor` with length, numeric, and character-policy filtering |
| Query processing | clause parsing, analyzed query terms, rewrite loading, validation, and alternatives |
| Python bindings | built-in analyzer, term extractor, and query module under `termflow.query` |
| CLI | `termflow analyze`, `termflow extract`, and `termflow analyze-query` for quick validation |
| C++ consumption | installable CMake package and external `find_package` example |

## Documentation

- [docs/usage.md](docs/usage.md) for day-to-day analyzer, term extraction, query, and Python usage
- [docs/customization.md](docs/customization.md) for pipeline tuning, query rewrites, and custom analyzers in C++
- [docs/installation.md](docs/installation.md) for Python and C++ installation paths
- [docs/installation-roadmap.md](docs/installation-roadmap.md) for packaging and distribution priorities

Runnable examples:

- [examples/analyze_text.cpp](examples/analyze_text.cpp)
- [examples/extract_terms.cpp](examples/extract_terms.cpp)
- [examples/custom_analyzer.cpp](examples/custom_analyzer.cpp)
- [examples/analyze_query.cpp](examples/analyze_query.cpp)
- [examples/find_package_consumer/CMakeLists.txt](examples/find_package_consumer/CMakeLists.txt)

## Scope

`termflow` currently focuses on:

- English text analysis
- Batch-oriented APIs
- Query parsing and rewrite preparation
- Reusable components for embedding in larger applications

`termflow` does not currently provide:

- indexing or retrieval
- ranking or scoring
- token graphs
- phrase execution logic
- multilingual analyzers

## Build From Source

Local build:

```bash
cmake -S . -B build -G Ninja
cmake --build build
ctest --test-dir build --output-on-failure
```

Build Python bindings from source:

```bash
cmake -S . -B build -G Ninja -DTERMFLOW_BUILD_PYTHON=ON
cmake --build build
PYTHONPATH=build/python python3 -c 'import termflow; print(termflow.EnglishAnalyzer().analyze_terms("Running Cars"))'
```

Build Python distributions:

```bash
python3 -m build --sdist --wheel
python3 -m twine check dist/*
```

## Project Status

`termflow` is early-stage and intentionally narrow in scope. The current focus is making the built-in English analysis and packaging story solid before expanding into more languages or broader IR features.

## Contributing

Issues and pull requests are welcome. If you want to make a larger API or packaging change, open an issue first so the direction is clear before implementation work starts.

## License

This repository does not yet include a `LICENSE` file. Until that is added, do not assume open source usage terms.
