Metadata-Version: 2.4
Name: arabinlp
Version: 2.1.1
Summary: A modern Python toolkit for Arabic Natural Language Processing (NLP).
Author: Ibtihal Makki
License: MIT
Project-URL: Homepage, https://github.com/IbtihalMakki/arabic-nlp-toolkit
Project-URL: Repository, https://github.com/IbtihalMakki/arabic-nlp-toolkit
Project-URL: Issues, https://github.com/IbtihalMakki/arabic-nlp-toolkit/issues
Keywords: arabic,arabic-nlp,nlp,text-processing,natural-language-processing,machine-learning,generative-ai
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# ArabiNLP

A lightweight, dependency-free Python toolkit for Arabic Natural Language Processing (NLP), providing preprocessing, normalization, morphology, stemming, lemmatization, spell checking, and information retrieval.

---

# Version 2.1.0

## Features

- Text cleaning
- Arabic normalization
- Tokenization
- Stopword removal
- Sentence splitting
- Text statistics
- Word and character n-grams
- Keyword extraction
- Language detection
- Jaccard similarity
- Arabic Light Stemmer
- Root Extraction
- Lemmatization
- Morphological Segmentation
- Morphological Analysis
- Arabic Spell Checker
- TF-IDF Vectorizer
- BM25 Ranking

---

# Installation

```bash
pip install arabinlp
```

---

# Quick Start

```python
from arabinlp import (
    normalize_arabic,
    remove_stopwords,
    tokenize,
    ArabicLightStemmer,
)

text = "أنا أحب تعلم الذكاء الاصطناعي في الجامعات."

# Normalize
text = normalize_arabic(text)

# Remove stopwords (expects text, returns filtered tokens)
filtered_tokens = remove_stopwords(text)

# Tokenize after converting tokens back to text
tokens = tokenize(" ".join(filtered_tokens))

# Stem
stemmer = ArabicLightStemmer()
stems = [stemmer.stem(token) for token in tokens]

print(stems)
```

Expected output

```text
['احب', 'تعلم', 'ذكاء', 'اصطناع', 'جامع']
```

---

# End-to-End NLP Pipeline

```python
from arabinlp import (
    normalize_arabic,
    remove_stopwords,
    tokenize,
    ArabicLightStemmer,
    ArabicRootExtractor,
    ArabicLemmatizer,
)

text = "والمكتبات جميلة"

text = normalize_arabic(text)
filtered_tokens = remove_stopwords(text)
tokens = tokenize(" ".join(filtered_tokens))

stemmer = ArabicLightStemmer()
extractor = ArabicRootExtractor()
lemmatizer = ArabicLemmatizer()

for token in tokens:
    print(
        token,
        stemmer.stem(token),
        extractor.extract(token),
        lemmatizer.lemmatize(token),
    )
```

---

# Python Support

- Python >= 3.9

Supported versions

- Python 3.9
- Python 3.10
- Python 3.11
- Python 3.12

---

# Project Links

Homepage

https://github.com/IbtihalMakki/arabic-nlp-toolkit

Repository

https://github.com/IbtihalMakki/arabic-nlp-toolkit

Issues

https://github.com/IbtihalMakki/arabic-nlp-toolkit/issues

---

# API Overview

| API | Description |
|------|-------------|
| preprocess | Complete preprocessing pipeline |
| normalize_arabic | Normalize Arabic text |
| tokenize | Tokenization |
| remove_stopwords | Stopword removal |
| split_sentences | Sentence segmentation |
| word_frequency | Word frequency |
| ngrams | Word n-grams |
| character_ngrams | Character n-grams |
| detect_language | Language detection |
| text_statistics | Text statistics |
| extract_keywords | Keyword extraction |
| jaccard_similarity | Text similarity |
| ArabicLightStemmer | Light stemming |
| ArabicRootExtractor | Root extraction |
| ArabicLemmatizer | Lemmatization |
| ArabicSegmenter | Morphological segmentation |
| ArabicMorphAnalyzer | Morphological analysis |
| ArabicSpellChecker | Spell checker |
| ArabicTfidfVectorizer | TF-IDF vectorizer |
| ArabicBM25 | BM25 ranking |

---

# Core Preprocessing

```python
from arabinlp import (
    preprocess,
    normalize_arabic,
    tokenize,
    remove_stopwords,
    split_sentences,
    word_frequency,
    ngrams,
    character_ngrams,
    detect_language,
    text_statistics,
    extract_keywords,
    jaccard_similarity,
)

text = "السَّلَامُ عَلَيْكُمْ 😊 زوروا https://example.com"

cleaned = preprocess(text)

normalized = normalize_arabic(
    "إسلام وآثار",
    level="light",
)

tokens = tokenize(
    "السلام عليكم ورحمة الله",
)

filtered = remove_stopwords(
    "أنا أحب تعلم الذكاء الاصطناعي في الجامعة",
)

sentences = split_sentences(
    "السلام عليكم. كيف حالك؟ أنا بخير!",
)

freq = word_frequency(
    "الذكاء الذكاء الاصطناعي رائع",
)

word_bigrams = ngrams(
    "السلام عليكم ورحمة الله",
    2,
)

char_trigrams = character_ngrams(
    "السلام",
    3,
)

lang = detect_language(
    "Hello مرحبا",
)

stats = text_statistics(
    "السلام عليكم. كيف حالك؟",
)

keywords = extract_keywords(
    "الذكاء الاصطناعي رائع. الذكاء يتطور بسرعة.",
    top_k=2,
)

sim = jaccard_similarity(
    "أنا أحب الذكاء الاصطناعي",
    "الذكاء الاصطناعي رائع",
)
```

---

# Cleaning Utilities

```python
from arabinlp import (
    remove_urls,
    remove_mentions,
    remove_hashtags,
    remove_extra_spaces,
    remove_diacritics,
    remove_tatweel,
    remove_emojis,
)

print(remove_urls("Visit https://example.com"))

print(remove_mentions("@ibtihal hello"))

print(remove_hashtags("#AI is awesome"))

print(remove_extra_spaces("Hello     World"))

print(remove_diacritics("السَّلَامُ عَلَيْكُمْ"))

print(remove_tatweel("الســــلام"))

print(remove_emojis("Hello 😊"))
```

---

# Stemming

```python
from arabinlp import ArabicLightStemmer

stemmer = ArabicLightStemmer()

print(stemmer.stem("والمكتبات"))
```

Expected output

```text
مكتب
```

---

# Root Extraction

```python
from arabinlp import ArabicRootExtractor

extractor = ArabicRootExtractor()

print(extractor.extract("المكتبات"))
```

Expected output

```text
كتب
```

---

# Lemmatization

```python
from arabinlp import ArabicLemmatizer

lemmatizer = ArabicLemmatizer()

print(lemmatizer.lemmatize("المهندسون"))

print(
    lemmatizer.lemmatize_sentence(
        "المهندسون في الجامعات",
    )
)

print(
    lemmatizer.analyze(
        "المهندسون",
    )
)
```

---

# Morphological Analysis

```python
from arabinlp import (
    ArabicSegmenter,
    ArabicMorphAnalyzer,
)

segmenter = ArabicSegmenter()

print(segmenter.segment("وبالمكتبات"))

print(
    segmenter.segment_sentence(
        "وبالمكتبات كتابهما",
    )
)

analyzer = ArabicMorphAnalyzer()

print(analyzer.analyze("المكتبات"))

print(
    analyzer.analyze_sentence(
        "وبالمكتبات كتابهما",
    )
)
```

---

# Spell Checking

```python
from arabinlp import ArabicSpellChecker

checker = ArabicSpellChecker()

print(checker.is_correct("رائع"))

print(
    checker.distance(
        "راءع",
        "رائع",
    )
)

print(checker.suggest("راءع"))

print(
    checker.correct(
        "اللغه العربيه جميله",
    )
)
```

Expected output

```text
اللغة العربية جميلة
```

---

# TF-IDF

```python
from arabinlp import ArabicTfidfVectorizer

docs = [
    "أنا أحب الذكاء الاصطناعي",
    "الذكاء الاصطناعي يعالج اللغة العربية",
    "معالجة اللغة الطبيعية مجال مهم",
]

vectorizer = ArabicTfidfVectorizer(
    ngram_range=(1, 2),
    min_df=1,
)

matrix = vectorizer.fit_transform(docs)

print(vectorizer.vocabulary_)
print(matrix)
```

---

# BM25

```python
from arabinlp import ArabicBM25

docs = [
    "أنا أحب الذكاء الاصطناعي",
    "الذكاء الاصطناعي يعالج اللغة العربية",
    "معالجة اللغة الطبيعية مجال مهم",
]

bm25 = ArabicBM25()

bm25.fit(docs)

print(
    bm25.get_scores(
        "الذكاء الاصطناعي",
    )
)

print(
    bm25.search(
        "الذكاء الاصطناعي",
        top_k=2,
    )
)
```

---

# Examples

Additional runnable examples are available in the GitHub repository:

- `examples/showcase.py`
- `examples/sample.txt`
- `examples/sample_errors.txt`

Repository:

```
https://github.com/IbtihalMakki/arabic-nlp-toolkit
```

---

# Running Tests

The automated test suite is available in the GitHub repository.

```bash
git clone https://github.com/IbtihalMakki/arabic-nlp-toolkit
cd arabic-nlp-toolkit
python -m pytest
```

---

## Current Features (v2.0.0)

<<<<<<< HEAD
✅ Arabic text preprocessing

- Text cleaning
- Arabic normalization
- Tokenization
- Stopword removal
- Sentence splitting
- Word frequency
- N-grams
- Character N-grams
- Keyword extraction
- Language detection
- Text statistics
- Jaccard similarity

✔ 40 unit tests
✔ GitHub Actions CI
✔ PyPI package
=======
## Implemented (v2.1.0)

- Complete preprocessing pipeline
- Arabic normalization
- Cleaning utilities
- Tokenization
- Stopword removal
- Morphological segmentation
- Morphological analysis
- Arabic Light Stemmer
- Root Extraction
- Lemmatization
- Spell Checker
- TF-IDF
- BM25

All public APIs exported from `arabinlp.__init__` are implemented and covered by automated tests.

---

## Planned

- Named Entity Recognition (NER)
- AraBERT integration
- Additional pretrained Arabic NLP pipelines
- Larger benchmarking datasets
>>>>>>> 90403ac (Fix README examples and documentation for v2.1.0)

---

# Contributing

Contributions, bug reports, feature requests, and pull requests are welcome.

---

# License

<<<<<<< HEAD
This project is licensed under the MIT License.

---

# Author

**Ibtihal Makki**

AI Engineer • NLP Researcher • Generative AI
=======
MIT License.
>>>>>>> 90403ac (Fix README examples and documentation for v2.1.0)
