Metadata-Version: 2.4
Name: LIAS_Labrary
Version: 0.2.0
Summary: A lightweight Python library for preprocessing Arabic dialect text written in Arabic script.
Author: Hasnae Sakhi
License: MIT
Keywords: arabic,dialect,darija,nlp,preprocessing,sentiment-analysis
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: pandas>=2.0

# LIAS_Labrary

**LIAS_Labrary** is a lightweight Python library for preprocessing Arabic dialect text written in **Arabic script**.  
It is designed to support both:

- **single-text preprocessing**
- **dataset / DataFrame column preprocessing**

The library focuses on **safe and practical preprocessing operations** commonly needed in Arabic dialect NLP and sentiment analysis workflows, while preserving the original Arabic letter forms.

---

## Main Features

LIAS_Labrary currently supports:

### Core text preprocessing
- Remove Arabic diacritics
- Remove tatweel (`ـ`)
- Reduce repeated letters
- Reduce repeated punctuation
- Normalize whitespace
- Remove or preserve selected special characters
- Convert digits between:
  - Western digits: `0 1 2 3 4 5 6 7 8 9`
  - Eastern Arabic digits: `٠ ١ ٢ ٣ ٤ ٥ ٦ ٧ ٨ ٩`

### Emoji handling
- `keep`
- `remove`
- `replace`

When using `replace`, users can choose:
- replacement position:
  - `inline`
  - `append`
  - `prepend`
- output format:
  - `plain`
  - `tagged`

### URL handling
- `keep`
- `remove`
- `replace`

When using `replace`, users can choose:
- replacement position:
  - `inline`
  - `append`
  - `prepend`
- output format:
  - `plain`
  - `tagged`
- replacement token:
  - default: `رابط`
  - customizable to any language or token

### Hashtag handling
- `keep`
- `normalize`
- `remove`

Example:
- `#هاد الفيلم زوين` → `هاد الفيلم زوين`

### HTML cleaning
- Remove HTML tags for web scraping / crawling / extracted text scenarios

### Dataset-level preprocessing
For each operation, LIAS_Labrary also provides `*_data` functions that:
- process a selected column in a pandas DataFrame
- preserve the full dataset
- return the dataset with the processed column added or updated

---

## Design Principles

LIAS_Labrary follows a conservative preprocessing philosophy:

- It **preserves original Arabic letters**
- It does **not perform aggressive Arabic letter normalization**
- It avoids transformations such as:
  - `أ / إ / آ → ا`
  - `ى → ي`
  - `ة → ه`
  - simplification of `ؤ / ئ`

This choice helps preserve orthographic information for:
- pretrained language models
- later lemmatization
- future transcoding or advanced linguistic processing

---

## Installation

### From PyPI
```bash
pip install LIAS_Labrary
