Metadata-Version: 2.1
Name: xinaprocessor
Version: 0.4
Summary: Xina processing library
Home-page: https://github.com/xina-ai/xinaprocessor
Author: Xina AI
License: UNKNOWN
Platform: UNKNOWN
Description-Content-Type: text/x-rst
Requires-Dist: emoji (==0.6.0)
Requires-Dist: tqdm (==4.55.0)

Xina Processor
----------------
.. image:: ./imgs/logo.png
   :width: 400
   :alt: Xina AI

Xina Processor is an open source library for cleaning Arabic text. It includes various cleaning functions as well as modules for streaming file and folder cleaning.

Installation
==============

=====
PIP
=====

If you use `pip`, you can install xinaprocessor with:

.. code:: bash

   pip install xinaprocessor

==============
From source
==============

You can directly clone this repo and install the library. First clone the repo with:

.. code:: bash

   git clone https://github.com/xina-ai/xinaprocessor.git

Then cd to the directory and install the library with:

.. code:: bash

   pip install -e .

Documentation
================

Documentation is still in process here: https://xina-ai.github.io/xinaprocessor/


Getting Started
================

.. code:: python

   from xinaprocessor import cleaners 



To clean text

.. code:: python

   Text = "نص عربي!"
   Cleaner = cleaners.TextCleaner(text=Text)
   Cleaner.keep_arabic_only()



To clean text File

.. code:: python

   # Creating File MyData.txt
   FilePath = "MyData.txt"
   with open(FilePath, "w") as f:
      f.write("Aالسطر الأول\nالسطر الثاني!")
   # Creating FileCleaner object
   Cleaner = cleaners.FileCleaner(filepath=FilePath)
   Cleaner.remove_english_text().remove_arabic_numbers().remove_punctuations()
   # To access the resulted data
   CleanedData = Cleaner.lines # the result will look like ['السطر الأول', 'السطر الثاني']
   CleanedText = Cleaner.text # the result will look like 'السطر الأول\nالسطر الثاني'
   # To save the proccessed/cleaned text to a file 
   Cleaner.save2file('CleanedData.txt', encoding='utf-8')

To clean large text File

.. code:: python

   # This Cleaner is used for large text files, the cleaned texts will be saved to CleanedFile.txt file
   FilePath = "MyData.txt"
   CleanedPath = "CleanedFile.txt"
   Cleaner = cleaners.FileStreamCleaner(filepath=FilePath, savepath=CleanedPath)
   Cleaner.remove_hashtags().remove_honorific_signs().drop_empty_lines().clean()



