.. _dataproc:

DataProcessing
==============

This module contains several transformer classes that aim to change or mitigate certain aspects of a dataset.
The goal of this module is to provide a unified interface for different mitigation methods scattered around
multiple machine learning libraries, such as scikit-learn, mlxtend, sdv, among others. The advantage of using
this module is that it allows users to create their data processing pipelines using a single library, without
worrying about making sure that the output from one mitigation (found in one library) is compatible with the
output of a second mitigation (found in a separate library). As this library grows, we aim to offer an even larger
amount of mitigation, and also offer complex solutions in a simple and ​easy-to-use interface.

Some of the classes available here are meant to be a wrapper for some other class found in one of these libraries,
with the goal of making simplifications to the usage of these approaches, while also automating certain steps. Other
classes offer unique solutions to a given aspect found in the area of data preprocessing for tabular data. All classes
offer a simple interface, with pre-determined parameter values, making it easier to use when the user doesn't know
(or doesn't want) to configure all of the available parameters. At the same time, the classes found here provide
several different customization options, making the module ideal for those with more experience.


Structure
---------

The **dataprocessing** module uses a base class (**DataProcessing**) that implements several tasks shared
across the different transformation classes. Whenever possible, the classes implemented in this module follow
the .fit() and .transform() behavior from scikit-learn, making these classes compatible with scikit-learn.
Each class aims to mitigate a certain aspect of tabular datasets. For each of these aspects, there might be one
or more different solutions for it. This way, we create a hierarchy, where each problem category has its own
abstract class (that inherits from the main abstract **DataProcessing** class), where each of these classes has
a set of concrete child classes that implements a different solution to the problem at hand. For example, we have
an abstract encoding class (:ref:`DataEncoding<DataEncoding>`), which has concrete child classes that implement
different encoding approaches (:ref:`EncoderOHE<EncoderOHE>` and :ref:`EncoderOrdinal<EncoderOrdinal>`).


API
---

.. toctree::
   :maxdepth: 2

   dataprocessing


Examples
--------

All notebooks listed here can be found in the folder **notebooks/dataprocessing/**.

.. toctree::
   :maxdepth: 1

   case_studies
   module_tests


scikit-learn's Pipeline Support
-------------------------------

.. toctree::
   :maxdepth: 1

   pipeline