Metadata-Version: 2.1
Name: pypmml-spark
Version: 1.5.5
Summary: Python PMML scoring library for PySpark as SparkML Transformer
Home-page: https://github.com/autodeployai/pypmml-spark
Download-URL: https://github.com/autodeployai/pypmml-spark/archive/v1.5.5.tar.gz
Author: AutoDeployAI
Author-email: autodeploy.ai@gmail.com
License: Apache License 2.0
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyspark>=3.0.0

# PyPMML-Spark

_PyPMML-Spark_ is a Python PMML scoring library for PySpark as SparkML Transformer, it really is the Python API for [PMML4S-Spark](https://github.com/autodeployai/pmml4s-spark).

## Prerequisites
 - Java >= 1.8
 - Python 2.7 or >= 3.5

## Dependencies
Module                    | PySpark
------------------------- | ----------------------
[pypmml-spark](https://github.com/autodeployai/pypmml-spark/tree/master)              | PySpark >= 3.0.0
[pypmml-spark2](https://github.com/autodeployai/pypmml-spark/tree/spark-2.x)          | PySpark >= 2.4.0, < 3.0.0
  
## Installation

```bash
pip install pypmml-spark
```

Or install the latest version from github:

```bash
pip install --upgrade git+https://github.com/autodeployai/pypmml-spark.git
```

After that, you need to do more to use it in Spark that must know those jars in the package `pypmml_spark.jars`. There are several ways to do that:

1. The easiest way is to run the script `link_pmml4s_jars_into_spark.py` that is delivered with `pypmml-spark`:

    ```bash
    link_pmml4s_jars_into_spark.py
    ```
    
2. Use those config options to specify dependent jars properly. e.g. `--jars`, or `spark.executor.extraClassPath` and `spark.executor.extraClassPath`. See [Spark](http://spark.apache.org/docs/latest/configuration.html) for details about those parameters.

## Usage

1. Load model from various sources, e.g. filename, string, or array of bytes.

    ```python
    from pypmml_spark import ScoreModel
    
    # The model is from http://dmg.org/pmml/pmml_examples/KNIME_PMML_4.1_Examples/single_iris_dectree.xml
    model = ScoreModel.fromFile('single_iris_dectree.xml')
    ```

2. Call `transform(dataset)` to run a batch score against an input dataset.

    ```python
    # The data is from http://dmg.org/pmml/pmml_examples/Iris.csv
    df = spark.read.csv('Iris.csv', header='true')
    score_df = model.transform(df)
    ```

## Use PMML in Scala or Java
See the [PMML4S](https://github.com/autodeployai/pmml4s) project. _PMML4S_ is a PMML scoring library for Scala. It provides both Scala and Java Evaluator API for PMML.

## Use PMML in Python
See the [PyPMML](https://github.com/autodeployai/pypmml) project. _PyPMML_ is a Python PMML scoring library, it really is the Python API for PMML4S.

## Use PMML in Spark
See the [PMML4S-Spark](https://github.com/autodeployai/pmml4s-spark) project. _PMML4S-Spark_ is a PMML scoring library for Spark as SparkML Transformer.

## Deploy PMML as REST API
See the [AI-Serving](https://github.com/autodeployai/ai-serving) project. _AI-Serving_ is serving AI/ML models in the open standard formats PMML and ONNX with both HTTP (REST API) and gRPC endpoints.

## Support
If you have any questions about the _PyPMML-Spark_ library, please open issues on this repository.

Feedback and contributions to the project, no matter what kind, are always very welcome. 

## License
_PyPMML-Spark_ is licensed under [APL 2.0](http://www.apache.org/licenses/LICENSE-2.0).
