Metadata-Version: 1.1
Name: sparkonda
Version: 0.2.1
Summary: Minimalistic utility library to manage conda environments for pyspark jobs on yarn clusters
Home-page: https://github.com/moutai/sparkonda
Author: Moussa Taifi
Author-email: @moutai
License: ISCL
Description: ===============================
        Sparkonda
        ===============================
        
        Minimalistic utility library to manage conda environments for PySpark jobs on yarn clusters.
        
        Features
        --------
        
        * Manage conda environments on PySpark executors to use specific
          packages on the remote workers without involving admins to install needed software on hadoop cluster.
        
        Docs
        --------
        
        http://sparkonda.readthedocs.org
        
        
        
        
        History
        -------
        
        0.1.0 (2015-11)
        ---------------------
        Initial version to manage the conda environments on pyspark cluster workers without involving your cluster admins too heavily.
        
        0.2.0 (2016-01)
        ---------------------
        Additional tests for the distributed version of remote package delivery
        
        Changed os files management to python based support (zip, rm)
        
        Use SparkFiles to detect files distributed to the workers.
        
        0.2.1 (2016-01)
        ---------------------
        Fix documentation, setup and usage methods
        
        Fix travis and setup.py configs
        
        Move zip to pack as an action
        
        Replace zip with tar to better preserve acls on conda env files
        
        Add configuration of error level for untar
        
        
        
        
Keywords: sparkonda
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: ISC License (ISCL)
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 2.7
