Metadata-Version: 2.1
Name: pandas-save-profiler
Version: 0.0.0
Summary: Tools to evaluate pandas performance when saving dataframes in different file formats.
Home-page: https://github.com/dmontaner/pandas_save_profiler
Author: David Montaner
Author-email: david.montaner@gmail.com
License: MIT
Project-URL: Source Code, https://github.com/dmontaner/pandas_save_profiler
Project-URL: Documentation, https://github.com/dmontaner/pandas_save_profiler
Project-URL: Issue Tracker, https://github.com/dmontaner/pandas_save_profiler/issues
Description: # Pandas Save Profiler
        
        `pandas_save_profiler` helps you evaluating and comparing the performance of different pandas read and write methods.
        
        ## Install
        
            pip install pandas-save-profiler
        
        ## Usage
        
        Load pandas and a dataframe you want to save.
        
            import pandas as pd
            data = pd.util.testing.makeMissingDataframe()
        
        Load `pandas_save_profiler` and use it to evaluate pandas performance saving a _pickle_ file:
        
            import pandas_save_profiler
            data.save_profiler('to_pickle')
        
        The output is a pandas series:
        
        ```
        format                                                 pickle
        writer                                              to_pickle
        reader                                            read_pickle
        writer_args                      {'path': '/tmp/tmppk7nkivk'}
        reader_args        {'filepath_or_buffer': '/tmp/tmppk7nkivk'}
        writer_time                                         0.0798338
        reader_time                                         0.0294895
        writer_memory                                     1.09087e+08
        reader_memory                                     1.09118e+08
        df_memory                                                 288
        file_size                                                1122
        writer_memory_h                                      109.1 MB
        reader_memory_h                                      109.1 MB
        df_memory_h                                         288 Bytes
        file_size_h                                            1.1 kB
        repeats                                                     5
        reads_the_same                                           True
        dtype: object
        ```
        
        Values in the series indicate:
        
        - The __format__ used to persist the dataframe and the writing and reading options.
        - Writing and reading __times__ in seconds.
        - Writing and reading __memory__ increment.
        - Size of the dataframe in memory.
        - __Size__ of the saved file.
        
        Memory values are in __bytes__ but a "humanized" version is also reported.
        The saving and reloading process is __repeated__ 5 times and average values are returned.
        The flag `reads_the_same` indicates whether the reloaded file is exactly the same as the original one or has some differences.
        
        
        To __compare several writing options__ you can use the `save_profiler` function on each of them
        and combine the results into a results dataframe:
        
        ```
        pd.DataFrame([
            data.save_profiler('to_csv'),
            data.save_profiler('to_pickle'),
            data.save_profiler('to_parquet'),
        ])
        
        ```
        
        returns:
        
        ```
            format      writer        reader                          writer_args  \
        0      csv      to_csv      read_csv  {'path_or_buf': '/tmp/tmpsedehjob'}   
        1   pickle   to_pickle   read_pickle         {'path': '/tmp/tmp_vhue2q7'}   
        2  parquet  to_parquet  read_parquet         {'path': '/tmp/tmp0zn8qsnk'}   
        
                                          reader_args  writer_time  reader_time  \
        0  {'filepath_or_buffer': '/tmp/tmpsedehjob'}     0.031842     0.039830   
        1  {'filepath_or_buffer': '/tmp/tmp_vhue2q7'}     0.025705     0.028469   
        2                {'path': '/tmp/tmp0zn8qsnk'}     0.039009     0.052447   
        
           writer_memory  reader_memory  df_memory  file_size writer_memory_h  \
        0    110149632.0    110599372.8        288        139        110.1 MB   
        1    110813184.0    110813184.0        288       1122        110.8 MB   
        2    116892467.2    118014771.2        288       3449        116.9 MB   
        
          reader_memory_h df_memory_h file_size_h  repeats  reads_the_same  
        0        110.6 MB   288 Bytes   139 Bytes        5           False  
        1        110.8 MB   288 Bytes      1.1 kB        5            True  
        2        118.0 MB   288 Bytes      3.4 kB        5            True  
        ```
        
Keywords: pandas,save,profile
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: Unix
Classifier: Operating System :: POSIX
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Topic :: Utilities
Classifier: Topic :: Scientific/Engineering
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Requires-Python: >=3.6
Description-Content-Type: text/markdown
