Metadata-Version: 2.1
Name: numereval
Version: 0.2
Summary: A small package for evaluating numer.ai model locally
Home-page: https://github.com/parmarsuraj99/numereval
Author: Suraj Parmar
Author-email: parmarsuraj99@gmail.com
License: UNKNOWN
Description: # A small library to reproduce the scores on numer.ai diagnistics dashboard.
        
        ## Installation
        
        `pip install numereval`
        
        ### Structure
        
        ![Structure](https://raw.githubusercontent.com/parmarsuraj99/numereval/master/images/numereval_structure.png)
        
        ## Numerai main tournament evaluation metrics
        
        ### numereval.numereval.evaluate
        
        A generic function to calculate basic per-era correlation stats with optional feature exposure and plotting.
        
        Useful for evaluating custom validation split from training data without MMC metrics and correlation with example predictions.
        
        ```
        from numereval.numereval import evaluate
        
        evaluate(training_data, plot=True, feature_exposure=False)
        ```
        
        Correlations plot      |  Returned metrics
        :-------------------------:|:-------------------------:
        ![Training Correlations](https://github.com/parmarsuraj99/numereval/raw/master/images/training_eval.png)  |  ![Metrics](https://raw.githubusercontent.com/parmarsuraj99/numereval/master/images/evaluate_metrics.png)
        
        ### numereval.numereval.diagnostics
        
        To reproduce the scores on diagnostics dashboard locally with optional plotting of per-era correlations.
        
        ```python
        from numereval.numereval import diagnostics
        
        validation_data = tournament_data[tournament_data.data_type == "validation"]
        
        diagnostics(
            validation_data,
            plot=True,
            example_preds_loc="numerai_dataset_244\example_predictions.csv",
        )
        
        ```
        
        Validation plot             |  Returned metrics
        :-------------------------:|:-------------------------:
        ![all eras validation plot](https://raw.githubusercontent.com/parmarsuraj99/numereval/master/images/nmr_eval.png)  |  ![all eras validation metrics](https://raw.githubusercontent.com/parmarsuraj99/numereval/master/images/numertest.png)
        
        #### Specific validation eras
        
        specify a list of eras in the format `eras = ["era121", "era122", "era209"]`
        
        ```python
        validation_data = tournament_data[tournament_data.data_type == "validation"]
        
        eras = validation_data.era.unique()[11:-2]
        
        numereval.numereval.diagnostics(
            validation_data,
            plot=True,
            example_preds_loc="numerai_dataset_244\example_predictions.csv",
            eras=eras,
        )
        
        ```
        
        Validation plot             |  Returned metrics
        :-------------------------:|:-------------------------:
        ![all eras validation plot](https://raw.githubusercontent.com/parmarsuraj99/numereval/master/images/nmr_eval_some_eras.png)  |  ![all eras validation metrics](https://raw.githubusercontent.com/parmarsuraj99/numereval/master/images/numertest_specific_eras.png)
        
        Docs will be updated soon!
        
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/markdown
