Metadata-Version: 2.4
Name: pereval
Version: 0.1.1
Summary: A toolkit for evaluating MIDI piano performances.
License-File: LICENSE
Author: Dmitrii Gavrilev
Author-email: dmitrygavrilyev@gmail.com
Requires-Python: >=3.12
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: miditok (>=3.0.6.post1)
Requires-Dist: numpy (<2.0.0)
Requires-Dist: pandas (>=2.3.3)
Requires-Dist: parangonar (==3.2.0)
Requires-Dist: partitura (>=1.6.0)
Requires-Dist: python-dotenv (>=1.2.2,<2.0.0)
Requires-Dist: scikit-learn (==1.7.1)
Requires-Dist: scipy (==1.16.0)
Requires-Dist: seaborn (>=0.13.2)
Requires-Dist: skops (>=0.13.0)
Requires-Dist: symusic (>=0.5.8)
Requires-Dist: torch (>=2.3)
Description-Content-Type: text/markdown

# Pereval: Piano Performance Evaluation toolkit

## Integrating Contextual Embeddings into Evaluation of Expressive MIDI Piano Performances

> [![arXiv](https://img.shields.io/badge/arXiv-2511.03425-b31b1b.svg?logo=arXiv)](https://arxiv.org/pdf/2607.27909)

**Dmitrii Gavrilev, Ilya Borovik, and Vladimir Viro**

This is an official repository for the paper ["Integrating Contextual Embeddings into Evaluation of Expressive MIDI Piano Performances"](https://arxiv.org/pdf/2607.27909) (**Accepted at ISMIR 2026**).

[Documentation (WIP)](https://realfolkcode.github.io/pereval/)

## Overview

In this paper, we tackle the problem of evaluating and comparing sets of piano MIDI performances.

Traditional metrics (which we coin **attribute-scoped**), such as *Pearson correlation* and *reconstruction error*, that are most commonly used in Expressive Performance Rendeering (EPR) have several drawbacks. Namely, they are limited by a single expressive attribute, such as tempo, articulation, and dynamics of a performance, disregarding other musical aspects and context. Second, they often require a note-level alignment between a score and a performance, hindering the scalability of evaluation.

We adapt recently proposed kernel-based methods from image and audio generation domains for EPR. **Deep feature metrics**, *Kernel Music Distance* (KMD) and *Kernel Performance Distance* (KPD), leverage rich contextual features from Aria and CLaMP3 symbolic music understanding models. The adapted metrics are scalable (do not require alignments) and contextual.

In short, this project aims to go beyond the traditional metrics for expressive MIDI piano performances and unify them under one toolkit to simplify evaluation routines for MIR practitioners.

## Installation

1. Prerequisites:
- Python >= 3.12 
- [torch](https://pytorch.org/) (only versions >=2.3 are supported officially)
- [Aria](https://github.com/EleutherAI/aria)
2. Install pereval:
```sh
pip install -U pereval
```

## Repository structure:

- `/pereval/` contains the main utilities (such as alignment, feature extraction, calculation of attribute-scoped and deep feature metrics) used in our research paper
- `/examples/` contains the notebooks for data analysis and examples of usage

**Command-line interface:**
- `pereval-save-alignment` - calculates the note alignment using Parangonar
- `pereval-correlation` - calculates attribute-scoped correlations between sets of performances
- `pereval-kde` - calculates Kernel Density Estimators for attribute-scoped KL-divergence
- `pereval-embed-aria` - extracts Aria embeddings
- `pereval-shift-notes` - shifts (truncates the first few) notes in performances
- `pereval-permute-velocity` - transfers the velocities of the notes between performances

**Notebooks:**
- `mahalanobis_pseudo_ratings.ipynb` - comparing Mahalanobis-based Aria pseudo-ratings with human MOS from the listening test 

## Acknowledgements

We would like to thank the authors of [FADtk](https://github.com/microsoft/fadtk), [KADtk](https://github.com/YoonjinXD/kadtk), [Aria](https://github.com/EleutherAI/aria), [CLaMP3](https://github.com/sanderwood/clamp3), [Parangonar](https://github.com/sildater/parangonar), and [symusic](https://github.com/Yikai-Liao/symusic) for their open source contributions in music and audio research.

