Metadata-Version: 2.1
Name: vendi-score
Version: 0.0.1
Summary: A diversity metric for machine learning
Author-email: Dan Friedman <dfriedman@princeton.edu>
License: MIT
Project-URL: Documentation, https://github.com/vertaix/Vendi-Score
Project-URL: Issues, https://github.com/vertaix/Vendi-Score/issues
Project-URL: Source, https://github.com/vertaix/Vendi-Score
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy (>=1.14)
Requires-Dist: scipy (>=1.3.2)
Requires-Dist: scikit-learn (>=1.1)
Provides-Extra: all
Requires-Dist: torch ; extra == 'all'
Requires-Dist: torchvision ; extra == 'all'
Requires-Dist: transformers ; extra == 'all'
Requires-Dist: datasets ; extra == 'all'
Requires-Dist: nltk ; extra == 'all'
Requires-Dist: rdkit ; extra == 'all'
Provides-Extra: images
Requires-Dist: torch ; extra == 'images'
Requires-Dist: torchvision ; extra == 'images'
Provides-Extra: molecules
Requires-Dist: rdkit ; extra == 'molecules'
Provides-Extra: text
Requires-Dist: torch ; extra == 'text'
Requires-Dist: transformers ; extra == 'text'
Requires-Dist: datasets ; extra == 'text'
Requires-Dist: nltk ; extra == 'text'

# Vendi Score: A diversity metric for machine learning

This repository contains the implementation of Vendi Score, a metric for evaluating the diversity of generative models.
The input to metric is a collection of samples and a pairwise similarity function, and the output is a number, which can be interpreted as the effective number of unique elements in the sample.
Specifically, given a positive semi-definite matrix $K \in \mathbb{R}^{n \times n}$ of similarity scores, the score is defined as:
$$\mathrm{vNd}(K) = \exp(-\mathrm{tr}(K/n \log K/n)) = \exp(-\sum_{i=1}^n \lambda_i \log \lambda_i),$$
where $\lambda_i$ are the eigenvalues of $K/n$ and $0 \log 0 = 0$.
That is, Vendi-Score is equal to the exponential of the von Neumann entropy of $K/n$, or the Shannon entropy of the eigenvalues, which is also known as the effective rank.


## Installation

You can install `vendi_score` from pip:
```
pip install vendi_score
```
or by cloning this repository:
```
git clone https://github.com/vertaix/Vendi-Score.git
cd Vendi-Score
pip install -e .
```
`vendi_score` includes some optional dependencies for computing predefined similarity score between images, text, or molecules. You can install these dependencies with a command as in the following:
```
pip install vendi_score[images]
pip install vendi_score[text,molecules]
pip install vendi_score[all]
```

## Usage

The input to `vendi_score` is a list of samples and a similarity function, `k`, mapping a pair of elements to a similarity score. `k` should be symmetric, and `k(x, x) = 1`:
```
import numpy as np
from vendi_score import vendi

samples = [0, 0, 10, 10, 20, 20]
k = lambda a, b: np.exp(-np.abs(a - b))

vendi.score(samples, k)

# 2.9999
```
If you already have precomputed a similarity matrix:
```
K = np.array([[1.0, 0.9, 0.0]
              [0.9, 1.0, 0.0],
              [0.0, 0.0, 1.0]])
vendi.score_K(K)

# 2.1573
```
If your similarity function is a dot product between normalized embeddings $X \in \mathbb{R}^{n \times d}$, and $d < n$, it is faster to compute the Vendi score using the covariance matrix, $\frac{1}{n} \sum_{i=1}^n x_i x_i^{\top}$:
```
vendi.score_dual(X)
```
If the rows of $X$ are not normalized, set `normalize = True`.


### Similarity functions

Some similarity functions are provided in `vendi_score.image_utils`, `vendi_score.text_utils`, and `vendi_score.molecule_utils`.
Example usage is illustrated in Jupyter notebooks in the `examples/` folder.
