Metadata-Version: 2.1
Name: few-shot-learning-nlp
Version: 1.0.2
Summary: This library provides tools and utilities for Few Shot Learning in Natural Language Processing (NLP).
Home-page: https://github.com/peulsilva/few-shot-learning-nlp
Author: Pedro Silva
Author-email: pedrolmssilva@gmail.com
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: absl-py ==1.4.0
Requires-Dist: accelerate ==0.24.1
Requires-Dist: aiohttp ==3.8.5
Requires-Dist: aiosignal ==1.3.1
Requires-Dist: alembic ==1.13.0
Requires-Dist: antlr4-python3-runtime ==4.9.3
Requires-Dist: asttokens ==2.4.0
Requires-Dist: async-timeout ==4.0.3
Requires-Dist: attrs ==23.1.0
Requires-Dist: backcall ==0.2.0
Requires-Dist: black ==23.9.1
Requires-Dist: box2d-py ==2.3.5
Requires-Dist: cachetools ==5.3.1
Requires-Dist: certifi ==2023.7.22
Requires-Dist: charset-normalizer ==3.2.0
Requires-Dist: click ==8.1.7
Requires-Dist: cloudpickle ==2.2.1
Requires-Dist: cmake ==3.27.4.1
Requires-Dist: colorlog ==6.8.0
Requires-Dist: comm ==0.1.4
Requires-Dist: contourpy ==1.1.0
Requires-Dist: cycler ==0.11.0
Requires-Dist: datasets ==2.16.1
Requires-Dist: debugpy ==1.8.0
Requires-Dist: decorator ==4.4.2
Requires-Dist: dill ==0.3.7
Requires-Dist: exceptiongroup ==1.1.3
Requires-Dist: executing ==1.2.0
Requires-Dist: Farama-Notifications ==0.0.4
Requires-Dist: filelock ==3.12.4
Requires-Dist: fonttools ==4.42.1
Requires-Dist: frozenlist ==1.4.0
Requires-Dist: fsspec ==2023.6.0
Requires-Dist: fvcore ==0.1.5.post20221221
Requires-Dist: greenlet ==3.0.1
Requires-Dist: grpcio ==1.58.0
Requires-Dist: h5py ==3.10.0
Requires-Dist: huggingface-hub ==0.20.2
Requires-Dist: hydra-core ==1.3.2
Requires-Dist: idna ==3.4
Requires-Dist: imageio ==2.32.0
Requires-Dist: imageio-ffmpeg ==0.4.9
Requires-Dist: importlib-metadata ==6.8.0
Requires-Dist: importlib-resources ==6.0.1
Requires-Dist: iopath ==0.1.9
Requires-Dist: ipykernel ==6.25.2
Requires-Dist: ipython ==8.15.0
Requires-Dist: ipywidgets ==8.1.1
Requires-Dist: jedi ==0.19.0
Requires-Dist: Jinja2 ==3.1.2
Requires-Dist: joblib ==1.3.2
Requires-Dist: jsonargparse ==4.27.1
Requires-Dist: jupyter-client ==8.3.1
Requires-Dist: jupyter-core ==5.3.1
Requires-Dist: jupyterlab-widgets ==3.0.9
Requires-Dist: kiwisolver ==1.4.5
Requires-Dist: lightning-utilities ==0.10.0
Requires-Dist: lit ==16.0.6
Requires-Dist: Mako ==1.3.0
Requires-Dist: Markdown ==3.4.4
Requires-Dist: MarkupSafe ==2.1.3
Requires-Dist: matplotlib ==3.7.3
Requires-Dist: matplotlib-inline ==0.1.6
Requires-Dist: moviepy ==1.0.3
Requires-Dist: mpmath ==1.3.0
Requires-Dist: multidict ==6.0.4
Requires-Dist: multiprocess ==0.70.15
Requires-Dist: mypy-extensions ==1.0.0
Requires-Dist: nest-asyncio ==1.5.7
Requires-Dist: networkx ==3.1
Requires-Dist: nltk ==3.8.1
Requires-Dist: numpy ==1.25.2
Requires-Dist: nvidia-cublas-cu11 ==11.10.3.66
Requires-Dist: nvidia-cuda-cupti-cu11 ==11.7.101
Requires-Dist: nvidia-cuda-nvrtc-cu11 ==11.7.99
Requires-Dist: nvidia-cuda-runtime-cu11 ==11.7.99
Requires-Dist: nvidia-cudnn-cu11 ==8.5.0.96
Requires-Dist: nvidia-cufft-cu11 ==10.9.0.58
Requires-Dist: nvidia-curand-cu11 ==10.2.10.91
Requires-Dist: nvidia-cusolver-cu11 ==11.4.0.1
Requires-Dist: nvidia-cusparse-cu11 ==11.7.4.91
Requires-Dist: nvidia-nccl-cu11 ==2.14.3
Requires-Dist: nvidia-nvtx-cu11 ==11.7.91
Requires-Dist: oauthlib ==3.2.2
Requires-Dist: omegaconf ==2.3.0
Requires-Dist: optuna ==3.4.0
Requires-Dist: packaging ==23.1
Requires-Dist: pandas ==2.1.0
Requires-Dist: parso ==0.8.3
Requires-Dist: pathspec ==0.11.2
Requires-Dist: pexpect ==4.8.0
Requires-Dist: pickleshare ==0.7.5
Requires-Dist: Pillow ==10.0.0
Requires-Dist: platformdirs ==3.10.0
Requires-Dist: portalocker ==2.8.2
Requires-Dist: proglog ==0.1.10
Requires-Dist: prompt-toolkit ==3.0.39
Requires-Dist: protobuf ==4.24.3
Requires-Dist: psutil ==5.9.5
Requires-Dist: ptyprocess ==0.7.0
Requires-Dist: pure-eval ==0.2.2
Requires-Dist: pyarrow ==13.0.0
Requires-Dist: pyarrow-hotfix ==0.6
Requires-Dist: pyasn1 ==0.5.0
Requires-Dist: pyasn1-modules ==0.3.0
Requires-Dist: pycocotools ==2.0.7
Requires-Dist: Pygments ==2.16.1
Requires-Dist: pyparsing ==3.1.1
Requires-Dist: python-dateutil ==2.8.2
Requires-Dist: pytz ==2023.3.post1
Requires-Dist: PyYAML ==6.0.1
Requires-Dist: pyzmq ==25.1.1
Requires-Dist: regex ==2023.8.8
Requires-Dist: requests ==2.31.0
Requires-Dist: requests-oauthlib ==1.3.1
Requires-Dist: rsa ==4.9
Requires-Dist: safetensors ==0.3.3
Requires-Dist: scikit-learn ==1.3.2
Requires-Dist: scipy ==1.11.4
Requires-Dist: seaborn ==0.12.2
Requires-Dist: sentence-transformers ==2.2.2
Requires-Dist: sentencepiece ==0.1.99
Requires-Dist: six ==1.16.0
Requires-Dist: SQLAlchemy ==2.0.23
Requires-Dist: stack-data ==0.6.2
Requires-Dist: swig ==4.1.1.post0
Requires-Dist: sympy ==1.12
Requires-Dist: tabulate ==0.9.0
Requires-Dist: tensorboard ==2.14.0
Requires-Dist: tensorboard-data-server ==0.7.1
Requires-Dist: tensordict ==0.1.2
Requires-Dist: termcolor ==2.3.0
Requires-Dist: threadpoolctl ==3.2.0
Requires-Dist: tokenizers ==0.15.0
Requires-Dist: tomli ==2.0.1
Requires-Dist: torch ==2.0.1
Requires-Dist: torcheval ==0.0.7
Requires-Dist: torchmetrics ==1.2.1
Requires-Dist: torchvision ==0.15.2
Requires-Dist: tornado ==6.3.3
Requires-Dist: tqdm ==4.66.1
Requires-Dist: traitlets ==5.9.0
Requires-Dist: transformers ==4.35.2
Requires-Dist: triton ==2.0.0
Requires-Dist: typing-extensions ==4.7.1
Requires-Dist: tzdata ==2023.3
Requires-Dist: urllib3 ==1.26.16
Requires-Dist: wcwidth ==0.2.6
Requires-Dist: Werkzeug ==2.3.7
Requires-Dist: widgetsnbextension ==4.0.9
Requires-Dist: xxhash ==3.3.0
Requires-Dist: yacs ==0.1.8
Requires-Dist: yarl ==1.9.2
Requires-Dist: zipp ==3.16.2

# few-shot-learning-nlp

This library provides tools and utilities for Few Shot Learning in Natural Language Processing (NLP).

## Overview

Few Shot Learning in NLP involves training and evaluating models on tasks with limited labeled data. This library offers functionalities to facilitate this process.

## Installation

You can install this library via pip:

```bash
pip install -U few-shot-learning-nlp
```

## Documentation

The documentation for this library is available [here](https://peulsilva.github.io/few-shot-learning-nlp/).

## Supported Approaches

### Text Classification
- Sentence Transformers Finetuning ([SetFit](https://arxiv.org/abs/2209.11055))
- Pattern Exploiting Training ([PET](https://arxiv.org/abs/2001.07676))

### Named Entity Recognition for Image Documents
- Pattern Exploiting Training ([PET](https://arxiv.org/abs/2001.07676))
- [Bio Technique](https://arxiv.org/abs/2305.04928)

### Classification Utils
- [Focal Loss function for imbalanced datasets](https://arxiv.org/abs/1708.02002)
- Stratified train test split

## Usage

To utilize this library, import the necessary classes and methods and follow the provided [documentation](https://peulsilva.github.io/few-shot-learning-nlp/) for each component.

Here is a short example of the SetFit implementation


```python
from datasets import load_dataset
import pandas as pd
from few_shot_learning_nlp.utils import stratified_train_test_split
from torch.utils.data import DataLoader
from few_shot_learning_nlp.few_shot_text_classification.setfit_dataset import SetFitDataset

# Load a dataset for text classification
ag_news_dataset = load_dataset("ag_news")

# Extract necessary information from the dataset
num_classes = len(ag_news_dataset['train'].features['label'].names)

# Perform few-shot learning by selecting a limited number of classes
n_shots = 50
train_validation, test_df = stratified_train_test_split(ag_news_dataset['train'], num_shots_per_class=n_shots)
train_df, val_df = stratified_train_test_split(pd.DataFrame(train_validation), num_shots_per_class=30)

# Create SetFitDataset objects for training and validation
set_fit_data_train = SetFitDataset(train_df['text'], train_df['label'], input_example_format=True)
set_fit_data_val = SetFitDataset(val_df['text'], val_df['label'], input_example_format=False)

# Create DataLoader objects for training and validation datasets
train_dataloader = DataLoader(set_fit_data_train.data, shuffle=False)
val_dataloader = DataLoader(set_fit_data_val)
```

### Defining Classifier

```python
import torch

class CLF(torch.nn.Module):
    def __init__(
        self,
        in_features : int,
        out_features : int, 
        *args, 
        **kwargs
    ) -> None:
        super().__init__(*args, **kwargs)

        self.layer1 = torch.nn.Linear(in_features, 128)
        self.relu = torch.nn.ReLU()
        self.layer2 = torch.nn.Linear(128, 32)
        self.layer3 = torch.nn.Linear(32, out_features)

    def forward(self, x : torch.Tensor):
        x = self.layer1(x)
        x = self.relu(x)
        x = self.layer2(x)
        x = self.relu(x)
        return self.layer3(x)
```

### Training the Embedding Model <a name="training-the-embedding-model"></a>

```python
import torch
from sentence_transformers import SentenceTransformer
from few_shot_learning_nlp.few_shot_text_classification.setfit import SetFitTrainer

# Load a pre-trained Sentence Transformer model
model = SentenceTransformer("whaleloops/phrase-bert")

# Initialize the SetFitTrainer with embedding model and classifier
embedding_model = model.to("cuda")
in_features = embedding_model.get_sentence_embedding_dimension()
clf = CLF(in_features, num_classes).to("cuda")
trainer = SetFitTrainer(embedding_model, clf, num_classes)

# Train the embedding model
trainer.train_embedding(train_dataloader, val_dataloader, n_epochs=10)
```

### Training the Classifier Model <a name="training-the-classifier-model"></a>

```python

# Shuffle training data
_, class_counts = np.unique(train_df['label'], return_counts=True)
X_train_shuffled, y_train_shuffled = shuffle_two_lists(train_df['text'], train_df['label'])

# Train the classifier
history, embedding_model, clf = trainer.train_classifier(
    X_train_shuffled, y_train_shuffled, val_df['text'], val_df['label'],
    clf=CLF(in_features, num_classes),
    n_epochs=15,
    lr=1e-4
)
```

### Testing the Models <a name="testing-the-models"></a>

```python
y_true, y_pred = trainer.test(test_df)
```


