Metadata-Version: 2.4
Name: textslinger
Version: 0.2.4
Summary: TextSlinger: Fast and accurate text predictions in Python
Author-email: Keith Vertanen <vertanen@mtu.edu>, Dylan Gaines <dgaine20@kennesaw.edu>, Soufia Bahmani <sbahmani@mtu.edu>
License-Expression: MIT
Project-URL: Homepage, https://github.com/kdv123/textslinger
Project-URL: Source, https://github.com/kdv123/textslinger
Platform: Linux
Platform: Windows
Platform: Mac OS-X
Classifier: Development Status :: 3 - Alpha
Classifier: Topic :: Scientific/Engineering :: Human Machine Interfaces
Classifier: Natural Language :: English
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: End Users/Desktop
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.10
Requires-Python: <3.11,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.6.0
Requires-Dist: torchvision>=0.21.0
Requires-Dist: torchaudio>=2.6.0
Requires-Dist: datasets==2.18.0
Requires-Dist: bitsandbytes==0.42.0
Requires-Dist: requests==2.32.3
Requires-Dist: kenlm==0.2.0
Requires-Dist: nlpaug==1.1.11
Requires-Dist: psutil==5.7.2
Requires-Dist: ipywidgets==8.1.3
Requires-Dist: sentencepiece==0.2.0
Requires-Dist: protobuf==4.25.3
Requires-Dist: evaluate==0.4.0
Requires-Dist: scikit-learn==1.2.2
Requires-Dist: accelerate>=1.1.0
Requires-Dist: transformers>=5.2.0
Requires-Dist: numpy~=1.26.3
Requires-Dist: tqdm==4.62.2
Requires-Dist: peft~=0.14.0
Provides-Extra: dev
Requires-Dist: coverage>=7.0; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: black; extra == "dev"
Provides-Extra: release
Requires-Dist: twine==5.0.0; extra == "release"
Requires-Dist: build>=1.4.0; extra == "release"
Requires-Dist: wheel==0.43.0; extra == "release"
Dynamic: license-file

<table border="0" cellpadding="0" cellspacing="0">
  <tr>
    <td width="120">
      <a href="images/textslinger.png"><img src="images/textslinger_small.jpg" width="236" alt="Cowboy drawing his two cell phone six shooters"></a>
    </td>
    <td align="center">
      <h2>TextSlinger: Fast and Accurate Text Predictions in Python</h2>
    </td>
  </tr>
</table>

This is a Python library for making text predictions using different types of language models.
Current features:
* Predict the distribution over the next character given the previous text.
* Predict the most likely next words given the previous text and prefix of current word.
* Supports:
  - N-gram language models via [KenLM](https://github.com/kpu/kenlm).
  - Subword tokenized large language models (LLMs) via [Hugging Face](https://huggingface.co/docs/hub/en/index).
  - Byte tokenized LLMs via Hugging Face and [Byte Latent Transformer](https://arxiv.org/abs/2412.09871).

## Developer setup
Our code style is whatever the Black formatter says it should be. 
You should [configure your IDE to format using Black when you save](https://black.readthedocs.io/en/stable/integrations/editors.html).

### Windows
Windows is not supported by the library dependencies of textslinger.
To use our library on Windows, you will need to use [Windows Subsystem for Linux (WSL)](https://learn.microsoft.com/en-gb/windows/wsl/).
Please follow the [installation directions](https://learn.microsoft.com/en-gb/windows/wsl/install) and use the Linux commands below.
You will also need to install pip to install Python libraries. Follow the 
[pip installation directions](https://pip.pypa.io/en/stable/installation/) before proceeding.

## Setting up a Python environment
If you don't have [Miniforge](https://conda-forge.org/download/) installed in your user account you'll first need to do that.

To install Miniforge on MacOS using Apple Silicon:
```
curl -LO https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
zsh Miniforge3-MacOSX-arm64.sh
~/miniforge3/bin/conda init zsh
```
To install Miniforge on Linux and WSL:
```
curl -LO https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh
```

When installing on WSL, enter "yes" when asked if you want to automatically initialize conda. This sets up easier access to conda in your WSL environment. You can disable the automatic start of the base environment using the following conda command after restarting your terminal:
```
conda config --set auto_activate_base false
```

After installing Miniforge, be sure to close your terminal and start a new one. Create an environment as follows:
```
conda config --remove-key channels
conda config --add channels conda-forge
conda config --set channel_priority strict
conda create -n textslinger python=3.10 -y
conda activate textslinger
```

### Installation of PyTorch

MacOS using Apple Silicon:
```
pip install torch torchvision torchaudio
```

Linux with CUDA support (GPU driver must support installed library version or greater. Run ```nvidia-smi``` to check driver support):
```
# CUDA 11.8 
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

# CUDA 12.1
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
```
Linux without CUDA support:
```
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
```

WSL with CUDA support (GPU driver must support installed library version or greater. Run ```nvidia-smi``` to check driver support).
Due to a security vulnerability in earlier versions, WSL requires at least torch 2.6:

```
# CUDA 12.6
python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
```
WSL without CUDA support:
```
python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
```

Test if the PyTorch installation worked:
```
python - <<'EOF'
import torch
print("Torch version:", torch.__version__)
print("MPS available:", torch.backends.mps.is_available())
print("MPS built:", torch.backends.mps.is_built())
print("CUDA available:", torch.cuda.is_available())
print("CUDA version:", torch.version.cuda)
print("CUDA device count:", torch.cuda.device_count())

if torch.backends.mps.is_available():
    x = torch.randn(2, 2, device="mps")
    print("Tensor device:", x.device)
elif torch.cuda.is_available():
    x = torch.randn(2, 2, device="cuda")
    print("Tensor device:", x.device)
else:
    x = torch.randn(2, 2)
    print("Tensor device:", x.device)
EOF
```

### Installation of libraries
Install transformers (5.2.0 or greater required).

MacOS and Linux:
```
pip install transformers
```

WSL:
```
python -m pip install transformers
```

Check transformers version and model support:
```
python - <<'EOF'
import transformers
from transformers import __version__
from transformers.utils import is_torch_available

print("Version:", __version__)
print("File:", transformers.__file__)

# Check for BLT symbols that do NOT exist in stable 5.0.0
try:
    from transformers.models.blt.modeling_blt import BltModel
    print("BLT model available ✅")
except Exception as e:
    print("BLT model missing ❌", e)
EOF
```

#### Hugging Face Authentication

You will need to log in to Hugging Face using the CLI. This CLI is installed with the `transformers` library. Please see the [Hugging Face CLI Login direction](https://huggingface.co/docs/huggingface_hub/en/guides/cli#hf-auth-login) for how to do this. We recommend [using a token](https://huggingface.co/docs/huggingface_hub/en/quick-start#authentication) so you do not need to manually enter your credentials.

```
hf auth login
```

Install other dependencies on MacOS and Linux:
```
pip install pytest scipy peft psutil datasets

# NOTE: increase MAX_ORDER if you plan to load n-gram models with longer context 
MAX_ORDER=12 pip install https://github.com/kpu/kenlm/archive/master.zip
```

Install other dependencies on WSL:
```
python -m pip install pytest scipy peft psutil datasets
```

If using the default WSL Ubuntu installation, the `build-essential` package is needed to build the KenLM library. You also need the `unzip` package to run the `download.sh` script later in this guide.
```
sudo apt-get update && sudo apt-get upgrade
sudo apt-get install build-essential unzip
# NOTE: increase MAX_ORDER if you plan to load n-gram models with longer context 
export MAX_ORDER=12
python -m pip install https://github.com/kpu/kenlm/archive/master.zip
```

Fix harmless warning message on MacOS and Linux:
```
pip install --upgrade --force-reinstall setuptools
```

Fix harmless warning message on WSL:

CUDA 12.6 does not support `setuptools` versions 82.0.0+.

```
python -m pip install --upgrade --force-reinstall setuptools==81.0.0
```

Install TextSlinger itself so scripts can import the `textslinger` package from any working directory. Run this from the repository root on all platforms, the directory containing `pyproject.toml`:
```
python -m pip install --no-deps -e .
```

### Testing installation
Download assets needed by the test suite and then run it. If you are on WSL, check if the `assets` directory is owned by `root` using `ls -l`. If so, restart your WSL session and rercheck the ownsership.
```
cd textslinger/assets
./download.sh
cd ..
pytest -v -rs
```

If you encounter an error with line endings (e.g., "\r: no such file or directory") when running `download.sh`, you can change the line endings using:

```
vim download.sh -c "set ff=unix" -c ":wq"
```
---
This material is based upon work supported by the NSF under Grant No. IIS-1909089 and IIS-2402876.
