Metadata-Version: 2.3
Name: problm-solver
Version: 1.5.1
Summary: A python library built on top of llama.cpp which generates and performs statistical analysis on language model outputs.
Author: Clio
Author-email: Clio <d.k.johnson@lancaster.ac.uk>
Requires-Dist: fsspec
Requires-Dist: huggingface-hub
Requires-Dist: llama-cpp-python
Requires-Dist: numpy
Requires-Dist: pandas>=3.0.3
Requires-Dist: tqdm>=4.67.3
Requires-Dist: poethepoet
Requires-Python: >=3.13, <4.0
Project-URL: homepage, https://github.com/muse-writes/probLM-solver
Project-URL: source, https://github.com/muse-writes/probLM-solver
Project-URL: changelog, https://github.com/muse-writes/probLM-solver/blob/main/CHANGELOG.md
Project-URL: releasenotes, https://github.com/muse-writes/probLM-solver/releases
Project-URL: documentation, https://muse-writes.github.io/probLM-solver
Project-URL: issues, https://github.com/muse-writes/probLM-solver/issues
Description-Content-Type: text/markdown

[![Open in Dev Containers](https://img.shields.io/static/v1?label=Dev%20Containers&message=Open&color=blue&logo=data:image/svg%2bxml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCI+PHBhdGggZmlsbD0iI2ZmZiIgZD0iTTE3IDE2VjdsLTYgNU0yIDlWOGwxLTFoMWw0IDMgOC04aDFsNCAyIDEgMXYxNGwtMSAxLTQgMmgtMWwtOC04LTQgM0gzbC0xLTF2LTFsMy0zIi8+PC9zdmc+)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/muse-writes/probLM-solver) [![Open in GitHub Codespaces](https://img.shields.io/static/v1?label=GitHub%20Codespaces&message=Open&color=blue&logo=github)](https://github.com/codespaces/new/muse-writes/probLM-solver) [![Documentation](https://img.shields.io/static/v1?label=Documentation&message=View&color=blue&logo=readme&logoColor=white)](https://muse-writes.github.io/probLM-solver)

# probLM-solver

A python library and CLI app built on top of llama.cpp which can generate and modify language model outputs based on internal probabilities. Included are utilities for generating large numbers of LLM outputs for performing statistical analysis, as well as outputting and modifying token-by-token conditional probabilities for a given output.

Of particular importance is the ability to define sampling functions for token generation, based off of the answer's power distribution, via iteratively sampling branches of future tokens.

This library is currently in active development, so expect large changes.

## Usage

### Bare Metal

This project is managed with uv, ensure you have Python, pip, and uv installed. To create and activate the virtual environment, run:

```sh
uv sync --python 3.13 --no-dev
source .venv/bin/activate
```

To install this package as a library, if you have cloned the repository, run:

```sh
python3 -m pip install /path/to/repository
```

else, if you only desire the library, run:

```sh
pip install git+https://github.com/muse-writes/probLM-solver
```

### Conda

An `environment.yml` is provided for conda users (Miniforge, Miniconda, or Anaconda). To create and activate the environment, run:

```sh
conda env create -f environment.yml
conda activate problm-solver
```

If you have an NVIDIA GPU and want hardware-accelerated inference, reinstall `llama-cpp-python` with CUDA support after activating the environment:

```sh
CMAKE_ARGS="-DGGML_CUDA=on" pip install "llama-cpp-python>=0.3.23" --force-reinstall
```

On the HEC, and other HPC systems which use Nvidia L40(S) and H200 GPUs, oftentimes the login node won't have native GPUs, so use the command:

```sh
CMAKE_ARGS="-DGGML_CUDA=on -DGGML_NATIVE=off -DCMAKE_CUDA_ARCHITECTURES=89;90 pip install --force-reinstall --no-cache-dir "llama-cpp-python>=0.3.23"
```

For complete and custom installation of `llama-cpp-python`, see [llama.cpp build docs](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#cuda).

### Docker

The CLI itself can be run in a docker container for testing the program.

To serve this app, run:

```sh
docker-compose run app --build
```

Within the Dev Container this is equivalent to:

```sh
poe serve
```

## Contributing

<details>
<summary>Prerequisites</summary>

1. [Generate an SSH key](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#generating-a-new-ssh-key) and [add the SSH key to your GitHub account](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account).
1. Configure SSH to automatically load your SSH keys:

    ```sh
    cat << EOF >> ~/.ssh/config
    
    Host *
      AddKeysToAgent yes
      IgnoreUnknown UseKeychain
      UseKeychain yes
      ForwardAgent yes
    EOF
    ```

1. [Install Docker Desktop](https://www.docker.com/get-started).
1. [Install VS Code](https://code.visualstudio.com/) and [VS Code's Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). Alternatively, install [PyCharm](https://www.jetbrains.com/pycharm/download/).
1. _Optional:_ install a [Nerd Font](https://www.nerdfonts.com/font-downloads) such as [FiraCode Nerd Font](https://github.com/ryanoasis/nerd-fonts/tree/master/patched-fonts/FiraCode) and [configure VS Code](https://github.com/tonsky/FiraCode/wiki/VS-Code-Instructions) or [PyCharm](https://github.com/tonsky/FiraCode/wiki/Intellij-products-instructions) to use it.

</details>

<details open>
<summary>Development environments</summary>

The following development environments are supported:

1. ⭐️ _GitHub Codespaces_: click on [Open in GitHub Codespaces](https://github.com/codespaces/new/muse-writes/probLM-solver) to start developing in your browser.
1. ⭐️ _VS Code Dev Container (with container volume)_: click on [Open in Dev Containers](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/muse-writes/probLM-solver) to clone this repository in a container volume and create a Dev Container with VS Code.
1. ⭐️ _uv_: clone this repository and run the following from root of the repository:

    ```sh
    # Create and install a virtual environment
    uv sync --python 3.13 --all-extras

    # Activate the virtual environment
    source .venv/bin/activate

    # Install the pre-commit hooks
    pre-commit install --install-hooks
    ```

1. _VS Code Dev Container_: clone this repository, open it with VS Code, and run <kbd>Ctrl/⌘</kbd> + <kbd>⇧</kbd> + <kbd>P</kbd> → _Dev Containers: Reopen in Container_.
1. _PyCharm Dev Container_: clone this repository, open it with PyCharm, [create a Dev Container with Mount Sources](https://www.jetbrains.com/help/pycharm/start-dev-container-inside-ide.html), and [configure an existing Python interpreter](https://www.jetbrains.com/help/pycharm/configuring-python-interpreter.html#widget) at `/opt/venv/bin/python`.

</details>

<details open>
<summary>Developing</summary>

- This project follows the [Conventional Commits](https://www.conventionalcommits.org/) standard to automate [Semantic Versioning](https://semver.org/) and [Keep A Changelog](https://keepachangelog.com/) with [Commitizen](https://github.com/commitizen-tools/commitizen).
- Run `poe` from within the development environment to print a list of [Poe the Poet](https://github.com/nat-n/poethepoet) tasks available to run on this project.
- Run `uv add {package}` from within the development environment to install a run time dependency and add it to `pyproject.toml` and `uv.lock`. Add `--dev` to install a development dependency.
- Run `uv sync --upgrade` from within the development environment to upgrade all dependencies to the latest versions allowed by `pyproject.toml`. Add `--only-dev` to upgrade the development dependencies only.
- Run `cz bump` to bump the app's version, update the `CHANGELOG.md`, and create a git tag. Then push the changes and the git tag with `git push origin main --tags`.
- Run `poe mutmut` to analyse test coverage via mutation.
- The library can be installed in editable mode using `python -m pip install -e .`.

</details>
