Metadata-Version: 2.1
Name: llm-http-api
Version: 0.0.1
Summary: HTTP API for LLM with OpenAI compatibility
Author-email: Alex Kwiatkowski <alex+python@fremantle.io>
License: MIT
Project-URL: Homepage, https://github.com/rupurt/llm-http-api
Project-URL: Issues, https://github.com/rupurt/llm-http-api/issues
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: llm >=0.0.12
Requires-Dist: fastapi >=0.104.0
Requires-Dist: pydantic >=2.5.0
Requires-Dist: pydantic-settings >=2.1.0
Requires-Dist: pyhamcrest >=2.1.0
Requires-Dist: pyinstaller >=6.2.0
Requires-Dist: uvicorn >=0.24.0
Provides-Extra: test
Requires-Dist: build >=1.0.3 ; extra == 'test'
Requires-Dist: httpx >=0.25.2 ; extra == 'test'
Requires-Dist: llm-clip >=0.1 ; extra == 'test'
Requires-Dist: llm-embed-jina >=0.1.2 ; extra == 'test'
Requires-Dist: llm-gpt4all >=0.2 ; extra == 'test'
Requires-Dist: llama-cpp-python >=0.2.23 ; extra == 'test'
Requires-Dist: llm-llama-cpp >=0.3 ; extra == 'test'
Requires-Dist: llm-mlc >=0.5 ; extra == 'test'
Requires-Dist: llm-sentence-transformers >=0.1.2 ; extra == 'test'
Requires-Dist: pytest ~=7.4.0 ; extra == 'test'
Requires-Dist: pytest-cov ~=4.1.0 ; extra == 'test'
Requires-Dist: ruff ~=0.1.0 ; extra == 'test'
Requires-Dist: twine >=4.0.2 ; extra == 'test'

# llm-http-api

HTTP API for [LLM](https://github.com/simonw/llm) with OpenAI compatibility

## Usage

```shell
> llm http-api --help
Usage: llm http-api [OPTIONS]

  Run a FastAPI HTTP server with OpenAI compatibility

Options:
  -h, --host TEXT       [default: 0.0.0.0]
  -p, --port INTEGER    [default: 8080]
  -l, --log-level TEXT  [default: info]
  --help                Show this message and exit.
```

```shell
> curl http://localhost:8080/v1/embeddings -X POST -H "Content-Type: application/json" -d '{
  "input": "Hello world",
  "model": "jina-embeddings-v2-small-en"
}'
{"object":"embedding","embedding":[-0.47561466693878174,-0.4471365511417389,...],"index":0}
```

## OpenAI Endpoints

### Embeddings

- [x] [`POST /v1/embeddings`](./docs/endpoints/EMBEDDINGS.md)

### Unimplemented

A detailed list of unimplemented OpenAI endpoints can be found [here](./docs/endpoints/UNIMPLEMENTED.md)

## Development

This repository manages the dev environment as a Nix flake and requires [Nix to be installed](https://github.com/DeterminateSystems/nix-installer)

```shell
nix develop -c $SHELL
```

```shell
make deps.install
make deps.install/test
```

```shell
make test
```

```shell
make coverage
```

```shell
make lint
```

```shell
make format
```
