<project title="Docs" summary="The purpose of this repository is to serve as the central, well-organized documentation hub for LM Studio, enabling users to understand how to install, configure, and use the platform effectively. It targets both casual users (e.g., those using the GUI) and developers (e.g., integrating via API or CLI). The documentation supports multiple programming languages and platforms, offering guides on model management, API usage, advanced features like speculative decoding and RAG, and even custom plugin development. It also facilitates community contributions through clear guidelines for adding new content.">**Remember:**
- LM Studio: a local AI model hosting platform
- Markdown-based documentation structure with hierarchical sections
- Frontmatter metadata for article ordering and sidebar display
- Multi-language code snippets with `lms_code_snippet` component
- API endpoints and OpenAI-compatible endpoints
- Plugin system for customizing behavior (tools, prompt preprocessors, generators)<docs><doc title="Authentication" desc="install &amp; quickstart.">---
title: Authentication
sidebar_title: Authentication
description: Using API Tokens in LM Studio
index: 3
---

##### Requires [LM Studio 0.4.0](/download) or newer.

LM Studio supports API Tokens for authentication, providing a secure and convenient way to access the LM Studio API.

By default, the LM Studio API runs **without enforcing authentication**. For production or shared environments, enable API Token authentication for secure access.

```lms_info
To enable API Token authentication, create tokens and control granular permissions, check [this guide](/docs/developer/core/authentication) for more details.
```

## Providing the API Token

The API Token can be provided in two ways:

1. **Environment Variable (Recommended)**: Set the `LM_API_TOKEN` environment variable, and the SDK will automatically read it.
2. **Function Argument**: Pass the token directly as the `api_token` parameter.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        # Configure the default client with an API token
        lms.configure_default_client(api_token="your-token-here")

        model = lms.llm()
        result = model.respond("What is the meaning of life?")
        print(result)

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        # Pass api_token to the Client constructor
        with lms.Client(api_token="your-token-here") as client:
            model = client.llm.model()
            result = model.respond("What is the meaning of life?")
            print(result)

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        # Pass api_token to the AsyncClient constructor
        async with lms.AsyncClient(api_token="your-token-here") as client:
            model = await client.llm.model()
            result = await model.respond("What is the meaning of life?")
            print(result)
```</doc><doc title="Project Setup" desc="install &amp; quickstart.">---
title: "Project Setup"
sidebar_title: "Project Setup"
description: "Set up your `lmstudio-python` app or script."
index: 2
---

`lmstudio` is a library published on PyPI that allows you to use `lmstudio-python` in your own projects.
It is open source and developed on GitHub.
You can find the source code [here](https://github.com/lmstudio-ai/lmstudio-python).

## Installing `lmstudio-python`

As it is published to PyPI, `lmstudio-python` may be installed using `pip`
or your preferred project dependency manager (`pdm` and `uv` are shown, but other
Python project management tools offer similar dependency addition commands).

```lms_code_snippet
  variants:
    pip:
      language: bash
      code: |
        pip install lmstudio
    pdm:
      language: bash
      code: |
        pdm add lmstudio
    uv:
      language: bash
      code: |
        uv add lmstudio
```

## Customizing the server API host and TCP port

All of the examples in the documentation assume that the server API is running locally
on one of the default application ports (Note: in Python SDK versions prior to 1.5.0, the
SDK also required that the optional HTTP REST server be enabled).

The network location of the server API can be overridden by
passing a `"host:port"` string when creating the client instance.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms
        SERVER_API_HOST = "localhost:1234"

        # This must be the *first* convenience API interaction (otherwise the SDK
        # implicitly creates a client that accesses the default server API host)
        lms.configure_default_client(SERVER_API_HOST)

        # Note: the dedicated configuration API was added in lmstudio-python 1.3.0
        # For compatibility with earlier SDK versions, it is still possible to use
        # lms.get_default_client(SERVER_API_HOST) to configure the default client

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms
        SERVER_API_HOST = "localhost:1234"

        # When using the scoped resource API, each client instance
        # can be configured to use a specific server API host
        with lms.Client(SERVER_API_HOST) as client:
            model = client.llm.model()

            for fragment in model.respond_stream("What is the meaning of life?"):
                print(fragment.content, end="", flush=True)
            print() # Advance to a new line at the end of the response

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms
        SERVER_API_HOST = "localhost:1234"

        # When using the asynchronous API, each client instance
        # can be configured to use a specific server API host
        async with lms.AsyncClient(SERVER_API_HOST) as client:
            model = await client.llm.model()

            for fragment in await model.respond_stream("What is the meaning of life?"):
                print(fragment.content, end="", flush=True)
            print() # Advance to a new line at the end of the response
```

### Checking a specified API server host is running

*Required Python SDK version*: **1.5.0**

While the most common connection pattern is to let the SDK raise an exception if it can't
connect to the specified API server host, the SDK also supports running the API check directly
without creating an SDK client instance first:

```lms_code_snippet
  variants:
    "Python (synchronous API)":
      language: python
      code: |
        import lmstudio as lms
        SERVER_API_HOST = "localhost:1234"

        if lms.Client.is_valid_api_host(SERVER_API_HOST):
            print(f"An LM Studio API server instance is available at {SERVER_API_HOST}")
        else:
            print("No LM Studio API server instance found at {SERVER_API_HOST}")

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms
        SERVER_API_HOST = "localhost:1234"

        if await lms.AsyncClient.is_valid_api_host(SERVER_API_HOST):
            print(f"An LM Studio API server instance is available at {SERVER_API_HOST}")
        else:
            print("No LM Studio API server instance found at {SERVER_API_HOST}")
```


### Determining the default local API server port

*Required Python SDK version*: **1.5.0**

When no API server host is specified, the SDK queries a number of ports on the local loopback
interface for a running API server instance. This scan is repeated for each new client instance
created. Rather than letting the SDK perform this scan implicitly, the SDK also supports running
the scan explicitly, and passing in the reported API server details when creating clients:

```lms_code_snippet
  variants:
    "Python (synchronous API)":
      language: python
      code: |
        import lmstudio as lms

        api_host = lms.Client.find_default_local_api_host()
        if api_host is not None:
            print(f"An LM Studio API server instance is available at {api_host}")
          else:
            print("No LM Studio API server instance found on any of the default local ports")

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        api_host = await lms.AsyncClient.find_default_local_api_host()
        if api_host is not None:
            print(f"An LM Studio API server instance is available at {api_host}")
          else:
            print("No LM Studio API server instance found on any of the default local ports")
```</doc><doc title="Quickstart" desc="install &amp; quickstart.">---
title: Get up and running with the LM Studio API
sidebar_title: Quickstart
description: Download a model and start a simple Chat session using the REST API
fullPage: false
index: 2
---

## Start the server

[Install](/download) and launch LM Studio.

Then ensure the server is running through the toggle at the top left of the Developer page, or through [lms](/docs/cli) in the terminal:

```bash
lms server start
```

By default, the server is available at `http://localhost:1234`.

If you don't have a model downloaded yet, you can download the model:

```bash
lms get ibm/granite-4-micro
```


## API Authentication

By default, the LM Studio API server does **not** require authentication. You can configure the server to require authentication by API token in the [server settings](/docs/developer/core/server/settings) for added security.

To authenticate API requests, generate an API token from the Developer page in LM Studio, and include it in the `Authorization` header of your requests as follows: `Authorization: Bearer $LM_API_TOKEN`. Read more about authentication [here](/docs/developer/core/authentication).


## Chat with a model

Use the chat endpoint to send a message to a model. By default, the model will be automatically loaded if it is not already.

The `/api/v1/chat` endpoint is stateful, which means you do not need to pass the full history in every request. Read more about it [here](/docs/developer/rest/stateful-chats).

```lms_code_snippet
variants:
  curl:
    language: bash
    code: |
      curl http://localhost:1234/api/v1/chat \
        -H "Authorization: Bearer $LM_API_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{
          "model": "ibm/granite-4-micro",
          "input": "Write a short haiku about sunrise."
        }'
  Python:
    language: python
    code: |
      import os
      import requests
      import json

      response = requests.post(
        "http://localhost:1234/api/v1/chat",
        headers={
          "Authorization": f"Bearer {os.environ['LM_API_TOKEN']}",
          "Content-Type": "application/json"
        },
        json={
          "model": "ibm/granite-4-micro",
          "input": "Write a short haiku about sunrise."
        }
      )
      print(json.dumps(response.json(), indent=2))
  TypeScript:
    language: typescript
    code: |
      const response = await fetch("http://localhost:1234/api/v1/chat", {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${process.env.LM_API_TOKEN}`,
          "Content-Type": "application/json"
        },
        body: JSON.stringify({
          model: "ibm/granite-4-micro",
          input: "Write a short haiku about sunrise."
        })
      });
      const data = await response.json();
      console.log(data);
```

See the full [chat](/docs/developer/rest/chat) docs for more details.

## Use MCP servers via API


Enable the model interact with ephemeral Model Context Protocol (MCP) servers in `/api/v1/chat` by specifying servers in the `integrations` field.

```lms_code_snippet
variants:
  curl:
    language: bash
    code: |
      curl http://localhost:1234/api/v1/chat \
        -H "Authorization: Bearer $LM_API_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{
          "model": "ibm/granite-4-micro",
          "input": "What is the top trending model on hugging face?",
          "integrations": [
            {
              "type": "ephemeral_mcp",
              "server_label": "huggingface",
              "server_url": "https://huggingface.co/mcp",
              "allowed_tools": ["model_search"]
            }
          ],
          "context_length": 8000
        }'
  Python:
    language: python
    code: |
      import os
      import requests
      import json

      response = requests.post(
        "http://localhost:1234/api/v1/chat",
        headers={
          "Authorization": f"Bearer {os.environ['LM_API_TOKEN']}",
          "Content-Type": "application/json"
        },
        json={
          "model": "ibm/granite-4-micro",
          "input": "What is the top trending model on hugging face?",
          "integrations": [
            {
              "type": "ephemeral_mcp",
              "server_label": "huggingface",
              "server_url": "https://huggingface.co/mcp",
              "allowed_tools": ["model_search"]
            }
          ],
          "context_length": 8000
        }
      )
      print(json.dumps(response.json(), indent=2))
  TypeScript:
    language: typescript
    code: |
      const response = await fetch("http://localhost:1234/api/v1/chat", {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${process.env.LM_API_TOKEN}`,
          "Content-Type": "application/json"
        },
        body: JSON.stringify({
          model: "ibm/granite-4-micro",
          input: "What is the top trending model on hugging face?",
          integrations: [
            {
              type: "ephemeral_mcp",
              server_label: "huggingface",
              server_url: "https://huggingface.co/mcp",
              allowed_tools: ["model_search"]
            }
          ],
          context_length: 8000
        })
      const data = await response.json();
      console.log(data);
```

You can also use locally configured MCP plugins (from your `mcp.json`) via the `integrations` field. Using locally run MCP plugins requires authentication via an API token passed through the `Authorization` header. Read more about authentication [here](/docs/developer/core/authentication).

```lms_code_snippet
variants:
  curl:
    language: bash
    code: |
      curl http://localhost:1234/api/v1/chat \
        -H "Authorization: Bearer $LM_API_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{
          "model": "ibm/granite-4-micro",
          "input": "Open lmstudio.ai",
          "integrations": [
            {
              "type": "plugin",
              "id": "mcp/playwright",
              "allowed_tools": ["browser_navigate"]
            }
          ],
          "context_length": 8000
        }'
  Python:
    language: python
    code: |
      import os
      import requests
      import json

      response = requests.post(
        "http://localhost:1234/api/v1/chat",
        headers={
          "Authorization": f"Bearer {os.environ['LM_API_TOKEN']}",
          "Content-Type": "application/json"
        },
        json={
          "model": "ibm/granite-4-micro",
          "input": "Open lmstudio.ai",
          "integrations": [
            {
              "type": "plugin",
              "id": "mcp/playwright".
              "allowed_tools": ["browser_navigate"]
            }
          ],
          "context_length": 8000
        }
      )
      print(json.dumps(response.json(), indent=2))
  TypeScript:
    language: typescript
    code: |
      const response = await fetch("http://localhost:1234/api/v1/chat", {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${process.env.LM_API_TOKEN}`,
          "Content-Type": "application/json"
        },
        body: JSON.stringify({
          model: "ibm/granite-4-micro",
          input: "Open lmstudio.ai",
          integrations: [
            {
              type: "plugin",
              id: "mcp/playwright",
              allowed_tools: ["browser_navigate"]
            }
          ],
          context_length: 8000
        })
      });
      const data = await response.json();
      console.log(data);
```

See the full [chat](/docs/developer/rest/chat) docs for more details.

## Download a model

Use the download endpoint to download models by identifier from the [LM Studio model catalog](https://lmstudio.ai/models), or by Hugging Face model URL.

```lms_code_snippet
variants:
  curl:
    language: bash
    code: |
      curl http://localhost:1234/api/v1/models/download \
        -H "Authorization: Bearer $LM_API_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{
          "model": "ibm/granite-4-micro"
        }'
  Python:
    language: python
    code: |
      import os
      import requests
      import json

      response = requests.post(
        "http://localhost:1234/api/v1/models/download",
        headers={
          "Authorization": f"Bearer {os.environ['LM_API_TOKEN']}",
          "Content-Type": "application/json"
        },
        json={"model": "ibm/granite-4-micro"}
      )
      print(json.dumps(response.json(), indent=2))
  TypeScript:
    language: typescript
    code: |
      const response = await fetch("http://localhost:1234/api/v1/models/download", {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${process.env.LM_API_TOKEN}`,
          "Content-Type": "application/json"
        },
        body: JSON.stringify({
          model: "ibm/granite-4-micro"
        })
      });
      const data = await response.json();
      console.log(data);
```

The response will return a `job_id` that you can use to track download progress.

```lms_code_snippet
variants:
  curl:
    language: bash
    code: |
      curl -H "Authorization: Bearer $LM_API_TOKEN" \
        http://localhost:1234/api/v1/models/download/status/{job_id}
  Python:
    language: python
    code: |
      import os
      import requests
      import json

      job_id = "your-job-id"
      response = requests.get(
        f"http://localhost:1234/api/v1/models/download/status/{job_id}",
        headers={"Authorization": f"Bearer {os.environ['LM_API_TOKEN']}"}
      )
      print(json.dumps(response.json(), indent=2))
  TypeScript:
    language: typescript
    code: |
      const jobId = "your-job-id";
      const response = await fetch(
        `http://localhost:1234/api/v1/models/download/status/${jobId}`,
        {
          headers: {
            "Authorization": `Bearer ${process.env.LM_API_TOKEN}`
          }
        }
      );
      const data = await response.json();
      console.log(data);
```

See the [download](/docs/developer/rest/download) and [download status](/docs/developer/rest/download-status) docs for more details.</doc><doc title="Repl" desc="install &amp; quickstart.">---
title: "Using `lmstudio-python` in REPL"
sidebar_title: "REPL Usage"
description: "You can use `lmstudio-python` in REPL (Read-Eval-Print Loop) to interact with LLMs, manage models, and more."
index: 2
---

To simplify interactive use, `lmstudio-python` offers a convenience API which manages
its resources via `atexit` hooks, allowing a default synchronous client session
to be used across multiple interactive commands.

This convenience API is shown in the examples throughout the documentation as the
`Python (convenience API)` tab (alongside the `Python (scoped resource API)` examples,
which use `with` statements to ensure deterministic cleanup of network communication
resources).

The convenience API allows the standard Python REPL, or more flexible alternatives like
Juypter Notebooks, to be used to interact with AI models loaded into LM Studio. For
example:

```lms_code_snippet
  title: "Python REPL"
  variants:
    "Interactive chat session":
      language: python
      code: |
        >>> import lmstudio as lms
        >>> loaded_models = lms.list_loaded_models()
        >>> for idx, model in enumerate(loaded_models):
        ...     print(f"{idx:>3} {model}")
        ...
          0 LLM(identifier='qwen2.5-7b-instruct')
        >>> model = loaded_models[0]
        >>> chat = lms.Chat("You answer questions concisely")
        >>> chat = lms.Chat("You answer questions concisely")
        >>> chat.add_user_message("Tell me three fruits")
        UserMessage(content=[TextData(text='Tell me three fruits')])
        >>> print(model.respond(chat, on_message=chat.append))
        Banana, apple, orange.
        >>> chat.add_user_message("Tell me three more fruits")
        UserMessage(content=[TextData(text='Tell me three more fruits')])
        >>> print(model.respond(chat, on_message=chat.append))
        Mango, strawberry, avocado.
        >>> chat.add_user_message("How many fruits have you told me?")
        UserMessage(content=[TextData(text='How many fruits have you told me?')])
        >>> print(model.respond(chat, on_message=chat.append))
        You asked for three initial fruits and three more, so I've listed a total of six fruits.

```

While not primarily intended for use this way, the SDK's asynchronous structured concurrency API
is compatible with the asynchronous Python REPL that is launched by `python -m asyncio`.
For example:

```lms_code_snippet
  title: "Python REPL"
  variants:
    "Asynchronous chat session":
      language: python
      code: |
        # Note: assumes use of the "python -m asyncio" asynchronous REPL (or equivalent)
        # Requires Python SDK version 1.5.0 or later
        >>> from contextlib import AsyncExitStack
        >>> import lmstudio as lms
        >>> resources = AsyncExitStack()
        >>> client = await resources.enter_async_context(lms.AsyncClient())
        >>> loaded_models = await client.llm.list_loaded()
        >>> for idx, model in enumerate(loaded_models):
        ...     print(f"{idx:>3} {model}")
        ...
          0 AsyncLLM(identifier='qwen2.5-7b-instruct-1m')
        >>> model = loaded_models[0]
        >>> chat = lms.Chat("You answer questions concisely")
        >>> chat.add_user_message("Tell me three fruits")
        UserMessage(content=[TextData(text='Tell me three fruits')])
        >>> print(await model.respond(chat, on_message=chat.append))
        Apple, banana, and orange.
        >>> chat.add_user_message("Tell me three more fruits")
        UserMessage(content=[TextData(text='Tell me three more fruits')])
        >>> print(await model.respond(chat, on_message=chat.append))
        Mango, strawberry, and pineapple.
        >>> chat.add_user_message("How many fruits have you told me?")
        UserMessage(content=[TextData(text='How many fruits have you told me?')])
        >>> print(await model.respond(chat, on_message=chat.append))
        You asked for three fruits initially, then three more, so I've listed six fruits in total.

```</doc></docs><tutorials><doc title="Examples" desc="worked example.">---
title: "Examples"
+description: "Example prompt preprocessors for LM Studio plugins"
index: 2
---

### Example: Inject Current Time

The following is an example preprocessor that injects the current time before each user message.

```lms_code_snippet
  title: "src/promptPreprocessor.ts"
  variants:
    TypeScript:
      language: typescript
      code: |
        import { type PromptPreprocessorController, type ChatMessage } from "@lmstudio/sdk";
        export async function preprocess(ctl: PromptPreprocessorController, userMessage: ChatMessage) {
          const textContent = userMessage.getText();
          const transformed = `Current time: ${new Date().toString()}\n\n${textContent}`;
          return transformed;
        }
```

### Example: Replace Trigger Words

Another example you can do it with simple text only processing is by replacing certain trigger words. For example, you can replace a `@init` trigger with a special initialization message.

```lms_code_snippet
  title: "src/promptPreprocessor.ts"
  variants:
    TypeScript:
      language: typescript
      code: |
        import { type PromptPreprocessorController, type ChatMessage, text } from "@lmstudio/sdk";

        const mySpecialInstructions = text`
          Here are some special instructions...
        `;

        export async function preprocess(ctl: PromptPreprocessorController, userMessage: ChatMessage) {
          const textContent = userMessage.getText();
          const transformed = textContent.replaceAll("@init", mySpecialInstructions);
          return transformed;
        }
```</doc></tutorials><api><doc title="Api Changelog" desc="API reference.">---
title: API Changelog
description: Updates and changes to the LM Studio API.
index: 2
---

---

###### LM Studio 0.3.29 • 2025‑10‑06

### OpenAI `/v1/responses` and variant listing

- New OpenAI‑compatible endpoint: `POST /v1/responses`.
  - Stateful interactions via `previous_response_id`.
  - Custom tool calling and Remote MCP support (opt‑in).
  - Reasoning support with `reasoning.effort` for `openai/gpt‑oss‑20b`.
  - Streaming via SSE when `stream: true`.
- CLI: `lms ls --variants` lists all variants for multi‑variant models.
- Docs: [/docs/developer/openai-compat](/docs/developer/openai-compat). Full release notes: [/blog/lmstudio-v0.3.29](/blog/lmstudio-v0.3.29).

---

###### LM Studio 0.3.27 • 2025‑09‑24

### CLI: model resource estimates, status, and interrupts

- New: `lms load --estimate-only <model>` prints estimated GPU and total memory before loading. Honors `--context-length` and `--gpu`, and uses an improved estimator that now accounts for flash attention and vision models.
- `lms chat`: press `Ctrl+C` to interrupt an ongoing prediction.
- `lms ps --json` now reports each model's generation status and the number of queued prediction requests.
- CLI color contrast improved for light mode.
- See docs: [/docs/cli/local-models/load](/docs/cli/local-models/load). Full release notes: [/blog/lmstudio-v0.3.27](/blog/lmstudio-v0.3.27).

---

###### LM Studio 0.3.26 • 2025‑09‑15

### CLI log streaming: server + model

- `lms log stream` now supports multiple sources and filters.
  - `--source server` streams HTTP server logs (startup, endpoints, status)
  - `--source model --filter input,output` streams formatted user input and model output
  - Append `--json` for machine‑readable logs; `--stats` adds tokens/sec and related metrics (model source)
- See usage and examples: [/docs/cli/serve/log-stream](/docs/cli/serve/log-stream). Full release notes: [/blog/lmstudio-v0.3.26](/blog/lmstudio-v0.3.26).

---

###### LM Studio 0.3.25 • 2025‑09‑04

### New model support (API)

- Added support for NVIDIA Nemotron‑Nano‑v2 with tool‑calling via the OpenAI‑compatible endpoints [‡](/blog/lmstudio-v0.3.25).
- Added support for Google EmbeddingGemma for the `/v1/embeddings` endpoint [‡](/blog/lmstudio-v0.3.25).

---

###### LM Studio 0.3.24 • 2025‑08‑28

### Seed‑OSS tool‑calling and template fixes

- Added support for ByteDance/Seed‑OSS including tool‑calling and prompt‑template compatibility fixes in the OpenAI‑compatible API [‡](/blog/lmstudio-v0.3.24).
- Fixed cases where tool calls were not parsed for certain prompt templates [‡](/blog/lmstudio-v0.3.24).

---

###### LM Studio 0.3.23 • 2025‑08‑12

### Reasoning content and tool‑calling reliability

- For `gpt‑oss` on `POST /v1/chat/completions`, reasoning content moves out of `message.content` and into `choices.message.reasoning` (non‑streaming) and `choices.delta.reasoning` (streaming), aligning with `o3‑mini` [‡](/blog/lmstudio-v0.3.23).
- Tool names are normalized (e.g., snake_case) before being provided to the model to improve tool‑calling reliability [‡](/blog/lmstudio-v0.3.23).
- Fixed errors for certain tools‑containing requests to `POST /v1/chat/completions` (e.g., "reading 'properties'") and non‑streaming tool‑call failures [‡](/blog/lmstudio-v0.3.23).

---

###### LM Studio 0.3.19 • 2025‑07‑21

### Bug fixes for streaming and tool calls

- Corrected usage statistics returned by OpenAI‑compatible streaming responses [‡](https://lmstudio.ai/blog/lmstudio-v0.3.19#:~:text=,OpenAI%20streaming%20responses%20were%20incorrect).
- Improved handling of parallel tool calls via the streaming API [‡](https://lmstudio.ai/blog/lmstudio-v0.3.19#:~:text=,API%20were%20not%20handled%20correctly).
- Fixed parsing of correct tool calls for certain Mistral models [‡](https://lmstudio.ai/blog/lmstudio-v0.3.19#:~:text=,Ryzen%20AI%20PRO%20300%20series).

---

###### LM Studio 0.3.18 • 2025‑07‑10

### Streaming options and tool‑calling improvements

- Added support for the `stream_options` object on OpenAI‑compatible endpoints. Setting `stream_options.include_usage` to `true` returns prompt and completion token usage during streaming [‡](https://lmstudio.ai/blog/lmstudio-v0.3.18#:~:text=%2A%20Added%20support%20for%20%60,to%20support%20more%20prompt%20templates).
- Errors returned from streaming endpoints now follow the correct format expected by OpenAI clients [‡](https://lmstudio.ai/blog/lmstudio-v0.3.18#:~:text=,with%20proper%20chat%20templates).
- Tool‑calling support added for Mistral v13 tokenizer models, using proper chat templates [‡](https://lmstudio.ai/blog/lmstudio-v0.3.18#:~:text=,with%20proper%20chat%20templates).
- The `response_format.type` field now accepts `"text"` in chat‑completion requests [‡](https://lmstudio.ai/blog/lmstudio-v0.3.18#:~:text=,that%20are%20split%20across%20multiple).
- Fixed bugs where parallel tool calls split across multiple chunks were dropped and where root‑level `$defs` in tool definitions were stripped [‡](https://lmstudio.ai/blog/lmstudio-v0.3.18#:~:text=,being%20stripped%20in%20tool%20definitions).

---

###### LM Studio 0.3.17 • 2025‑06‑25

### Tool‑calling reliability and token‑count updates

- Token counts now include the system prompt and tool definitions [‡](https://lmstudio.ai/blog/lmstudio-v0.3.17#:~:text=,have%20a%20URL%20in%20the). This makes usage reporting more accurate for both the UI and the API.
- Tool‑call argument tokens are streamed as they are generated [‡](https://lmstudio.ai/blog/lmstudio-v0.3.17#:~:text=Build%206), improving responsiveness when using streamed function calls.
- Various fixes improve MCP and tool‑calling reliability, including correct handling of tools that omit a `parameters` object and preventing hangs when an MCP server reloads [‡](https://lmstudio.ai/blog/lmstudio-v0.3.17#:~:text=,tool%20calls%20would%20hang%20indefinitely).

---

###### LM Studio 0.3.16 • 2025‑05‑23

### Model capabilities in `GET /models`

- The OpenAI‑compatible REST API (`/api/v0`) now returns a `capabilities` array in the `GET /models` response. Each model lists its supported capabilities (e.g. `"tool_use"`) [‡](https://lmstudio.ai/blog/lmstudio-v0.3.16#:~:text=,response) so clients can programmatically discover tool‑enabled models.
- Fixed a streaming bug where an empty function name string was appended after the first packet of streamed tool calls [‡](https://lmstudio.ai/blog/lmstudio-v0.3.16#:~:text=%2A%20Bugfix%3A%20%5BOpenAI,packet%20of%20streamed%20function%20calls).

---

###### [👾 LM Studio 0.3.15](/blog/lmstudio-v0.3.15) • 2025-04-24

### Improved Tool Use API Support

OpenAI-like REST API now supports the `tool_choice` parameter:

```json
{
  "tool_choice": "auto" // or "none", "required"
}
```

- `"tool_choice": "none"` — Model will not call tools
- `"tool_choice": "auto"` — Model decides
- `"tool_choice": "required"` — Model must call tools (llama.cpp only)

Chunked responses now set `"finish_reason": "tool_calls"` when appropriate.

---

###### [👾 LM Studio 0.3.14](/blog/lmstudio-v0.3.14) • 2025-03-27

### [API/SDK] Preset Support

RESTful API and SDKs support specifying presets in requests.

_(example needed)_

###### [👾 LM Studio 0.3.10](/blog/lmstudio-v0.3.10) • 2025-02-18

### Speculative Decoding API

Enable speculative decoding in API requests with `"draft_model"`:

```json
{
  "model": "deepseek-r1-distill-qwen-7b",
  "draft_model": "deepseek-r1-distill-qwen-0.5b",
  "messages": [ ... ]
}
```

Responses now include a `stats` object for speculative decoding:

```json
"stats": {
  "tokens_per_second": ...,
  "draft_model": "...",
  "total_draft_tokens_count": ...,
  "accepted_draft_tokens_count": ...,
  "rejected_draft_tokens_count": ...,
  "ignored_draft_tokens_count": ...
}
```

---

###### [👾 LM Studio 0.3.9](blog/lmstudio-v0.3.9) • 2025-01-30

### Idle TTL and Auto Evict

Set a TTL (in seconds) for models loaded via API requests (docs article: [Idle TTL and Auto-Evict](/docs/developer/core/ttl-and-auto-evict))

```diff
curl http://localhost:1234/api/v0/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-r1-distill-qwen-7b",
    "messages": [ ... ]
+   "ttl": 300,
}'
```

With `lms`:

```
lms load --ttl <seconds>
```

### Separate `reasoning_content` in Chat Completion responses

For DeepSeek R1 models, get reasoning content in a separate field. See more [here](/blog/lmstudio-v0.3.9#separate-reasoningcontent-in-chat-completion-responses).

Turn this on in App Settings > Developer.

---

###### [👾 LM Studio 0.3.6](blog/lmstudio-v0.3.6) • 2025-01-06

### Tool and Function Calling API

Use any LLM that supports Tool Use and Function Calling through the OpenAI-like API.

Docs: [Tool Use and Function Calling](/docs/developer/core/tools).

---

###### [👾 LM Studio 0.3.5](blog/lmstudio-v0.3.5) • 2024-10-22

### Introducing `lms get`: download models from the terminal

You can now download models directly from the terminal using a keyword

```bash
lms get deepseek-r1
```

or a full Hugging Face URL

```bash
lms get <hugging face url>
```

To filter for MLX models only, add `--mlx` to the command.

```bash
lms get deepseek-r1 --mlx
```</doc><doc title="Act" desc="API reference.">---
title: "`.act()`"
sidebar_title: "`.act()`"
description: ".act() - API reference for automatic tool use in a multi-turn chat conversation"
index: 3
---

`.act()` is a method that generates automatic tool use in a multi-turn chat conversation.</doc><doc title="Act" desc="API reference.">---
title: "`.act()`"
sidebar_title: "`.act()`"
description: ".act() - API reference for automatic tool use in a multi-turn chat conversation"
index: 3
---

`.act()` is a method that generates automatic tool use in a multi-turn chat conversation.</doc><doc title="Chat" desc="API reference.">---
title: "`Chat`"
sidebar_title: "`Chat`"
description: "`Chat` - API reference for representing a chat conversation with an LLM"
index: 5
---

...</doc><doc title="Chat" desc="API reference.">---
title: "`Chat`"
sidebar_title: "`Chat`"
description: "`Chat` - API reference for representing a chat conversation with an LLM"
index: 5
---

...</doc><doc title="Complete" desc="API reference.">---
title: "`.complete()`"
sidebar_title: "`.complete()`"
description: ".complete() - API reference for generating text completions from a loaded language model"
index: 4
---

`.complete()` is a method that generates text completions from a loaded language model.</doc><doc title="Complete" desc="API reference.">---
title: "`.complete()`"
sidebar_title: "`.complete()`"
description: ".complete() - API reference for generating text completions from a loaded language model"
index: 4
---

`.complete()` is a method that generates text completions from a loaded language model.</doc><doc title="Count Tokens" desc="API reference.">---
title: "`.countTokens()`"
sidebar_title: "`.countTokens()`"
description: ".countTokens() - API reference for counting tokens in a string using a model's tokenizer"
---

`.countTokens()` is a method that counts the number of tokens in a string using a model's tokenizer. This method is useful for situations such as assessing whether a string could fit in the model's remaining context window.</doc><doc title="Count Tokens" desc="API reference.">---
title: "`.countTokens()`"
sidebar_title: "`.countTokens()`"
description: ".countTokens() - API reference for counting tokens in a string using a model's tokenizer"
---

`.countTokens()` is a method that counts the number of tokens in a string using a model's tokenizer. This method is useful for situations such as assessing whether a string could fit in the model's remaining context window.</doc><doc title="Embed" desc="API reference.">---
title: "`.embed()`"
sidebar_title: "`.embed()`"
description: ".embed() - API reference for generating embeddings from a loaded embedding model"
---

`.embed()` is a method that generates embeddings from a loaded embedding model.</doc></api><configuration><doc title="Inference" desc="docs page.">---
title: Inference parameters
description: Configurable parameters for inference in LM Studio
---

### Page

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam auctor, nunc nec suscipit ultricies, nunc nunc ultricies nunc, nec suscipit nunc nunc nec. Nullam.</doc><doc title="Lm Runtimes" desc="docs page.">---
title: LM Runtimes
description: Downloading, installing, and using different LM Runtimes in LM Studio
---

### Page

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam auctor, nunc nec suscipit ultricies, nunc nunc ultricies nunc, nec suscipit nunc nunc nec. Nullam.</doc><doc title="Load" desc="docs page.">---
title: Load parameters
description: Configurable parameters for load in LM Studio
---

### Page

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam auctor, nunc nec suscipit ultricies, nunc nunc ultricies nunc, nec suscipit nunc nunc nec. Nullam.</doc></configuration><0 app><doc title="0 Root" desc="docs page.">---
title: Welcome to LM Studio Docs!
sidebar_title: Welcome
description: Learn how to run Llama, DeepSeek, Qwen, Phi, and other LLMs locally with LM Studio.
index: 1
---

To get LM Studio, head over to the [Downloads page](/download) and download an installer for your operating system.

LM Studio is available for macOS, Windows, and Linux.

## What can I do with LM Studio?

1. Download and run local LLMs like gpt-oss or Llama, Qwen
2. Use a simple and flexible chat interface
3. Connect MCP servers and use them with local models
4. Search & download functionality (via Hugging Face 🤗)
5. Serve local models on OpenAI-like endpoints, locally and on the network
6. Manage your local models, prompts, and configurations

## System requirements

LM Studio generally supports Apple Silicon Macs, x64/ARM64 Windows PCs, and x64 Linux PCs.

Consult the [System Requirements](app/system-requirements) page for more detailed information.

## Run llama.cpp (GGUF) or MLX models

LM Studio supports running LLMs on Mac, Windows, and Linux using [`llama.cpp`](https://github.com/ggerganov/llama.cpp).

On Apple Silicon Macs, LM Studio also supports running LLMs using Apple's [`MLX`](https://github.com/ml-explore/mlx).

To install or manage LM Runtimes, press `⌘` `Shift` `R` on Mac or `Ctrl` `Shift` `R` on Windows/Linux.

## LM Studio as an MCP client

You can install MCP servers in LM Studio and use them with your local models.

See the docs for more: [Use MCP server](/docs/app/plugins/mcp).

If you're develping an MCP server, check out [Add to LM Studio Button](/docs/app/plugins/mcp/deeplink).

## Run an LLM like `gpt-oss`, `Llama`, `Qwen`, `Mistral`, or `DeepSeek R1` on your computer

To run an LLM on your computer you first need to download the model weights.

You can do this right within LM Studio! See [Download an LLM](app/basics/download-model) for guidance.

## Chat with documents entirely offline on your computer

You can attach documents to your chat messages and interact with them entirely offline, also known as "RAG".

Read more about how to use this feature in the [Chat with Documents](app/basics/rag) guide.

## Use LM Studio's API from your own apps and scripts

LM Studio provides a REST API that you can use to interact with your local models from your own apps and scripts.

- [OpenAI Compatibility API](api/openai-api)
- [LM Studio REST API (beta)](api/rest-api)

<br />

## Community

Join the LM Studio community on [Discord](https://discord.gg/aPQfnNkxGC) to ask questions, share knowledge, and get help from other users and the LM Studio team.</doc><doc title="1 Basics" desc="docs page.">---
title: Get started with LM Studio
sidebar_title: Overview
description: Download and run Large Language Models like Qwen, Mistral, Gemma, or gpt-oss in LM Studio.
index: 1
---

Double check computer meets the minimum [system requirements](/docs/system-requirements).

```lms_info
You might sometimes see terms such as `open-source models` or `open-weights models`. Different models might be released under different licenses and varying degrees of 'openness'. In order to run a model locally, you need to be able to get access to its "weights", often distributed as one or more files that end with `.gguf`, `.safetensors` etc.
```

<hr>

## Getting up and running

First, **install the latest version of LM Studio**. You can get it from [here](/download).

Once you're all set up, you need to **download your first LLM**.

### 1. Download an LLM to your computer

Head over to the Discover tab to download models. Pick one of the curated options or search for models by search query (e.g. `"Llama"`). See more in-depth information about downloading models [here](/docs/basics/download-models).

<img src="/assets/docs/discover.png" style="width: 500px; margin-top:30px" data-caption="The Discover tab in LM Studio" />

### 2. Load a model to memory

Head over to the **Chat** tab, and

1. Open the model loader
2. Select one of the models you downloaded (or [sideloaded](/docs/advanced/sideload)).
3. Optionally, choose load configuration parameters.

<img src="/assets/docs/loader.png" data-caption="Quickly open the model loader with `cmd` + `L` on macOS or `ctrl` + `L` on Windows/Linux" />

##### What does loading a model mean?

Loading a model typically means allocating memory to be able to accommodate the model's weights and other parameters in your computer's RAM.

### 3. Chat!

Once the model is loaded, you can start a back-and-forth conversation with the model in the Chat tab.

<img src="/assets/docs/chat.png" data-caption="LM Studio on macOS" />

<hr>

### Community

Chat with other LM Studio users, discuss LLMs, hardware, and more on the [LM Studio Discord server](https://discord.gg/aPQfnNkxGC).</doc><doc title="2 Mcp" desc="docs page.">---
title: Use MCP Servers
description: Connect MCP servers to LM Studio
index: 1
---

Starting LM Studio 0.3.17, LM Studio acts as an **Model Context Protocol (MCP) Host**. This means you can connect MCP servers to the app and make them available to your models.

### Be cautious

Never install MCPs from untrusted sources.

```lms_warning
Some MCP servers can run arbitrary code, access your local files, and use your network connection. Always be cautious when installing and using MCP servers. If you don't trust the source, don't install it.
```

# Use MCP servers in LM Studio

Starting 0.3.17 (b10), LM Studio supports both local and remote MCP servers. You can add MCPs by editing the app's `mcp.json` file or via the ["Add to LM Studio" Button](mcp/deeplink), when available. LM Studio currently follows Cursor's `mcp.json` notation.

## Install new servers: `mcp.json`

Switch to the "Program" tab in the right hand sidebar. Click `Install > Edit mcp.json`.

<img src="/assets/docs/install-mcp.png"  data-caption="" style="width: 80%;" className="" />

This will open the `mcp.json` file in the in-app editor. You can add MCP servers by editing this file.

<img src="/assets/docs/mcp-editor.png"  data-caption="Edit mcp.json using the in-app editor" style="width: 100%;" className="" />

### Example MCP to try: Hugging Face MCP Server

This MCP server provides access to functions like model and dataset search.

<div className="w-fit">
  <a style="background: rgb(255,255,255)" href="https://lmstudio.ai/install-mcp?name=hf-mcp-server&config=eyJ1cmwiOiJodHRwczovL2h1Z2dpbmdmYWNlLmNvL21jcCIsImhlYWRlcnMiOnsiQXV0aG9yaXphdGlvbiI6IkJlYXJlciA8WU9VUl9IRl9UT0tFTj4ifX0%3D">
    <LightVariant>
      <img src="https://files.lmstudio.ai/deeplink/mcp-install-light.svg" alt="Add MCP Server hf-mcp-server to LM Studio" />
    </LightVariant>
    <DarkVariant>
      <img src="https://files.lmstudio.ai/deeplink/mcp-install-dark.svg" alt="Add MCP Server hf-mcp-server to LM Studio" />
    </DarkVariant>
  </a>
</div>

```json
{
  "mcpServers": {
    "hf-mcp-server": {
      "url": "https://huggingface.co/mcp",
      "headers": {
        "Authorization": "Bearer <YOUR_HF_TOKEN>"
      }
    }
  }
}
```

###### You will need to replace `<YOUR_HF_TOKEN>` with your actual Hugging Face token. Learn more [here](https://huggingface.co/docs/hub/en/security-tokens).

Use the [deeplink button](mcp/deeplink), or copy the JSON snippet above and paste it into your `mcp.json` file.

---

## Gotchas and Troubleshooting

- Never install MCP servers from untrusted sources. Some MCPs can have far reaching access to your system.

- Some MCP servers were designed to be used with Claude, ChatGPT, Gemini and might use excessive amounts of tokens.

  - Watch out for this. It may quickly bog down your local model and trigger frequent context overflows.

- When adding MCP servers manually, copy only the content after `"mcpServers": {` and before the closing `}`.</doc><doc title="3 Modelyaml" desc="docs page.">---
title: "Introduction to `model.yaml`"
description: Describe models with the cross-platform `model.yaml` specification.
index: 5
socialCard: 
  url: https://files.lmstudio.ai/modelyaml-card.jpg
  alt: "model.yaml logo"
---

`Draft`

[`model.yaml`](https://modelyaml.org) describes a model and all of its variants in a single portable file. Models in LM Studio's [model catalog](https://lmstudio.ai/models) are all implemented using model.yaml.

This allows abstracting away the underlying format (GGUF, MLX, etc) and presenting a single entry point for a given model. Furthermore, the model.yaml file supports baking in additional metadata, load and inference options, and even custom logic (e.g. enable/disable thinking).

**You can clone existing model.yaml files on the LM Studio Hub and even [publish your own](./modelyaml/publish)!**

## Core fields

### `model`

The canonical identifier in the form `publisher/model`.

```yaml
model: qwen/qwen3-8b
```

### `base`

Points to the "concrete" model files or other virtual models. Each entry uses a unique `key` and one or more `sources` from which the file can be fetched.

The snippet below demonstrates a case where the model (`qwen/qwen3-8b`) can resolve to one of 3 different concrete models.

```yaml
model: qwen/qwen3-8b
base:
  - key: lmstudio-community/qwen3-8b-gguf
    sources:
      - type: huggingface
        user: lmstudio-community
        repo: Qwen3-8B-GGUF
  - key: lmstudio-community/qwen3-8b-mlx-4bit
    sources:
      - type: huggingface
        user: lmstudio-community
        repo: Qwen3-8B-MLX-4bit
  - key: lmstudio-community/qwen3-8b-mlx-8bit
    sources:
      - type: huggingface
        user: lmstudio-community
        repo: Qwen3-8B-MLX-8bit
```

Concrete model files refer to the actual weights.

### `metadataOverrides`

Overrides the base model's metadata. This is useful for presentation purposes, for example in LM Studio's model catalog or in app model search. It is not used for any functional changes to the model.

```yaml
metadataOverrides:
  domain: llm
  architectures:
    - qwen3
  compatibilityTypes:
    - gguf
    - safetensors
  paramsStrings:
    - 8B
  minMemoryUsageBytes: 4600000000
  contextLengths:
    - 40960
  vision: false
  reasoning: true
  trainedForToolUse: true
```

### `config`

Use this to "bake in" default runtime settings (such as sampling parameters) and even load time options.
This works similarly to [Per Model Defaults](/docs/app/advanced/per-model).

- `operation:` inference time parameters
- `load:` load time parameters

```yaml
config:
  operation:
    fields:
      - key: llm.prediction.topKSampling
        value: 20
      - key: llm.prediction.temperature
        value: 0.7
  load:
    fields:
      - key: llm.load.contextLength
        value: 42690
```

### `customFields`

Define model-specific custom fields.

```yaml
customFields:
  - key: enableThinking
    displayName: Enable Thinking
    description: Controls whether the model will think before replying
    type: boolean
    defaultValue: true
    effects:
      - type: setJinjaVariable
        variable: enable_thinking
```

In order for the above example to work, the jinja template needs to have a variable named `enable_thinking`.

## Complete example

Taken from https://lmstudio.ai/models/qwen/qwen3-8b

```yaml
# model.yaml is an open standard for defining cross-platform, composable AI models
# Learn more at https://modelyaml.org
model: qwen/qwen3-8b
base:
  - key: lmstudio-community/qwen3-8b-gguf
    sources:
      - type: huggingface
        user: lmstudio-community
        repo: Qwen3-8B-GGUF
  - key: lmstudio-community/qwen3-8b-mlx-4bit
    sources:
      - type: huggingface
        user: lmstudio-community
        repo: Qwen3-8B-MLX-4bit
  - key: lmstudio-community/qwen3-8b-mlx-8bit
    sources:
      - type: huggingface
        user: lmstudio-community
        repo: Qwen3-8B-MLX-8bit
metadataOverrides:
  domain: llm
  architectures:
    - qwen3
  compatibilityTypes:
    - gguf
    - safetensors
  paramsStrings:
    - 8B
  minMemoryUsageBytes: 4600000000
  contextLengths:
    - 40960
  vision: false
  reasoning: true
  trainedForToolUse: true
config:
  operation:
    fields:
      - key: llm.prediction.topKSampling
        value: 20
      - key: llm.prediction.minPSampling
        value:
          checked: true
          value: 0
customFields:
  - key: enableThinking
    displayName: Enable Thinking
    description: Controls whether the model will think before replying
    type: boolean
    defaultValue: true
    effects:
      - type: setJinjaVariable
        variable: enable_thinking
```

The [GitHub specification](https://github.com/modelyaml/modelyaml) contains further details and the latest schema.</doc><doc title="3 Presets" desc="docs page.">---
title: Config Presets
sidebar_title: Overview
description: Save your system prompts and other parameters as Presets for easy reuse across chats.
index: 1
---

Presets are a way to bundle together a system prompt and other parameters into a single configuration that can be easily reused across different chats.

New in 0.3.15: You can [import](/docs/app/presets/import) Presets from file or URL, and even [publish](/docs/app/presets/publish) your own Presets to share with others on to the LM Studio Hub.
<hr>

## Saving, resetting, and deselecting Presets

Below is the anatomy of the Preset manager:

<img src="/assets/docs/preset-widget-anatomy.png" style="width:70%" data-caption="The anatomy of the Preset manager in the settings sidebar.">

## Importing, Publishing, and Updating Downloaded Presets

Presets are JSON files. You can share them by sending around the JSON, or you can share them by publishing them to the LM Studio Hub.
You can also import Presets from other users by URL. See the [Import](/docs/app/presets/import) and [Publish](/docs/app/presets/publish) sections for more details.

## Example: Build your own Prompt Library

You can create your own prompt library by using Presets.

<video autoplay loop muted playsinline style="width:60vh;" data-caption="Save collections of parameters as a Preset for easy reuse." class="border border-border">
  <source src="https://files.lmstudio.ai/presets.mp4" type="video/mp4">
  Your browser does not support the video tag.
</video>

In addition to system prompts, every parameter under the Advanced Configuration sidebar can be recorded in a named Preset.

For example, you might want to always use a certain Temperature, Top P, or Max Tokens for a particular use case. You can save these settings as a Preset (with or without a system prompt) and easily switch between them.

#### The Use Case for Presets

- Save your system prompts, inference parameters as a named `Preset`.
- Easily switch between different use cases, such as reasoning, creative writing, multi-turn conversations, or brainstorming.

## Where Presets are stored

Presets are stored in the following directory:

#### macOS or Linux

```xml
~/.lmstudio/config-presets
```

#### Windows

```xml
%USERPROFILE%\.lmstudio\config-presets
```

### Migration from LM Studio 0.2.\* Presets

- Presets you've saved in LM Studio 0.2.\* are automatically readable in 0.3.3 with no migration step needed.
- If you save **new changes** in a **legacy preset**, it'll be **copied** to a new format upon save.
  - The old files are NOT deleted.
- Notable difference: Load parameters are not included in the new preset format.
  - Favor editing the model's default config in My Models. See [how to do it here](/docs/configuration/per-model).

<hr>

### Community

Chat with other LM Studio users, discuss LLMs, hardware, and more on the [LM Studio Discord server](https://discord.gg/aPQfnNkxGC).</doc><doc title="Branching" desc="docs page.">---
title: Branch conversation
description: Fork, clone, and branch a conversation in LM Studio
---

### Page

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam auctor, nunc nec suscipit ultricies, nunc nunc ultricies nunc, nec suscipit nunc nunc nec. Nullam.</doc><doc title="Chat" desc="docs page.">---
title: Manage chats
description: Manage conversation threads with LLMs
index: 2
---

LM Studio has a ChatGPT-like interface for chatting with local LLMs. You can create many different conversation threads and manage them in folders.

<hr>

### Create a new chat

You can create a new chat by clicking the "+" button or by using a keyboard shortcut: `⌘` + `N` on Mac, or `ctrl` + `N` on Windows / Linux.

### Create a folder

Create a new folder by clicking the new folder button or by pressing: `⌘` + `shift` + `N` on Mac, or `ctrl` + `shift` + `N` on Windows / Linux.

### Drag and drop

You can drag and drop chats in and out of folders, and even drag folders into folders!

### Duplicate chats

You can duplicate a whole chat conversation by clicking the `•••` menu and selecting "Duplicate". If the chat has any files in it, they will be duplicated too.

## FAQ

#### Where are chats stored in the file system?

Right-click on a chat and choose "Reveal in Finder" / "Show in File Explorer".
Conversations are stored in JSON format. It is NOT recommended to edit them manually, nor to rely on their structure.

#### Does the model learn from chats?

The model doesn't 'learn' from chats. The model only 'knows' the content that is present in the chat or is provided to it via configuration options such as the "system prompt".

## Conversations folder filesystem path

Mac / Linux:

```shell
~/.lmstudio/conversations/
```

Windows:

```ps
%USERPROFILE%\.lmstudio\conversations
```

<hr>

### Community

Chat with other LM Studio users, discuss LLMs, hardware, and more on the [LM Studio Discord server](https://discord.gg/aPQfnNkxGC).</doc><doc title="Connect Apps" desc="docs page.">---
title: Connect apps to LM Studio
description: Getting started with connecting applications to LM Studio
---

LM Studio comes with a few built-in themes for app-wide color palettes.

<hr>

### Selecting a Theme

You can choose a theme in the Settings tab.

Choosing the "Auto" option will automatically switch between Light and Dark themes based on your system settings.

```lms_protip
You can jump to Settings from anywhere in the app by pressing `cmd` + `,` on macOS or `ctrl` + `,` on Windows/Linux.
```
###### To get to the Settings page, you need to be on [Power User mode](/docs/modes) or higher.

<hr>

### Community
Chat with other LM Studio users, discuss LLMs, hardware, and more on the [LM Studio Discord server](https://discord.gg/aPQfnNkxGC).</doc><doc title="Context" desc="docs page.">---
title: Context construction
description: Inserting Assistant and User messages to construct a conversation context for a given purpose
---

### Page

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam auctor, nunc nec suscipit ultricies, nunc nunc ultricies nunc, nec suscipit nunc nunc nec. Nullam.</doc><doc title="Deeplink" desc="docs page.">---
title: "`Add to LM Studio` Button"
description: Add MCP servers to LM Studio using a deeplink
index: 2
---

You can install MCP servers in LM Studio with one click using a deeplink.

Starting with version 0.3.17 (10), LM Studio can act as an MCP host. Learn more about it [here](../mcp).

---

# Generate your own MCP install link

Enter your MCP JSON entry to generate a deeplink for the `Add to LM Studio` button.

```lms_mcp_deep_link_generator

```

## Try an example

Try to copy and paste the following into the link generator above.

```json
{
  "hf-mcp-server": {
    "url": "https://huggingface.co/mcp",
    "headers": {
      "Authorization": "Bearer <YOUR_HF_TOKEN>"
    }
  }
}
```

### Deeplink format

```bash
lmstudio://add_mcp?name=hf-mcp-server&config=eyJ1cmwiOiJodHRwczovL2h1Z2dpbmdmYWNlLmNvL21jcCIsImhlYWRlcnMiOnsiQXV0aG9yaXphdGlvbiI6IkJlYXJlciA8WU9VUl9IRl9UT0tFTj4ifX0%3D
```

#### Parameters

```lms_params
- name: "lmstudio://"
  type: "protocol"
  description: "The protocol scheme to open LM Studio"
- name: "add_mcp"
  type: "path"
  description: "The action to install an MCP server"
- name: "name"
  type: "query parameter"
  description: "The name of the MCP server to install"
- name: "config"
  type: "query parameter"
  description: "Base64 encoded JSON configuration for the MCP server"
```</doc></0 app><1 developer><doc title="1 Developer" desc="docs page.">---
title: LM Studio Developer Docs
sidebar_title: Introduction
description: Build with LM Studio's local APIs and SDKs — TypeScript, Python, REST, and OpenAI‑compatible endpoints.
index: 1
---

```lms_hstack
## Get to know the stack

- TypeScript SDK: [lmstudio-js](/docs/typescript)
- Python SDK: [lmstudio-python](/docs/python)
- OpenAI‑compatible: [Chat, Responses, Embeddings](/docs/developer/openai-compat)
- LM Studio CLI: [`lms`](/docs/cli)

:::split:::

## What you can build

- Chat and text generation with streaming
- Structured output (JSON schema)
- Tool calling and local agents
- Embeddings and tokenization
- Model management (JIT load, TTL, auto‑evict)
```

## Super quick start

### TypeScript (`lmstudio-js`)

```bash
npm install @lmstudio/sdk
```

```ts
import { LMStudioClient } from "@lmstudio/sdk";

const client = new LMStudioClient();
const model = await client.llm.model("openai/gpt-oss-20b");
const result = await model.respond("Who are you, and what can you do?");

console.info(result.content);
```

Full docs: [lmstudio-js](/docs/typescript), Source: [GitHub](https://github.com/lmstudio-ai/lmstudio-js)

### Python (`lmstudio-python`)

```bash
pip install lmstudio
```

```python
import lmstudio as lms

with lms.Client() as client:
    model = client.llm.model("openai/gpt-oss-20b")
    result = model.respond("Who are you, and what can you do?")
    print(result)
```

Full docs: [lmstudio-python](/docs/python), Source: [GitHub](https://github.com/lmstudio-ai/lmstudio-python)

### Try a minimal HTTP request (OpenAI‑compatible)

```bash
lms server start --port 1234
```

```bash
curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-20b",
    "messages": [{"role": "user", "content": "Who are you, and what can you do?"}]
  }'
```

Full docs: [OpenAI‑compatible endpoints](/docs/developer/openai-compat)

## Helpful links

- API Changelog: [/docs/developer/api-changelog](/docs/developer/api-changelog)
- Local server basics: [/docs/developer/core](/docs/developer/core)
- CLI reference: [/docs/cli](/docs/cli)
- Community: [Discord](https://discord.gg/lmstudio)</doc><doc title="3 Openai Compat" desc="docs page.">---
title: OpenAI Compatibility Endpoints
sidebar_title: Overview
description: Send requests to Responses, Chat Completions (text and images), Completions, and Embeddings endpoints.
index: 1
---

### Supported endpoints

<table class="flexible-cols">
  <thead>
    <tr>
      <th>Endpoint</th>
      <th>Method</th>
      <th>Docs</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code>/v1/models</code></td>
      <td><apimethod method="GET" /></td>
      <td><a href="/docs/developer/openai-compat/models">Models</a></td>
    </tr>
    <tr>
      <td><code>/v1/responses</code></td>
      <td><apimethod method="POST" /></td>
      <td><a href="/docs/developer/openai-compat/responses">Responses</a></td>
    </tr>
    <tr>
      <td><code>/v1/chat/completions</code></td>
      <td><apimethod method="POST" /></td>
      <td><a href="/docs/developer/openai-compat/chat-completions">Chat Completions</a></td>
    </tr>
    <tr>
      <td><code>/v1/embeddings</code></td>
      <td><apimethod method="POST" /></td>
      <td><a href="/docs/developer/openai-compat/embeddings">Embeddings</a></td>
    </tr>
    <tr>
      <td><code>/v1/completions</code></td>
      <td><apimethod method="POST" /></td>
      <td><a href="/docs/developer/openai-compat/completions">Completions</a></td>
    </tr>
  </tbody>
</table>

<hr>

## Set the `base url` to point to LM Studio

You can reuse existing OpenAI clients (in Python, JS, C#, etc) by switching up the "base URL" property to point to your LM Studio instead of OpenAI's servers.

Note: The following examples assume the server port is `1234`

### Python Example

```diff
from openai import OpenAI

client = OpenAI(
+    base_url="http://localhost:1234/v1"
)

# ... the rest of your code ...
```

### Typescript Example

```diff
import OpenAI from 'openai';

const client = new OpenAI({
+  baseUrl: "http://localhost:1234/v1"
});

// ... the rest of your code ...
```

### cURL Example

```diff
- curl https://api.openai.com/v1/chat/completions \
+ curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
-     "model": "gpt-4o-mini",
+     "model": "use the model identifier from LM Studio here",
     "messages": [{"role": "user", "content": "Say this is a test!"}],
     "temperature": 0.7
   }'
```

---

Other OpenAI client libraries should have similar options to set the base URL.

If you're running into trouble, hop onto our [Discord](https://discord.gg/lmstudio) and enter the `#🔨-developers` channel.</doc><doc title="0 Server" desc="docs page.">---
title: LM Studio as a Local LLM API Server
sidebar_title: Running the Server
description: Run an LLM API server on `localhost` with LM Studio
fullPage: false
index: 1
---

You can serve local LLMs from LM Studio's Developer tab, either on `localhost` or on the network.

LM Studio's APIs can be used through [REST API](/docs/developer/rest), client libraries like [lmstudio-js](/docs/typescript) and [lmstudio-python](/docs/python), and [OpenAI compatibility endpoints](/docs/developer/openai-compat)

<img src="/assets/docs/server.png" style="" data-caption="Load and serve LLMs from LM Studio" />

### Running the server

To run the server, go to the Developer tab in LM Studio, and toggle the "Start server" switch to start the API server.

<img src="/assets/docs/server-start.png" style="" data-caption="Start the LM Studio API Server" />



Alternatively, you can use `lms` ([LM Studio's CLI](/docs/cli)) to start the server from your terminal:

```bash
lms server start
```


### API options

- [LM Studio REST API](/docs/developer/rest)
- [TypeScript SDK](/docs/typescript) - `lmstudio-js`
- [Python SDK](/docs/python) - `lmstudio-python`
- [OpenAI compatibility endpoints](/docs/developer/openai-compat)</doc><doc title="Embeddings" desc="docs page.">---
title: Embeddings
description: Use embedding models in LM Studio
---

### Page

lorem ipsum dolor sit amet, consectetur adipiscing elit. nullam auctor, nunc nec suscipit ultricies, nunc nunc ultricies nunc, nec suscipit nunc nunc nec. nullam.</doc><doc title="Authentication" desc="docs page.">---
title: Authentication
sidebar_title: Authentication
description: Using API Tokens in LM Studio
index: 1
---

##### Requires [LM Studio 0.4.0](/download) or newer.

LM Studio supports API Tokens for authentication, providing a secure and convenient way to access the LM Studio API.

### Require Authentication for each request

By default, LM Studio does not require authentication for API requests. To enable authentication so that only requests with a valid API Token are accepted, toggle the switch in the Developers Page > Server Settings.

```lms_info
Once enabled, all requests made through the REST API, Python SDK, or Typescript SDK will need to include a valid API Token. See usage [below](#api-token-usage).
```

<img src="/assets/docs/require-auth.png" style="width: 75%;" data-caption="Enable authentication to require valid API tokens for all requests" />

<img src="/assets/docs/multiple-tokens.png" style="width: 75%;" data-caption="Managing tokens in the server settings" />

### Creating API Tokens

To create API Tokens, click on Manage Tokens in the Server Settings. It will open the API Tokens modal where you can create, view, and delete API Tokens.

<img src="/assets/docs/tokens-empty-modal.png" style="width: 75%;" data-caption="API Tokens Modal" />

Create a token by clicking on the Create Token button. Provide a name for the token and select the desired permissions.

<img src="/assets/docs/create-dave-token.png" style="width: 75%;" data-caption="Creating an API Token" />

Once created, make sure to copy the token as it will not be shown again.

<img src="/assets/docs/created-dave-token.png" style="width: 75%;" data-caption="API token created" />

### Configuring API Token Permissions

To edit the permissions of an existing API Token, click on the Edit button next to the token in the API Tokens modal. You can modify the name and permissions of the token.

<img src="/assets/docs/edit-token.png" style="width: 75%;" data-caption="Editing an API Token" />

## API Token Usage

### Using API Tokens with REST API:

```lms_noticechill
The example below requires [allowing calling servers from mcp.json](/docs/developer/core/server/settings) to be enabled and the [tiktoken MCP](https://gitmcp.io/openai/tiktoken) in mcp.json.
```

```bash
curl -X POST \
  http://localhost:1234/api/v1/chat \
  -H "Authorization: Bearer $LM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ibm/granite-4-micro",
    "input": "What is the first line in the tiktoken documentation?",
    "integrations": [
      {
        "type": "plugin",
        "id": "mcp/tiktoken",
        "allowed_tools": [
          "fetch_tiktoken_documentation"
        ]
      }
    ]
  }'
```

### Using API Tokens with Python SDK

To use API tokens with the Python SDK, see the [Python SDK guide](/docs/python/getting-started/authentication).

### Using API Tokens with TypeScript SDK

To use API tokens with the TypeScript SDK, see the [TS SDK guide](/docs/typescript/authentication).</doc><doc title="Chat" desc="docs page.">---
title: "Chat with a model"
description: "Send a message to a model and receive a response. Supports MCP integration."
fullPage: true
index: 5
api_info:
  method: POST
---

````lms_hstack
`POST /api/v1/chat`

**Request body**
```lms_params
- name: model
  type: string
  optional: false
  description: Unique identifier for the model to use.
- name: input
  type: string
  optional: false
  description: Message to send to the model.
- name: system_prompt
  type: string
  optional: true
  description: System message that sets model behavior or instructions.
- name: integrations
  type: array<string | object>
  optional: true
  description: List of integrations (plugins, ephemeral MCP servers, etc...) to enable for this request.
  children:
    - name: Plugin id
      unstyledName: true
      type: string
      description: Unique identifier of a plugin to use. Plugins contain `mcp.json` installed MCP servers (id `mcp/<server_label>`). Shorthand for plugin object with no custom configuration.
    - name: Plugin
      unstyledName: true
      type: object
      description: Specification of a plugin to use. Plugins contain `mcp.json` installed MCP servers (id `mcp/<server_label>`).
      children:
        - name: type
          type: '"plugin"'
          optional: false
          description: Type of integration.
        - name: id
          type: string
          optional: false
          description: Unique identifier of the plugin.
        - name: allowed_tools
          type: array<string>
          optional: true
          description: List of tool names the model can call from this plugin. If not provided, all tools from the plugin are allowed.
    - name: Ephemeral MCP server specification
      unstyledName: true
      type: object
      description: Specification of an ephemeral MCP server. Allows defining MCP servers on-the-fly without needing to pre-configure them in your `mcp.json`.
      children:
        - name: type
          type: '"ephemeral_mcp"'
          optional: false
          description: Type of integration.
        - name: server_label
          type: string
          optional: false
          description: Label to identify the MCP server.
        - name: server_url
          type: string
          optional: false
          description: URL of the MCP server.
        - name: allowed_tools
          type: array<string>
          optional: true
          description: List of tool names the model can call from this server. If not provided, all tools from the server are allowed.
        - name: headers
          type: object
          optional: true
          description: Custom HTTP headers to send with requests to the server.
- name: stream
  type: boolean
  optional: true
  description: Whether to stream partial outputs via SSE. Default `false`. See [streaming events](/docs/developer/rest/streaming-events) for more information.
- name: temperature
  type: number
  optional: true
  description: Randomness in token selection. 0 is deterministic, higher values increase creativity [0,1].
- name: top_p
  type: number
  optional: true
  description: Minimum cumulative probability for the possible next tokens [0,1].
- name: top_k
  type: integer
  optional: true
  description: Limits next token selection to top-k most probable tokens.
- name: min_p
  type: number
  optional: true
  description: Minimum base probability for a token to be selected for output [0,1].
- name: repeat_penalty
  type: number
  optional: true
  description: Penalty for repeating token sequences. 1 is no penalty, higher values discourage repetition.
- name: max_output_tokens
  type: integer
  optional: true
  description: Maximum number of tokens to generate.
- name: reasoning
  type: '"off" | "low" | "medium" | "high" | "on"'
  optional: true
  description: Reasoning setting. Will error if the model being used does not support the reasoning setting using. Defaults to the automatically chosen setting for the model.
- name: context_length
  type: integer
  optional: true
  description: Number of tokens to consider as context. Higher values recommended for MCP usage.
- name: store
  type: boolean
  optional: true
  description: Whether to store the chat. If set, response will return a `"response_id"` field. Default `true`.
- name: previous_response_id
  type: string
  optional: true
  description: Identifier of existing response to append to. Must start with `"resp_"`.
```
:::split:::
```lms_code_snippet
title: Example Request
variants:
  curl:
    language: bash
    code: |
      curl http://localhost:1234/api/v1/chat \
        -H "Authorization: Bearer $LM_API_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{
          "model": "ibm/granite-4-micro",
          "input": "Tell me the top trending model on hugging face and navigate to https://lmstudio.ai",
          "integrations": [
            {
              "type": "ephemeral_mcp",
              "server_label": "huggingface",
              "server_url": "https://huggingface.co/mcp",
              "allowed_tools": [
                "model_search"
              ]
            },
            {
              "type": "plugin",
              "id": "mcp/playwright",
              "allowed_tools": [
                "browser_navigate"
              ]
            }
          ],
          "context_length": 8000,
          "temperature": 0
        }'
```
````

---

````lms_hstack
**Response fields**
```lms_params
- name: model_instance_id
  type: string
  description: Unique identifier for the loaded model instance that generated the response.
- name: output
  type: array<object>
  description: Array of output items generated. Each item can be one of three types.
  children:
    - name: Message
      unstyledName: true
      type: object
      description: A text message from the model.
      children:
        - name: type
          type: '"message"'
          description: Type of output item.
        - name: content
          type: string
          description: Text content of the message.
    - name: Tool call
      unstyledName: true
      type: object
      description: A tool call made by the model.
      children:
        - name: type
          type: '"tool_call"'
          description: Type of output item.
        - name: tool
          type: string
          description: Name of the tool called.
        - name: arguments
          type: object
          description: Arguments passed to the tool. Can have any keys/values depending on the tool definition.
        - name: output
          type: string
          description: Result returned from the tool.
        - name: provider_info
          type: object
          description: Information about the tool provider.
          children:
            - name: type
              type: '"plugin" | "ephemeral_mcp"'
              description: Provider type.
            - name: plugin_id
              type: string
              optional: true
              description: Identifier of the plugin (when `type` is `"plugin"`).
            - name: server_label
              type: string
              optional: true
              description: Label of the MCP server (when `type` is `"ephemeral_mcp"`).
    - name: Reasoning
      unstyledName: true
      type: object
      description: Reasoning content from the model.
      children:
        - name: type
          type: '"reasoning"'
          description: Type of output item.
        - name: content
          type: string
          description: Text content of the reasoning.
- name: stats
  type: object
  description: Token usage and performance metrics.
  children:
    - name: input_tokens
      type: number
      description: Number of input tokens. Includes formatting, tool definitions, and prior messages in the chat.
    - name: total_output_tokens
      type: number
      description: Total number of output tokens generated.
    - name: reasoning_output_tokens
      type: number
      description: Number of tokens used for reasoning.
    - name: tokens_per_second
      type: number
      description: Generation speed in tokens per second.
    - name: time_to_first_token_seconds
      type: number
      description: Time in seconds to generate the first token.
    - name: model_load_time_seconds
      type: number
      optional: true
      description: Time taken to load the model for this request in seconds. Present only if the model was not already loaded.
- name: response_id
  type: string
  optional: true
  description: Identifier of the response for subsequent requests. Starts with `"resp_"`. Present when `store` is `true`.
```
:::split:::
```lms_code_snippet
title: Response
variants:
  json:
    language: json
    code: |
      {
        "model_instance_id": "ibm/granite-4-micro",
        "output": [
          {
            "type": "tool_call",
            "tool": "model_search",
            "arguments": {
              "sort": "trendingScore",
              "query": "",
              "limit": 1
            },
            "output": "...",
            "provider_info": {
              "server_label": "huggingface",
              "type": "ephemeral_mcp"
            }
          },
          {
            "type": "message",
            "content": "..."
          },
          {
            "type": "tool_call",
            "tool": "browser_navigate",
            "arguments": {
              "url": "https://lmstudio.ai"
            },
            "output": "...",
            "provider_info": {
              "plugin_id": "mcp/playwright",
              "type": "plugin"
            }
          },
          {
            "type": "message",
            "content": "**Top Trending Model on Hugging Face** ... Below is a quick snapshot of what’s on the landing page ... more details on the model or LM Studio itself!"
          }
        ],
        "stats": {
          "input_tokens": 646,
          "total_output_tokens": 586,
          "reasoning_output_tokens": 0,
          "tokens_per_second": 29.753900615398926,
          "time_to_first_token_seconds": 1.088,
          "model_load_time_seconds": 2.656
        },
        "response_id": "resp_4ef013eba0def1ed23f19dde72b67974c579113f544086de"
      }
```
````</doc><doc title="Chat Completions" desc="docs page.">---
title: Chat Completions
description: Send a chat history and get the assistant's response.
index: 4
api_info:
  method: POST
---

- Method: `POST`
- Prompt template is applied automatically for chat‑tuned models
- Provide inference parameters (temperature, top_p, etc.) in the payload
- See OpenAI docs: https://platform.openai.com/docs/api-reference/chat
- Tip: keep a terminal open with [`lms log stream`](/docs/cli/serve/log-stream) to inspect model input

##### Python example

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")

completion = client.chat.completions.create(
  model="model-identifier",
  messages=[
    {"role": "system", "content": "Always answer in rhymes."},
    {"role": "user", "content": "Introduce yourself."}
  ],
  temperature=0.7,
)

print(completion.choices[0].message)
```

### Supported payload parameters

See https://platform.openai.com/docs/api-reference/chat/create for parameter semantics.

```py
model
top_p
top_k
messages
temperature
max_tokens
stream
stop
presence_penalty
frequency_penalty
logit_bias
repeat_penalty
seed
```</doc><doc title="Completions" desc="docs page.">---
title: Completions (Legacy)
description: Text completion for base models (legacy OpenAI endpoint).
index: 6
api_info:
  method: POST
---

```lms_warning
This endpoint is no longer supported by OpenAI. LM Studio continues to support it.

Using this endpoint with chat‑tuned models may produce unexpected tokens. Prefer base models.
```

- Method: `POST`
- Prompt template is not applied
- See OpenAI docs: https://platform.openai.com/docs/api-reference/completions</doc><doc title="Download" desc="docs page.">---
title: "Download a model"
description: "Download LLMs and embedding models"
fullPage: true
index: 8
api_info:
  method: POST
---

````lms_hstack
`POST /api/v1/models/download`

**Request body**
```lms_params
- name: model
  type: string
  optional: false
  description: The model to download. Accepts [model catalog](https://lmstudio.ai/models) identifiers (e.g., `openai/gpt-oss-20b`) and exact Hugging Face links (e.g., `https://huggingface.co/lmstudio-community/gpt-oss-20b-GGUF`)
- name: quantization
  type: string
  optional: true
  description: Quantization level of the model to download (e.g., `Q4_K_M`). Only supported for Hugging Face links.
```
:::split:::
```lms_code_snippet
title: Example Request
variants:
  curl:
    language: bash
    code: |
      curl http://localhost:1234/api/v1/models/download \
        -H "Authorization: Bearer $LM_API_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{
          "model": "ibm/granite-4-micro"
        }'
```
````

````lms_hstack
**Response fields**

Returns a download job status object. The response varies based on the download status.

```lms_params
- name: job_id
  type: string
  optional: true
  description: Unique identifier for the download job. Absent when `status` is `already_downloaded`.
- name: status
  type: '"downloading" | "paused" | "completed" | "failed" | "already_downloaded"'
  description: Current status of the download.
- name: bytes_per_second
  type: number
  optional: true
  description: Current download speed in bytes per second. Present when `status` is `downloading`.
- name: estimated_completion
  type: string
  optional: true
  description: Estimated completion time in ISO 8601 format. Present when `status` is `downloading`.
- name: completed_at
  type: string
  optional: true
  description: Download completion time in ISO 8601 format. Present when `status` is `completed`.
- name: total_size_bytes
  type: number
  optional: true
  description: Total size of the download in bytes. Absent when `status` is `already_downloaded`.
- name: downloaded_bytes
  type: number
  optional: true
  description: Number of bytes downloaded so far. Absent when `status` is `already_downloaded`.
- name: started_at
  type: string
  optional: true
  description: Download start time in ISO 8601 format. Absent when `status` is `already_downloaded`.
```
:::split:::
```lms_code_snippet
title: Response
variants:
  json:
    language: json
    code: |
      {
        "job_id": "job_493c7c9ded",
        "status": "downloading",
        "total_size_bytes": 2279145003,
        "downloaded_bytes": 948,
        "bytes_per_second": 7834.710743801653,
        "estimated_completion": "2025-10-07T00:21:47.030Z",
        "started_at": "2025-10-03T15:33:23.496Z"
      }
```
````</doc><doc title="Download Status" desc="docs page.">---
title: "Get download status"
description: "Get the status of model downloads"
fullPage: true
index: 9
api_info:
  method: GET
---

````lms_hstack
`GET /api/v1/models/download/status/:job_id`

**Path parameters**
```lms_params
- name: job_id
  type: string
  optional: false
  description: The unique identifier of the download job. `job_id` is returned by the [download](/docs/developer/rest/download) endpoint when a download is initiated.
```
:::split:::
```lms_code_snippet
title: Example Request
variants:
  curl:
    language: bash
    code: |
      curl -H "Authorization: Bearer $LM_API_TOKEN" \
        http://localhost:1234/api/v1/models/download/status/job_493c7c9ded
```
````

````lms_hstack
**Response fields**

Returns a single download job status object. The response varies based on the download status.

```lms_params
- name: job_id
  type: string
  description: Unique identifier for the download job.
- name: status
  type: '"downloading" | "paused" | "completed" | "failed"'
  description: Current status of the download.
- name: bytes_per_second
  type: number
  optional: true
  description: Current download speed in bytes per second. Present when `status` is `downloading`.
- name: estimated_completion
  type: string
  optional: true
  description: Estimated completion time in ISO 8601 format. Present when `status` is `downloading`.
- name: completed_at
  type: string
  optional: true
  description: Download completion time in ISO 8601 format. Present when `status` is `completed`.
- name: total_size_bytes
  type: number
  optional: true
  description: Total size of the download in bytes.
- name: downloaded_bytes
  type: number
  optional: true
  description: Number of bytes downloaded so far.
- name: started_at
  type: string
  optional: true
  description: Download start time in ISO 8601 format.
```
:::split:::
```lms_code_snippet
title: Response
variants:
  json:
    language: json
    code: |
      {
        "job_id": "job_493c7c9ded",
        "status": "completed",
        "total_size_bytes": 2279145003,
        "downloaded_bytes": 2279145003,
        "started_at": "2025-10-03T15:33:23.496Z",
        "completed_at": "2025-10-03T15:43:12.102Z"
      }
```
````</doc></1 developer><1 python><doc title="1 Python" desc="docs page.">---
title: "`lmstudio-python` (Python SDK)"
sidebar_title: "Introduction"
description: "Getting started with LM Studio's Python SDK"
---

`lmstudio-python` provides you a set APIs to interact with LLMs, embeddings models, and agentic flows.

## Installing the SDK

`lmstudio-python` is available as a PyPI package. You can install it using pip.

```lms_code_snippet
  variants:
    pip:
      language: bash
      code: |
        pip install lmstudio
```

For the source code and open source contribution, visit [lmstudio-python](https://github.com/lmstudio-ai/lmstudio-python) on GitHub.

## Features

- Use LLMs to [respond in chats](./python/llm-prediction/chat-completion) or predict [text completions](./python/llm-prediction/completion)
- Define functions as tools, and turn LLMs into [autonomous agents](./python/agent) that run completely locally
- [Load](./python/manage-models/loading), [configure](./python/llm-prediction/parameters), and [unload](./python/manage-models/loading) models from memory
- Generate embeddings for text, and more!

## Quick Example: Chat with a Llama Model

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm("qwen/qwen3-4b-2507")
        result = model.respond("What is the meaning of life?")

        print(result)

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model("qwen/qwen3-4b-2507")
            result = model.respond("What is the meaning of life?")

            print(result)

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model("qwen/qwen3-4b-2507")
            result = await model.respond("What is the meaning of life?")

            print(result)
```

### Getting Local Models

The above code requires the [qwen3-4b-2507](https://lmstudio.ai/models/qwen/qwen3-4b-2507) model.
If you don't have the model, run the following command in the terminal to download it.

```bash
lms get qwen/qwen3-4b-2507
```

Read more about `lms get` in LM Studio's CLI [here](./cli/get).

# Interactive Convenience, Deterministic Resource Management, or Structured Concurrency?

As shown in the example above, there are three distinct approaches for working
with the LM Studio Python SDK.

The first is the interactive convenience API (listed as "Python (convenience API)"
in examples), which focuses on the use of a default LM Studio client instance for
convenient interactions at a synchronous Python prompt, or when using Jupyter notebooks.

The second is a synchronous scoped resource API (listed as "Python (scoped resource API)"
in examples), which uses context managers to ensure that allocated resources
(such as network connections) are released deterministically, rather than
potentially remaining open until the entire process is terminated.

The last is an asynchronous structured concurrency API (listed as "Python (asynchronous API)" in
examples), which is designed for use in asynchronous programs that follow the design principles of
["structured concurrency"](https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/)
in order to ensure the background tasks handling the SDK's connections to the API server host
are managed correctly. Asynchronous applications which do not adhere to those design principles
will need to rely on threaded access to the synchronous scoped resource API rather than attempting
to use the SDK's native asynchronous API. Python SDK version 1.5.0 is the first version to fully
support the asynchronous API.

Some examples are common between the interactive convenience API and the synchronous scoped
resource API. These examples are listed as "Python (synchronous API)".

## Timeouts in the synchronous API

_Required Python SDK version_: **1.5.0**

Starting in Python SDK version 1.5.0, the synchronous API defaults to timing out after 60 seconds
with no activity when waiting for a response or streaming event notification from the API server.

The number of seconds to wait for responses and event notifications can be adjusted using the
`lmstudio.set_sync_api_timeout()` function. Setting the timeout to `None` disables the timeout
entirely (restoring the behaviour of previous SDK versions).

The current synchronous API timeout can be queried using the `lmstudio.get_sync_api_timeout()`
function.

## Timeouts in the asynchronous API

_Required Python SDK version_: **1.5.0**

As asynchronous coroutines support cancellation, there is no specific timeout support implemented
in the asynchronous API. Instead, general purpose async timeout mechanisms, such as
[`asyncio.wait_for()`](https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for) or
[`anyio.move_on_after()`](https://anyio.readthedocs.io/en/stable/cancellation.html#timeouts),
should be used.</doc><doc title="3 Embedding" desc="docs page.">---
title: Embedding
sidebar_title: Generating embedding vectors
description: Generate text embeddings from input text
---

Generate embeddings for input text. Embeddings are vector representations of text that capture semantic meaning. Embeddings are a building block for RAG (Retrieval-Augmented Generation) and other similarity-based tasks.

### Prerequisite: Get an Embedding Model

If you don't yet have an embedding model, you can download a model like `nomic-ai/nomic-embed-text-v1.5` using the following command:

```bash
lms get nomic-ai/nomic-embed-text-v1.5
```

## Create Embeddings

To convert a string to a vector representation, pass it to the `embed` method on the corresponding embedding model handle.

```lms_code_snippet
  title: "example.py"
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.embedding_model("nomic-embed-text-v1.5")

        embedding = model.embed("Hello, world!")

```</doc><doc title="4 Tokenization" desc="docs page.">---
title: Tokenization
sidebar_title: Tokenizing text
description: Tokenize text using a model's tokenizer
---

Models use a tokenizer to internally convert text into "tokens" they can deal with more easily. LM Studio exposes this tokenizer for utility.

## Tokenize

You can tokenize a string with a loaded LLM or embedding model using the SDK.
In the below examples, the LLM reference can be replaced with an
embedding model reference without requiring any other changes.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()

        tokens = model.tokenize("Hello, world!")

        print(tokens) # Array of token IDs.
```

## Count tokens

If you only care about the number of tokens, simply check the length of the resulting array.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        token_count = len(model.tokenize("Hello, world!"))
        print("Token count:", token_count)
```

### Example: count context

You can determine if a given conversation fits into a model's context by doing the following:

1. Convert the conversation to a string using the prompt template.
2. Count the number of tokens in the string.
3. Compare the token count to the model's context length.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        def does_chat_fit_in_context(model: lms.LLM, chat: lms.Chat) -> bool:
            # Convert the conversation to a string using the prompt template.
            formatted = model.apply_prompt_template(chat)
            # Count the number of tokens in the string.
            token_count = len(model.tokenize(formatted))
            # Get the current loaded context length of the model
            context_length = model.get_context_length()
            return token_count < context_length

        model = lms.llm()

        chat = lms.Chat.from_history({
            "messages": [
                { "role": "user", "content": "What is the meaning of life." },
                { "role": "assistant", "content": "The meaning of life is..." },
                # ... More messages
            ]
        })

        print("Fits in context:", does_chat_fit_in_context(model, chat))

```</doc><doc title="Act" desc="docs page.">---
title: The `.act()` call
description: How to use the `.act()` call to turn LLMs into autonomous agents that can perform tasks on your local machine.
index: 1
---

## Automatic tool calling

We introduce the concept of execution "rounds" to describe the combined process of running a tool, providing its output to the LLM, and then waiting for the LLM to decide what to do next.

**Execution Round**

```
 • run a tool ->
 ↑   • provide the result to the LLM ->
 │       • wait for the LLM to generate a response
 │
 └────────────────────────────────────────┘ └➔ (return)
```

A model might choose to run tools multiple times before returning a final result. For example, if the LLM is writing code, it might choose to compile or run the program, fix errors, and then run it again, rinse and repeat until it gets the desired result.

With this in mind, we say that the `.act()` API is an automatic "multi-round" tool calling API.

### Quick Example

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        def multiply(a: float, b: float) -> float:
            """Given two numbers a and b. Returns the product of them."""
            return a * b

        model = lms.llm("qwen2.5-7b-instruct")
        model.act(
          "What is the result of 12345 multiplied by 54321?",
          [multiply],
          on_message=print,
        )
```

### What does it mean for an LLM to "use a tool"?

LLMs are largely text-in, text-out programs. So, you may ask "how can an LLM use a tool?". The answer is that some LLMs are trained to ask the human to call the tool for them, and expect the tool output to to be provided back in some format.

Imagine you're giving computer support to someone over the phone. You might say things like "run this command for me ... OK what did it output? ... OK now click there and tell me what it says ...". In this case you're the LLM! And you're "calling tools" vicariously through the person on the other side of the line.

### Running multiple tool calls in parallel

By default, version 1.4.0 and later of the Python SDK will only run a single tool call request at a time,
even if the model requests multiple tool calls in a single response message. This ensures the requests will
be processed correctly even if the tool implementations do not support multiple concurrent calls.

When the tool implementations are known to be thread-safe, and are both slow and frequent enough to be worth
running in parallel, the `max_parallel_tool_calls` option specifies the maximum number of tool call requests
that will be processed in parallel from a single model response. This value defaults to 1 (waiting for each
tool call to complete before starting the next one). Setting this value to `None` will automatically scale
the maximum number of parallel tool calls to a multiple of the number of CPU cores available to the process.

### Important: Model Selection

The model selected for tool use will greatly impact performance.

Some general guidance when selecting a model:

- Not all models are capable of intelligent tool use
- Bigger is better (i.e., a 7B parameter model will generally perform better than a 3B parameter model)
- We've observed [Qwen2.5-7B-Instruct](https://model.lmstudio.ai/download/lmstudio-community/Qwen2.5-7B-Instruct-GGUF) to perform well in a wide variety of cases
- This guidance may change

### Example: Multiple Tools

The following code demonstrates how to provide multiple tools in a single `.act()` call.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import math
        import lmstudio as lms

        def add(a: int, b: int) -> int:
            """Given two numbers a and b, returns the sum of them."""
            return a + b

        def is_prime(n: int) -> bool:
            """Given a number n, returns True if n is a prime number."""
            if n < 2:
                return False
            sqrt = int(math.sqrt(n))
            for i in range(2, sqrt):
                if n % i == 0:
                    return False
            return True

        model = lms.llm("qwen2.5-7b-instruct")
        model.act(
          "Is the result of 12345 + 45668 a prime? Think step by step.",
          [add, is_prime],
          on_message=print,
        )
```

### Example: Chat Loop with Create File Tool

The following code creates a conversation loop with an LLM agent that can create files.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import readline # Enables input line editing
        from pathlib import Path

        import lmstudio as lms

        def create_file(name: str, content: str):
            """Create a file with the given name and content."""
            dest_path = Path(name)
            if dest_path.exists():
                return "Error: File already exists."
            try:
                dest_path.write_text(content, encoding="utf-8")
            except Exception as exc:
                return "Error: {exc!r}"
            return "File created."

        def print_fragment(fragment, round_index=0):
            # .act() supplies the round index as the second parameter
            # Setting a default value means the callback is also
            # compatible with .complete() and .respond().
            print(fragment.content, end="", flush=True)

        model = lms.llm()
        chat = lms.Chat("You are a task focused AI assistant")

        while True:
            try:
                user_input = input("You (leave blank to exit): ")
            except EOFError:
                print()
                break
            if not user_input:
                break
            chat.add_user_message(user_input)
            print("Bot: ", end="", flush=True)
            model.act(
                chat,
                [create_file],
                on_message=chat.append,
                on_prediction_fragment=print_fragment,
            )
            print()

```

### Progress Callbacks

Complex interactions with a tool using agent may take some time to process.

The regular progress callbacks for any prediction request are available,
but the expected capabilities differ from those for single round predictions.

* `on_prompt_processing_progress`: called during prompt processing for each
  prediction round. Receives the progress ratio (as a float) and the round
  index as positional arguments.
* `on_first_token`: called after prompt processing is complete for each prediction round.
  Receives the round index as its sole argument.
* `on_prediction_fragment`: called for each prediction fragment received by the client.
  Receives the prediction fragment and the round index as positional arguments.
* `on_message`: called with an assistant response message when each prediction round is
  complete, and with tool result messages as each tool call request is completed.
  Intended for appending received messages to a chat history instance, and hence
  does *not* receive the round index as an argument.

The following additional callbacks are available to monitor the prediction rounds:

* `on_round_start`: called before submitting the prediction request for each round.
  Receives the round index as its sole argument.
* `on_prediction_completed`: called after the prediction for the round has been completed,
  but before any requested tool calls have been initiated. Receives the round's prediction
  result as its sole argument. A round prediction result is a regular prediction result
  with an additional `round_index` attribute.
* `on_round_end`: called after any tool call requests for the round have been resolved.

Finally, applications may request notifications when agents emit invalid tool requests:

* `handle_invalid_tool_request`: called when a tool request was unable to be processed.
  Receives the exception that is about to be reported, as well as the original tool
  request that resulted in the problem. When no tool request is given, this is
  purely a notification of an unrecoverable error before the agent interaction raises
  the given exception (allowing the application to raise its own exception instead).
  When a tool request is given, it indicates that rather than being raised locally,
  the text description of the exception is going to be passed back to the agent
  as the result of that failed tool request. In these cases, the callback may either
  return `None` to indicate that the error description should be sent to the agent,
  raise the given exception (or a different exception) locally, or return a text
  string that should be sent to the agent instead of the error description.

For additional details on defining tools, and an example of overriding the invalid
tool request handling to raise all exceptions locally instead of passing them to
back the agent, refer to [Tool Definition](./tools.md).</doc><doc title="Apply Prompt Template" desc="docs page.">---
title: Apply Prompt Template
description: Apply a model's prompt template to a conversation
---

## Overview

LLMs (Large Language Models) operate on a text-in, text-out basis. Before processing conversations through these models, the input must be converted into a properly formatted string using a prompt template. If you need to inspect or work with this formatted string directly, the LM Studio SDK provides a streamlined way to apply a model's prompt template to your conversations.

```lms_info
You do not need to use this method when using `.respond`. It will automatically apply the prompt template for you.
```

## Usage with a Chat

You can apply a prompt template to a `Chat` by using the `applyPromptTemplate` method. This method takes a `Chat` object as input and returns a formatted string.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import { Chat, LMStudioClient } from "@lmstudio/sdk";

        client = new LMStudioClient()
        model = client.llm.model() # Use any loaded LLM

        chat = Chat.createEmpty()
        chat.append("system", "You are a helpful assistant.")
        chat.append("user", "What is LM Studio?")
        
        formatted = model.applyPromptTemplate(chat)
        print(formatted)
```

## Usage with an Array of Messages

The same method can also be used with any object that can be converted to a `Chat`, for example, an array of messages.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import { LMStudioClient } from "@lmstudio/sdk";

        client = new LMStudioClient()
        model = client.llm.model() # Use any loaded LLM

        formatted = model.applyPromptTemplate([
          { role: "system", content: "You are a helpful assistant." },
          { role: "user", content: "What is LM Studio?" },
        ])
        print(formatted)
```</doc><doc title="Cancelling Predictions" desc="docs page.">---
title: Cancelling Predictions
description: Stop an ongoing prediction in `lmstudio-python`
index: 4
---

One benefit of using the streaming API is the ability to cancel the
prediction request based on criteria that can't be represented using
the `stopStrings` or `maxPredictedTokens` configuration settings.

The following snippet illustrates cancelling the request in response
to an application specification cancellation condition (such as polling
an event set by another thread).

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms
        model = lms.llm()

        prediction_stream = model.respond_stream("What is the meaning of life?")
        cancelled = False
        for fragment in prediction_stream:
            if ...: # Cancellation condition will be app specific
                cancelled = True
                prediction_stream.cancel()
                # Note: it is recommended to let the iteration complete,
                # as doing so allows the partial result to be recorded.
                # Breaking the loop *is* permitted, but means the partial result
                # and final prediction stats won't be available to the client
        # The stream allows the prediction result to be retrieved after iteration
        if not cancelled:
            print(prediction_stream.result())

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model()

            prediction_stream = model.respond_stream("What is the meaning of life?")
            cancelled = False
            for fragment in prediction_stream:
                if ...: # Cancellation condition will be app specific
                    cancelled = True
                    prediction_stream.cancel()
                    # Note: it is recommended to let the iteration complete,
                    # as doing so allows the partial result to be recorded.
                    # Breaking the loop *is* permitted, but means the partial result
                    # and final prediction stats won't be available to the client
            # The stream allows the prediction result to be retrieved after iteration
            if not cancelled:
                print(prediction_stream.result())

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model()

            prediction_stream = await model.respond_stream("What is the meaning of life?")
            cancelled = False
            async for fragment in prediction_stream:
                if ...: # Cancellation condition will be app specific
                    cancelled = True
                    await prediction_stream.cancel()
                    # Note: it is recommended to let the iteration complete,
                    # as doing so allows the partial result to be recorded.
                    # Breaking the loop *is* permitted, but means the partial result
                    # and final prediction stats won't be available to the client
            # The stream allows the prediction result to be retrieved after iteration
            if not cancelled:
                print(prediction_stream.result())

```</doc><doc title="Chat Completion" desc="docs page.">---
title: Chat Completions
sidebar_title: Chat
description: APIs for a multi-turn chat conversations with an LLM
index: 2
---

Use `llm.respond(...)` to generate completions for a chat conversation.

## Quick Example: Generate a Chat Response

The following snippet shows how to obtain the AI's response to a quick chat prompt.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()
        print(model.respond("What is the meaning of life?"))

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model()
            print(model.respond("What is the meaning of life?"))

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model()
            print(await model.respond("What is the meaning of life?"))

```

## Streaming a Chat Response

The following snippet shows how to stream the AI's response to a chat prompt,
displaying text fragments as they are received (rather than waiting for the
entire response to be generated before displaying anything).

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms
        model = lms.llm()

        for fragment in model.respond_stream("What is the meaning of life?"):
            print(fragment.content, end="", flush=True)
        print() # Advance to a new line at the end of the response

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model()

            for fragment in model.respond_stream("What is the meaning of life?"):
                print(fragment.content, end="", flush=True)
            print() # Advance to a new line at the end of the response

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model()

            async for fragment in model.respond_stream("What is the meaning of life?"):
                print(fragment.content, end="", flush=True)
            print() # Advance to a new line at the end of the response

```

## Cancelling a Chat Response

See the [Cancelling a Prediction](./cancelling-predictions) section for how to cancel a prediction in progress.

## Obtain a Model

First, you need to get a model handle.
This can be done using the top-level `llm` convenience API,
or the `model` method in the `llm` namespace when using the scoped resource API.
For example, here is how to use Qwen2.5 7B Instruct.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm("qwen2.5-7b-instruct")

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model("qwen2.5-7b-instruct")

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model("qwen2.5-7b-instruct")

```

There are other ways to get a model handle. See [Managing Models in Memory](./../manage-models/loading) for more info.

## Manage Chat Context

The input to the model is referred to as the "context".
Conceptually, the model receives a multi-turn conversation as input,
and it is asked to predict the assistant's response in that conversation.

```lms_code_snippet
  variants:
    "Constructing a Chat object":
      language: python
      code: |
        import lmstudio as lms

        # Create a chat with an initial system prompt.
        chat = lms.Chat("You are a resident AI philosopher.")

        # Build the chat context by adding messages of relevant types.
        chat.add_user_message("What is the meaning of life?")
        # ... continued in next example

  "From chat history data":
      language: python
      code: |
        import lmstudio as lms

        # Create a chat object from a chat history dict
        chat = lms.Chat.from_history({
            "messages": [
                { "role": "system", "content": "You are a resident AI philosopher." },
                { "role": "user", "content": "What is the meaning of life?" },
            ]
        })
        # ... continued in next example

```

See [Working with Chats](./working-with-chats) for more information on managing chat context.


## Generate a response

You can ask the LLM to predict the next response in the chat context using the `respond()` method.

```lms_code_snippet
  variants:
    "Non-streaming (synchronous API)":
      language: python
      code: |
        # The `chat` object is created in the previous step.
        result = model.respond(chat)

        print(result)

    "Streaming (synchronous API)":
      language: python
      code: |
        # The `chat` object is created in the previous step.
        prediction_stream = model.respond_stream(chat)

        for fragment in prediction_stream:
            print(fragment.content, end="", flush=True)
        print() # Advance to a new line at the end of the response

    "Non-streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        # The `chat` object is created in the previous step.
        result = await model.respond(chat)

        print(result)

    "Streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        # The `chat` object is created in the previous step.
        prediction_stream = await model.respond_stream(chat)

        async for fragment in prediction_stream:
            print(fragment.content, end="", flush=True)
        print() # Advance to a new line at the end of the response

```

## Customize Inferencing Parameters

You can pass in inferencing parameters via the `config` keyword parameter on `.respond()`.

```lms_code_snippet
  variants:
    "Non-streaming (synchronous API)":
      language: python
      code: |
        result = model.respond(chat, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

    "Streaming (synchronous API)":
      language: python
      code: |
        prediction_stream = model.respond_stream(chat, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

    "Non-streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        result = await model.respond(chat, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

    "Streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        prediction_stream = await model.respond_stream(chat, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

```

See [Configuring the Model](./parameters) for more information on what can be configured.

## Print prediction stats

You can also print prediction metadata, such as the model used for generation, number of generated
tokens, time to first token, and stop reason.

```lms_code_snippet
  variants:
    "Non-streaming":
      language: python
      code: |
        # `result` is the response from the model.
        print("Model used:", result.model_info.display_name)
        print("Predicted tokens:", result.stats.predicted_tokens_count)
        print("Time to first token (seconds):", result.stats.time_to_first_token_sec)
        print("Stop reason:", result.stats.stop_reason)

    "Streaming":
      language: python
      code: |
        # After iterating through the prediction fragments,
        # the overall prediction result may be obtained from the stream
        result = prediction_stream.result()

        print("Model used:", result.model_info.display_name)
        print("Predicted tokens:", result.stats.predicted_tokens_count)
        print("Time to first token (seconds):", result.stats.time_to_first_token_sec)
        print("Stop reason:", result.stats.stop_reason)

```

Both the non-streaming and streaming result access is consistent across the synchronous and
asynchronous APIs, as `prediction_stream.result()` is a non-blocking API that raises an exception
if no result is available (either because the prediction is still running, or because the
prediction request failed). Prediction streams also offer a blocking (synchronous API) or
awaitable (asynchronous API) `prediction_stream.wait_for_result()` method that internally handles
iterating the stream to completion before returning the result.

## Example: Multi-turn Chat

```lms_code_snippet
  title: "chatbot.py"
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()
        chat = lms.Chat("You are a task focused AI assistant")

        while True:
            try:
                user_input = input("You (leave blank to exit): ")
            except EOFError:
                print()
                break
            if not user_input:
                break
            chat.add_user_message(user_input)
            prediction_stream = model.respond_stream(
                chat,
                on_message=chat.append,
            )
            print("Bot: ", end="", flush=True)
            for fragment in prediction_stream:
                print(fragment.content, end="", flush=True)
            print()

```

### Progress Callbacks

Long prompts will often take a long time to first token, i.e. it takes the model a long time to process your prompt.
If you want to get updates on the progress of this process, you can provide a float callback to `respond`
that receives a float from 0.0-1.0 representing prompt processing progress.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        llm = lms.llm()

        response = llm.respond(
            "What is LM Studio?",
            on_prompt_processing_progress = (lambda progress: print(f"{progress*100}% complete")),
        )

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            llm = client.llm.model()

            response = llm.respond(
                "What is LM Studio?",
                on_prompt_processing_progress = (lambda progress: print(f"{progress*100}% complete")),
            )

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            llm = await client.llm.model()

            response = await llm.respond(
                "What is LM Studio?",
                on_prompt_processing_progress = (lambda progress: print(f"{progress*100}% complete")),
            )


```

In addition to `on_prompt_processing_progress`, the other available progress callbacks are:

- `on_first_token`: called after prompt processing is complete and the first token is being emitted.
  Does not receive any arguments (use the streaming iteration API or `on_prediction_fragment`
  to process tokens as they are emitted).
- `on_prediction_fragment`: called for each prediction fragment received by the client.
  Receives the same prediction fragments as iterating over the stream iteration API.
- `on_message`: called with an assistant response message when the prediction is complete.
  Intended for appending received messages to a chat history instance.</doc><doc title="Completion" desc="docs page.">---
title: Text Completions
description: "Provide a string input for the model to complete"
---

Use `llm.complete(...)` to generate text completions from a loaded language model.
Text completions mean sending a non-formatted string to the model with the expectation that the model will complete the text.

This is different from multi-turn chat conversations. For more information on chat completions, see [Chat Completions](./chat-completion).

## 1. Instantiate a Model

First, you need to load a model to generate completions from.
This can be done using the top-level `llm` convenience API,
or the `model` method in the `llm` namespace when using the scoped resource API.
For example, here is how to use Qwen2.5 7B Instruct.


```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm("qwen2.5-7b-instruct")

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model("qwen2.5-7b-instruct")

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model("qwen2.5-7b-instruct")

```

## 2. Generate a Completion

Once you have a loaded model, you can generate completions by passing a string to the `complete` method on the `llm` handle.

```lms_code_snippet
  variants:
    "Non-streaming (synchronous API)":
      language: python
      code: |
        # The `chat` object is created in the previous step.
        result = model.complete("My name is", config={"maxTokens": 100})

        print(result)

    "Streaming (synchronous API)":
      language: python
      code: |
        # The `chat` object is created in the previous step.
        prediction_stream = model.complete_stream("My name is", config={"maxTokens": 100})

        for fragment in prediction_stream:
            print(fragment.content, end="", flush=True)
        print() # Advance to a new line at the end of the response

    "Non-streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        # The `chat` object is created in the previous step.
        result = await model.complete("My name is", config={"maxTokens": 100})

        print(result)

    "Streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        # The `chat` object is created in the previous step.
        prediction_stream = await model.complete_stream("My name is", config={"maxTokens": 100})

        async for fragment in prediction_stream:
            print(fragment.content, end="", flush=True)
        print() # Advance to a new line at the end of the response

```

## 3. Print Prediction Stats

You can also print prediction metadata, such as the model used for generation, number of generated tokens, time to first token, and stop reason.

```lms_code_snippet
  variants:
    "Non-streaming":
      language: python
      code: |
        # `result` is the response from the model.
        print("Model used:", result.model_info.display_name)
        print("Predicted tokens:", result.stats.predicted_tokens_count)
        print("Time to first token (seconds):", result.stats.time_to_first_token_sec)
        print("Stop reason:", result.stats.stop_reason)

    "Streaming":
      language: python
      code: |
        # After iterating through the prediction fragments,
        # the overall prediction result may be obtained from the stream
        result = prediction_stream.result()

        print("Model used:", result.model_info.display_name)
        print("Predicted tokens:", result.stats.predicted_tokens_count)
        print("Time to first token (seconds):", result.stats.time_to_first_token_sec)
        print("Stop reason:", result.stats.stop_reason)

```

Both the non-streaming and streaming result access is consistent across the synchronous and
asynchronous APIs, as `prediction_stream.result()` is a non-blocking API that raises an exception
if no result is available (either because the prediction is still running, or because the
prediction request failed). Prediction streams also offer a blocking (synchronous API) or
awaitable (asynchronous API) `prediction_stream.wait_for_result()` method that internally handles
iterating the stream to completion before returning the result.

## Example: Get an LLM to Simulate a Terminal

Here's an example of how you might use the `complete` method to simulate a terminal.

```lms_code_snippet
  title: "terminal-sim.py"
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()
        console_history = []

        while True:
            try:
                user_command = input("$ ")
            except EOFError:
                print()
                break
            if user_command.strip() == "exit":
                break
            console_history.append(f"$ {user_command}")
            history_prompt = "\n".join(console_history)
            prediction_stream = model.complete_stream(
                history_prompt,
                config={ "stopStrings": ["$"] },
            )
            for fragment in prediction_stream:
                print(fragment.content, end="", flush=True)
            print()
            console_history.append(prediction_stream.result().content)

```

## Customize Inferencing Parameters

You can pass in inferencing parameters via the `config` keyword parameter on `.complete()`.

```lms_code_snippet
  variants:
    "Non-streaming (synchronous API)":
      language: python
      code: |
        result = model.complete(initial_text, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

    "Streaming (synchronous API)":
      language: python
      code: |
        prediction_stream = model.complete_stream(initial_text, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

    "Non-streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        result = await model.complete(initial_text, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

    "Streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        prediction_stream = await model.complete_stream(initial_text, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

```

See [Configuring the Model](./parameters) for more information on what can be configured.

### Progress Callbacks

Long prompts will often take a long time to first token, i.e. it takes the model a long time to process your prompt.
If you want to get updates on the progress of this process, you can provide a float callback to `complete`
that receives a float from 0.0-1.0 representing prompt processing progress.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        llm = lms.llm()

        completion = llm.complete(
            "My name is",
            on_prompt_processing_progress = (lambda progress: print(f"{progress*100}% complete")),
        )

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            llm = client.llm.model()

            completion = llm.complete(
                "My name is",
                on_prompt_processing_progress = (lambda progress: print(f"{progress*100}% processed")),
            )

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            llm = await client.llm.model()

            completion = await llm.complete(
                "My name is",
                on_prompt_processing_progress = (lambda progress: print(f"{progress*100}% processed")),
            )

```

In addition to `on_prompt_processing_progress`, the other available progress callbacks are:

* `on_first_token`: called after prompt processing is complete and the first token is being emitted.
  Does not receive any arguments (use the streaming iteration API or `on_prediction_fragment`
  to process tokens as they are emitted).
* `on_prediction_fragment`: called for each prediction fragment received by the client.
  Receives the same prediction fragments as iterating over the stream iteration API.
* `on_message`: called with an assistant response message when the prediction is complete.
  Intended for appending received messages to a chat history instance.</doc><doc title="Download Models" desc="docs page.">---
title: Download Models
description: Download models to the machine running the LM Studio server
---

TODO: model downloading is available, but the current API is a bit awkward, so hold
      off on documenting it until the interface is nicer to use

## Overview

You can browse and download models using the LM Studio SDK just like you would
in the Discover tab of the app itself. Once a model is downloaded, you can
[load it](/docs/api/sdk/load-and-access-models) for inference.

### Usage

Downloading models consists of three steps:

1. Search for the model you want;
2. Find the download option you want (e.g. quantization) and
3. Download the model!


TODO: Actually translate this example code from TS to Python

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import { LMStudioClient } from "@lmstudio/sdk";

        const client = new LMStudioClient()

        # 1. Search for the model you want
        # Specify any/all of searchTerm, limit, compatibilityTypes
        const searchResults = client.repository.searchModels({
          searchTerm: "llama 3.2 1b",    # Search for Llama 3.2 1B
          limit: 5,                      # Get top 5 results
          compatibilityTypes: ["gguf"],  # Only download GGUFs
        })

        # 2. Find download options
        const bestResult = searchResults[0];
        const downloadOptions = bestResult.getDownloadOptions()

        # Let's download Q4_K_M, a good middle ground quantization
        const desiredModel = downloadOptions.find(option => option.quantization === 'Q4_K_M')

        # 3. Download it!
        const modelKey = desiredModel.download()

        # This returns a path you can use to load the model
        const loadedModel = client.llm.model(modelKey)
```

## Advanced Usage

### Progress callbacks

TODO: TS/python differ in callback names

Model downloading can take a very long time, depending on your local network speed.
If you want to get updates on the progress of this process, you can provide callbacks to `download`:
one for progress updates and/or one when the download is being finalized
(validating checksums, etc.)

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import { LMStudioClient, type DownloadProgressUpdate } from "@lmstudio/sdk";

        function printProgressUpdate(update: DownloadProgressUpdate) {
          process.stdout.write(`Downloaded ${update.downloadedBytes} bytes of ${update.totalBytes} total \
                                at ${update.speed_bytes_per_second} bytes/sec`)
        }

        const client = new LMStudioClient()

        # ... Same code as before ...

        modelKey = desiredModel.download({
          onProgress: printProgressUpdate,
          onStartFinalizing: () => console.log("Finalizing..."),
        })

        const loadedModel = client.llm.model(modelKey)

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        def print_progress_update(update: lmstudio.DownloadProgressUpdate) -> None:
            print(f"Downloaded {update.downloaded_bytes} bytes of {update.total_bytes} total \
                    at {update.speed_bytes_per_second} bytes/sec")

        with lms.Client() as client:
            # ... Same code as before ...

            model_key = desired_model.download(
                on_progress=print_progress_update,
                on_finalize: lambda: print("Finalizing download...")
            )

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

```</doc><doc title="Get Context Length" desc="docs page.">---
title: Get Context Length
description: API to get the maximum context length of a model.
---

LLMs and embedding models, due to their fundamental architecture, have a property called `context length`, and more specifically a **maximum** context length. Loosely speaking, this is how many tokens the models can "keep in memory" when generating text or embeddings. Exceeding this limit will result in the model behaving erratically.

## Use the `get_context_length()` function on the model object

It's useful to be able to check the context length of a model, especially as an extra check before providing potentially long input to the model.

```lms_code_snippet
  title: "example.py"
  variants:
    "Python (convenience API)":
      language: python
      code: |
        context_length = model.get_context_length()
```

The `model` in the above code snippet is an instance of a loaded model you get from the `llm.model` method. See [Manage Models in Memory](../manage-models/loading) for more information.

### Example: Check if the input will fit in the model's context window

You can determine if a given conversation fits into a model's context by doing the following:

1. Convert the conversation to a string using the prompt template.
2. Count the number of tokens in the string.
3. Compare the token count to the model's context length.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        def does_chat_fit_in_context(model: lms.LLM, chat: lms.Chat) -> bool:
            # Convert the conversation to a string using the prompt template.
            formatted = model.apply_prompt_template(chat)
            # Count the number of tokens in the string.
            token_count = len(model.tokenize(formatted))
            # Get the current loaded context length of the model
            context_length = model.get_context_length()
            return token_count < context_length

        model = lms.llm()

        chat = lms.Chat.from_history({
            "messages": [
                { "role": "user", "content": "What is the meaning of life." },
                { "role": "assistant", "content": "The meaning of life is..." },
                # ... More messages
            ]
        })

        print("Fits in context:", does_chat_fit_in_context(model, chat))

```</doc></1 python><2 typescript><doc title="2 Typescript" desc="docs page.">---
title: "`lmstudio-js` (TypeScript SDK)"
sidebar_title: "Introduction"
description: "Getting started with LM Studio's Typescript / JavaScript SDK"
---

The SDK provides you a set of programmatic tools to interact with LLMs, embeddings models, and agentic flows.

## Installing the SDK

`lmstudio-js` is available as an npm package. You can install it using npm, yarn, or pnpm.

```lms_code_snippet
  variants:
    npm:
      language: bash
      code: |
        npm install @lmstudio/sdk --save
    yarn:
      language: bash
      code: |
        yarn add @lmstudio/sdk
    pnpm:
      language: bash
      code: |
        pnpm add @lmstudio/sdk
```

For the source code and open source contribution, visit [lmstudio-js](https://github.com/lmstudio-ai/lmstudio-js) on GitHub.

## Features

- Use LLMs to [respond in chats](./typescript/llm-prediction/chat-completion) or predict [text completions](./typescript/llm-prediction/completion)
- Define functions as tools, and turn LLMs into [autonomous agents](./typescript/agent/act) that run completely locally
- [Load](./typescript/manage-models/loading), [configure](./typescript/llm-prediction/parameters), and [unload](./typescript/manage-models/loading) models from memory
- Supports for both browser and any Node-compatible environments
- Generate embeddings for text, and more!

## Quick Example: Chat with a Llama Model

```lms_code_snippet
  title: "index.ts"
  variants:
    TypeScript:
      language: typescript
      code: |
        import { LMStudioClient } from "@lmstudio/sdk";
        const client = new LMStudioClient();

        const model = await client.llm.model("qwen/qwen3-4b-2507");
        const result = await model.respond("What is the meaning of life?");

        console.info(result.content);
```

### Getting Local Models

The above code requires the [qwen3-4b-2507](https://lmstudio.ai/models/qwen/qwen3-4b-2507). If you don't have the model, run the following command in the terminal to download it.

```bash
lms get qwen/qwen3-4b-2507
```

Read more about `lms get` in LM Studio's CLI [here](./cli/get).</doc><doc title="3 Plugins" desc="docs page.">---
title: "Introduction to Plugins"
description: "A brief introduction to making plugins for LM Studio using TypeScript."
index: 1
---

Plugins extend LM Studio's functionality by providing "hook functions" that execute at specific points during operation.

Plugins are currently written in JavaScript/TypeScript and run on Node.js v20.18.0. Python support is in development.

## Getting Started

LM Studio includes Node.js, so no separate installation is required.

### Create a new plugin

To create a new plugin, navigate to LM Studio... [TO BE CONTINUED]

### Run a plugin in development mode

Once you've created a plugin, run this command in the plugin directory to start development mode:

```bash
lms dev
```

Your plugin will appear in LM Studio's plugin list. Development mode automatically rebuilds and reloads your plugin when you make code changes.

You only need `lms dev` during development. When the plugin is installed, LM Studio automatically runs them as needed. Learn more about distributing and installing plugins in the [Sharing Plugins](./plugins/sharing) section.

## Next Steps

- [Tools Providers](./plugins/tools-provider)

  Give models extra capabilities by creating tools they can use during generation, like accessing external APIs or performing calculations.

- [Prompt Preprocessors](./plugins/prompt-preprocessor)

  Modify user input before it reaches the model - handle file uploads, inject context, or transform queries.

- [Generators](./plugins/generator)

  Create custom text generation sources that replace the local model, perfect for online model adapters.

- [Custom Configurations](./plugins/custom-configuration)

  Add configuration UIs so users can customize your plugin's behavior.

- [Third-Party Dependencies](./plugins/dependencies)

  Use npm packages to leverage existing libraries in your plugins.

- [Sharing Plugins](./plugins/publish-plugins)

  Package and share your plugins with the community.</doc><doc title="4 Embedding" desc="docs page.">---
title: Embedding
sidebar_title: Generating embedding vectors
description: Generate text embeddings from input text
---

Generate embeddings for input text. Embeddings are vector representations of text that capture semantic meaning. Embeddings are a building block for RAG (Retrieval-Augmented Generation) and other similarity-based tasks.

### Prerequisite: Get an Embedding Model

If you don't yet have an embedding model, you can download a model like `nomic-ai/nomic-embed-text-v1.5` using the following command:

```bash
lms get nomic-ai/nomic-embed-text-v1.5
```

## Create Embeddings

To convert a string to a vector representation, pass it to the `embed` method on the corresponding embedding model handle.

```lms_code_snippet
  title: "index.ts"
  variants:
    TypeScript:
      language: typescript
      code: |
        import { LMStudioClient } from "@lmstudio/sdk";
        const client = new LMStudioClient();

        const model = await client.embedding.model("nomic-embed-text-v1.5");

        const { embedding } = await model.embed("Hello, world!");
```</doc><doc title="5 Tokenization" desc="docs page.">---
title: Tokenization
sidebar_title: Tokenizing text
description: Tokenize text using a model's tokenizer
---

Models use a tokenizer to internally convert text into "tokens" they can deal with more easily. LM Studio exposes this tokenizer for utility.

## Tokenize

You can tokenize a string with a loaded LLM or embedding model using the SDK. In the below examples, `llm` can be replaced with an embedding model `emb`.

```lms_code_snippet
  variants:
    TypeScript:
      language: typescript
      code: |
        import { LMStudioClient } from "@lmstudio/sdk";

        const client = new LMStudioClient();
        const model = await client.llm.model();

        const tokens = await model.tokenize("Hello, world!");

        console.info(tokens); // Array of token IDs.
```

## Count tokens

If you only care about the number of tokens, you can use the `.countTokens` method instead.

```lms_code_snippet
  variants:
    TypeScript:
      language: typescript
      code: |
        const tokenCount = await model.countTokens("Hello, world!");
        console.info("Token count:", tokenCount);
```

### Example: Count Context

You can determine if a given conversation fits into a model's context by doing the following:

1. Convert the conversation to a string using the prompt template.
2. Count the number of tokens in the string.
3. Compare the token count to the model's context length.

```lms_code_snippet
  variants:
    TypeScript:
      language: typescript
      code: |
        import { Chat, type LLM, LMStudioClient } from "@lmstudio/sdk";

        async function doesChatFitInContext(model: LLM, chat: Chat) {
          // Convert the conversation to a string using the prompt template.
          const formatted = await model.applyPromptTemplate(chat);
          // Count the number of tokens in the string.
          const tokenCount = await model.countTokens(formatted);
          // Get the current loaded context length of the model
          const contextLength = await model.getContextLength();
          return tokenCount < contextLength;
        }

        const client = new LMStudioClient();
        const model = await client.llm.model();

        const chat = Chat.from([
          { role: "user", content: "What is the meaning of life." },
          { role: "assistant", content: "The meaning of life is..." },
          // ... More messages
        ]);

        console.info("Fits in context:", await doesChatFitInContext(model, chat));
```

<!-- ### Context length comparisons

The below examples check whether a conversation is over a LLM's context length
(replace `llm` with `emb` to check for an embedding model).

```lms_code_snippet
  variants:
    TypeScript:
      language: typescript
      code: |
        import { LMStudioClient, Chat } from "@lmstudio/sdk";

        const client = new LMStudioClient();
        const llm = await client.llm.model();

        // To check for a string, simply tokenize
        var tokens = await llm.tokenize("Hello, world!");

        // To check for a Chat, apply the prompt template first
        const chat = Chat.createEmpty().withAppended("user", "Hello, world!");
        const templatedChat = await llm.applyPromptTemplate(chat);
        tokens = await llm.tokenize(templatedChat);

        // If the prompt's length in tokens is less than the context length, you're good!
        const contextLength = await llm.getContextLength()
        const isOkay = (tokens.length < contextLength)
``` --></doc><doc title="1 Tools Provider" desc="docs page.">---
title: "Introduction to Tools Provider"
description: "Writing tools providers for LM Studio plugins using TypeScript"
index: 1
---

Tools provider is a function that returns an array of tools that the model can use during generation.

## Examples

The following are some plugins that make use of tools providers:

- [lmstudio/wikipedia](https://lmstudio.ai/lmstudio/wikipedia)

  Gives the LLM tools to search and read Wikipedia articles.

- [lmstudio/js-code-sandbox](https://lmstudio.ai/lmstudio/js-code-sandbox)

  Gives the LLM tools to run JavaScript/TypeScript code in a sandbox environment using [deno](https://deno.com/).

- [lmstudio/dice](https://lmstudio.ai/lmstudio/dice)

  Allows the LLM to generate random numbers using "dice".</doc><doc title="2 Prompt Preprocessor" desc="docs page.">---
title: "Introduction"
description: "Writing prompt preprocessors for LM Studio plugins using TypeScript"
index: 1
---

Prompt Preprocessor is a function that is called upon the user hitting the "Send" button. It receives the user input and can modify it before it reaches the model. If multiple prompt preprocessors are registered, they will be chained together, with each one receiving the output of the previous one.

The modified result will be saved in the chat history, meaning that even if your plugin is disabled afterwards, the modified input will still be used.

Prompt preprocessors will only be triggered for the current user input. It will not be triggered for previous messages in the chat history even if they were not preprocessed.

Prompt preprocessors takes in a `ctl` object for controlling the preprocessing and a `userMessage` it needs to preprocess. It returns either a string or a message object which will replace the user message.

### Examples

The following are some plugins that make use of prompt preprocessors:

- [lmstudio/rag-v1](https://lmstudio.ai/lmstudio/rag-v1)

  Retrieval Augmented Generation (RAG) for LM Studio. This is the plugin that gives document handling capabilities to LM Studio.</doc><doc title="3 Generator" desc="docs page.">---
title: "Introduction"
description: "Writing generators for LM Studio plugins using TypeScript"
index: 1
---

Generators are replacement for local LLMs. They act like a token source. When a plugin with a generator is used, LM Studio will no longer use the local model to generate text. The generator will be used instead.

Generators are useful for implementing adapters for external models, such as using a remote LM Studio instance or other online models.

If a plugin contains a generator, it will no longer show up in the plugins list. Instead, it will show up in the model dropdown and act as a model. If your plugins contains [Tools Provider](./tools-providers.md) or [Prompt Preprocessor](./prompt-preprocessors.md), they will be used when your generator is being selected.

## Examples

The following are some plugins that make use of generators:

- [lmstudio/remote-lmstudio](https://lmstudio.ai/lmstudio/remote-lmstudio)

  Basic support for using a remote LM Studio instance to generate text.

- [lmstudio/openai-compat-endpoint](https://lmstudio.ai/lmstudio/openai-compat-endpoint)

  Use any OpenAI-compatible API in LM Studio.</doc><doc title="4 Custom Configuration" desc="docs page.">---
title: "Introduction"
description: "Add custom configurations to LM Studio plugins using TypeScript"
index: 1
---

LM Studio plugins support custom configurations. That is, you can define a configuration schema and LM Studio will present a UI to the user so they can configure your plugin without having to edit any code.

There are two types of configurations:

- **Per-chat configuration**: tied to a specific chat. Different chats can have different configurations. Most configurations that affects the behavior of the plugin should be of this type.
- **Global configuration**: apply to _all_ chats and are shared across the application. This is useful for global settings such as API keys.

## Types of Configurations

You can define configurations in TypeScript using the `createConfigSchematics` function from the `@lmstudio/sdk` package. This function allows you to define fields with various types and options.

Supported types include:

- `string`: A text input field.
- `numeric`: A number input field with optional validation and slider UI.
- `boolean`: A checkbox or toggle input field.
- `stringArray`: An array of string values with configurable constraints.
- `select`: A dropdown selection field with predefined options.

See the [Defining New Fields](./custom-configuration/defining-new-fields) section for more details on how to define these fields.

## Examples

The following are some plugins that make use of custom configurations

- [lmstudio/wikipedia](https://lmstudio.ai/lmstudio/wikipedia)

  Gives the LLM tools to search and read Wikipedia articles.

- [lmstudio/openai-compat-endpoint](https://lmstudio.ai/lmstudio/openai-compat-endpoint)

  Use any OpenAI-compatible API in LM Studio.</doc><doc title="5 Publish Plugins" desc="docs page.">---
title: "Sharing Plugins"
description: "How to publish your LM Studio plugins so they can be used by others"
index: 7
---

To share publish your LM Studio plugin, open the plugin directory in a terminal and run:

```bash
lms push
```

This command will package your plugin and upload it to the LM Studio Hub. You can use this command to create new plugins or update existing ones.

### Changing Plugin Names

If you wish to change the name of the plugin, you can do so by editing the `manifest.json` file in the root of your plugin directory. Look for the `name` field and update it to your desired plugin name. Note the `name` must be kebab-case.

When you `lms push` the plugin, it will be treated as a new plugin if the name has changed. You can delete the old plugin from the LM Studio Hub if you no longer need it.

### Publishing Plugins to an Organization

If you are in an organization and wish to publish the plugin to the organization, you can do so by editing the `manifest.json` file in the root of your plugin directory. Look for the `owner` field and set it to the name of your organization. When you run `lms push`, the plugin will be published to the organization instead of your personal account.

### Private Plugins

If your account supports private plugins, you can publish your plugins privately by using the `--private` flag when running `lms push`:

```bash
lms push --private
```

Private artifact is in test. Get in touch if you are interested.</doc><doc title="Authentication" desc="docs page.">---
title: Authentication
sidebar_title: Authentication
description: Using API Tokens in LM Studio
index: 2
---

##### Requires [LM Studio 0.4.0](/download) or newer.

LM Studio supports API Tokens for authentication, providing a secure and convenient way to access the LM Studio API.

By default, the LM Studio API runs **without enforcing authentication**. For production or shared environments, enable API Token authentication for secure access.

```lms_info
To enable API Token authentication, create tokens and control granular permissions, check [this guide](/docs/developer/core/authentication) for more details.
```

## Providing the API Token

There are two ways to provide the API Token when creating an instance of `LMStudioClient`:

1. **Environment Variable (Recommended)**: Set the `LM_API_TOKEN` environment variable, and the SDK will automatically read it.
2. **Function Argument**: Pass the token directly as the `apiToken` parameter in the constructor.

```lms_code_snippet
  variants:
    Environment Variable:
      language: typescript
      code: |
        // Set environment variables in your terminal before running the code:
        // export LM_API_TOKEN="your-token-here"

        import { LMStudioClient } from "@lmstudio/sdk";
        // The SDK automatically reads from LM_API_TOKEN environment variable
        const client = new LMStudioClient();

        const model = await client.llm.model("qwen/qwen3-4b-2507");
        const result = await model.respond("What is the meaning of life?");

        console.info(result.content);
    Function Argument:
      language: typescript
      code: |
        import { LMStudioClient } from "@lmstudio/sdk";
        const client = new LMStudioClient({
          apiToken: "your-token-here",
        });

        const model = await client.llm.model("qwen/qwen3-4b-2507");
        const result = await model.respond("What is the meaning of life?");

        console.info(result.content);
```</doc></2 typescript><3 cli><doc title="3 Cli" desc="docs page.">---
title: "`lms` — LM Studio's CLI"
sidebar_title: "Introduction"
description: Get starting with the `lms` command line utility.
index: 1
---

## Install `lms`

`lms` ships with LM Studio, so you don't need to do any additional installation steps if you have LM Studio installed.

Just open a terminal window and run `lms`:

```shell
lms --help
```

## Open source

`lms` is **MIT Licensed** and is developed in this repository on GitHub: https://github.com/lmstudio-ai/lms

## Command quick links

| Command | Syntax | Docs |
| --- | --- | --- |
| Chat in the terminal | `lms chat` | [Guide](/docs/cli/local-models/chat) |
| Download models | `lms get` | [Guide](/docs/cli/local-models/get) |
| List your models | `lms ls` | [Guide](/docs/cli/local-models/ls) |
| See models loaded into memory | `lms ps` | [Guide](/docs/cli/local-models/ps) |
| Control the server | `lms server start` | [Guide](/docs/cli/serve/server-start) |
| Manage the inference runtime | `lms runtime` | [Guide](/docs/cli/runtime) |


### Verify the installation

```lms_info
👉 You need to run LM Studio _at least once_ before you can use `lms`.
```

Open a terminal window and run `lms`.

```lms_terminal
$ lms

lms is LM Studio's CLI utility for your models, server, and inference runtime. (v0.0.47)

Usage: lms [options] [command]

Local models
   chat               Start an interactive chat with a model
   get                Search and download models
   load               Load a model
   unload             Unload a model
   ls                 List the models available on disk
   ps                 List the models currently loaded in memory
   import             Import a model file into LM Studio

Serve
   server             Commands for managing the local server
   log                Log incoming and outgoing messages

Runtime
   runtime            Manage and update the inference runtime

Develop & Publish (Beta)
   clone              Clone an artifact from LM Studio Hub to a local folder
   push               Uploads the artifact in the current folder to LM Studio Hub
   dev                Starts a plugin dev server in the current folder
   login              Authenticate with LM Studio

Learn more:           https://lmstudio.ai/docs/developer
Join our Discord:     https://discord.gg/lmstudio
```

## Use `lms` to automate and debug your workflows

### Start and stop the local server

```bash
lms server start
lms server stop
```

Learn more about [`lms server`](/docs/cli/serve/server-start).

### List the local models on the machine

```bash
lms ls
```

Learn more about [`lms ls`](/docs/cli/local-models/ls).

This will reflect the current LM Studio models directory, which you set in **📂 My Models** tab in the app.

### List the currently loaded models

```bash
lms ps
```

Learn more about [`lms ps`](/docs/cli/local-models/ps).

### Load a model (with options)

```bash
lms load [--gpu=max|auto|0.0-1.0] [--context-length=1-N]
```

`--gpu=1.0` means 'attempt to offload 100% of the computation to the GPU'.

- Optionally, assign an identifier to your local LLM:

```bash
lms load openai/gpt-oss-20b --identifier="my-model-name"
```

This is useful if you want to keep the model identifier consistent.

### Unload a model
```
lms unload [--all]
```

Learn more about [`lms load and unload`](/docs/cli/local-models/load).</doc><doc title="Lms Load" desc="docs page.">---
title: "`lms load`"
description: Use the lms CLI to load or unload models
---

### `lms load`

```bash
lms load --model <model-name> --path <path-to-model>
```</doc><doc title="Chat" desc="docs page.">---
title: "`lms chat`"
sidebar_title: "`lms chat`"
description: Start a chat session with a local model from the command line.
index: 1
---

Use `lms chat` to talk to a local model directly in the terminal. This is handy for quick experiments or scripting.

### Flags 
```lms_params
- name: "[model]"
  type: "string"
  optional: true
  description: "Identifier of the model to use. If omitted, you will be prompted to pick one."
- name: "-p, --prompt"
  type: "string"
  optional: true
  description: "Send a one-off prompt and print the response to stdout before exiting"
- name: "-s, --system-prompt"
  type: "string"
  optional: true
  description: "Custom system prompt for the chat"
- name: "--stats"
  type: "flag"
  optional: true
  description: "Show detailed prediction statistics after each response"
- name: "--ttl"
  type: "number"
  optional: true
  description: "Seconds to keep the model loaded after the chat ends (default: 3600)"
```

### Start an interactive chat

```shell
lms chat
```

You will be prompted to pick a model if one is not provided.

### Chat with a specific model

```shell
lms chat my-model
```

### Send a single prompt and exit

Use `-p` to print the response to stdout and exit instead of staying interactive:

```shell
lms chat my-model -p "Summarize this release note"
```

### Set a system prompt

```shell
lms chat my-model -s "You are a terse assistant. Reply in two sentences."
```

### Keep the model loaded after chatting

```shell
lms chat my-model --ttl 600
```

### Pipe input from another command

`lms chat` reads from stdin, so you can pipe content directly into a prompt:

```shell
cat my_file.txt | lms chat -p "Summarize this, please"
```</doc><doc title="Clone" desc="docs page.">---
title: "`lms clone`"
sidebar_title: "`lms clone`"
description: Clone an artifact from LM Studio Hub to a local folder (beta).
index: 1
---

Use `lms clone` to copy an artifact from LM Studio Hub onto your machine.

### Flags
```lms_params
- name: "<artifact>"
  type: "string"
  optional: false
  description: "Artifact identifier in the form owner/name"
- name: "[path]"
  type: "string"
  optional: true
  description: "Destination folder. Defaults to a new folder named after the artifact."
```

If no path is provided, `lms clone owner/name` creates a folder called `name` in the current directory. The command exits if the target path already exists.

### Clone the latest revision

```shell
lms clone alice/sample-plugin
```

### Clone into a specific directory

```shell
lms clone alice/sample-plugin ./my-folder
```</doc><doc title="Dev" desc="docs page.">---
title: "`lms dev` (Beta)"
sidebar_title: "`lms dev`"
description: Start a plugin dev server or install a local plugin (beta).
index: 3
---

Use `lms dev` inside a plugin project to run a local dev server that rebuilds and reloads on file changes.

This feature is a part of LM Studio [Plugins](/docs/typescript/plugins), currently in private beta.

### Run the dev plugin server

```shell
lms dev
```

This verifies `manifest.json`, installs dependencies if needed, and starts a watcher that rebuilds the plugin on changes. Supported runners: Node/ECMAScript and Deno.

### Install the plugin instead of running dev

```shell
lms dev --install
```

### Flags
```lms_params
- name: "-i, --install"
  type: "flag"
  optional: true
  description: "Install the plugin into LM Studio instead of running the dev server"
- name: "--no-notify"
  type: "flag"
  optional: true
  description: "Do not show the \"Plugin started\" notification in LM Studio"
```</doc><doc title="Get" desc="docs page.">---
title: "`lms get`"
sidebar_title: "`lms get`"
description: Search and download models from the command line.
index: 2
---

The `lms get` command allows you to search and download models from online repositories. If no model is specified, it shows staff-picked recommendations.

Models you download via `lms get` will be stored in your LM Studio model directory. 

### Flags 
```lms_params
- name: "[modelName]"
  type: "string"
  optional: true
  description: "The model to download. If omitted, staff picks are shown. For models with multiple quantizations, append '@' (e.g., 'llama-3.1-8b@q4_k_m')."
- name: "--mlx"
  type: "flag"
  optional: true
  description: "Include only MLX models in search results. If either '--mlx' or '--gguf' is set, only matching formats are shown; otherwise results match installed runtimes."
- name: "--gguf"
  type: "flag"
  optional: true
  description: "Include only GGUF models in search results. If either '--mlx' or '--gguf' is set, only matching formats are shown; otherwise results match installed runtimes."
- name: "-n, --limit"
  type: "number"
  optional: true
  description: "Limit the number of model options shown."
- name: "--always-show-all-results"
  type: "flag"
  optional: true
  description: "Always prompt you to choose from search results, even when there's an exact match."
- name: "-a, --always-show-download-options"
  type: "flag"
  optional: true
  description: "Always prompt you to choose a quantization, even when an exact match is auto-selected."
```

## Download a model

Download a model by name:

```shell
lms get llama-3.1-8b
```

### Specify quantization

Download a specific model quantization:

```shell
lms get llama-3.1-8b@q4_k_m
```

### Filter by format

Show only MLX or GGUF models:

```shell
lms get --mlx
lms get --gguf
```

### Control search results

Limit the number of results:

```shell
lms get --limit 5
```

Always show all options:

```shell
lms get --always-show-all-results
lms get --always-show-download-options
```</doc><doc title="Import" desc="docs page.">---
title: "`lms import`"
sidebar_title: "`lms import`"
description: Import a local model file into your LM Studio models directory.
index: 6
---

Use `lms import` to bring an existing model file into LM Studio without downloading it.

### Flags 
```lms_params
- name: "<file-path>"
  type: "string"
  optional: false
  description: "Path to the model file to import"
- name: "--user-repo"
  type: "string"
  optional: true
  description: "Set the target folder as <user>/<repo>. Skips the categorization prompts."
- name: "-y, --yes"
  type: "flag"
  optional: true
  description: "Skip confirmations and try to infer the model location from the file name"
- name: "-c, --copy"
  type: "flag"
  optional: true
  description: "Copy the file instead of moving it"
- name: "-L, --hard-link"
  type: "flag"
  optional: true
  description: "Create a hard link instead of moving or copying the file"
- name: "-l, --symbolic-link"
  type: "flag"
  optional: true
  description: "Create a symbolic link instead of moving or copying the file"
- name: "--dry-run"
  type: "flag"
  optional: true
  description: "Do not perform the import, just show what would be done"
```

Only one of `--copy`, `--hard-link`, or `--symbolic-link` can be used at a time. If none is provided, `lms import` moves the file by default.

### Import a model file

```shell
lms import ~/Downloads/model.gguf
```

### Keep the original file

```shell
lms import ~/Downloads/model.gguf --copy
```

### Pick the target folder yourself

Use `--user-repo` to skip prompts and place the model in the chosen namespace:

```shell
lms import ~/Downloads/model.gguf --user-repo my-user/custom-models
```

### Dry run before importing

```shell
lms import ~/Downloads/model.gguf --dry-run
```</doc><doc title="Load" desc="docs page.">---
title: "`lms load`"
sidebar_title: "`lms load`"
description: Load or unload models, set context length, GPU offload, TTL, or estimate memory usage without loading.
index: 3
---

The `lms load` command loads a model into memory. You can optionally set parameters such as context length, GPU offload, and TTL. This guide also covers unloading models with `lms unload`.

### Flags 
```lms_params
- name: "[path]"
  type: "string"
  optional: true
  description: "The path of the model to load. If not provided, you will be prompted to select one"
- name: "--ttl"
  type: "number"
  optional: true
  description: "If provided, when the model is not used for this number of seconds, it will be unloaded"
- name: "--gpu"
  type: "string"
  optional: true
  description: "How much to offload to the GPU. Values: 0-1, off, max"
- name: "--context-length"
  type: "number"
  optional: true
  description: "The number of tokens to consider as context when generating text"
- name: "--identifier"
  type: "string"
  optional: true
  description: "The identifier to assign to the loaded model for API reference"
- name: "--estimate-only"
  type: "boolean"
  optional: true
  description: "Print a resource (memory) estimate and exit without loading the model"
```

## Load a model

Load a model into memory by running the following command:

```shell
lms load <model_key>
```

You can find the `model_key` by first running [`lms ls`](/docs/cli/local-models/ls) to list your locally downloaded models.

### Set a custom identifier

Optionally, you can assign a custom identifier to the loaded model for API reference:

```shell
lms load <model_key> --identifier "my-custom-identifier"
```

You will then be able to refer to this model by the identifier `my_model` in subsequent commands and API calls (`model` parameter).

### Set context length

You can set the context length when loading a model using the `--context-length` flag:

```shell
lms load <model_key> --context-length 4096
```

This determines how many tokens the model will consider as context when generating text.

### Set GPU offload

Control GPU memory usage with the `--gpu` flag:

```shell
lms load <model_key> --gpu 0.5    # Offload 50% of layers to GPU
lms load <model_key> --gpu max    # Offload all layers to GPU
lms load <model_key> --gpu off    # Disable GPU offloading
```

If not specified, LM Studio will automatically determine optimal GPU usage.

### Set TTL

Set an auto-unload timer with the `--ttl` flag (in seconds):

```shell
lms load <model_key> --ttl 3600   # Unload after 1 hour of inactivity
```

### Estimate resources without loading

Preview memory requirements before loading a model using `--estimate-only`:

```shell
lms load --estimate-only <model_key>
```

Optional flags such as `--context-length` and `--gpu` are honored and reflected in the estimate. The estimator accounts for factors like context length, flash attention, and whether the model is vision‑enabled.

Example:

```bash
$ lms load --estimate-only gpt-oss-120b
Model: openai/gpt-oss-120b
Estimated GPU Memory:   65.68 GB
Estimated Total Memory: 65.68 GB

Estimate: This model may be loaded based on your resource guardrails settings.
```

## Unload models

Use `lms unload` to remove models from memory.

### Flags 
```lms_params
- name: "[model_key]"
  type: "string"
  optional: true
  description: "The key of the model to unload. If not provided, you will be prompted to select one"
- name: "--all"
  type: "flag"
  optional: true
  description: "Unload all currently loaded models"
- name: "--host"
  type: "string"
  optional: true
  description: "The host address of a remote LM Studio instance to connect to"
```

### Unload a specific model

```shell
lms unload <model_key>
```

If no model key is provided, you will be prompted to select from currently loaded models.

### Unload all models

```shell
lms unload --all
```

### Unload from a remote LM Studio instance

```shell
lms unload <model_key> --host <host>
```

## Operate on a remote LM Studio instance

`lms load` supports the `--host` flag to connect to a remote LM Studio instance. 

```shell
lms load <model_key> --host <host>
```

For this to work, the remote LM Studio instance must be running and accessible from your local machine, e.g. be accessible on the same subnet.</doc><doc title="Log Stream" desc="docs page.">---
title: "`lms log stream`"
sidebar_title: "`lms log stream`"
description: Stream logs from LM Studio. Useful for debugging prompts sent to the model.
index: 4
---

`lms log stream` lets you inspect the exact strings LM Studio sends to and receives from models, and (new in 0.3.26) stream server logs. This is useful for debugging prompt templates, model IO, and server operations.

### Flags

```lms_params
- name: "-s, --source"
  type: "string"
  optional: true
  description: "Source of logs: model or server (default: model)"
- name: "--stats"
  type: "flag"
  optional: true
  description: "Print prediction stats when available"
- name: "--filter"
  type: "string"
  optional: true
  description: "Filter for model source: input, output, or both"
- name: "--json"
  type: "flag"
  optional: true
  description: "Output logs as JSON (newline separated)"
```

### Quick start

Stream model IO (default):

```shell
lms log stream
```

Stream server logs:

```shell
lms log stream --source server
```

### Filter model logs

```bash
# Only the formatted user input
lms log stream --source model --filter input

# Only the model output (emitted once the message completes)
lms log stream --source model --filter output

# Both directions
lms log stream --source model --filter input,output
```

### JSON output and stats

Emit JSON:

  ```shell
  lms log stream --source model --filter input,output --json
  ```

Include prediction stats:

  ```shell
  lms log stream --source model --filter output --stats
  ```</doc><doc title="Login" desc="docs page.">---
title: "`lms login`"
sidebar_title: "`lms login`"
description: Authenticate with LM Studio Hub (beta).
index: 4
---

Use `lms login` to authenticate the CLI with LM Studio Hub.

### Sign in with the browser

```shell
lms login
```

The CLI opens a browser window for authentication. If a browser cannot be opened automatically, copy the printed URL into your browser.

### "CI style" login with pre-authenticated keys

```bash
lms login --with-pre-authenticated-keys \
  --key-id <KEY_ID> \
  --public-key <PUBLIC_KEY> \
  --private-key <PRIVATE_KEY> 
```

### Advanced Flags
```lms_params
- name: "--with-pre-authenticated-keys"
  type: "flag"
  optional: true
  description: "Authenticate using pre-generated keys (CI/CD). Requires --key-id, --public-key, and --private-key."
- name: "--key-id"
  type: "string"
  optional: true
  description: "Key ID to use with --with-pre-authenticated-keys"
- name: "--public-key"
  type: "string"
  optional: true
  description: "Public key to use with --with-pre-authenticated-keys"
- name: "--private-key"
  type: "string"
  optional: true
  description: "Private key to use with --with-pre-authenticated-keys"
```</doc></3 cli></project>
