---
title: ML Models & Primitives
amsdal_docs:
  target: framework/plugins/amsdal-ml/ml-models.md
  nav_title: ML Models
  nav_section: Framework
  nav_path: ["Plugins & Extensions", "ML", "ML Models"]
  nav_order: 3
---

# ML Models & Primitives

`amsdal_ml` provides a unified interface for interacting with different LLM providers (like OpenAI and Anthropic) using common primitives. This makes it easy to switch providers or pass complex conversation histories, including images and files, without rewriting your code.

## Available Models

Currently, two primary LLM wrappers are supported:

- `OpenAIModel` (using the OpenAI Responses API)
- `ClaudeModel` (using the Anthropic Messages API)

Both models inherit from a common `MLModel` interface and support both synchronous (`invoke`, `stream`) and asynchronous (`ainvoke`, `astream`) operations. 

> **Important:** You must call `.setup()` on a model before executing calls. In async contexts, use `await model.setup()`.

### Basic Example

```python
from amsdal_ml.ml_models.openai.openai_model import OpenAIModel
from amsdal_ml.ml_models.claude.claude_model import ClaudeModel

# OpenAI Example
openai_model = OpenAIModel(model_name='gpt-5.1', temperature=0.7)
await openai_model.setup()

output = await openai_model.ainvoke("What is the speed of light?")
print(output.content)

# Anthropic Claude Example
claude_model = ClaudeModel(model_name='claude-sonnet-4-6', temperature=0.7)
await claude_model.setup()

output = await claude_model.ainvoke("What is the speed of light?")
print(output.content)
```

---

## Primitives

To interact seamlessly across different providers, `amsdal_ml` uses canonical primitives to represent messages, tools, and attachments. These are defined in `amsdal_ml.ml_models.primitives`.

### `MessageRole`
An enumeration of allowed roles in a conversation: `USER`, `ASSISTANT`, `SYSTEM`, and `TOOL`.

### `ChatMessage`
The canonical representation of a message. It encapsulates role, text content, tool calls, and file attachments in a provider-agnostic way.

### MCP Events (`MCPItem`, `MCPToolCall`, `MCPToolResult`)
Base classes and events used to track tools provided by external Model Context Protocol (MCP) servers. The `ChatMessage` model includes a list of `MCPItem` objects to keep track of interactions with remote servers (like a `call_id` and the final execution `content`).

---

## Conversation Memory

### `MessageBuffer`
A helper class used to manage a history of `ChatMessage` objects. It is imported from `amsdal_ml.agents.memory.buffer`. It is used extensively by AI Agents and provides a convenient API for managing conversation flow manually.

---

## Using Memory & File Attachments

One of the most powerful features of `amsdal_ml` models is their native support for file attachments—such as sending images or documents as part of the context.

To upload files, you can use built-in loaders like `OpenAIFileLoader`. Once uploaded, they become `FileAttachment` objects which can be embedded directly into a `ChatMessage`.

### Example: Analyzing Images

The following example demonstrates how to set up a `MessageBuffer`, inject a system prompt, attach two images to a user message, and invoke the model:

```python
import asyncio
from openai import AsyncOpenAI
from amsdal_ml.ml_models.openai.openai_model import OpenAIModel
from amsdal_ml.ml_models.primitives import ChatMessage, MessageRole
from amsdal_ml.agents.memory.buffer import MessageBuffer
from amsdal_ml.fileio.openai_loader import OpenAIFileLoader
from amsdal_ml.fileio.base_loader import FileItem

async def analyze_images():
    # 1. Initialize the model
    model = OpenAIModel(model_name='gpt-5.1', temperature=0.2)
    await model.setup()

    # 2. Upload images using a loader
    client = AsyncOpenAI()
    loader = OpenAIFileLoader(client, purpose='vision')

    with open('image_a.jpg', 'rb') as f1, open('image_b.jpg', 'rb') as f2:
        att1 = await loader.load(FileItem(file=f1, filename='image_a.jpg'))
        att2 = await loader.load(FileItem(file=f2, filename='image_b.jpg'))

    # 3. Build conversation memory using MessageBuffer
    buffer = MessageBuffer()
    
    # Add system instructions
    buffer.append(ChatMessage(
        role=MessageRole.SYSTEM,
        content="You are an expert image analyst."
    ))

    # Add the user query with the attachments
    buffer.append(ChatMessage(
        role=MessageRole.USER,
        content="Please compare these two images and describe the differences.",
        attachments=[att1, att2]
    ))

    # 4. Invoke the model with the entire history
    output = await model.ainvoke(input=buffer.snapshot())
    
    print("Analysis:")
    print(output.content)

# Run the async function
# asyncio.run(analyze_images())
```

### Passing Buffer History to Claude

The exact same primitives work flawlessly with `ClaudeModel`. You only need to change the model initialization:

```python
from amsdal_ml.ml_models.claude.claude_model import ClaudeModel

async def claude_example():
    # Use Claude instead of OpenAI
    model = ClaudeModel(model_name='claude-sonnet-4-6')
    await model.setup()

    buffer = MessageBuffer()
    buffer.add_user("Write a haiku about a robot.", attachments=[])
    
    # Use the buffer's snapshot (list of ChatMessage) as input
    output = await model.ainvoke(input=buffer.snapshot())
    print(output.content)
```

## `LLModelOutput`

When you call `invoke` or `ainvoke`, the model returns an `LLModelOutput` object. This contains:
- `content`: The final text output from the assistant.
- `raw_response`: The raw, provider-specific response object (e.g., the native OpenAI or Anthropic response class), allowing you to access metadata like token usage or finish reasons if needed.
