# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.6.0] - 2026-05-22

### Added
- **AMSDAL Skills:** Added the `AmsdalSkill` model for skill management with tools for skill discovery (`list_available_skills`). Includes `use_when` (which allows defining conditions for when a skill should be used) and `requires_user_approval` (which allows requiring manual user confirmation before execution). In the MCP flow, skills are now called first in the chat.
- Updated `get_model_schema` MCP tool flow: The LLM is now instructed to call the tool every time a model is mentioned to maintain accurate context.

### Changed
- Replaced property extraction mechanism: Switched from `get_property_source` to `model_to_object_schema` and removed the unused `get_property_source` functions entirely.
- Removed cell length truncation limits in the NLQ Retriever, allowing the full content of database records and columns to be returned without being cut off.
- Enhanced MCP tool prompts and descriptions, specifically for entity resolution, synthesis, and transaction tools.

## [1.6.0-rc1] - 2026-05-12

### Added
- **Expanded Query Support:** Added support for queries using primary keys (PK), including full support for composite keys.
- **NLQuery Retriever Enhancements:**
    - Introduced stage-aware retriever analysis with confidence scoring and field pruning.
    - Added support for complex query filters with `AND/OR` logical conditions via `filter_group`.
- **NLQuery Updater Improvements:**
    - Enhanced preview support for related (nested) model updates.
    - Improved robustness for update operations by validating related changes in empty payloads.
- **Agent & MCP Refactoring:**
    - Introduced `MCPServerConfig` and unified `BaseTool` hierarchy in `ml_models/primitives.py`.
    - LLM models now accept tools inheriting from the common `BaseTool` base class and MCP servers via `MCPServerConfig`.
    - Provider-specific parameters for MCP and tools are now encapsulated in `extra_params`.
    - Updated `perform_crud_operation` tool descriptions and enforced stricter analysis payload validation.

### Changed
- **Model Adapters:** Refactored `ClaudeAdapter` and `OpenAIAdapter` to support the new `BaseTool` and `extra_params` architecture.
- **MCP Connectors:** Unified connector logic and removed redundant provider-specific implementations in favor of `MCPServerConfig`.

## [1.5.0] - 2026-04-29

### Added
- Added attachment and message history support (`ChatMessage`, `MessageBuffer`) to `FunctionalCallingAgent`, allowing attachments to persist across LLM turns.
- Enhanced ML Models to handle and pass attachment inputs as segmented parts during API calls, preserving accurate message history natively.
- Exposed explicit `object_id` fields in `nlq_retriever` outputs, enabling `nlquery/creator`, `updater`, and `deleter` to directly reference, link, update, or remove existing records by ID.
- Added robust validation, schemas, and error handling for NLQ components.

### Changed
- **NLQ Refactoring:** Extracted Natural Language Query (NLQ) logic from `ml_retrievers` into a new, dedicated `amsdal_ml/nlquery/` package, organized into phase-specific sub-packages (`creator/`, `updater/`, `deleter/`, `retriever/`), with structured core modules and a shared NLQ base class.
- **ML Models Restructuring:** Unified model primitives (`base_model.py`, `base_adapter.py`, `primitives.py`) and relocated schema adapters into provider-specific model packages (`amsdal_ml/ml_models/openai/`, `amsdal_ml/ml_models/claude/`).
- **Agents Reorganization:** Cleaned up `amsdal_ml/agents/` into a domain-driven structure (core logic, schemas, memory, and specialized tooling/integration categories like `structured_tools` and `mcp` connectors).
- Reorganized prompt files into structured subdirectories matching the codebase architecture.
- Standardized Foreign Key/M2M Discovery processes utilizing official AMSDAL introspection utilities.
- Extracted and grouped AMSDAL-specific utility functions into a dedicated `amsdal_ml/utils/amsdal_internal` package (previously scattered in `amsdal_ml/utils`).

### Removed
- Removed deprecated `MultipleRecordsError` and outdated query retriever/updater components.
- Removed deprecated tool adapters and retriever adapters, merging their functionality into the new adapter architecture.
- Removed the `get_model_field_values_by_ids` MCP tool, as its functionality is now supported natively by `CRUD` operations via the enhanced `nlq_retriever`.

## [1.4.0] - 2026-04-10

### Added
- Added MCP connector support in `FunctionalCallingAgent`, so remote MCP servers can be attached directly to function-calling flows.
- Added file attachment support in `FunctionalCallingAgent` (including provider file IDs for images and documents).
- Added provider-specific MCP connectors in `amsdal_ml/agents/mcp_connectors.py` for both OpenAI and Claude payload formats.
- Added `amsdal_ml/agents/message_buffer.py` with canonical message history types (`AgentMessage`, `MessageBuffer`) and MCP event models.
- Added `MCPItem` hierarchy for structured MCP request/response handling in agent adapters and connector flows.
- Added new Anthropic model implementation in `amsdal_ml/ml_models/claude`.
- Added Claude file loader in `amsdal_ml/fileio/claude_loader.py`.
- Added Claude adapters for function/tool calling and structured response schema handling.

### Changed
- Reorganized model implementations into provider-specific subpackages; OpenAI model code now lives under `amsdal_ml/ml_models/openai`.
- Fully rewrote `OpenAIModel` to use the OpenAI Responses API as the single pathway for sync/async and streaming calls.
- `setup()` in model implementations is now asynchronous (`await model.setup()`) via the `MLModel` base interface.
- Agents now support passing chat history between LLM calls in a normalized, provider-agnostic format.

### Fixed
- Fixed NLQ retriever behavior and related query execution edge cases.

## [1.3.0] - 2026-03-25

### Added
- Added `list_available_transactions` MCP tool to discover available transactions by scope (`user`, `contrib`, `all`) with parameter metadata.
- Added `perform_transaction_operation` MCP tool for natural language transaction execution with model argument modes (`existing_ref`, `new_object`, `explicit_id`) and primitive/dict support.

### Changed
- Transaction execution now follows strict two-step confirmation: run analysis with `confirm=false`, then execute with `confirm=true` using the returned `analysis` payload.

## [1.2.1] - 2026-03-06

### Changed
- Replaced `pymupdf` with `pypdf` in PDF ingestion loader implementation.
- Updated project dependencies and lockfile to remove `pymupdf` and use `pypdf`.
- Updated documentation references for `PdfLoader` to reflect `pypdf` usage.

## [1.2.0] - 2026-03-04

### Added
- Added MCP OAuth integration in `amsdal_ml/mcp_server/mcp_oauth`, enabling MCP server authorization via AMSDAL authentication.

### Security
- MCP OAuth flow now preserves AMSDAL permission checks during authenticated MCP access.

## [1.1.0] - 2026-02-25

### Changed
- Upgraded to `amsdal[cli]>=0.8.0`, consolidating `amsdal`, `amsdal_cli`, and `amsdal_server` into a single dependency.

## [1.0.1] - 2026-02-18

### Changed
- Documentation: recommend Chatbox AI (desktop) as the primary MCP client and document setup steps (provider token + custom MCP server + auth headers).
- Documentation: clarify LibreChat usage as local-only via `ngrok` tunnel (deployed server URLs may fail), and note `ngrok-skip-browser-warning=1` header for tunnels.

## [1.0.0] - 2026-02-17

### Added
- First MCP server release for AMSDAL apps via `server_model_explorer`, including:
	- `list_available_models` for model discovery,
	- `get_model_schema` for full JSON schema + computed property source,
	- `get_model_relationships` for outgoing and incoming FK graph inspection,
	- `get_model_field_values_by_ids` for id-based field inspection (with history support),
	- `perform_crud_operation` for NL-driven read/create/update/delete with two-step confirmation.
- Added `NLQueryCreatorExecutor` with a 3-phase create pipeline:
	- resolution planning for nested entities,
	- context retrieval for candidate related objects,
	- payload synthesis + persistence with required-field checks and reference/file preprocessing.
- Added `NLQueryUpdaterExecutor` with advanced update flow:
	- split of natural language into `find_query` + `update_query`,
	- filter generation + preview of direct/FK/JSONB changes,
	- intent detection (`UPDATE_DIRECT`, `REASSIGN_FK`, `UPDATE_RELATED`, `MIXED`),
	- related-entity update handling and transactional apply.
- Added `NLQueryDeleterExecutor` with split/analyze/execute delete flow, preview support, and transactional deletion.
- Expanded `NLQueryExecutor` (retriever) behavior for robust filter parsing, related-queryset application, preview summaries, and strict single-record execution checks.
- Added support for ingester workflows with new HTTP MCP functionality.

### Changed
- This is the first major MCP release line (`1.x`) for AMSDAL ML.
- Query execution in `perform_crud_operation` uses an explicit two-step confirmation flow:
	- `confirm=false` returns analysis/preview only,
	- `confirm=true` executes using the returned analysis payload.
- Context resolution pipeline was reworked to better resolve nested entities and references in NLQ payloads.
- `get_model_field_values_by_ids` is the primary MCP tool for explicit "show object by id" verification before user confirmation.
- Dynamic dimension sizing for embeddings is now configurable from environment variables.
- Improved flexible-filter behavior in the React agent.
- Global improvements:
	- Dynamic dimension sizing in the embedding model (env),
	- Improvements to the flexible filter in the React agent,
	- Fixes for the MCP stdio client,
	- Support for the ingester with new HTTP MCP functionality,
	- Tenacity-based retry logic in the OpenAI model / OpenAI retriever,
	- New tests added.

### Security
- Permission checks are enforced at two levels in MCP operations:
	- class-level authorization before operation planning/execution,
	- object-level authorization while objects are loaded, saved, or deleted.
- Request context bridging (`with_amsdal_context`) propagates MCP request user/auth into AMSDAL context for permission evaluation in execution hooks.

## [0.3.2] - 2026-02-06

### Added
- AMSDAL EULA license
- CHANGELOG.md in Keep a Changelog format
- Consolidated CI/CD workflows

## [0.3.1] - 2026-02-06

### Fixed
- Fixed on_setup for AppConfig

## [0.3.0] - 2026-02-06

### Changed
- Update to amsdal v0.6.0

## [0.2.2] - 2026-01-19

### Fixed
- Added support for `JSON_SCHEMA` response format when using attachments in OpenAI Responses API
- Support for images in OpenAI Responses API (automatically selects `input_image` based on MIME type)
- Added `mime_type` support to `FileAttachment` and `OpenAIFileLoader`

## [0.2.1] - 2025-12-23

### Changed
- Upgraded dependencies: pydantic~=2.12, pydantic-settings~=2.12

## [0.2.0] - 2025-12-16

### Added
- `PythonTool`: Tool for executing Python functions within agents
- `FunctionalCallingAgent`: Agent specialized in functional calling with configurable tools
- `NLQueryRetriever`: Retriever for natural language queries on AMSDAL querysets
- `DefaultIngestionPipeline`: Pipeline for document ingestion including loader, cleaner, splitter, embedder, and store
- `ModelIngester`: High-level ingester for processing models with customizable pipelines and metadata
- `PdfLoader`: Document loader using pymupdf for PDF processing
- `TextCleaner`: Processor for cleaning and normalizing text
- `TokenSplitter`: Splitter for dividing text into chunks based on token count
- `OpenAIEmbedder`: Embedder for generating embeddings via OpenAI API
- `EmbeddingDataStore`: Store for saving embedding data linked to source objects

## [0.1.4] - 2025-10-15

### Fixed
- Fixed lazy initialization of OpenAIRetriever to ensure env vars are loaded
- Added missing env parameter to stdio_client for non-persistent sessions
- Environment variables now properly passed to MCP stdio subprocesses
- Updated README.md to be production-ready
- Added RELEASE.md with step-by-step release guide

## [0.1.3] - 2025-10-13

### Fixed
- Pass env vars into stdio server
- Cleanup of app.py

## [0.1.2] - 2025-10-08

### Fixed
- Fix for UserWarning: Field name "object_id" in "EmbeddingModel" shadows an attribute in parent "Model"

## [0.1.1] - 2025-10-08

### Added
- BaseFileLoader interface and OpenAI Files API implementation

## [0.1.0] - 2025-09-22

### Added
- Interfaces and default OpenAI-based implementations