Metadata-Version: 2.4
Name: xlmtec
Version: 3.27.0
Summary: Production-grade LLM fine-tuning framework with CLI
Author: Abdur Rahman
License: MIT
Project-URL: Homepage, https://github.com/Abdur-azure/xlmtec
Project-URL: Documentation, https://Abdur-azure.github.io/xlmtec
Project-URL: Issues, https://github.com/Abdur-azure/xlmtec/issues
Keywords: llm,fine-tuning,lora,qlora,transformers
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pyyaml>=6.0.0
Requires-Dist: typer>=0.9.0
Requires-Dist: rich>=13.0.0
Requires-Dist: tqdm>=4.65.0
Requires-Dist: rouge-score>=0.1.2
Requires-Dist: nltk>=3.8.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: huggingface-hub>=0.19.0
Requires-Dist: sentencepiece>=0.1.99
Requires-Dist: protobuf>=3.20.0
Provides-Extra: ml
Requires-Dist: torch>=2.0.0; extra == "ml"
Requires-Dist: transformers>=4.35.0; extra == "ml"
Requires-Dist: datasets>=2.14.0; extra == "ml"
Requires-Dist: peft>=0.7.0; extra == "ml"
Requires-Dist: accelerate>=0.24.0; extra == "ml"
Requires-Dist: bitsandbytes>=0.41.0; extra == "ml"
Provides-Extra: tui
Requires-Dist: textual>=0.52.0; extra == "tui"
Provides-Extra: dpo
Requires-Dist: trl>=0.7.0; extra == "dpo"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: pytest-timeout>=2.0.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs>=1.5.0; extra == "docs"
Requires-Dist: mkdocs-material>=9.0.0; extra == "docs"
Requires-Dist: mkdocs-minify-plugin>=0.7.0; extra == "docs"
Requires-Dist: pymdown-extensions>=10.0.0; extra == "docs"
Provides-Extra: onnx
Requires-Dist: optimum[exporters]>=1.16.0; extra == "onnx"
Requires-Dist: onnx>=1.14.0; extra == "onnx"
Requires-Dist: onnxruntime>=1.16.0; extra == "onnx"
Provides-Extra: gguf
Requires-Dist: gguf>=0.6.0; extra == "gguf"
Provides-Extra: notify
Requires-Dist: plyer>=2.1.0; extra == "notify"
Provides-Extra: claude
Requires-Dist: anthropic>=0.20.0; extra == "claude"
Provides-Extra: gemini
Requires-Dist: google-genai>=0.8.0; extra == "gemini"
Provides-Extra: codex
Requires-Dist: openai>=1.10.0; extra == "codex"
Provides-Extra: ai
Requires-Dist: anthropic>=0.20.0; extra == "ai"
Requires-Dist: google-genai>=0.8.0; extra == "ai"
Requires-Dist: openai>=1.10.0; extra == "ai"
Provides-Extra: full
Requires-Dist: torch>=2.0.0; extra == "full"
Requires-Dist: transformers>=4.35.0; extra == "full"
Requires-Dist: datasets>=2.14.0; extra == "full"
Requires-Dist: peft>=0.7.0; extra == "full"
Requires-Dist: accelerate>=0.24.0; extra == "full"
Requires-Dist: bitsandbytes>=0.41.0; extra == "full"
Requires-Dist: textual>=0.52.0; extra == "full"
Requires-Dist: trl>=0.7.0; extra == "full"
Requires-Dist: optimum[exporters]>=1.16.0; extra == "full"
Requires-Dist: onnx>=1.14.0; extra == "full"
Requires-Dist: onnxruntime>=1.16.0; extra == "full"
Requires-Dist: gguf>=0.6.0; extra == "full"
Requires-Dist: pytest>=7.0.0; extra == "full"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "full"
Requires-Dist: pytest-cov>=4.0.0; extra == "full"
Requires-Dist: pytest-timeout>=2.0.0; extra == "full"
Requires-Dist: ruff>=0.1.0; extra == "full"
Requires-Dist: mypy>=1.0.0; extra == "full"
Provides-Extra: sweep
Requires-Dist: optuna>=3.0.0; extra == "sweep"
Dynamic: license-file

# xlmtec

[![PyPI version](https://img.shields.io/pypi/v/xlmtec.svg)](https://pypi.org/project/xlmtec/)
[![Python](https://img.shields.io/pypi/pyversions/xlmtec.svg)](https://pypi.org/project/xlmtec/)
[![License](https://img.shields.io/pypi/l/xlmtec.svg)](https://github.com/Abdur-azure/xlmtec/blob/main/LICENSE)

**xlmtec** is a command-line toolkit for fine-tuning large language models. Describe your task in plain English, get a ready-to-run config, browse HuggingFace models, and train — all from the terminal.

---

## Features

- **AI-powered config generation** — describe your task, get a YAML config from Claude, Gemini, or GPT
- **Model Hub browser** — search and inspect HuggingFace models without leaving the terminal
- **5 fine-tuning methods** — LoRA, QLoRA, Full, Instruction, DPO
- **Config validation** — catch errors before training starts
- **Dry-run mode** — preview your training plan without loading a model
- **Rich terminal UI** — progress bars, panels, colour output throughout

---

## Installation

```bash
# Core (lightweight — no ML deps)
pip install xlmtec

# With training support
pip install xlmtec[ml]

# With AI suggestions (pick your provider)
pip install xlmtec[claude]    # Anthropic
pip install xlmtec[gemini]    # Google
pip install xlmtec[codex]     # OpenAI
pip install xlmtec[ai]        # All three

# Everything
pip install xlmtec[full]
```

---

## Quickstart

### 1. Get an AI-generated config

```bash
xlmtec ai-suggest "fine-tune a small model for customer support" --provider claude
```

Outputs a ready-to-run YAML config and the exact command to run.

### 2. Browse models on HuggingFace

```bash
xlmtec hub search "bert" --task text-classification --limit 5
xlmtec hub trending
xlmtec hub info google/bert-base-uncased
```

### 3. Validate your config

```bash
xlmtec config validate config.yaml
```

### 4. Train

```bash
# Preview without loading model
xlmtec train --config config.yaml --dry-run

# Start training
xlmtec train --config config.yaml
```

---

## Commands

| Command | Description |
|---------|-------------|
| `xlmtec ai-suggest "<task>"` | Generate a config from plain English |
| `xlmtec hub search "<query>"` | Search HuggingFace models |
| `xlmtec hub info <model-id>` | Show model details |
| `xlmtec hub trending` | Top trending models |
| `xlmtec config validate <file>` | Validate a YAML config |
| `xlmtec train --config <file>` | Fine-tune a model |
| `xlmtec train --config <file> --dry-run` | Preview training plan |
| `xlmtec recommend` | Get method recommendation for your hardware |
| `xlmtec evaluate` | Evaluate a fine-tuned model |
| `xlmtec benchmark` | Compare multiple runs |
| `xlmtec merge` | Merge LoRA adapter into base model |
| `xlmtec upload` | Upload model to HuggingFace Hub |
| `xlmtec --version` | Show installed version |

---

## Fine-tuning methods

| Method | VRAM | Best for |
|--------|------|----------|
| `lora` | Low (4–8 GB) | Most tasks, fast convergence |
| `qlora` | Very low (4 GB) | Large models on limited hardware |
| `full` | High (24 GB+) | Best quality, small models |
| `instruction` | Low (4–8 GB) | Prompt/response style tasks |
| `dpo` | Low (4–8 GB) | Preference learning from pairs |

---

## AI Providers

Set your API key as an environment variable, then pass `--provider`:

```bash
export ANTHROPIC_API_KEY=sk-ant-...
xlmtec ai-suggest "summarise legal documents" --provider claude

export GEMINI_API_KEY=...
xlmtec ai-suggest "summarise legal documents" --provider gemini

export OPENAI_API_KEY=sk-...
xlmtec ai-suggest "summarise legal documents" --provider codex
```

---

## Example config

```yaml
model:
  name: gpt2

dataset:
  source: local_file
  path: data/train.jsonl

lora:
  r: 16
  alpha: 32
  target_modules: [c_attn]

training:
  output_dir: output/run1
  num_epochs: 3
  batch_size: 4
  learning_rate: 2e-4
```

---

## Development

```bash
git clone https://github.com/Abdur-azure/xlmtec.git
cd xlmtec
pip install -e ".[full,dev]"
pytest tests/ -v --ignore=tests/test_integration.py
```

---

## Changelog

See [CHANGELOG.md](CHANGELOG.md) for full release history.

## License

MIT
