Metadata-Version: 2.4
Name: ameva-runtime
Version: 2.0.2
Summary: Unified Next-Gen Hardware Orchestration & AI Acceleration Runtime for Mobile & Edge
Home-page: https://github.com/uno-km/ameva-runtime
Author: Eunho Kim
Author-email: Eunho Kim <contact@uno-km.com>
License: Apache-2.0
Project-URL: Homepage, https://uno-km.vercel.app/lib/vulkan/
Project-URL: Repository, https://github.com/uno-km/ameva-runtime
Project-URL: Documentation, https://uno-km.vercel.app/lib/vulkan/
Keywords: vulkan,opencl,npu,cpu-neon,android,termux,acceleration,adreno,mali,hal,ameva
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Android
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: stt
Requires-Dist: termux-stt>=1.1.1; extra == "stt"
Provides-Extra: diffusion
Requires-Dist: termux-diffusion>=1.3.1; extra == "diffusion"
Provides-Extra: bitnet
Requires-Dist: termux-bitnet>=1.0.0; extra == "bitnet"
Provides-Extra: llamacpp
Requires-Dist: termux-llamacpp>=1.0.0; extra == "llamacpp"
Provides-Extra: tts
Requires-Dist: termux-tts>=1.0.0; extra == "tts"
Provides-Extra: vision
Requires-Dist: termux-vision>=1.0.0; extra == "vision"
Provides-Extra: all
Requires-Dist: termux-stt>=1.1.1; extra == "all"
Requires-Dist: termux-diffusion>=1.3.1; extra == "all"
Requires-Dist: termux-bitnet>=1.0.0; extra == "all"
Requires-Dist: termux-llamacpp>=1.0.0; extra == "all"
Requires-Dist: termux-tts>=1.0.0; extra == "all"
Requires-Dist: termux-vision>=1.0.0; extra == "all"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# AMEVA-Runtime (Python)

[![PyPI](https://img.shields.io/pypi/v/ameva-runtime.svg?style=flat-square&color=0369a1)](https://pypi.org/project/ameva-runtime/)
[![Python](https://img.shields.io/pypi/pyversions/ameva-runtime.svg?style=flat-square)](https://pypi.org/project/ameva-runtime/)
[![License](https://img.shields.io/badge/License-Apache_2.0-004499.svg?style=flat-square)](https://github.com/uno-km/ameva-runtime)

> **모바일 및 엣지 환경을 위한 차세대 통합 온디바이스 하드웨어 오케스트레이션 및 6대 멀티모달 가속 런타임**  
> *Next-Gen Unified On-Device Hardware Orchestration & 6-Modality AI Acceleration Runtime for Mobile & Edge*

## Installation

```bash
pip install ameva-runtime
```

## Quickstart

```python
import ameva_runtime as ameva
from ameva_runtime import vulkan

# 1. Execute LLM inference directly with optimal on-device hardware dispatch
result = ameva.run(
    model="qwen2.5-0.5b",
    prompt="Space in Korean is:",
    max_tokens=32
)
print(f"Generated text: {result.text}")
print(f"Hardware backend: {result.backend_used} ({result.tokens_per_second:.2f} t/s)")

# 2. Hardware diagnostic inspection via Vulkan engine
doc = vulkan.Doctor()
report = doc.run_self_test(verbose=False)
print(f"GPU Target: {report.device_name} (Passed: {report.passed_stages}/{report.total_stages})")

```

## Description
AMEVA Runtime is a unified on-device hardware orchestration and AI acceleration engine engineered for mobile and edge systems. It dynamically inspects SoC topology and driver environments, routing compute graphs between Qualcomm Adreno, ARM Mali, and ARM Cortex CPU-NEON.

### 1. Empirical Real-Device LLM Benchmarks (Qwen2.5-0.5B-Instruct, GGUF Q4_K_M)

| Device & Processor | GPU Architecture | Active Backend | Layers in VRAM | Generation Speed (t/s) | Prompt Speed (t/s) | UI Freeze | Acceleration |
| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **Galaxy S25** (Snapdragon 8 Elite) | Qualcomm Adreno 830 (Vulkan 1.3) | **VULKAN** | **25/25 (100%)** | **35.80 t/s** (27.9 ms/t) | **4.53 t/s** | **0% (Stable)** | **35.8x (vs CPU)** |
| **Galaxy A35** (Exynos 1380) | ARM Mali-G68 MP5 (Vulkan 1.3) | **VULKAN (Medium MatMul)** | **25/25 (100%)** | **4.44 t/s** (225 ms/t) | **6.12 t/s** | **0% (Stable)** | **+26.9% (vs NEON)** |
| **Galaxy A35** (Exynos 1380) | Cortex-A78 CPU-NEON (3 Threads) | CPU-NEON | 0/25 | 3.55 t/s (281 ms/t) | 8.05 t/s | 0% (Stable) | Baseline |

### 2. Empirical Real-Device STT Benchmarks (Whisper Large-v3-Turbo Q5_0, 548MB)

- **Test Device**: Samsung Galaxy A35 5G (Exynos 1380, ARM Mali-G68 MP5, 8GB RAM, Android 16 Termux)
- **Audio Source**: John F. Kennedy 1-minute speech sample (`jfk_1min.wav`)

| Execution Mode | Target Hardware | Elapsed Time | GPU Clock / Load | CPU Utilization | Accuracy | Speedup |
| :--- | :--- | :---: | :---: | :---: | :---: | :---: |
| **CPU NEON Mode** (`-dev -1`, 4 threads) | Cortex-A78 x4 cores | **816.48s (13m 36s)** | 0% (Idle) | 291% (Active) | Standard | Baseline |
| **Vulkan GPU Mode** (`-dev 0`, Mali Quirk) | Mali-G68 MP5 | **360.60s (6m 00s)** | **949 MHz (100%)** | **20~30% (Low)** | Standard | **2.26x (56% time reduction)** |

- **Galaxy S25 (Adreno 830) STT**: Completed 4,401 ms inference on speech input (JIT compiler bug resolved).

### 3. Root-Cause Defect Resolution (Ground Truth)

#### (1) ARM Mali-G68 Valhall Integer Truncation Infinite Loop Elimination
- **Defect**: Executing `mul_mm.comp` on Mali-G68 (subgroup size 16) caused GPU hangs and hardware watchdog TDR resets (`VK_ERROR_DEVICE_LOST`).
- **Root Cause**: The stride calculation `loadstride_b = gl_WorkGroupSize.x * LOAD_VEC_B / BK = 16 * 1 / 32 = 0` truncated to zero in integer division, producing an infinite loop `for (uint l = 0; l < BN; l += 0)`.
- **Resolution**: Enforced Medium MatMul kernels (`_m`, workgroup size 128, `loadstride_b = 4 > 0`) via `enforce_medium_matmul: true`, enabling stable 25/25 layer GPU offloading.

#### (2) Qualcomm Adreno 830 JIT Compiler Bug Isolation
- **Defect**: Whisper STT pipeline compilation failed on Snapdragon 8 Elite with `VK_ERROR_UNKNOWN (-13)` during `mul_mat_vec` dispatch.
- **Root Cause**: Qualcomm's Adreno JIT compiler failed register allocation when Specialization Constant `NUM_COLS >= 3`.
- **Resolution**: Bound `mul_mat_vec_max_cols = 2` for Adreno 830, achieving stable GPU inference in 4,401 ms on speech input.

### 4. 6-Modality Acceleration Roadmap

| Modality | Engine & Architecture | Status | Hardware Acceleration Mechanism |
| :--- | :--- | :---: | :--- |
| **1. LLM (Text)** | Llama.cpp (Qwen2.5, Llama 3.2) | **[v2.0.0 Stable]** | Vulkan 25/25 layer full VRAM offload (Adreno 35.8 t/s, Mali 4.44 t/s) |
| **2. STT (Speech)** | Whisper.cpp (Large-v3-Turbo) | **[v2.0.0 Stable]** | Vulkan on-device STT acceleration (Adreno 4.4s, Mali 2.26x speedup) |
| **3. Vision (VLM)** | CLIP, MobileVLM, LLaVA | **[v2.1.0 WIP]** | GGML Vulkan image encoder tensor engine binding |
| **4. Diffusion (Image)** | Stable Diffusion v1.5 / FLUX.1 | **[v2.2.0 WIP]** | On-device Vulkan UNet & DiT tensor offload engine |
| **5. TTS (Audio)** | Piper / Sherpa-ONNX / Kokoro | **[v2.3.0 WIP]** | Real-time low-latency neural TTS streaming pipeline via Vulkan/NPU |
| **6. Train (Training)** | On-Device LoRA / QLoRA | **[v2.4.0 WIP]** | Smartphone local Vulkan gradient descent backpropagation engine |

### 5. Architectural Principles
- **Fail-Fast & Zero-Silent-Fallback**: Never disguise GPU failures as CPU success. If a hardware backend fails, immediate explicit exceptions and telemetry are raised.
- **Consolidated Single Package**: Distributed cleanly via `pip install ameva-runtime` and `npm install @ameva/runtime`.


## Documentation
- [Official Documentation & API Reference](https://uno-km.vercel.app/lib/ameva-runtime/)
- [GitHub Repository](https://github.com/uno-km/ameva-runtime)

## License
Apache-2.0 License. Copyright (c) 2026 Eunho Kim (@uno-km).
