Metadata-Version: 2.4
Name: metrik-optimize
Version: 0.4.0
Summary: Metrik Optimization Lab — quantize a model, and account honestly for what it cost.
Project-URL: Homepage, https://github.com/Asmodeus14/Metrik
Project-URL: Repository, https://github.com/Asmodeus14/Metrik
Author: The Metrik Authors
License-Expression: Apache-2.0
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: metrik-sdk==0.4.0
Provides-Extra: torch
Requires-Dist: torch>=2.2; extra == 'torch'
Requires-Dist: transformers>=4.40; extra == 'torch'
Provides-Extra: torchao
Requires-Dist: torch>=2.2; extra == 'torchao'
Requires-Dist: torchao>=0.9; extra == 'torchao'
Requires-Dist: transformers>=4.40; extra == 'torchao'
Description-Content-Type: text/markdown

# metrik-optimize

Quantize a model, and account honestly for what it cost.

Part of [Metrik](https://github.com/Asmodeus14/Metrik).

## The number this package exists to get right

Ask for a 4-bit format and you do not get a 4-bit model. Two reasons, both routinely omitted:

- **Scales are real bytes.** INT4 with FP16 scales at `group_size=128` is **4.125 bits**; at
  `group_size=32` it is **4.5**. Calling both "4-bit" compares two things you have not
  distinguished, which is why a format here is identified *including* its group size.
- **Not everything gets quantized.** Embeddings, the output head and normalisation scales stay
  in full precision, because quantizing them costs real accuracy for almost no size. On a small
  model with a large vocabulary that can pull the whole-model average well above 6 bits while
  every weight the policy touched really is 4.

`avg_bits_naive` is the 4. `avg_bits_effective` is what the file actually costs. The distance
between them is the point.

## What is checked rather than promised

- `avg_bits_effective` is **computed** from bytes over parameters. A declared value that
  disagrees is rejected.
- `method="serialized"` requires the size of a file that exists. A claim that disagrees with
  the bytes on disk fails at the store boundary, in both directions.
- `AppliedFidelity.faithful` is derived from the record, not asserted by the backend. Backends
  round unsupported group sizes and skip layers whose shapes do not divide; unrecorded, those
  turn a claim about a policy into a claim about the policy plus undocumented behaviour.

## Install

```console
pip install metrik-optimize              # the policy: reads a graph, writes a plan. No ML stack.
pip install "metrik-optimize[torchao]"   # adds torch and a backend that can apply one
```

The split is deliberate: a plan contains no weights, so it can be inspected, diffed and shared
on a machine that could never load the model it describes.

## License

Apache-2.0
