Metadata-Version: 2.4
Name: range-coding-cython
Version: 0.2.0
Summary: A range coding lib implemented in Cython
Author-email: Namoe <yangziming@buaa.edu.cn>
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# range_coding_cython

A range coding lib implemented in Cython.

## Install

```bash
pip install range_coding_cython
```

## Usage

```python
import numpy as np
from range_coding_cython import decode_nxk, encode_nxk

syms = np.array([0, 1, 2, 3, 3, 3], dtype=np.uint32)

cdf = [0.0, 0.1, 0.3, 0.6, 1.0]
PRECISION = 16
cdf_table = np.array([cdf]) * (1 << PRECISION)  # (1, alphabet_size + 1)
cdf_table = cdf_table.astype(np.uint32)

cdf_indices = np.array([0, 0, 0, 0, 0, 0], dtype=np.uint32)

# Encode to a zero-copy view backed by a bytearray.
data = encode_nxk(syms, cdf_table, cdf_indices)
assert isinstance(data, memoryview)

# Decode directly from the memoryview.
buff = np.zeros_like(syms)
dec_syms = decode_nxk(buff, cdf_table, cdf_indices, data)

assert np.array_equal(dec_syms, syms)
```

`decode_nxk` accepts any one-dimensional contiguous unsigned-byte buffer,
including `bytes`, `bytearray`, `memoryview`, and NumPy `uint8` arrays. Convert
to an immutable `bytes` object only when needed:

```python
data_bytes = data.tobytes()
```

## PyTorch interoperability

The encoded memoryview can be exposed to PyTorch as a zero-copy CPU tensor:

```python
import torch

data_tensor = torch.frombuffer(data, dtype=torch.uint8)
```

The tensor and encoded memoryview share the same writable CPU memory. Keep the
bitstream unchanged while decoding or storing it. Moving the tensor to a CUDA
device requires a device copy.

## References

- [Range Coding](https://en.wikipedia.org/wiki/Range_coding)
- [Arithmetic Coder](https://marknelson.us/posts/2014/10/19/data-compression-with-arithmetic-coding.html)
- [NeuralCompression](https://github.com/facebookresearch/NeuralCompression)
- [Yet-Another-Entropy-Coding-Library](https://github.com/tongdaxu/YAECL-Yet-Another-Entropy-Coding-Library)
