Metadata-Version: 2.4
Name: cattensors
Version: 0.1.0
Summary: Fast safetensors loading for distributed filesystems
Requires-Python: >=3.11
Requires-Dist: safetensors
Requires-Dist: torch
Provides-Extra: dev
Requires-Dist: flashpack; extra == 'dev'
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: pytest>=9.0.2; extra == 'dev'
Requires-Dist: safetensors>=0.7.0; extra == 'dev'
Requires-Dist: torch>=2.10.0; extra == 'dev'
Requires-Dist: ty>=0.0.19; extra == 'dev'
Description-Content-Type: text/markdown

# cattensors

Faster `safetensors` loading for JuiceFS.

> Note: This library was developed and benchmarked against JuiceFS, but in theory should work with any network backed distributed filesystem.

`cattensors` monkey-patches `safetensors` to pre-warm the OS page cache before tensor reads. On network backed filesystems, the first read of a large model file is slow. `cattensors` reads the file into page cache using parallel threads so subsequent `mmap` based access hits RAM instead of the network.

## Compatibility

- POSIX platforms (Linux/macOS)
- Not supported on Windows (`fcntl.flock` is used for cross-process coordination)

## Installation

```bash
pip install git+https://github.com/badayvedat/cattensors@16ebb9ab66a59747c810530c2d4ca1b53bb41539
```

After installation, `cattensors` is auto-activated at interpreter startup via a `.pth` file, so no code changes are required.

To disable this behavior, set `CATTENSORS_AUTO_PATCH` environment variable to `false`

> [!WARNING]
>
> Installing `cattensors` ships a `.pth` file into your `site-packages`. Python evaluates `.pth` files at **every interpreter startup**, which means `import cattensors.startup` runs in *every* Python process in that environment; before your code, before `python -c`, before `pytest`, etc. It then monkey-patches `safetensors` globally, and also `flashpack` if it is installed.
>
> This is intentional and is what makes the library zero-config, but it has two implications you should know about:
>
> 1. **Global side effects.** Any process in the env that imports `safetensors` gets the patched version, even if it never imports `cattensors`. To opt out per-process, set `CATTENSORS_AUTO_PATCH=false` before Python starts. To opt out entirely, uninstall the package or remove `cattensors.pth` from `site-packages`.
> 2. **Install-time trust.** Because `.pth` runs unconditionally, installing this package means trusting the source you installed it from to run code in every future Python invocation in that environment. Prefer pinning to a specific commit/tag, and avoid installing into shared/system Python environments.


## Configuration

Environment variables:

| Variable | Default | Description |
|---|---|---|
| `CATTENSORS_AUTO_PATCH` | `true` | Auto-patch `safetensors` (and `flashpack`, if installed) at interpreter startup |
| `CATTENSORS_WARM_THREADS` | `8` | Parallel reader threads per file |
| `CATTENSORS_PARALLEL_SHARDS` | `false` | Pre-warm sibling shards in background when loading sharded models |
| `HF_ENABLE_PARALLEL_LOADING` | `false` | If `true`, `cattensors` skips sibling-shard parallel warming to avoid overlapping I/O |
| `CATTENSORS_DEBUG` | `false` | Enable debug logs to stderr |

Behavior notes:

- `CATTENSORS_AUTO_PATCH` is evaluated once at interpreter startup (set it before Python starts).
- Other flags are read when used, so changes affect subsequent load calls.
