Metadata-Version: 2.2
Name: clipparse
Version: 0.2.0
Summary: Fast CLIP STUDIO PAINT (.clip) reader/writer — C++17 core with pybind11 bindings
Keywords: clip,clip studio paint,csp,parser,image,graphics
Author-Email: wamsoft <wtnbgo@gmail.com>
License: MIT License
         
         Copyright (c) 2026 wamsoft
         
         Permission is hereby granted, free of charge, to any person obtaining a copy
         of this software and associated documentation files (the "Software"), to deal
         in the Software without restriction, including without limitation the rights
         to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
         copies of the Software, and to permit persons to whom the Software is
         furnished to do so, subject to the following conditions:
         
         The above copyright notice and this permission notice shall be included in all
         copies or substantial portions of the Software.
         
         THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
         IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
         FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
         AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
         LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
         OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
         SOFTWARE.
         
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: C++
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Multimedia :: Graphics
Classifier: Topic :: Multimedia :: Graphics :: Graphics Conversion
Project-URL: Homepage, https://github.com/wamsoft/clipparse
Project-URL: Repository, https://github.com/wamsoft/clipparse
Project-URL: Issues, https://github.com/wamsoft/clipparse/issues
Requires-Python: >=3.9
Provides-Extra: image
Requires-Dist: numpy; extra == "image"
Requires-Dist: pillow; extra == "image"
Provides-Extra: psd
Requires-Dist: psdparse>=0.10; extra == "psd"
Requires-Dist: numpy; extra == "psd"
Provides-Extra: all
Requires-Dist: psdparse>=0.10; extra == "all"
Requires-Dist: numpy; extra == "all"
Requires-Dist: pillow; extra == "all"
Description-Content-Type: text/markdown

# clipparse

[日本語版 README](README.ja.md) — Japanese version of this page.

C++17 library for CLIP STUDIO PAINT `.clip` files — **reads, composites, edits and
writes them** — with pybind11-based Python bindings on PyPI.

- **Lazy I/O.** Parsing touches only the embedded SQLite metadata. Pixels are
  decompressed per **256x256 block**, on demand, straight out of the mmapped file.
- **Partial reads.** `layer_region()` expands only the tiles that overlap the
  rectangle you ask for — something the row-RLE format of PSD cannot do.
- **Compositing.** All 27 CLIP blend modes, folders (including pass-through),
  masks, clipping and 5 kinds of adjustment layer, matched pixel-for-pixel against
  the preview CLIP STUDIO itself stores in the file.
- **Writing.** An unmodified round-trip is **byte-identical** (sha256, even for a
  60 MB file). Editing layer attributes, replacing pixels, adding/removing layers
  and rebuilding a canvas from scratch are all verified in **CLIP STUDIO PAINT PRO
  5.0.4** on real hardware.
- **CLIP to PSD and back.** Round-trip conversion via
  [psdparse](https://github.com/wamsoft/psdparse), in both C++ and Python.
- **No runtime dependencies.** zlib and sqlite3 are compiled in; the Python wheel
  is a single extension module.

The format was reverse-engineered from real files; what was verified by measurement
and what is still inferred are kept apart in [docs/CLIP_FORMAT.md](docs/CLIP_FORMAT.md).

## Install (Python)

```bash
pip install clipparse
```

Wheels are published for Python 3.9-3.14 (free-threaded builds included) on
Linux / Windows / macOS (x86_64 + arm64). From source — a C++17 compiler and
CMake 3.16+ is all you need, **no package manager**:

```bash
pip install .
```

## Quick start

```python
import clipparse

f = clipparse.ClipFile()
f.load("artwork.clip")

print(f.width, f.height, f.resolution)         # canvas size in pixels, DPI
for layer in f.layers:                          # flat list, bottom-to-top
    print(layer.index, layer.name, layer.opacity, layer.is_group)

bgra = f.merged_image()                         # every layer composited, BGRA bytes
one  = f.layer_image(2)                         # one layer, BGRA bytes
part = f.layer_region(2, 100, 120, 64, 48)      # only the overlapping tiles

png, w, h = f.preview_png()                     # the preview CLIP STUDIO stored
```

Pixels always come back as **BGRA bytes with straight (un-premultiplied) alpha**,
the same convention psdparse uses:

```python
from PIL import Image
img = Image.frombytes("RGBA", (f.width, f.height), f.merged_image())
b, g, r, a = img.split()
Image.merge("RGBA", (r, g, b, a)).save("merged.png")
```

Editing. The writer addresses layers by **`Layer.MainId`** (`layer.main_id`), not
by the list index used for reading:

```python
w = clipparse.ClipWriter()
w.load("artwork.clip")

w.set_layer_attr(main_id, name="renamed", opacity=128)   # opacity is 0..256 here
w.set_pixels(main_id, bgra, f.width, f.height)           # replace a layer's pixels
new_id = w.add_layer(main_id, "new layer", bgra, f.width, f.height)
w.delete_layer(other_id)

w.save("out.clip")
assert clipparse.validate("out.clip") == []              # run before opening in CSP
```

Full reference: **[docs/PYTHON_API.md](docs/PYTHON_API.md)**
([日本語](docs/PYTHON_API.ja.md)).

## Command-line tools

The utilities ship with the wheel, so `pip install clipparse` also gives you
these commands (no extra dependencies — compositing and pixel decoding run in
the bundled C++ extension):

```
clip-probe    file.clip [--blocks]        # structure dump: chunks, tables, layer tree, blocks
clip-validate file.clip                   # referential-integrity check — run before opening in CSP
clip-doctor   file.clip [--deep]          # per-layer diagnosis; --fix/--remove excises broken layers
clip-export   file.clip [-o out.png]      # merged PNG; --layers DIR exports every layer + manifest
clip-write    roundtrip in.clip out.clip  # writing: round-trip, attributes, pixels, add layer
```

`clip-doctor` tells you **which layer** is bad, where `clip-validate` only
gives a pass/fail for the file:

```
clip-doctor file.clip [--deep]                   # diagnose: layer tree + findings
clip-doctor file.clip --fix --out fixed.clip     # repair, and excise broken layers
clip-doctor file.clip --remove 7 --out out.clip  # remove specific layers (folders take their subtree)
```

- Checks every layer's mipmap chains (render and mask), the attribute and
  block-stream structure, and whether the referenced pixel chunks actually
  exist in the file; `--deep` additionally inflates every zlib block.
- Findings come in three grades. **Removal candidates**: the layer's pixel
  data itself is broken (severed chain, corrupt blocks, missing chunks) —
  nothing left to repair, so `--fix` removes the layer. **Repairable**:
  references or counters are off but the data is intact (`MipmapCount`
  mismatches, dead links, storage types, a broken mask or thumbnail only) —
  `--fix` repairs these in place, cutting away just the broken mask or
  thumbnail. **Info**: harmless states that CSP's own files also contain.
- Layer numbers are `MainId` (shown in the tree and by `clip-probe`), not the
  indices `clip-export` uses.
- After writing it recomposes the embedded preview, re-runs the
  `clip-validate` checks, and every kind of surgery it performs has been
  verified to open in a real CLIP STUDIO PAINT installation (PRO 5.0.4).

Some features activate when optional libraries are present ("extras" —
the base install stays dependency-free):

```
pip install clipparse[psd]    # psdparse + numpy  → clip-to-psd / psd-to-clip converters
pip install clipparse[image]  # numpy + Pillow    → clip-write setpixels / addlayer
pip install clipparse[all]    # both — also enables CanvasPreview recomposition on edits
```

Without the extra installed, the command explains what to install and exits.

```
clip-to-psd in.clip out.psd  [--verify] [--flat]
psd-to-clip in.psd  out.clip [--verify] [--paper] [--template empty.clip]
```

- `--verify` reads the output back and checks the pixels against the input
  (layer-exact for CLIP→PSD; merged-composite match for PSD→CLIP).
- CLIP→PSD keeps the folder tree (pass-through maps to PSD `pass`) and maps
  all 27 blend modes; `--flat` skips folders. **Masks and clipping are baked
  into the alpha** (they look right but are no longer editable), and
  **adjustment / vector layers are not exported**. Translucent parts of
  glow-dodge layers differ because PSD has no equivalent alpha behaviour.
- PSD→CLIP rebuilds the canvas from a bundled blank template (a fresh CSP
  document; pass `--template` to use your own — its size does not matter).
  `--paper` keeps the template's white paper layer. The result opens in
  CLIP STUDIO PAINT — verified on a real installation (PRO 5.0.4).

The scripts under `tools/` are the same code; `tools/` is the reference
implementation and stays in the repository.

```
# structure dump — chunk layout, tables, layer tree, block list (stdlib only)
python tools/clip_probe.py file.clip [--blocks]

# lazy-reference prototype: composite and compare against a reference PNG
python tools/clip_lazy_demo.py file.clip -o out.png --compare reference.png

# writing (an unmodified round-trip must be byte-identical)
python tools/clip_write.py roundtrip in.clip out.clip
python tools/clip_write.py set       in.clip out.clip --layer 5 --opacity 64 --composite 2
python tools/clip_write.py setpixels in.clip out.clip --layer 3 --png patch.png
python tools/clip_write.py addlayer  in.clip out.clip --copy-from 3 --name new --png patch.png

# referential-integrity check — ALWAYS run this before opening a written file in CSP
python tools/clip_validate.py out.clip

# per-layer diagnosis and excision of broken layers / masks / thumbnails
python tools/clip_doctor.py file.clip --deep
python tools/clip_doctor.py file.clip --fix --out fixed.clip

# CLIP <-> PSD (needs the psdparse Python bindings)
python tools/clip_to_psd.py input.clip output.psd  --verify
python tools/psd_to_clip.py input.psd  output.clip --verify
```

`clip_probe.py` needs nothing but the standard library; `clip_lazy_demo.py` needs
numpy, plus Pillow when comparing.

## Build (C++ library / CLI)

```powershell
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release

build\clipparse\Release\clip_cli.exe file.clip --check
build\clipparse\Release\clip_cli.exe file.clip --validate
build\clipparse\Release\clip_cli.exe in.clip --set 5 --opacity 64 --out out.clip
build\clipparse\Release\clip_cli.exe in.clip --set-pixels 3 rgba.raw out.clip
build\clipparse\Release\clip_cli.exe in.clip --add-layer  3 rgba.raw out.clip --name new
```

The only dependencies are **zlib and sqlite3**, both fetched from source by CMake
(`FetchContent`), so vcpkg and friends are unnecessary. SQLite is attached with
`sqlite3_deserialize(..., SQLITE_DESERIALIZE_READONLY)` directly on the mmapped
bytes — no temporary file is ever created.

```cpp
clip::ClipFile f;
f.load("artwork.clip");
clip::Image img;
f.mergedImage(img);                       // RGBA8, straight alpha

clip::ClipWriter w;
w.load("artwork.clip");
w.addLayer(3, "new layer", rgba, 300, 400);
w.save("out.clip");
```

C++ and Python produce **byte-identical chunk payloads** when writing, down to the
zlib output; that equivalence is what the test suite checks.

## CLIP to PSD conversion

`examples/clipconv/` is a standalone command that links both clipparse and
psdparse, using nothing but their public APIs.

```powershell
cmake -S examples/clipconv -B build-conv -DCMAKE_BUILD_TYPE=Release
cmake --build build-conv --config Release

build-conv\Release\clipconv.exe in.clip out.psd  --verify
build-conv\Release\clipconv.exe in.psd  out.clip --verify
```

Layer pixels, the folder tree and blend modes survive the round-trip; masks and
clipping are baked into alpha, and adjustment/vector layers are not exported.
Details in [examples/clipconv/README.md](examples/clipconv/README.md).

## What works, and what does not

| | |
|---|---|
| Reading | RGBA / gray / monochrome / mask planes, folders, masks, clipping, text and rasterized vector layers |
| Compositing | 27 blend modes, pass-through folders, 5 adjustment-layer kinds. Of 28 samples, 13 are pixel-exact against CSP's own preview and 22 are within rounding error |
| Writing | attributes, pixel replacement, add/delete layer, canvas rebuild — all confirmed in CSP 5.0.4 |
| Not supported | vector layers (a brush engine would be needed), some adjustment kinds (levels, colour balance, posterize, gradient map) |

Writing has traps a tolerant reader cannot see — per-table storage types, a
checksum CSP actually verifies, a mipmap count that crashes it when wrong. They
were flushed out over five rounds of testing on real CLIP STUDIO and are checked
mechanically by `clipparse.validate()` / `clip_cli --validate`.
**Run the validator before opening anything you wrote.**

## Tests

```powershell
python -m pytest tests -q
```

The tests cross-check the C++ extension against the pure-Python reference
implementation (pixels must match byte for byte) and the write path against
CLIP STUDIO's own rules. Real `.clip` samples are not committed — see
[docs/STATUS.md](docs/STATUS.md) for what goes into `samples/`.

## How the format works, in five lines

```
[CSFCHUNK header][CHNKHead][CHNKExta ...][CHNKSQLi][CHNKFoot]
                            ^ pixel data  ^ all metadata (a SQLite3 database)
```

`CHNKHead.binary_section_size` points straight at the SQLite chunk, so the first
64 bytes are enough to reach the metadata. From there, `ExternalChunk.Offset` plus
the prefix sum of `Offscreen.Attribute.BlockSize[]` gives the absolute position of
any pixel block — the binary area is never scanned.

## Documentation

| | |
|---|---|
| [docs/PYTHON_API.md](docs/PYTHON_API.md) ([ja](docs/PYTHON_API.ja.md)) | Python API reference |
| [docs/CLIP_FORMAT.md](docs/CLIP_FORMAT.md) | `.clip` format specification, measured facts kept apart from inferred ones |
| [docs/DESIGN.md](docs/DESIGN.md) | Design of the lazy-reference scheme, the API shared with psdparse, roadmap |
| [docs/STATUS.md](docs/STATUS.md) | Development status, how to resume, what is next |
| [docs/CLIP_TOOLS_REPORT.md](docs/CLIP_TOOLS_REPORT.md) | Feedback for clip-tools: three reproducible bugs and two spec corrections |

## Credits

- [animeops/clip-tools](https://github.com/animeops/clip-tools) — the Python
  implementation this analysis started from.
- [psdparse](https://github.com/wamsoft/psdparse) — the library whose design
  clipparse follows.

## License

MIT — see [LICENSE](LICENSE).
