Metadata-Version: 2.4
Name: pdfmarks
Version: 0.1.0
Summary: Read the data back out of a vector figure in a PDF
Author: Iris
License: MIT
Project-URL: Homepage, https://github.com/savecharlie/iris-the-maker
Project-URL: Source, https://github.com/savecharlie/iris-the-maker/tree/main/work/pdfplot
Keywords: pdf,figure,plot,data extraction,digitizer,content stream,scientific figures,reproducibility
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Text Processing
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pypdf>=4.0
Dynamic: license-file

# pdfmarks

Read the data back out of a vector figure in a PDF.

A scatter plot in a paper looks like a picture of the data, and if you want the
numbers the usual answer is to email the authors or trace the dots by hand. But
a vector PDF does not contain a picture. It contains a list of instructions for
placing marks, and the coordinate in each instruction is where the measurement
was. The numbers never left. They are sitting in the file at full precision.

`pdfmarks` walks a page's drawing instructions and hands you every mark on it.

```
pip install pdfmarks
```

```python
from pdfmarks import marks, affine

ms = marks('paper.pdf', 4)                       # 0-based page index

# a series is a colour and a marker shape. matplotlib draws a circle with 26
# path points, MATLAB with 15; a triangle is 5, a square or diamond is 4.
series = [m for m in ms
          if m['fill'] == (0.121569, 0.466667, 0.705882) and m['npts'] == 26]
series.sort(key=lambda m: m['seq'])              # drawing order = data order

# calibrate from two marks whose data values you know -- axis tick stubs are
# ideal, they are short two-point strokes on the frame.
fx, fy = affine((225.28, 292.21), (0.0, 0.0),
                (410.30, 427.67), (1.0, 1.0))
data = [(fx(m['x']), fy(m['y'])) for m in series]
```

Each mark is a dict: `x, y` (centre, PDF user space, points, y upwards),
`w, h` (bounding box), `fill`, `stroke`, `npts`, `op` (the paint operator, or
`Do` for a placed image), `seq` (position in drawing order), and `pts` (every
vertex, so a line series comes back whole).

There is a command line too:

```
pdfmarks paper.pdf 5 --colours          # what series are on page 5
pdfmarks paper.pdf 5 --long 40          # paths long enough to be a line series
pdfmarks paper.pdf 5 --path 27          # every vertex of that path
pdfmarks paper.pdf 5 --csv marks.csv --max-size 12
```

## How exact is it

Exact. It reads numbers, not pixels.

The test suite downloads two real papers and checks two things whose answers
exist independently of this code. In one (a MATLAB figure) two panels carry
copies of a third panel's markers; matched in drawing order the recovered
coordinates agree to **0.0012 pt**. In the other (matplotlib) three series are
plotted against a parameter running 0 to 1, and calibrated from nothing but the
tick stubs the recovered values land on **k/24** — the authors' `linspace`,
which the reader is never told about — to **3.3 × 10⁻⁵**.

## What it does not do

**It does not find your axes.** Every plotting library writes them differently,
and a wrong guess produces numbers that are confidently wrong to four decimal
places, which is worse than no numbers. You supply two reference points and
`affine` does the rest. Tick stubs are easy to spot: two-point strokes, a few
points long, sitting on the frame.

It also cannot help with a raster figure. If the plot was saved as PNG and
pasted in, there are no instructions to read and you want a tracing tool
instead.

## Three things worth knowing

**Series separate by shape as well as colour.** Two panels that reuse one
colour still come apart if their markers differ, because `npts` differs. Run
`--colours` and check the counts against what the figure should contain; if a
series is short, something is merging.

**Drawing order is usually data order, and often time order.** Trajectories
come out in the order they were measured. It is also how you tell a legend
swatch from data: the swatch is the same marker in the same colour, drawn in a
separate run.

**A filled marker is often drawn twice**, once filled and once as a stroked
outline at the same centre. De-duplicate by adjacency in the stream, never by
proximity in the plane: in a dense trajectory, neighbouring real data points sit
closer together than a marker is wide, and a proximity rule quietly eats them.

## Why it exists

It came out of wanting the numbers behind one figure — twelve lab earthquakes
published as a scatter plot with no data file. Reading them back out took an
afternoon and turned three of the paper's qualitative sentences into
measurements. It also caught something invisible: the copies of one panel's data
laid into the other two had been translated rather than replotted, so they sat
1.4 % too far right. Harmless, and the sort of thing only the instructions know.

MIT licensed. Written by Iris.
