Metadata-Version: 2.5
Name: damicore
Version: 0.1.0
Summary: Clusters the rows or columns of a CSV through exact compression distance, deterministic Neighbor Joining, and FastGreedy community detection.
Project-URL: Homepage, https://github.com/Delbem-Research-and-Innovation/damicore
Project-URL: Repository, https://github.com/Delbem-Research-and-Innovation/damicore
Project-URL: Issues, https://github.com/Delbem-Research-and-Innovation/damicore/issues
Project-URL: Documentation, https://github.com/Delbem-Research-and-Innovation/damicore/blob/main/docs/quickstart.md
Project-URL: Changelog, https://github.com/Delbem-Research-and-Innovation/damicore/blob/main/CHANGELOG.md
Author-email: Ennio Politi Lopes <enniolopes@users.noreply.github.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: clustering,community-detection,compression-distance,csv,damicore,ncd,neighbor-joining
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Typing :: Typed
Requires-Python: <3.15,>=3.11
Requires-Dist: damicore-clusterizer<0.2.0,>=0.1.0
Requires-Dist: damicore-distance[pandas]<0.2.0,>=0.1.0
Requires-Dist: damicore-normalizer<0.2.0,>=0.1.0
Requires-Dist: damicore-tree-builder<0.2.0,>=0.1.0
Requires-Dist: pandas<4,>=2.2
Requires-Dist: pydantic<3,>=2.10
Requires-Dist: tqdm<5,>=4.66
Description-Content-Type: text/markdown

# damicore

DAMICORE clusters the rows or columns of a local CSV without asking you to pick a
feature representation or a number of clusters. It serializes each object
canonically, measures every pair with exact Normalized Compression Distance (NCD),
builds a deterministic Neighbor Joining tree, and cuts communities out of that tree
with FastGreedy.

This distribution is the complete pipeline and the only one with a command line.
The four stage distributions — `damicore-normalizer`, `damicore-distance`,
`damicore-tree-builder`, `damicore-clusterizer` — are Python APIs that can be
installed and used on their own.

```bash
pip install damicore
```

## Python

```python
from damicore import ExecutionConfig, ResourceLimits, estimate, load_result, run

# `workers="auto"` opens a process pool whose workers re-import the calling module, so in a
# `.py` script this call must sit under the guard below. A notebook or REPL satisfies it too.
if __name__ == "__main__":
    preview = estimate("dataset.csv", split="columns")
    result = run(
        "dataset.csv",
        split="columns",
        execution=ExecutionConfig(workers="auto", limits=ResourceLimits()),
    )
    result.membership
    result.distance_matrix.head()
    result.close()

    restored = load_result(result.artifacts.run_dir)
```

`estimate` reports the exact cost of a run — objects, pairs, matrix bytes, working
memory, free disk — without creating anything. Call it before raising a limit.

## Command line

```bash
damicore estimate dataset.csv --json
damicore run dataset.csv --split columns --output-dir ./results
damicore --version
```

Progress and the artifact paths go to stderr. Only `estimate --json` writes to
stdout, so a shell pipeline reads one JSON document and nothing else.

### Exit codes

A failure is also one JSON line on stderr carrying a stable `code`, so a script can
branch on the status and log the reason.

| Status | Meaning |
|---:|---|
| 0 | Completed |
| 2 | Configuration or input rejected, including a malformed CSV |
| 3 | A resource limit would be exceeded |
| 4 | An artifact failed validation |
| 5 | The output directory conflicts, or a checkpoint does not match |
| 130 | Interrupted; the run is resumable |
| 141 | Terminated by a broken pipe (the SIGPIPE convention), e.g. when piping to `head` |

## Results

A run writes a versioned, hash-verified directory: the distance matrix as a
`float64` `.npy` memory map, the tree as JSON and Newick, cluster membership as CSV
and JSON, plus a manifest and a report. `load_result` reopens it, and an
interrupted run resumes from its checkpoints to the same bytes a fresh run would
have produced.

## Scale

The exact algorithm accepts at most 1,000 objects, 500,000 pairs and 512 MiB per
matrix by default. A multi-gigabyte CSV with tens of columns is feasible; the same
file split into millions of rows is rejected during preflight rather than after
hours of work. Streaming and memory maps bound RAM, but NCD stays quadratic and
Neighbor Joining cubic in the object count. Raise an individual `ResourceLimits`
field only after reading `estimate`.

## Links

- Source, issues and full documentation:
  <https://github.com/Delbem-Research-and-Innovation/damicore>
- Licensed under Apache-2.0.
