Metadata-Version: 2.4
Name: satellome
Version: 1.16.0
Summary: A comprehensive tool for satellite DNA analysis in T2T genome assemblies
Home-page: https://github.com/aglabx/Satellome
Author: Aleksey Komissarov
Author-email: Aleksey Komissarov <ad3002@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/aglabx/satellome
Project-URL: Documentation, https://github.com/aglabx/satellome/wiki
Project-URL: Repository, https://github.com/aglabx/satellome
Project-URL: Issues, https://github.com/aglabx/satellome/issues
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.6
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: PyExp
Requires-Dist: numpy
Requires-Dist: pyyaml
Requires-Dist: simplejson
Requires-Dist: intervaltree
Requires-Dist: tqdm
Requires-Dist: plotly
Requires-Dist: editdistance
Requires-Dist: networkx
Requires-Dist: matplotlib
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# Satellome

[![Tests](https://github.com/aglabx/satellome/workflows/Tests/badge.svg)](https://github.com/aglabx/satellome/actions?query=workflow%3ATests)
[![codecov](https://codecov.io/gh/aglabx/satellome/branch/main/graph/badge.svg)](https://codecov.io/gh/aglabx/satellome)
[![Python Version](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11-blue)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![PyPI version](https://badge.fury.io/py/satellome.svg)](https://badge.fury.io/py/satellome)

A comprehensive bioinformatics tool for analyzing satellite DNA (tandem repeats) in telomere-to-telomere (T2T) genome assemblies.

## Overview

Satellome uses **FasTAN** (Fast Tandem Repeat Finder) as its default tandem repeat detection engine, providing fast and accurate identification of repetitive DNA sequences. The tool classifies and visualizes tandem repeats with a focus on centromeric and telomeric regions.

The tool is designed to work with various genome assembly projects including:
- T2T (Telomere-to-Telomere) Consortium assemblies
- DNA Zoo chromosome-length assemblies
- VGP (Vertebrate Genome Project) assemblies
- NCBI RefSeq and GenBank assemblies

## Features

- **Fast Tandem Repeat Detection**: Uses FasTAN by default for rapid, accurate detection
- **Smart Classification**: Categorizes repeats into microsatellites, complex repeats, and other types
- **Rich Visualizations**: Generates karyotype plots and chromosome-level visualizations
- **Annotation Integration**: Supports GFF3 and RepeatMasker annotations
- **Parallel Processing**: Efficient handling of large genomes
- **Smart Pipeline**: Automatically skips completed steps (override with `--force`)
- **Compressed File Support**: Direct processing of .gz compressed FASTA files
- **Optional TRF Support**: Traditional TRF analysis available with `--run-trf` flag

## Quick Start

```bash
# Install from PyPI
pip install satellome

# Install required binaries (FasTAN, tanbed, Rust helpers)
satellome --install-all

# Run on a genome
satellome -i genome.fasta -o output_dir -p project_name -t 8
```

## Installation

### From PyPI (Recommended)

```bash
pip install satellome

# Install external tools
satellome --install-all
```

### Checking the installation

```bash
satellome --doctor
```

`--doctor` reports where this install lives, whether your shell can actually
see the launcher, and where each external tool resolves from. It exits 0 when
everything is healthy and 1 when it finds a problem, so it can be used as a
setup gate in a driver script.

Missing tools are reported by **consequence**, not merely as "not installed",
because that is the part that decides whether a finished run is complete:

| Tool | Missing means |
|------|---------------|
| `fastan`, `tanbed`, `arraysplitter` | the analysis cannot run |
| `sat-family` | family clustering is **skipped** — no families output |
| `telomere-check` | the telomere check is **skipped** — no telomere output |
| `find-gaps`, `bed-extract`, `genome-size` | a Python fallback gives the **same result**, slower |
| `trf` | only needed with `--run-trf` |

A tool whose absence removes results makes `--doctor` exit non-zero, and is
announced at the start of a run — before hours of compute, rather than as a
line that scrolls past in the middle of it. Tools that only cost speed are
listed separately and are not treated as problems.

Install the Rust helpers:

```bash
satellome --install-rust-tools    # or: satellome --install-all
```

This downloads binaries built by CI and attached to the GitHub Release for your
version, verifying each against the `SHA256SUMS-<platform>.txt` published
alongside it — a binary whose checksum does not match is refused rather than
installed, because a wrong binary is worse than a missing one: the pipeline
would run it. Prebuilt binaries are published for `linux-x86_64`
and `macos-arm64`. Intel macOS is not published — GitHub's Intel runners no
longer schedule in reasonable time — so it uses the source fallback below.

On any other platform, or if the assets are unreachable, it falls back to
building the `rust/` crates with cargo, which then requires a Rust toolchain.
Set `SATELLOME_NO_PREBUILT=1` to always build from source.

It is the answer to the most common post-install surprise:

```
WARNING: The script satellome is installed in '/home/user/.local/bin'
which is not on PATH.
```

That means `pip` used a user install and your shell cannot see the launcher —
`satellome` will report `command not found` even though the package is fine.
Satellome repairs this itself:

```bash
python -m satellome --fix-path
```

`python -m satellome` always works, because it does not depend on PATH at all —
which is what makes it the way in when the `satellome` command is exactly the
thing you cannot reach. `--fix-path` appends one marked block to the startup
file your shell already reads:

```bash
# >>> satellome path >>>
export PATH="$HOME/.local/bin:$PATH"
# <<< satellome path <<<
```

It is idempotent (the marker means it is written once and never again), it will
not duplicate an entry you added by hand, and it appends to a file that already
exists rather than creating a new one — creating `~/.bash_profile` where none
existed would stop a login shell from reading `~/.profile`.

A process cannot change its parent shell's environment, so the block applies to
**new** shells. For the shell you are in: `source ~/.bashrc`.

The same repair runs automatically at the start of any run that detects the
problem, so it does not have to be found first. To keep the warning but never
let satellome touch a shell file, set `SATELLOME_NO_PATH_FIX=1`. Manual
equivalents, if you prefer:

```bash
export PATH="$HOME/.local/bin:$PATH"                       # this shell only
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc   # permanent
python -m satellome ...                                    # no PATH change needed
```

Two competing installations (a `satellome` on PATH that is not the one your
active interpreter owns) are reported but never "fixed" — deciding which one to
delete is yours to make.

`python -m satellome` is the most robust form in batch scripts: it always
follows the active environment, whereas a console script keeps the interpreter
its shebang was written with. Satellome resolves pip-installed companion tools
(such as `arraysplitter`) through the same interpreter-aware lookup, so a
directory missing from `PATH` degrades nothing silently — it is reported and
the tool is still used.

Set `SATELLOME_NO_ENV_CHECK=1` to silence the startup check entirely on machines
where this layout is deliberate; `--doctor` still reports it.

### From Source

```bash
git clone https://github.com/aglabx/satellome.git
cd satellome
pip install -e .
satellome --install-all
```

### External Tools

Satellome requires **FasTAN** and **tanbed** for default operation. Install them automatically:

```bash
# Install all tools (FasTAN, tanbed, modified TRF)
satellome --install-all

# Or install individually
satellome --install-fastan
satellome --install-tanbed
satellome --install-trf-large  # For genomes with chromosomes >2GB
```

**Build requirements:** git, make, C compiler (gcc/clang)

```bash
# Ubuntu/Debian
sudo apt-get install build-essential git

# macOS
xcode-select --install
```

## Usage

### Basic Command

```bash
satellome -i genome.fasta -o output_dir -p project_name -t 8
```

### Input formats

The genome may be given as plain FASTA, as an archive, or as a UCSC `.2bit`:

| Input | Handling |
|---|---|
| `.fasta` / `.fa` / `.fna` | used directly |
| `.gz`, `.bz2`, `.xz` | decompressed once into `<output>/input/` |
| `.zst` | same, if the `zstandard` package is installed |
| `.zip` | the single FASTA inside is extracted; an archive with several is refused rather than guessed at |
| `.2bit` | converted to FASTA, preserving N runs and soft-masking |

The format is detected **by content, not by extension** — a gzip named
`genome.fasta` is common enough to be worth handling. The conversion is
announced, written atomically, and cached: a second run on the same compressed
genome reuses it, and a source file newer than the cache invalidates it.

Downstream tools (FasTAN, TRF, the Rust helpers) are handed a path and read
plain FASTA, which is why conversion happens once up front rather than in each
reader. A corrupt archive stops the run — decompressing it as far as it goes
would otherwise produce a genome with no tandem repeats and no visible reason.

### Re-running steps on a finished analysis

The tandem-repeat search dominates a run, and its result does not change when
you later decide you also want a browser track, a redrawn report, or
annotations from a GFF that arrived afterwards. Those steps can be re-run
against an existing output directory:

```bash
satellome --list-steps                              # what can be re-run
satellome --rerun ucsc_track -o /path/to/output     # add a track, recompute nothing
satellome --rerun drawing,ucsc_track -o /path/to/output
```

| Step | Does |
|---|---|
| `classification` | re-classify repeats and rewrite the classified `.sat` |
| `annotations` | intersect with the GFF / RepeatMasker file the run used |
| `sat_family` | re-cluster satellite families |
| `drawing` | redraw plots and the HTML report |
| `ucsc_track` | write the UCSC track files |

The original options — cutoffs, `--gff`, taxon, genome size — are recovered
from `run_manifest.json`, which records the run's own command line. You do not
retype the command, and a re-run therefore cannot quietly apply *different*
parameters than the run it is amending, which would leave an output directory
whose files disagree with each other.

A re-run takes the same output-directory lock as a full run, updates the
statuses of the steps it ran while keeping the others, refreshes the file
inventory, and appends to a `reruns` history in the manifest — so
`satellome --verify-run` keeps working on an amended directory.

It refuses, with the reason, when the directory has no usable manifest, when a
step's inputs are missing (naming which file), or when a step name is unknown.
The search steps are deliberately not re-runnable this way: recomputing them is
what `--force` on a normal run is for, and putting hours of work behind a
one-word option would be a trap.

### UCSC Genome Browser track

```bash
satellome -i genome.fasta -o out -t 8 --ucsc-track
satellome -i genome.fasta -o out -t 8 --ucsc-track --ucsc-min-length 1000
```

Writes, on request:

* `<project>.ucsc.bed` — BED9 custom track, one feature per repeat, coloured by
  period class (microsatellite 1–6 bp, short SSR 7–9, minisatellite 10–100,
  satellite 101–1000, macrosatellite >1000). Score is percent identity. Ready
  to paste into *add custom track*.
* `<project>.chrom.sizes` — sequence lengths for the same assembly.
* `<project>.ucsc.bb` — bigBed, when `bedToBigBed` is on PATH. A whole-genome
  BED exceeds the browser's custom-track size limit; bigBed does not, and it is
  what a track hub needs.

Sequence names are written exactly as the assembly spells them (the first token
of the FASTA header), not the normalised form used internally for grouping — a
track whose names disagree with `chrom.sizes` displays nothing and makes
`bedToBigBed` fail. Rows are sorted by position, which the browser tolerates
either way but `bedToBigBed` requires.

### Common Options

```bash
# With GFF3 annotations
satellome -i genome.fasta -o output_dir -p project_name -t 8 --gff annotations.gff3

# With RepeatMasker annotations
satellome -i genome.fasta -o output_dir -p project_name -t 8 --rm repeatmasker.out

# Force rerun all steps
satellome -i genome.fasta -o output_dir -p project_name -t 8 --force

# Also run traditional TRF analysis
satellome -i genome.fasta -o output_dir -p project_name -t 8 --run-trf
```

### Parameters

| Parameter | Description | Default |
|-----------|-------------|---------|
| `-i, --input` | Input FASTA file (.fa, .fasta, .gz) | Required |
| `-o, --output` | Output directory | Required |
| `-p, --project` | Project name | Required |
| `-t, --threads` | Number of threads | 1 |
| `--gff` | GFF3 annotation file | None |
| `--rm` | RepeatMasker output file | None |
| `--run-trf` | Also run TRF analysis | False |
| `--force` | Force rerun all steps | False |
| `--taxid` | NCBI taxonomy ID | None |

## Output Structure

```
output_dir/
├── genome.sat                    # Main SAT output (all arrays)
├── genome.1kb.sat                # Arrays >1kb
├── genome.3kb.sat                # Arrays >3kb
├── genome.10kb.sat               # Arrays >10kb
├── genome.micro.sat              # Microsatellites (1-9 bp monomers)
├── genome.complex.sat            # Complex repeats (>9 bp monomers)
├── genome.pmicro.sat             # Potential microsatellites
├── genome.tssr.sat               # Tandem simple sequence repeats
├── genome.gaps.bed               # Gaps annotation
├── results.yaml                  # Analysis statistics
├── run_manifest.json             # What the run produced (written last) + step statuses
├── fastan/                       # FasTAN intermediate files
│   ├── genome.1aln               # FasTAN alignment output
│   └── genome.bed                # FasTAN BED format
├── fasta/                        # FASTA sequences
│   └── genome.arrays.fasta       # All array sequences
├── gff3/                         # GFF3 annotations
│   ├── genome.1kb.gff
│   ├── genome.complex.gff
│   └── ...
├── images/                       # Visualizations
│   └── *.png
└── reports/                      # HTML reports
    └── satellome_report.html
```

### Taxon Names in File Names

Karyotype charts are named after the taxon (`--taxon`, or the name resolved from
`--taxid`). Organism names are free text and NCBI strain designations often
contain slashes — `Leishmania braziliensis MHOM/BR/75/M2904`, `Chlorella
vulgaris CCAP 1055/1`. Such a name is reduced to a single safe file-name
component (`..._MHOM_BR_75_M2904.karyo.*`) and the substitution is logged; the
plot titles keep the original name. Without this the slash was read as a path
separator and the drawing step died with `FileNotFoundError` after the whole
pipeline had already written its data.

## Verifying a Run

Do not decide that an output directory is complete by checking that some files
exist. A file that was read or copied while satellome was still writing it exists
just as hard as a complete one, and a `gzip` of such a partial read is a valid
archive that `gzip -t` accepts — so truncated data can enter downstream analysis
unnoticed.

Every run writes `run_manifest.json` **last**, recording each file it produced
with its byte size plus the status of every step. Verify against it:

```bash
satellome --verify-run output_dir
```

* exit `0` — the directory matches its manifest and no step failed
* exit `1` — not a verifiably complete run: no/corrupt manifest, a failed step, a
  missing file, a leftover `*.partial`, or a file whose size no longer matches
  what the run wrote (the truncated-copy case)
* exit `2` — the argument is not a directory

Files already compressed by your own pipeline are still checked: for a missing
`X` with an `X.gz` next to it, the gzip ISIZE trailer (the uncompressed length
the compressor actually consumed) is compared to the recorded size, which catches
a `.gz` made from an incomplete read. Above 4 GiB that comparison is modulo
4 GiB and the report says so.

Two other guarantees back this up:

* **Atomic outputs** — files are written as `<path>.partial` and renamed into
  place, so a final name never refers to a half-written file. If you compress or
  copy an output directory concurrently, you either get the complete file or no
  file, never a truncated one.
* **Output-directory lock** — a second satellome run into the same `-o` is
  refused, naming the pid and host that holds it, instead of overwriting the
  first run's files mid-write. Override with `--ignore-lock` only if you are sure.

A run whose drawing step fails still writes a manifest — with `drawing: failed` —
and exits non-zero. The data files are complete and usable; the failed step is
recorded in the run's own artifact rather than only in an exit code.

## SAT File Format

The SAT format is a tab-delimited file with the following columns:

| Column | Description |
|--------|-------------|
| project | Project name |
| trf_id | Unique array ID |
| trf_head | Chromosome/scaffold name |
| trf_l_ind | Left coordinate (1-based) |
| trf_r_ind | Right coordinate |
| trf_period | Monomer period length |
| trf_n_copy | Number of copies |
| trf_pmatch | Percent match |
| trf_pvar | Percent variation |
| trf_entropy | Shannon entropy |
| trf_consensus | Consensus monomer sequence |
| trf_array | Full array sequence |
| trf_array_gc | Array GC content |
| trf_consensus_gc | Consensus GC content |
| trf_array_length | Array length in bp |
| trf_joined | Join status |
| trf_family | Repeat family |
| trf_ref_annotation | Reference annotation |

## Classification System

Satellome classifies tandem repeats into four categories:

| Category | Description | Criteria |
|----------|-------------|----------|
| **micro** | Microsatellites | Monomer 1-9 bp |
| **complex** | Complex repeats | Monomer >9 bp, entropy >1.82 |
| **pmicro** | Potential microsatellites | Intermediate characteristics |
| **tssr** | Tandem simple sequence repeats | Simple patterns |

## Example Results

Analysis of CHM13 v2.0 human genome (3.1 GB):

| Category | Arrays | % Genome |
|----------|--------|----------|
| Total | 614,616 | - |
| Complex | 20,373 | 5.27% |
| Microsatellites | 319,489 | 1.96% |
| TSSR | 296,475 | 0.47% |
| >1kb | 14,438 | 7.69% |
| >10kb | 1,223 | 6.67% |

## Utility Scripts

### Format Conversion
```bash
python scripts/trf_to_fasta.py -i repeats.sat -o repeats.fasta
python scripts/trf_to_gff3.py -i repeats.sat -o repeats.gff3
```

### Analysis Tools
```bash
python scripts/trf_get_large.py -i repeats.sat -m 1000 -o large_repeats.sat
python scripts/trf_get_micro_stat.py -i repeats.sat -o micro_stats.txt
python scripts/check_telomeres.py -i genome.fasta -t repeats.sat
```

## Testing

```bash
pytest tests/unit/ -v
```

## Contributing

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

## Citation

If you use Satellome in your research, please cite:

```
Komissarov A. et al. (2026). Satellome: A comprehensive tool for satellite DNA
analysis in T2T genome assemblies. [Publication details]
```

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Support

- **Issues**: [GitHub Issues](https://github.com/aglabx/satellome/issues)
- **Documentation**: [Wiki](https://github.com/aglabx/satellome/wiki)
- **Email**: ad3002@gmail.com

## Acknowledgments

- [FasTAN](https://github.com/thegenemyers/FASTAN) by Gene Myers
- [Tandem Repeat Finder](https://github.com/Benson-Genomics-Lab/TRF) by Gary Benson
- [T2T Consortium](https://www.genome.gov/about-nhgri/telomere-to-telomere)
- [DNA Zoo](https://www.dnazoo.org/)
- [Vertebrate Genome Project](https://vertebrategenomesproject.org/)
