Metadata-Version: 2.4
Name: arrakis-admin
Version: 0.7.0
Summary: Admin CLI for the Arrakis low-latency timeseries data distribution platform
Project-URL: Homepage, https://git.ligo.org/ngdd/arrakis-admin
Project-URL: Repository, https://git.ligo.org/ngdd/arrakis-admin.git
Project-URL: Issues, https://git.ligo.org/ngdd/arrakis-admin/issues
Author-email: Olivia Godwin <olivia.godwin@ligo.org>
Maintainer-email: Olivia Godwin <olivia.godwin@ligo.org>
License-Expression: GPL-3.0-or-later
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Topic :: Scientific/Engineering :: Physics
Requires-Python: >=3.11
Requires-Dist: arrakis-backend-kafka>=0.10
Requires-Dist: arrakis-server>=0.17
Requires-Dist: arrakis>=0.15
Requires-Dist: confluent-kafka>=2.2
Requires-Dist: gpstime
Requires-Dist: rich>=13.0
Requires-Dist: typer>=0.12
Provides-Extra: dev
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: mypy-extensions; extra == 'dev'
Requires-Dist: pip; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Provides-Extra: lint
Requires-Dist: mypy; extra == 'lint'
Requires-Dist: mypy-extensions; extra == 'lint'
Requires-Dist: pip; extra == 'lint'
Requires-Dist: ruff; extra == 'lint'
Provides-Extra: test
Requires-Dist: pytest; extra == 'test'
Requires-Dist: pytest-cov; extra == 'test'
Description-Content-Type: text/markdown

<h1 align="center">arrakis-admin</h1>

<p align="center">Admin CLI for the Arrakis low-latency timeseries data distribution platform</p>

<p align="center">
  <a href="https://git.ligo.org/ngdd/arrakis-admin/-/pipelines/latest">
    <img alt="ci" src="https://git.ligo.org/ngdd/arrakis-admin/badges/main/pipeline.svg" />
  </a>
  <a href="https://git.ligo.org/ngdd/arrakis-admin/-/pipelines/latest">
    <img alt="coverage" src="https://git.ligo.org/ngdd/arrakis-admin/badges/main/coverage.svg" />
  </a>
  <a href="https://ngdd.docs.ligo.org/arrakis-admin/">
    <img alt="documentation" src="https://img.shields.io/badge/docs-mkdocs%20material-blue.svg?style=flat" />
  </a>
  <a href="https://pypi.org/project/arrakis-admin/">
    <img alt="pypi version" src="https://img.shields.io/pypi/v/arrakis-admin.svg" />
  </a>
</p>

---

Administrative tooling for operating Arrakis deployments: manage the
Kafka topics backing data partitions, manage the server's static
publisher tokens, probe data topics for debugging, and migrate server
metadata caches.

## Resources

* [Documentation](https://docs.ligo.org/ngdd/arrakis-admin)
* [Source Code](https://git.ligo.org/ngdd/arrakis-admin)
* [Issue Tracker](https://git.ligo.org/ngdd/arrakis-admin/-/issues)

## Installation

With `pip`:

```
pip install arrakis-admin
```

## Quickstart

Every command supports `--help`. Commands that modify the broker show
what they will do and prompt for confirmation, and support `--dry-run`
to preview changes without applying them.

### Manage topics

Create Kafka topics for a publisher's data partitions, with partition
assignments queried from the Arrakis server:

```
➜ arrakis-admin topics create H1-lldd -b kafka1:9092 -s grpc://arrakis:31206

topics to create (2):
  broker: kafka1:9092
  retention: 7d (604_800_000ms), replication factor: 1

  + arrakis-H1-lldd_float32
  + arrakis-H1-lldd_float64

metadata topic (compact, indefinite retention):
  + arrakis-metadata-H1-lldd

Is this ok [y/N]:
```

Partition assignments can instead be read from a publisher TOML config
with `--config`, and replay-context publishers are supported via
`--replay-id`.

Sync data topics against the channel metadata published on the broker,
creating missing topics and removing stale ones:

```
➜ arrakis-admin topics sync -b kafka1:9092 --dry-run
```

Remove a publisher's topics (or all Arrakis topics with `--all`):

```
➜ arrakis-admin topics remove -b kafka1:9092 -p H1-lldd
```

### Manage static publisher tokens

Manage the static token file of an arrakis-server running with
`--auth publish --static-token-file` (see the arrakis-server
authentication documentation). Register a publisher by minting a
fresh 256-bit token bound to it:

```
➜ arrakis-admin tokens add H1-lldd -f /etc/arrakis/tokens.toml
token added for H1-lldd in /etc/arrakis/tokens.toml (fingerprint 3fa4b2c1)
f3a9c2...
```

The token is printed to stdout, once — deliver it to the publisher's
host (e.g. an `ARRAKIS_TOKEN=` environment file for its service, which
`--env` emits directly) and nowhere else. Edits touch only the target
publisher's line, are written atomically, and are validated with the
server's own parser, so a hand-maintained file's comments and layout
survive; the server hot-reloads the file, so changes need no restart.

Rotate a token in two phases — `tokens rotate` mints a new token while
keeping the old one valid, then `tokens prune` drops the old one once
the publisher is redeployed. `tokens remove` revokes a publisher,
`tokens list` shows entries as fingerprints (never secrets), and
`tokens check` validates the file.

### Probe data topics

A debugging tool that consumes a partition's data topic directly from
Kafka and unpacks each message's Arrow payload, reporting block times,
per-channel content, and latencies:

```
➜ arrakis-admin probe -p H1-lldd_float64 -b kafka1:9092 -v
probing topic 'arrakis-H1-lldd_float64' on kafka1:9092 (Ctrl-C to stop)

offset=5231 time=1418063158.000000000 latency=0.412s transport=0.198s size=125.4kB channels=42 samples=131072
  [   0]  float64  16384 samples
  [   1]  float64  16384 samples
  ...
```

`latency` is the delay between the GPS block time and receipt;
`transport` is the delay relative to the Kafka message timestamp,
separating publisher lag from broker lag.

A channel name can be given instead of a partition ID, in which case
its partition is resolved through the Arrakis server and the channel's
rows are highlighted in the output:

```
➜ arrakis-admin probe -c H1:GDS-CALIB_STRAIN -s grpc://arrakis:31206 -b kafka1:9092
```

### Migrate metadata caches

Convert an arrakis-server TOML metadata cache to SQLite, or upgrade an
existing SQLite cache to the current schema:

```
➜ arrakis-admin cache migrate metadata.toml
migrated 12483 channels from metadata.toml to metadata.db

➜ arrakis-admin cache migrate metadata.db
upgraded metadata.db from schema v2 to v3
```

Clear all data from a cache while preserving its schema:

```
➜ arrakis-admin cache clear metadata.db
```

Remove publishers from a cache. The server keeps a publisher whose
config file disappears, so moving publishers to another server means
taking them out of this one's cache (their channels, the partition
counters only they used, their publisher info and sync offsets), with
the server stopped:

```
➜ arrakis-admin cache remove metadata.db H1-CAL-O4E2E_202409 H1-CALCLEAN-O4E2E_202409
```
