Metadata-Version: 2.4
Name: hfbr
Version: 0.2
Summary: High Frequency Backup and Retention
Keywords: backup,retention,cli,snapshot
Author: Liz Balbuena
License-Expression: Apache-2.0
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: System :: Archiving :: Backup
Classifier: Topic :: Utilities
Requires-Dist: pyyaml>=5.0
Requires-Python: >=3.14
Project-URL: Homepage, https://github.com/Liz4v/hfbr
Project-URL: Repository, https://github.com/Liz4v/hfbr.git
Project-URL: Documentation, https://github.com/Liz4v/hfbr/blob/main/README.md
Project-URL: Issues, https://github.com/Liz4v/hfbr/issues
Project-URL: Changelog, https://github.com/Liz4v/hfbr/releases
Description-Content-Type: text/markdown

# High Frequency Backup and Retention

Backup important small files under a high frequency schedule,
avoiding duplicates, and pruning less relevant snapshots as they get older.

Licensed under the [Apache License 2.0](LICENSE).

## Installation

```
uv tool install hfbr
```

Or, if you prefer pip:

```
pip install hfbr
```

1.  Copy `example_settings.yaml` into `settings.yaml`.
2.  Edit the newly created [Settings File](#settings-file) according to your needs, or do some basic testing with [CLI Mode](#cli-mode).
3.  Add `hfbr` to your crontab. My schedule is `*/20 * * * *`, which means every 20 minutes.

## Settings File

The settings file is a YAML file named `settings.yaml` placed in your working directory.
It has three top-level keys: `targets` and `plans`, described below,
and `logging`, following the [dictConfig schema](https://docs.python.org/3/library/logging.config.html).

### targets

This is the heart of your settings.
In it you have a list of targets, each representing a backup and/or a retention task:

```yaml
targets:
  - target_path: "/home/django/honesty/db.sqlite3"

  - backup_dir: "/home/backup/kindness"
    retention_plan: "important"

  - target_path: "/home/django/loyalty/db.sqlite3"
    backup_dir: "/home/backup/loyalty"
    retention_plan: "meh"

  - target_path: "/home/django/generosity/db.sqlite3"
    backup_dir: "/home/backup/generosity"
    retention_plan: "important"
    prune: false

  - target_path: "/home/django/laughter/db.sqlite3"
    backup_dir: "/home/backup/laughter"
    retention_plan: "important"
    pin:
      - "20150717-1155.sq3.bz2"

  - target_path: "/home/django/magic/db.sqlite3"
    backup_dir: "/home/backup/magic"
    retention_plan: [[null, 10], [month, 3]]
    prune: false
```

Available settings are:

- `target_path`: Full path to the file to be backed up. It is read without any kind of locking or waiting.
  If not given, only the retention policy is performed at `backup_dir`.
- `backup_dir`: Full path to the directory where the backups are stored.
  If not given, it is backed up in place, alongside the same directory of `target_path`.
- `retention_plan`: Name or inline description of the retention plan.
  See [plans](#plans) for details. If not given, no files are pruned.
- `pin`: A list of filenames that are not to be pruned.
  Pinned files fulfill the retention slots they fall in.
- `prune`: Set to `false` to run the retention plan in pretend mode. Results go in the logs.

### plans

This is a mapping of named retention plans, to be referenced by the targets:

```yaml
plans:
  important:
    - [year, null]       # permanent yearly snapshots
    - [month, 9]         # 9 monthly snapshots
    - ["1 week", 6]      # 6 weekly snapshots
    - ["1 day", 5]       # 5 daily snapshots
    - ["1 hour", 18]     # 18 hourly snapshots
    - [null, 10]         # 10 latest snapshots
  meh:
    - [null, 5]          # 5 latest snapshots
    - ["1 day", 8]       # 8 daily snapshots
```

A retention plan is a list of retention slots.
A retention slot is a pair of *granularity* and *quantity*.

- *Granularity* is how far apart we're interested the backups to be, so that they remain relevant as they get older.
  It is a human-readable duration string like `"1 week"`, `"5 days"`, or `"1 hour"`.
  There are also three special values: `year`, `month`, or `null`. `null` means look at all backups.

- *Quantity* is how many backups to keep when looking within that granularity.
  The special value `null` means keep all of them [forever](https://www.youtube.com/watch?v=ofvJU3AFOOo),
  and it's a useful quantity to give to your largest granularity.

Every time the retention plan is applied to a backup directory, it will run each slot on all files.
For each desireable slot it will try to find a file to be retained, and pin the earliest one if none is found.

Because we don't always make backups (only when the file changes),
we will only count the slots when at least one file can fulfill a slot.
For example, let's say your file only changes on weekdays, and you have 7 day retention.
No files will be generated on weekends, and to fulfill 7 days we'll need to go over to the previous week until
at least 7 files can be found, each in a different day. This way, 7 non-consecutive days will be retained.

It's recommended to define retention slots from the biggest granularity to the smallest.
The reason is that if you define slots from the smallest to the biggest,
you will lose your earliest backups because later backups will fulfill the same granularity.

## CLI Mode

```
hfbr                               # reads ./settings.yaml
hfbr -c /etc/hfbr/settings.yaml    # reads given config
hfbr target_path [backup_dir]      # CLI mode (no config)
```

If you don't have a settings file, you can use just the command line interface (CLI)
for simple change-detection backup without a retention plan.
To do that, simply define the origin and destination.
As when defined using the [Settings File](#settings-file), if `backup_dir` is not provided, it'll back up in place.

## Roadmap

- Maybe detect changes based on mtime and size instead? Checksum seems a bit overkill...
- Special cases for `target_path`:
  - Detect sqlite databases and use their backup function.
  - Detect directories and `tar` them.
    - Use GNU differential tar (`-g`) based on retention granularities. Will this work on non-Linux?
- Ability to push backups to a remote server or something. What makes sense, `scp`, e-mail, or what?
  In my scenario, pulling was way easier to implement.
