Metadata-Version: 2.4
Name: flyfile
Version: 0.3.0
Summary: Agent-native data transfer: push/pull/send anything between agents and machines
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.110
Requires-Dist: flaxkv2>=0.2.14
Requires-Dist: httpx>=0.27
Requires-Dist: pyyaml>=6.0
Requires-Dist: typer>=0.12
Requires-Dist: uvicorn[standard]>=0.29
Requires-Dist: zstandard>=0.22
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Description-Content-Type: text/markdown

# flyfile

Agent-native data transfer. Push/pull anything (text, files, directories) through a central
server, or stream it directly client-to-client. Built for AI agents: JSON output everywhere,
stable exit codes, content-addressed dedup, burn-after-read.

```bash
pip install flyfile

# server (single worker; data lives in LMDB via flaxkv2)
flyfile serve --port 8632 --token SECRET

# client
export FLYFILE_SERVER=http://host:8632 FLYFILE_TOKEN=SECRET
echo "build log" | flyfile push - --name buildlog --tag ci --ttl 2h
flyfile push ./model.bin --reads 1            # burn after one read
flyfile push ./dataset/                       # dirs stream as tar, no temp files
flyfile push ./release.tar --keep             # objects expire in 7d unless --keep
flyfile ls --tag ci --json                    # → {"total": N, "items": [...]}
flyfile rm --until 30d --dry-run              # bulk delete by filters, preview first
flyfile preview k3x9m2pq                      # peek without consuming reads
flyfile pull k3x9m2pq -o ./model.bin

# client → client (server relays the stream, nothing is stored)
flyfile send ./results/          # prints: code: amber-falcon
flyfile recv amber-falcon        # on the other machine
```

## Agent contract

- **JSON everywhere**: `--json`, or automatic when stdout is not a TTY. Progress goes to
  stderr, data to stdout. Never prompts.
- **Exit codes (stable)**: 0 ok · 2 usage · 3 not found · 4 auth · 5 conflict ·
  6 expired/burned · 7 network (retryable) · 1 other.
- **Errors** are JSON on stderr: `{"error", "message", "retryable", "suggestion"}`.
- **Idempotent push**: content-addressed (sha256). Re-pushing the same bytes is instant
  (`"deduped": true`).
- **Default retention is 7 days**, enforced server-side (so the HTTP API and the Python
  client behave the same). `--ttl` adjusts it; `--keep` stores forever.
- **`flyfile preview <id>`** reads the head of an object (or a dir's file manifest)
  without downloading and without consuming burn-after-read counts. Because it does not
  consume a read, the response is capped at 64 KiB and sets `truncated` when it clips.
- **`flyfile schema [cmd]`** dumps the command tree as JSON for introspection.
- **`flyfile send --json`** emits NDJSON events; the `code` event arrives before the
  transfer starts, so an agent can hand it to the receiver immediately.

## Breaking changes in 0.3.0

- **The on-disk format changed and is not backward compatible.** Chunks are now
  namespaced by upload rather than by content hash, which is what makes concurrent
  uploads of the same content safe and makes abandoned chunks reclaimable. A server
  started against a 0.2.x data directory **fails at startup** with instructions rather
  than corrupting or silently leaking data. To upgrade: pull anything you still need
  with 0.2.x, then delete the data directory (default `~/.local/share/flyfile`).
- `GET /objects/{id}/chunks/{idx}` for `idx > 0` now requires the `x-ff-lease` header
  issued with chunk 0. Previously any caller could fetch later chunks without claiming
  a read, which made `--reads 1` meaningless for multi-chunk objects. The bundled
  client handles this transparently; custom clients that fetch chunks directly must
  pass the lease through.

## Breaking changes in 0.2.0

- `GET /objects` (and `FlyfileClient.ls()`) now returns `{"total": N, "items": [...]}`
  instead of a bare list, so a paged query can report how many objects matched.
  `flyfile ls --json` changes shape accordingly. Upgrade server and clients together.
- Uploads without an explicit TTL now expire after 7 days instead of being kept forever.
  Pass `--keep` (CLI) or `ttl=0` (client/`x-ff-ttl: 0`) for the old behavior.

## Design notes

- Storage is [flaxkv2](https://github.com/KenyonY/flaxkv) (LMDB): metadata and 8 MiB
  content chunks in one env. The LMDB file does not shrink after deletes (free pages are
  reused; file size ≈ historical peak).
- Compression (zstd-3) happens on the *client*; the server stores/relays compressed bytes.
  Each 8 MiB chunk of a large upload is an independent zstd frame, so parallel upload,
  resume, and parallel download all work per-chunk.
- Burn-after-read: the read is claimed atomically when chunk 0 (or `/content`) is fetched.
  That claim issues a short-lived lease (`x-ff-lease`, default 5 min) which an in-flight
  parallel download presents to fetch the remaining chunks, so a burnt object stays
  unreadable to everyone else while the legitimate download finishes. A sweeper grace
  period (default 15 min) keeps the chunks around for the life of the lease.
- Every chunk belongs to exactly one owner at all times — an upload ledger entry or a
  blob record — and ownership moves in a single transaction. That invariant is what
  makes crashes, aborted uploads, and concurrent uploads of the same content unable to
  strand or clobber data. See `docs/architecture.md`.
- Run exactly **one** uvicorn worker: relay pairing and the burn-claim lock are in-process.

## Development

```bash
uv venv && uv pip install -e ".[dev]"
.venv/bin/pytest
scripts/bench.sh   # 1 GiB loopback throughput smoke test
```
