Metadata-Version: 2.4
Name: bili-downloader-from-scratch
Version: 0.1.0
Summary: A from-scratch Bilibili downloader learning project.
Author: Codex
License-Expression: MIT AND GPL-3.0-only
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: tools/ffmpeg-8.1-essentials_build/LICENSE
Requires-Dist: qrcode[pil]>=7.4
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: ruff>=0.6.0; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Dynamic: license-file

# bili-downloader-from-scratch

A from-scratch learning project for building a Bilibili downloader step by step.

This repository is intentionally scoped as an MVP path, not a clone of a full desktop app. The goal is to grow the downloader in layers: parsing first, stream selection second, downloading third, merging fourth, and GUI last.

## Current status

Implemented:

- Runnable CLI entrypoint
- `inspect` command with real metadata parsing for normal Bilibili video URLs and Bangumi episode/season URLs
- `streams` command with real DASH stream manifest parsing for normal videos and Bangumi episodes
- `plan` command that writes metadata + stream based JSON plans
- `download` command that downloads media files, with optional subtitle and danmaku sidecars
- `batch-download` command for multi-link downloads, with configurable `--jobs` concurrency, list-source expansion, and optional multi-page expansion
- `history list` / `history show` for querying saved records and mutable run state
- `resume` command for restarting failed or partial `plan` / `download` / `batch-download` runs
- `pause` command for pausing active runs, queued runs, or one batch item by index
- `retry-failed` command for retrying failed items in a saved batch run
- `cancel` command for requesting cancellation of active or queued runs
- `app_service` facade for GUI/CLI task creation, listing, detail lookup, pause, cancellation, retry, and resume
- `auth-check` command for validating whether configured Bilibili cookies are loaded and logged in
- Lightweight `tkinter` GUI MVP with task list, new download, task detail, Cookie, and settings pages
- Support for `BV...`, `av...`, Bangumi `ss...` / `ep...` / `md...`, full video URLs, and `b23.tv` short URLs
- Batch expansion for favorites, uploader collections, Bangumi seasons, and UP space video lists
- Optional subtitle sidecar download in `srt` or raw `json` format
- Optional danmaku XML sidecar download
- WBI signing flow for the live video detail API
- Best-available quality selection, with cookie-enabled higher tiers when the account can access them
- Premium audio parsing for Dolby and Hi-Res lossless tracks
- Built-in Windows `ffmpeg` runtime for merging, with config/PATH overrides and automatic fallback to separate audio/video files when `ffmpeg` is unavailable
- CLI progress output during downloads
- `--overwrite` support for repeated downloads
- Cookie file loading for authenticated requests
- Immutable per-run JSON records saved under the local `history_dir`, including failures
- Mutable run snapshots saved under the local `runs_dir` for task resume and future GUI polling
- Batch item phase/progress snapshots in `run.json` for active GUI polling
- Request-style pause checkpoints for active download and batch runs
- Request-style cancellation checkpoints for active download and batch runs
- Structured error categories for `auth`, `network`, `risk`, `quality`, `ffmpeg`, `io`, `parser`, `cancelled`, `paused`, and `unknown`
- Retry/backoff for retryable metadata, manifest, and media download failures
- Range-based resume from retained `.part` files
- Config loading and log initialization
- `pytest`, `ruff`, and GitHub Actions CI configuration
- Unit and integration-style tests for parser, stream, downloader, task, and real-scenario flows

GUI MVP:

- Task list with status, phase, progress, speed, ETA, cancel, resume, and open-folder actions
- New download page for single URL or multi-line batch downloads
- Task detail page with raw `run.json` and batch item rows
- Cookie configuration page with save and login-check actions
- Settings page for output directory, default concurrency, and ffmpeg path

## Support scope

Current backend coverage is no longer limited to plain single-video downloads:

- normal videos, including optional multi-page expansion via `batch-download --expand-pages`
- Bangumi episode URLs and season/media URLs
- favorites, uploader collections, and UP space video lists as batch sources
- optional subtitle and danmaku sidecar downloads during `plan` / `download` / `batch-download`

Still not positioned as a full product-grade desktop downloader for every Bilibili surface. Live streams, paid courses, broader GUI controls for the new sidecar/list features, and deeper real-device compatibility work remain future work.

## Structure

```text
bili-downloader-from-scratch/
|-- main.py
|-- pyproject.toml
|-- requirements.txt
|-- LICENSE
|-- THIRD_PARTY_NOTICES.md
|-- docs/
|-- src/
|   `-- bili_downloader/
|       |-- __init__.py
|       |-- __main__.py
|       |-- app_service.py
|       |-- auth.py
|       |-- cli.py
|       |-- config.py
|       |-- downloader.py
|       |-- errors.py
|       |-- logging_setup.py
|       |-- models.py
|       |-- network.py
|       |-- parser.py
|       |-- records.py
|       |-- services/
|       |   |-- __init__.py
|       |   |-- history.py
|       |   `-- tasks.py
|       `-- streams.py
|-- tools/
|   `-- ffmpeg-8.1-essentials_build/
|       |-- LICENSE
|       |-- README.txt
|       `-- bin/
|           `-- ffmpeg.exe
`-- tests/
    |-- test_app_service.py
    |-- test_auth.py
    |-- test_cli.py
    |-- test_downloader.py
    |-- test_errors.py
    |-- test_history.py
    |-- test_network.py
    |-- test_parser.py
    |-- test_records.py
    |-- test_streams.py
    `-- test_tasks.py
```

## Run

```powershell
cd D:\pycode\bili-downloader-from-scratch
python -m pip install -e ".[dev]"
bili-downloader --help
python .\main.py --help
python .\main.py inspect "https://www.bilibili.com/video/BV1xx411c7mD"
python .\main.py inspect "https://www.bilibili.com/bangumi/play/ss1129"
python .\main.py streams "BV1xx411c7mD"
python .\main.py plan "BV1xx411c7mD" --with-subtitles --with-danmaku
python .\main.py download "BV1xx411c7mD" --audio-only --overwrite
python .\main.py download "https://www.bilibili.com/bangumi/play/ep2001" --with-subtitles --with-danmaku
python .\main.py batch-download BV1xx411c7mD av170001 --audio-only --overwrite
python .\main.py batch-download BV1xx411c7mD av170001 --jobs 2 --overwrite
python .\main.py batch-download BV1xx411c7mD --expand-pages
python .\main.py batch-download "https://www.bilibili.com/list/ml123456"
python .\main.py batch-download "https://space.bilibili.com/2/video" --with-subtitles
python .\main.py history list --limit 10
python .\main.py history show <record-id-or-run-id>
python .\main.py resume <run-id> --jobs 2
python .\main.py pause <run-id> --reason "pause from GUI"
python .\main.py pause <run-id> --item 1 --reason "hold one item"
python .\main.py retry-failed <run-id> --jobs 2
python .\main.py cancel <run-id> --reason "stop from GUI"
python .\main.py gui
python .\main.py config
python .\main.py auth-check
python .\main.py --cookie-file .\cookies.txt streams "BV1xx411c7mD"
python -m ruff check .
python -m pytest
```

The `bili-downloader` console script is available after installing the project. `python .\main.py ...`
remains useful when running directly from a source checkout.

## FFmpeg policy

Releases include a bundled Windows `ffmpeg.exe` under `tools/ffmpeg-8.1-essentials_build/bin`.
The downloader resolves `ffmpeg` in this order:

1. `BILI_DOWNLOADER_FFMPEG`, set from the config file or GUI settings page
2. the bundled `ffmpeg.exe` from an installed package or source checkout
3. `ffmpeg` found on `PATH`

If no usable `ffmpeg` is found, downloads still succeed by saving separate video and audio files. The
release does not auto-download `ffmpeg` on first start.

## License

Project source code is MIT licensed. Release artifacts also include a bundled Windows FFmpeg build under
GPL v3; see `THIRD_PARTY_NOTICES.md` and `tools/ffmpeg-8.1-essentials_build/LICENSE`.

Cookie files can be provided as:

- raw header text such as `SESSDATA=...; bili_jct=...`
- JSON object such as `{"SESSDATA": "...", "bili_jct": "..."}`
- JSON array with `name` / `value` entries
- Netscape `cookies.txt`

By default the CLI looks for cookies at the `cookie_file` path printed by:

```powershell
python .\main.py config
```

Create that file with your own Bilibili browser cookies, then verify login state with:

```powershell
python .\main.py auth-check
```

For a one-off cookie file, use:

```powershell
python .\main.py --cookie-file .\cookies.txt auth-check
python .\main.py --cookie-file .\cookies.txt download "BV1xx411c7mD" --quality 1080p --overwrite
```

`auth-check` only prints cookie names and account status. It intentionally does not print cookie values.

Each `plan`, `download`, and `batch-download` run saves two local artifacts:

- a final JSON record in `history_dir`, with `status`, `exit_code`, request context, result payload, and any error details
- a mutable `run.json` snapshot in `runs_dir`, with `run_id`, phase, attempts, item-level batch state, active progress, and links to emitted history records

This split keeps immutable audit records and resumable task state separate, which is a better base for GUI task lists and interrupted-run recovery.

Errors are serialized with a stable `category`, `retryable` flag, and `stage`. Retryable metadata and manifest failures are retried before the command fails. Media track downloads retry each URL with exponential backoff and still fall through to backup URLs when available. Bilibili HTTP 412 and request-frequency failures are reported as `risk`, while expired or missing login cookies are reported as `auth`.

Batch concurrency is capped at 3 jobs, and all network opens pass through a small per-host rate limiter to reduce request bursts. Media downloads preflight known `Content-Length` values against available disk space, reject illegal Windows path components, keep `.part` files for resumable failures, and remove completed merge work directories after successful ffmpeg output.

Batch resume skips successful items, reruns failed or pending items, and resumes paused items. `resume --jobs N` can override the stored concurrency for a resumed batch. `retry-failed <run-id>` only requeues failed batch items and leaves successful or paused items untouched.

Pause is request-based. `pause <run-id>` writes `pause_requested` and moves active runs to `pause_requested` until the running process reaches a checkpoint. Active runs check this flag during phase transitions, progress updates, and media chunk downloads, then finalize as `paused` while keeping `.part` files for Range resume. `pause <run-id> --item N` pauses one batch item without pausing the rest of the batch.

Cancellation is request-based. `cancel <run-id>` writes `cancel_requested` into the run state. Active runs check this flag during phase transitions, progress updates, and media chunk downloads, then finalize as `cancelled` with exit code `130`.

GUI and other application clients should use `bili_downloader.app_service` instead of depending on CLI argument parsing. The facade exposes `create_download_task`, `create_batch_download_task`, `list_tasks`, `get_task_detail`, `pause_task`, `cancel_task`, `retry_failed_task`, and `resume_task`; the CLI now uses the same facade.

The GUI saves its settings to the default app config file under `app_home`. Use the settings page for `output_dir`, default batch concurrency, and an optional ffmpeg executable path to override the bundled runtime. The newly added subtitle / danmaku / list-expansion controls are currently exposed through the CLI and `app_service` API first.

## Next step

1. Expose subtitle, danmaku, and list-expansion options in the GUI.
2. Expand real-device regression coverage across more account types, regions, member-only videos, and larger list sources.
3. Package the GUI as a standalone executable.
