Metadata-Version: 2.4
Name: SpiderKeeper-3
Version: 0.1
Summary: Admin ui for spider service
Author-email: Dormy Mo <modongming91@gmail.com>
License-Expression: MIT
Classifier: Development Status :: 4 - Beta
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.13
Description-Content-Type: text/markdown
Requires-Dist: aniso8601==9.0.1
Requires-Dist: APScheduler==3.11.3
Requires-Dist: click==8.1.8
Requires-Dist: Flask==3.1.3
Requires-Dist: Flask-BasicAuth==0.2.0
Requires-Dist: Flask-RESTful==0.3.10
Requires-Dist: Flask-SQLAlchemy==3.1.1
Requires-Dist: itsdangerous==2.2.0
Requires-Dist: Jinja2==3.1.6
Requires-Dist: MarkupSafe==3.0.3
Requires-Dist: PyMySQL==1.1.1
Requires-Dist: python-dateutil==2.9.0.post0
Requires-Dist: pytz==2024.2
Requires-Dist: requests==2.32.3
Requires-Dist: six==1.17.0
Requires-Dist: SQLAlchemy==2.0.36
Requires-Dist: tzlocal==5.2
Requires-Dist: Werkzeug==3.1.3

# SpiderKeeper-3

[中文文档](README.zh-CN.md)

SpiderKeeper-3 is a web administration console for [Scrapyd](https://github.com/scrapy/scrapyd). It provides a single place to manage Scrapy projects, deploy spiders, schedule jobs, inspect execution logs, and review basic runtime statistics.

## Features

- Project and spider management
- One-off and periodic job scheduling
- Scrapyd deployment and server selection
- Job execution status and logs
- Spider and project runtime statistics
- Basic authentication and a REST API
- Swagger-style API documentation

## Requirements

- Python 3.13 or newer
- [uv](https://docs.astral.sh/uv/)
- A running Scrapyd service, usually at `http://localhost:6800`

## Installation

Clone the repository and let uv create or update the project virtual environment:

```bash
uv sync
```

The package metadata is `SpiderKeeper-3` at version `0.1`. Dependencies are installed into the repository's `.venv` directory.

## Run Locally

Start the web application with the installed console script:

```bash
uv run spiderkeeper --server=http://localhost:6800 --verbose
```

By default the server binds to `0.0.0.0:5000`; on the same machine, open `http://127.0.0.1:5000` and sign in with the development defaults:

```text
username: admin
password: admin
```

Change these values with `--username` and `--password` when starting a shared instance. For local troubleshooting only, `--no-auth` disables authentication.

Useful options:

```text
--host HOST                 Bind address (default: 0.0.0.0)
--port PORT                 Port (default: 5000)
--server URL                Scrapyd endpoint; may be repeated
--database-url URL          SQLAlchemy database URL
--type SERVER_TYPE          Spider service type (default: scrapyd)
--verbose                   Enable debug logging
```

## Using the API

- API documentation: `http://127.0.0.1:5000/api.html`
- OpenAPI-style JSON: `http://127.0.0.1:5000/api`
- Example project endpoint: `GET /api/projects`

API requests use the same Basic Authentication credentials as the web UI.

## Development

Run the main checks with the project environment:

```bash
uv pip check
uv run python -m compileall -q SpiderKeeper
uv build
```

Application code is under `SpiderKeeper/`. Flask routes and API resources are in `SpiderKeeper/app/`, templates are in `SpiderKeeper/app/templates/`, and static assets are in `SpiderKeeper/app/static/`.

## License

SpiderKeeper-3 is released under the MIT License.
