Metadata-Version: 2.4
Name: p1-taskqueue
Version: 0.2.2
Summary: A Task Queue Wrapper for Dekoruma Backend
Author-email: Chalvin <engineering@dekoruma.com>
Project-URL: Homepage, https://github.com/Dekoruma/p1-taskqueue
Project-URL: Repository, https://github.com/Dekoruma/p1-taskqueue.git
Project-URL: Issues, https://github.com/Dekoruma/p1-taskqueue/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: celery>=5.5.3
Requires-Dist: redis>=6.4.0
Requires-Dist: kombu>=5.5.4
Requires-Dist: django>=4.0.0
Requires-Dist: django-celery-results>=2.6.0
Requires-Dist: django-celery-beat>=2.8.1
Requires-Dist: requests>=2.32.3
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: flake8>=6.0.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"

# TaskQueue

A Task Queue Wrapper for Dekoruma Backend

## Description

TaskQueue is a Python package that provides a  wrapper around Celery for task queue management. It includes automatic queue setup, Dead Letter Queue (DLQ) routing, and dynamic task execution capabilities.

With a RabbitMQ broker, set `TASKQUEUE_RETRY_DELAY_SECONDS = 3600` to hold failed tasks in
`retry.<queue>` while workers process other tasks. Provision the retry queues before
starting workers by running the Django management command with that setting enabled:

```bash
python manage.py setup_taskqueue
```

A task retries three times, then follows the existing DLQ route. If the setting is
absent, retry behavior is unchanged.

### Retry queue and DLQ

The retry queue is a temporary waiting area. RabbitMQ keeps a failed message
there for the configured delay, then returns it to its original work queue.
Workers do not consume retry queues, so a delayed retry does not occupy a
worker slot.

The dead-letter queue (DLQ) is the terminal failure area. A message enters
`dlq.<queue>` after it exhausts its retries or cannot be routed safely. It
stays there until an operator or the existing DLQ replay process handles it.

| Queue | Purpose | Leaves automatically? | Consumed by workers? |
| --- | --- | --- | --- |
| `retry.<queue>` | Wait before another attempt | Yes, after the RabbitMQ TTL | No |
| `dlq.<queue>` | Preserve an exhausted or unroutable task | No | No |

### Retry flow

```mermaid
flowchart TD
    A1[A attempt 1 fails] --> R1[retry.default waits for broker delay]
    A1 --> B[B succeeds immediately]
    R1 --> A2[A attempt 2 fails]
    A2 --> R2[retry.default waits for broker delay]
    R2 --> A3[A attempt 3 fails]
    A3 --> R3[retry.default waits for broker delay]
    R3 --> A4[A attempt 4 fails]
    A4 --> DLQ[dlq.default]
```

With one worker, the observed order is:

```text
A attempt 1 — failed
B succeeded  — ran immediately
A attempt 2 — failed after broker delay
A attempt 3 — failed after broker delay
A attempt 4 — failed after broker delay
A → dlq.default
```

The final queue state is:

```text
default:       0
retry.default: 0
dlq.default:   1
```


## Deploy
- Push changes to main
- Create new TAG with version name (i.e 0.1.3)
- Check status on the Actions page on Github
