Metadata-Version: 2.4
Name: al-devops-kit
Version: 0.4.0
Summary: Airflow installer DAG and CLI: sync DAG repos, install requirements and package DAGs into a running Airflow.
Author-email: AL Data Labs <pypi@aldatalabs.com>
License-Expression: MIT
Project-URL: Homepage, https://aldatalabs.com
Keywords: airflow,devops,dags,deployment,ubuntu
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: POSIX :: Linux
Classifier: Topic :: System :: Installation/Setup
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: al-data-utils>=0.3.2
Dynamic: license-file

# al-devops-kit

Airflow bootstrap tooling: a versioned installer DAG plus a CLI that deploys it.

## Install

```bash
pip install al-devops-kit
al-devops-kit copy-dags /opt/airflow/dags
```

Python 3.10+. Install into the Airflow virtualenv. `al-data-utils` is pulled in
automatically; `apache-airflow` 3.x is expected from the host environment.

## CLI

```bash
al-devops-kit copy-dags <dags_dir> [--version vX.Y.Z] [--force]
al-devops-kit list-versions
```

`copy-dags` copies the installer DAG template into the given dags folder.
Existing files are overwritten only when their content differs.

## Installer DAG

`_airflow_installer_dag.py` runs daily (or on demand) and:

1. Reads `/opt/airflow/.repos/repos.yml`.
2. Clones or pulls each listed repo into `/opt/airflow/.repos/<name>/`.
3. Merges the repos' `requirements.txt` files and pip-installs them into the Airflow venv.
4. Installs DAGs from Python packages listed in each repo's `requirements_post_install.yaml`.
5. Syncs each repo's `dags_path` into `/opt/airflow/dags/` (copy, update, delete).

`repos.yml` format:

```yaml
venv_pip: /opt/airflow/.venv/bin/pip
repos:
  - name: my-dags
    url: https://<token>@github.com/<org>/<repo>.git
    branch: main
    dags_path: dags/
    requirements: requirements.txt
```

`requirements_post_install.yaml` (in a repo root, optional):

```yaml
package_dags:
  - name: al_data_core
    version: latest
    files:
      - data_core_dag.py
      - data_core_config_engine_dag.py
```

Each package listed must be installed in the Airflow venv and expose a
`dags_installer` module with `_copy_dags(target, version, force)`.

## Repository extras

The source repository also contains an Ansible playbook for a single-VM
Airflow install on Ubuntu. It is not part of the PyPI package.

## Changelog

See `CHANGELOG.md` in the repository for release notes.
