Metadata-Version: 2.4
Name: dtaas
Version: 2.1.0
Summary: DTaaS CLI
License: INTO-CPS-Association
Author: Astitva Sehgal
Requires-Python: >=3.10,<4.0
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: PyYAML (>=6.0.3,<7.0.0)
Requires-Dist: click (>=8.4.2,<9.0.0)
Requires-Dist: cryptography (>=49.0.0,<50.0.0)
Requires-Dist: email-validator (>=2.2.0,<3.0.0)
Requires-Dist: fqdn (>=1.5.1,<2.0.0)
Requires-Dist: python-gitlab (>=8.3.0,<9.0.0)
Requires-Dist: python-on-whales (>=0.81.0,<0.82.0)
Requires-Dist: tomlkit (>=0.15.0,<0.16.0)
Description-Content-Type: text/markdown

# DTaaS Command Line Interface

Command-line tool for the
[INTO-CPS-Association](https://github.com/INTO-CPS-Association/DTaaS)
Digital Twin as a Service platform. Use it to generate deployment projects,
manage users, and operate a running DTaaS instance.  All from a single `dtaas`
entry point.

---

## ⚡ Quick Start

From a clean machine to a running DTaaS deployment in seven steps.

```bash
# 1. Create and activate a virtual environment (recommended)
python -m venv .venv && source .venv/bin/activate   # Linux / macOS
# python -m venv .venv && .venv\Scripts\activate    # Windows

# 2. Install the package
pip install dtaas

# 3. Generate dtaas.toml + a sample users.csv to fill in
dtaas config generate

# 4. Open dtaas.toml and fill in your server DNS, paths, and credentials
#    (see Configuration Reference below for all fields)

# 5. Validate the configuration fix any reported errors before continuing
dtaas config validate

# 6. Generate deployment files for your chosen scenario
dtaas deployment generate --type secure-server

# 7. Bring the deployment up
dtaas platform install

# Tear it down when done
dtaas platform uninstall
```

> **Deployment type cheat-sheet**
>
> | `--type` | When to use |
> |---|---|
> | `localhost` | Local dev / demo only |
> | `insecure-server` | Multi-user HTTP demo: **not internet-facing** |
> | `secure-server` | Multi-user HTTPS: production-ready |
> | `secure-server-gitlab` | HTTPS + bundled GitLab: production-ready |
> | `workspace-localhost` | Workspace + Dex on localhost |
> | `workspace-secure-server` | Workspace + Keycloak: production-ready |

---

## 📋 Table of Contents

- [DTaaS Command Line Interface](#dtaas-command-line-interface)
  - [⚡ Quick Start](#-quick-start)
  - [📋 Table of Contents](#-table-of-contents)
  - [📦 Installation](#-installation)
  - [🛠 Commands](#-commands)
    - [🗒️ config](#️-config)
    - [deployment generate](#deployment-generate)
      - [Configuration substitution](#configuration-substitution)
      - [TLS certificate placement](#tls-certificate-placement)
    - [🚀 platform install](#-platform-install)
    - [🧹 platform uninstall](#-platform-uninstall)
    - [Lifecycle operations: status / stop / start / pause / resume](#lifecycle-operations)
    - [🔁 platform update --certs](#-platform-update---certs)
    - [🧩 platform update --config](#-platform-update---config)
    - [Deprecated command spellings](#deprecated-command-spellings)
    - [➕ user add](#-user-add)
    - [➖ user delete](#-user-delete)
    - [⏯️ user pause / stop / resume](#️-user-pause--stop--resume)
    - [🔍 config reconcile](#-config-reconcile)
  - [👥 User files](#-user-files)
  - [⚙️ Configuration Reference dtaas.toml](#️-configuration-reference-dtaastoml)
    - [Which sections does my deployment need?](#which-sections-does-my-deployment-need)
    - [Annotated dtaas.toml](#annotated-dtaastoml)

---

## 📦 Installation

Installation inside a virtual environment is strongly recommended to avoid
conflicts with system-wide packages.

```bash
python -m venv .venv
source .venv/bin/activate     # Linux / macOS
# .venv\Scripts\activate      # Windows

pip install dtaas
```

Verify the install:

```bash
dtaas --help
```

---

## 🛠 Commands

### 🗒️ config

Manage `dtaas.toml` independently of the rest of the project. This is the
first step in the setup workflow generate a template, fill it in, then
validate before running any other command.

#### Generate a fresh template

```bash
dtaas config generate
```

This writes `dtaas.toml` and a sample `users.csv` (bulk input for
`dtaas user add --file`) into the target directory.

#### Validate an existing file

```bash
dtaas config validate
```

`validate` reads `dtaas.toml` (from `--output-dir` first, then the current
directory) and reports all problems at once:

| Field | Rule |
|---|---|
| `git-repo` | Must be an `http(s)` URL |
| `[common].server-dns` | Must be `localhost`, an IP, or a fully qualified hostname |
| `[common].path` | Must be an absolute path to an existing directory |
| `[common.security].certs-src` | When present, must be an absolute path to an existing directory |
| `[common.resources].set_limits` | When present, `true` or `false` (default `true`) |
| `[common.resources].cpus` | Positive number (e.g. `4` or `0.5`) |
| `[common.resources].pids_limit` | Integer |
| `[common.resources].mem_limit`, `shm_size` | Byte size with required unit (e.g. `4G`, `512m`) |
| `[[users]]` | When present, must be an array of tables; usernames must be unique |
| `[[users]].username` | Required, valid username |
| `[[users]].email` | Required, valid RFC 5321/5322 address (no DNS lookup) |
| `[[users]].groups` | When present, must be a list of strings |
| `[[users]].load_balance` | When present, must be `true` or `false` |
| `[[users]].password` | When present, must be a string |
| Deployment-section URLs | When present, must be `http(s)` URLs |
| Deployment-section `default-user` | When present, must be a valid username |

Deployment-section URLs include `react-app-oauth-url`, `oauth-url`,
`auth-authority`, and `keycloak-issuer-url` across `[frontend]`,
`[localhost]`, `[insecure-server]`, `[secure-server]`, `[workspace-localhost]`,
and `[workspace-secure-server]`; each is checked only when its section is
present.

`path` and `certs-src` are checked against the local filesystem, run
`validate` on the deployment host. If a check fails with a permission error
(e.g. `certs-src` points under `/etc/letsencrypt`, which is root-owned), the
CLI says so and prints the elevated form to re-run:
`sudo -E env PATH="$PATH" dtaas <command>`.

The `[common.resources]` limit fields (`cpus`, `pids_limit`, `mem_limit`,
`shm_size`) are required only when `set_limits` is `true` (the default). With
`set_limits = false` they are optional and ignored; any value still present is
validated.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | For `generate`: target directory (created if missing). For `validate`: search location |
| `--force` | off | (`generate` only) Overwrite an existing `dtaas.toml` |

---

### deployment generate

Copies the full project structure for a specific deployment scenario
`docker-compose.yml`, config examples, and supporting files, into a target
directory ready to be customised.

```bash
dtaas deployment generate --type <name>
```

**Options:**

| Option | Default | Description |
|---|---|---|
| `--type NAME` | *(required)* | Deployment scenario (see table below) |
| `--output-dir PATH` | `.` | Target directory (must already exist) |
| `--force` | off | Overwrite files that already exist |

#### Available types

| `--type` | Deployment scenario | Support level |
|---|---|---|
| `localhost` | Single-machine Docker deployment | dev/demo only |
| `insecure-server` | Multi-user HTTP server | insecure/demo only |
| `secure-server` | Multi-user HTTPS/TLS server | **production-supported** |
| `secure-server-gitlab` | HTTPS/TLS + integrated GitLab | **production-supported** |
| `workspace-localhost` | Workspace service with Dex on localhost | dev/demo only |
| `workspace-secure-server` | Workspace service with Keycloak | **production-supported** |

> ⚠️ **Warning** Types marked *dev/demo only* or *insecure/demo only* run
> over plain HTTP with default or static credentials. **Do not expose them to
> the internet or shared networks.** Production-supported types still require
> manual hardening steps documented in the `README.md` and `CONFIGURATION.md`
> shipped with each generated project.

**Examples:**

```bash
# Localhost demo in the current directory
dtaas deployment generate --type localhost

# Production HTTPS server in a subdirectory
dtaas deployment generate --type secure-server --output-dir ./my-server

# Regenerate, overwriting existing files
dtaas deployment generate --type insecure-server --output-dir ./demo --force
```

#### Configuration substitution

When `dtaas.toml` is present, `deployment generate` reads
deployment-specific values from it and substitutes them into the generated
files automatically.

The CLI searches `--output-dir` first, then falls back to the current working
directory. Each `--type` reads from its matching top-level section in
`dtaas.toml`. Values are written into dotenv files (`config/.env`,
`config/conf.server`) and the React client config (`config/client.js`).

The `[frontend]` section supplies `REACT_APP_CLIENT_ID` and
`REACT_APP_AUTH_AUTHORITY` for the DTaaS web client, these are a separate
OAuth application from the traefik-forward-auth credentials configured in
`[insecure-server]` / `[secure-server]`. The `[common]` and `[[users]]`
sections are substituted across all types.

If `dtaas.toml` is not found, a note is printed and generated files keep
their default placeholder values.

#### TLS certificate placement

For the TLS types (`secure-server`, `secure-server-gitlab`,
`workspace-secure-server`), `deployment generate` also populates the
`certs/` directory in the output. It reads `[common.security].certs-src` from
`dtaas.toml` and copies the latest `fullchain.pem` and `privkey.pem` there.

---

### 🚀 platform install

Brings a generated deployment up with a single command.

```bash
dtaas platform install
```

Internally runs `docker compose up -d` against the `docker-compose.yml` in
the installation directory. Before starting, it ensures per-user workspace
directories for every `[[users]]` record exist, recreating each from
`files/template/` if missing and sets ownership to `1000:100`.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory containing the generated deployment |

The CLI looks for `dtaas.toml` in `--output-dir` first, then the current
working directory, so a single top-level `dtaas.toml` can serve a deployment
generated into a subdirectory:

```bash
dtaas platform install --output-dir ./insecure
```

The command fails with a clear error if `docker-compose.yml` is missing,
`dtaas.toml` cannot be found, or the Docker daemon is unreachable.

---

### 🧹 platform uninstall

Tears the deployment down, stopping and removing containers and networks.

```bash
dtaas platform uninstall
```

User containers added with `user add` run as a separate Compose project;
they are torn down first so they do not hold the shared network open.
**Per-user workspace files are preserved by default.**

To also delete the generated per-user workspace directories **and** the
CLI-owned `dtaas.users.registry.json` / `.dtaas.state.json`:

```bash
dtaas platform uninstall --remove-user-files
```

This is destructive, so the command prompts for confirmation. Skip the prompt
in non-interactive scripts with `--yes`:

```bash
dtaas platform uninstall --remove-user-files --yes
```

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory |
| `--remove-user-files` | off | Also delete per-user workspace dirs + registry/state files |
| `--yes` / `-y` | off | Skip the confirmation prompt for `--remove-user-files` |

> `--remove-user-files` removes only the per-user directories inside
> `<output-dir>/files/`, preserving `files/common/` and `files/template/` so
> a later `platform install` can recreate user directories. It refuses to follow
> a symlinked `files/`. Double-check `--output-dir` before using this flag.

---

### Lifecycle operations

Operational controls for an **already-installed** deployment: observe it with
`status`, and suspend or resume it with `stop`/`start` and `pause`/`resume`.
None of these remove containers or networks, that is `uninstall`'s job. They
exit `0` on success (including the idempotent "nothing installed" case) and
non-zero on failure, so they are safe to call from CI/ops scripts.

`platform stop`/`start`/`pause`/`resume` act on the **core services only** —
they never touch per-user containers. Suspend or resume individual additional
users with [`dtaas user stop`/`pause`/`resume`](#️-user-pause--stop--resume)
instead. `platform status`, being read-only, still reports the whole
installation (core services **and** user containers).

**Lifecycle command matrix:**

| Command | `docker compose` verb | Effect | Containers kept? | Reverse with |
|---|---|---|:---:|---|
| `platform install` | `up -d` | Create and start every service | n/a | `platform stop` / `platform uninstall` |
| `platform status` | `ps` (read-only) | Report per-service state; no change | n/a | n/a |
| `platform stop` | `stop` | Terminate the processes, keep the containers | yes | `platform start` |
| `platform start` | `start` | Start previously stopped containers | yes | `platform stop` |
| `platform pause` | `pause` | Freeze the processes (memory preserved) | yes | `platform resume` |
| `platform resume` | `unpause` | Thaw previously paused processes | yes | n/a |
| `platform uninstall` | `down` | Stop **and remove** containers and networks | no | `platform install` |

> **`stop` vs `pause`.** `stop` sends `SIGTERM`/`SIGKILL`: processes end, and a
> restart re-runs them from scratch (reverse with `platform start`). `pause` uses
> the kernel cgroup freezer: processes are suspended in place with their memory
> intact and resume instantly (reverse with `platform resume`), but a paused
> container still holds its resources. Use `stop` to free CPU; use `pause` for
> a brief, instantly reversible suspension. `pause` expects running containers
> and will error if the deployment is already stopped.

#### 📊 platform status

Reports the state of every service, for both the deployment and user
workloads.

```bash
dtaas platform status
```

```text
PROJECT     SERVICE            STATE        HEALTH
deployment  traefik            running      healthy
deployment  client             running      -
deployment  gitlab             not created  -
users       user-alice         paused       -
```

State values are `running`, `paused`, `stopped` (a terminated container, what
Docker calls `exited`), `restarting`, or `not created` (a service defined in
`docker-compose.yml` that has no container yet). `HEALTH` shows the container
healthcheck status, or `-` when the service has none.

For automation, `--json` emits the same records as machine-readable JSON:

```bash
dtaas platform status --json
```

```json
[
  {"project": "deployment", "service": "traefik", "state": "running", "health": "healthy"}
]
```

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory |
| `--json` | off | Emit machine-readable JSON instead of the table |

#### ⏹️ platform stop / ▶️ platform start

`stop` stops all services with `docker compose stop`; `start` brings the
stopped containers back with `docker compose start`. Containers and networks
are **kept**, so `stop` is not an uninstall.

```bash
dtaas platform stop
dtaas platform start
```

Both report `no existing DTaaS / Workspace installation` and exit `0` when
nothing is installed (no containers in any state), so they are safe to call
repeatedly.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory |

#### ⏸️ platform pause / ▶️ platform resume

`pause` freezes every running container in place with `docker compose pause`;
`resume` thaws them with `docker compose unpause`. Memory is preserved and
resume is near-instant.

```bash
dtaas platform pause
dtaas platform resume
```

Both report the absent-installation case and exit `0` when nothing is
installed (no containers in any state).

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory |

---

### 🔁 platform update --certs

Rotates TLS certificates of a running deployment in place: no project
regeneration or manual file copying required.

```bash
dtaas platform update --certs
```

The command reads `[common.security].certs-src` from `dtaas.toml`, then:

1. **Validates** the new certificate pair (parseable, private key matches cert,
   no expired intermediates).
2. **Stops** the `traefik` service to release open file handles.
3. **Swaps** the validated files into `<output-dir>/certs/`, backing up the
   live pair and restoring it on any failure.
4. **Restricts** the private key to `0600` (POSIX; prints a warning on Windows).
5. **Restarts** `traefik` and waits for it to come back up.

If validation fails, live certificates are left untouched. The command is safe
to run repeatedly.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--certs` | *(required)* | Refresh the deployment's TLS certificates |
| `--output-dir PATH` | `.` | Installation directory |

---

### 🧩 platform update --config

Re-applies the values in `dtaas.toml` to an already-installed deployment's
service config files without regenerating the project.

```bash
dtaas platform update --config
```

Treats `dtaas.toml` as the single source of truth, re-runs the same
substitution as `deployment generate`, and if anything changed, recreates all
deployment services with `docker compose up -d --force-recreate`. The
deployment type is auto-detected from `docker-compose.yml`.

```bash
# Preview changes without writing or restarting
dtaas platform update --config --dry-run

# Apply changes and restart
dtaas platform update --config

# Update a deployment in a subdirectory
dtaas platform update --config --output-dir ./my-server
```

`--config` validates `dtaas.toml` before making any changes and refuses to
apply if problems are found. It is **idempotent** a second run with no
`dtaas.toml` changes reports `No configuration changes` and restarts nothing.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--config` | *(required)* | Re-apply `dtaas.toml` to installed service config files |
| `--dry-run` | off | Report what would change without writing or restarting |
| `--output-dir PATH` | `.` | Installation directory |

> `--certs` and `--config` may be combined in a single invocation.

---

### Deprecated command spellings

Version 2.0.0 moved every command to a single `dtaas <noun> <verb>` grammar.
The old spellings still work for **one release** as hidden aliases that print a
deprecation notice to stderr and forward to the new command; they will be
removed in the next major version. Update scripts to the new spellings:

| Old spelling | New spelling |
|---|---|
| `dtaas generate-project` | `dtaas config generate` + `dtaas deployment generate` |
| `dtaas generate-deployment --type <t>` | `dtaas deployment generate --type <t>` |
| `dtaas admin config <verb>` | `dtaas config <verb>` |
| `dtaas admin install\|uninstall\|update` | `dtaas platform install\|uninstall\|update` |
| `dtaas admin status` | `dtaas platform status` |
| `dtaas admin stop\|start\|pause\|resume` | `dtaas platform stop\|start\|pause\|resume` ⚠️ scope narrowed, see below |
| `dtaas admin user add\|delete\|pause\|stop\|resume` | `dtaas user add\|delete\|pause\|stop\|resume` |

> **Scope change: `admin stop`/`start`/`pause`/`resume`.** Pre-2.0, these acted
> on the **whole installation** (core services and every additional user).
> Their 2.0 replacements act on the **core services only**; per-user
> containers are left untouched. To reproduce the old whole-installation
> effect, also target every additional user with
> [`dtaas user stop`/`pause`/`resume --all`](#️-user-pause--stop--resume)
> (`admin start` maps to `user resume --all`, since there is no `user start`).
> The running CLI also prints this as a warning when you use the old spelling.
>
> **`user status` has no old spelling.** It is new in 2.0 (the closest
> pre-2.0 equivalent, whole-installation `admin status`, is now `platform
> status`), so there was never an `admin user status` to alias.

The old `generate-project` folded into two commands: `dtaas config generate`
now solely writes `dtaas.toml`, while `dtaas deployment generate` writes the
Docker Compose user-workspace templates (`users.server.yml`,
`users.server.secure.yml`, `users.resources.yml`) and the `files/template/`
skeleton, alongside the deployment compose tree.

> **Tip: verify the Docker image tag**
> `users.server.yml` and `users.server.secure.yml` contain a pinned workspace
> image tag (e.g. `intocps/workspace:main-56c6f68`). Check
> [Docker Hub](https://hub.docker.com/r/intocps/workspace/tags) and update the
> tag to a current, stable version before deploying.

---

### ➕ user add

Provisions users on a running DTaaS instance. Additional users are recorded in
the CLI-owned `dtaas.users.registry.json`
(see [User files](#-user-files)), not in `dtaas.toml`.

**Options:**

| Option | Default | Description |
|---|---|---|
| `USERNAME` | — | Add one user (requires `--email`) |
| `--file PATH` / `-f` | — | Bulk-add users from a CSV |
| `--email TEXT` | — | Email for `USERNAME` (enables forward-auth routing) |
| `--group TEXT` | `additional` | Group tag for `USERNAME`; repeat the flag for multiple groups, e.g. `--group dtaas --group testers` |
| `--load-balance / --no-load-balance` | on | Mark `USERNAME` for load balancing |
| `--password TEXT` | — | GitLab password for `USERNAME`; only used when GitLab provisioning is enabled (see below). Visible in shell history and the process list, prefer the `users.csv` `password` column (`chmod 600` it) or the interactive prompt for non-interactive/scripted use |

Add a single user:

```bash
dtaas user add --email alice@intocps.org --group dtaas --load-balance alice
```

> Click accepts `--email`, `--group`, and `--load-balance` in any position
> relative to `USERNAME` the form above is the recommended convention,
> matching `useradd [options] LOGIN`.

`--group` is repeatable, not comma-separated pass it once per group to add a
user to multiple groups:

```bash
dtaas user add --email alice@intocps.org --group dtaas --group testers alice
```

Or bulk-add from a CSV:

```bash
dtaas user add --file users.csv
```

`dtaas config generate` writes a sample `users.csv` next to `dtaas.toml`:

```csv
username,email,groups,load_balance
alice,alice@intocps.org,additional,true
bob,bob@intocps.org,additional;beta-testers,false
```

`groups` is a `;`-separated list and `load_balance` is `true`/`false`. Both
forms merge into the registry (never hand-edited) with `desired_status` set to
`running`, then **only the newly-added users are started** — already-running
users are left untouched, so adding one user never recreates the rest. A
username already declared in `dtaas.toml`'s `[[users]]` or the registry is
**skipped with a warning**: it is never added twice or overwritten.

#### GitLab provisioning (optional)

When `[gitlab].provision = true` in `dtaas.toml` (off by default), `user add`
also creates each new user's GitLab account and a Personal Access Token:

```toml
[gitlab]
provision = true
api_url = "https://gitlab.example.com"
```

The provisioning token must be able to create users (an admin token), so it is
read from the `DTAAS_GITLAB_PAT` environment variable and is deliberately not
part of the generated template:

```bash
export DTAAS_GITLAB_PAT="glpat-xxxxxxxxxxxxxxxxxxxx"
```

A `[gitlab].pat` key is still honoured if you prefer to set one, and takes
precedence over the environment variable.

For a self-hosted GitLab behind an internal CA, set `[gitlab].ssl_verify` to
the CA bundle's path instead of leaving TLS verification on the system trust
store (which will fail) or disabling it outright:

```toml
[gitlab]
ssl_verify = "/etc/ssl/certs/corp-ca.pem"  # or true (default) / false
```

Setting `ssl_verify = false` disables certificate verification for all
GitLab API traffic, including the admin PAT and every provisioned user's
password, the CLI prints a warning whenever it is disabled.

Each provisioned user needs an initial GitLab password, supplied via
`--password` (prompted interactively with hidden input if omitted, for a
single-user add) or via a `password` column in `users.csv`:

```csv
username,email,groups,load_balance,password
alice,alice@intocps.org,additional,true,S3cur3-p4ss
bob,bob@intocps.org,additional;beta-testers,false,An0ther-p4ss
```

`--password` is visible in shell history and to anyone on the host who can
list processes (`ps`), since the OS records a command's actual arguments.
For a single interactive add, omit it and use the hidden prompt instead. For
scripted/non-interactive use, prefer the CSV `password` column above and
`chmod 600 users.csv` -- the CLI does not manage permissions on a CSV path
you supply yourself (it only chmods the sample file `dtaas config generate`
writes).

Re-running `dtaas user add` with the same `USERNAME`/CSV row and password
retries GitLab provisioning for an already-registered user without touching
their container e.g. after account creation failed outright (a network
blip), or after the account was created but PAT issuance failed. The CLI
tracks the GitLab account id it created internally, so a retry reissues a
token directly rather than asking GitLab to create the account again (whose
"already exists" response can't be trusted to mean "created by this CLI"
it could just as easily be someone else's account with the same name).

A retry only fills gaps: once a token has been issued for a user, the CLI
records that in the registry (`gitlab_pat_issued`) and a further re-run
issues no second token, reporting the user as skipped. This keeps a repeated
`user add` (the natural response to a partial failure) from minting a fresh
365-day PAT on every pass and leaving the earlier ones live but untracked. If
a token really is lost or revoked, issue a replacement from GitLab directly.

The password is used only to create the GitLab account and is never written
to `dtaas.users.registry.json`, `.dtaas.state.json`, or logs. A user missing
a password when provisioning is enabled has their GitLab step skipped with a
warning. An already-existing GitLab account is left with its current
password and issued no new token, and is reported with an explicit warning
(its credentials were not created by this run and are unknown to it) rather
than as an unremarkable success. Container provisioning is unaffected by any
GitLab outcome containers are already up by the time GitLab provisioning
runs but a GitLab failure (a missing/skipped password does not count) now
makes the `user add` command itself exit non-zero, so scripts can detect it.

Issued tokens are saved to `gitlab_user_tokens.json` as `{"username":
"token"}`, in the current working directory alongside `dtaas.toml` and
`dtaas.users.registry.json` (not fixed to `--output-dir`). Each `user add`
run merges newly issued tokens into the existing file rather than
overwriting it, so it accumulates one token per user across runs (the
`gitlab_pat_issued` guard above stops a second token being issued for a user
who already has one). If a user's entry is ever replaced, the previous value
is kept under a `"<username> (superseded <timestamp>)"` key and a warning is
printed the old token is still live on GitLab and needs manual revocation.
Treat the file as a credential store, the same as `dtaas.toml`.

`dtaas.toml`, `users.csv`, and `gitlab_user_tokens.json` are all written mode
`0600` and are gitignored, since each can hold a credential: the provisioning
PAT, user passwords, and the issued user tokens respectively.

A `USERNAME` or `--file` is required (not both) — a bare `dtaas user add`
with neither is rejected rather than silently reprovisioning the whole
registry. To (re)provision **every** registry user at once (e.g. after
`compose.users.yml` was lost), use `dtaas config reconcile --fix`
instead.

**Options:**

| Option | Default | Description |
|---|---|---|
| `USERNAME` | — | Add one user (requires `--email`) |
| `--file PATH` / `-f` | — | Bulk-add users from a CSV |
| `--email TEXT` | — | Email for `USERNAME` (enables forward-auth routing) |
| `--group TEXT` | `additional` | Group tag for `USERNAME`; repeat the flag for multiple groups, e.g. `--group dtaas --group testers` |
| `--load-balance / --no-load-balance` | on | Mark `USERNAME` for load balancing |
| `--password TEXT` | — | GitLab password for `USERNAME`; only used when GitLab provisioning is enabled (see below). Visible in shell history and the process list, prefer the `users.csv` `password` column (`chmod 600` it) or the interactive prompt for non-interactive/scripted use |

For each username the CLI checks whether `files/<username>/` already exists.
If not, a new directory with the correct structure is created from
`files/template/`. The directory, if it already exists, must be owned by the
user running the `dtaas` command; otherwise the command fails.

When an `email` is provided for a user in `dtaas.toml`, the CLI automatically
adds a traefik-forward-auth routing rule to `config/conf.server`. Restart
the container for the change to take effect:

```bash
docker compose --env-file config/.env up -d --force-recreate traefik-forward-auth
```

#### Resource limits (optional)

By default each user container is created with the CPU, memory, process, and
shared-memory caps from `[common.resources]`, merged in from the
`users.resources.yml` overlay. To onboard users without any caps, set
`set_limits = false` in that section:

```toml
# Constrained users (default): limits enforced
[common.resources]
set_limits = true
cpus       = 4
mem_limit  = "4G"
pids_limit = 4960
shm_size   = "512m"
```

```toml
# Unconstrained users: no caps written, limit fields optional
[common.resources]
set_limits = false
```

The flag is read on every `user add`, so a deployment can host both constrained
and unconstrained users by toggling `set_limits` between runs.

> **Notes**
>
> - `user add` starts a container for a new user or restarts a stopped one; it
>   reports *Running* for containers already up without restarting them.
> - Provisioning is idempotent: re-running `user add` reprovisions every
>   registry user without duplicating work. An empty registry is a no-op.
> - Usernames may include '.', '_' and '-' (must start with a letter or digit).
>   (Whitespace, path separators, and shell metacharacters are rejected.)
> - This command does not enable AuthMS authentication.

---

### ➖ user delete

Removes one or more users from a running DTaaS instance, like `userdel`.

**Options:**

| Option | Default | Description |
|---|---|---|
| `USERNAMES` | — | One or more usernames to remove |
| `--file PATH` / `-f` | — | Bulk-delete users listed in a CSV (only the `username` column is used) |
| `--dry-run` | off | Preview the removal without making any changes |

Pass the usernames as arguments:

```bash
dtaas user delete username1 username2
```

Or bulk-delete from a CSV (the same `users.csv` format used by
`user add --file` other columns are ignored):

```bash
dtaas user delete --file users.csv
```

`USERNAMES` and `--file` are mutually exclusive, and one of them is required.

Each user is deprovisioned (its container stopped, its compose service and
forward-auth rule removed) and dropped from `dtaas.users.registry.json`. Users
that are not currently provisioned are reported and skipped, but are still
removed from the registry.

Preview a removal without making any changes with `--dry-run`:

```bash
dtaas user delete username1 username2 --dry-run
```

It lists which users would be deprovisioned and removed from the registry, then
exits without stopping containers or editing any file.

The CLI automatically removes the traefik-forward-auth routing rules for
deleted users from `config/conf.server`. Restart the container for the change
to take effect:

```bash
docker compose --env-file config/.env up -d --force-recreate traefik-forward-auth
```

---

### ⏯️ user pause / stop / resume

Suspend or resume **specific additional (registry) users** without touching
the rest of the installation, targeting one or more `USERNAMES`, a
`--file`/`-f users.csv` (only the `username` column is read, the same way
`user delete` does), or `--all` for every additional user at once.

```bash
dtaas user pause alice bob
dtaas user stop alice
dtaas user resume alice bob
dtaas user pause --file users.csv
dtaas user stop --all
```

`--all` is also how to reproduce the pre-2.0 whole-installation
`admin stop`/`pause`/`resume` for the additional-users half: pair it with the
core-only [`dtaas platform stop`/`pause`/`resume`](#lifecycle-operations) (see
[Deprecated command spellings](#deprecated-command-spellings)).

| Command | `docker compose` verb | Effect | Reverse with |
|---|---|---|---|
| `user pause` | `pause` | Freeze the named users' containers (memory preserved) | `user resume` |
| `user stop` | `stop` | Terminate the named users' containers, keep them | `user resume` |
| `user resume` | `unpause` or `start`, as needed | Thaw a paused user, or restart a stopped one | — |

Each command also writes a `desired_status` (`"paused"`/`"stopped"`/`"running"`)
into `dtaas.users.registry.json` for the users it acted on. This is what makes
the suspension durable: a later `dtaas user add` (which idempotently
re-provisions every registry user on every run) or `dtaas config
reconcile --fix` checks `desired_status` and will **not** silently restart a
user you paused or stopped. Only `user resume` (or hand-editing the
registry) clears it back to `"running"`.

Only additional users can be targeted here. Naming a `dtaas.toml` starting
user is rejected with an error: suspend or resume the whole installation
(starting users included) with [`dtaas platform pause`/`stop`/`resume`](#lifecycle-operations)
instead.

A username not found in the registry, or found but not currently provisioned
(e.g. `user add` was never run for them), is reported and skipped rather than
aborting the whole batch:

```text
'carol' is not a registered user, skipping
alice, bob paused successfully
```

**Options:**

| Option | Default | Description |
|---|---|---|
| `USERNAMES` | — | One or more usernames to target |
| `--file PATH` / `-f` | — | Bulk-target users listed in a CSV (only the `username` column is used) |
| `--all` | off | Target every additional (registry) user; mutually exclusive with `USERNAMES`/`--file` |

---

### 🔍 config reconcile

Reports drift between `dtaas.users.registry.json` (which **should** be
provisioned) and the live `compose.users.yml` services (which **are**
provisioned).

```bash
dtaas config reconcile
```

It lists:

- **missing** registered but not currently provisioned;
- **unexpected** provisioned but not in the registry (investigate: may be a
  manual edit or a partial delete);
- **drifted** provisioned, but the live config no longer matches what
  `.dtaas.state.json` recorded when it was last provisioned;
- **desired-status drift** provisioned, but the live container state does not
  match the user's registry `desired_status` (e.g. `desired 'paused' but
  container is 'running'`).

When everything matches, it prints `In sync: no drift detected.`

Without `--fix`, this is read-only. Pass `--fix` to reprovision **missing** and
**drifted** users and to pause/stop/start every provisioned user to match its
`desired_status` (equivalent to running `dtaas user add`, so it acts on
the current directory, not `--output-dir`):

```bash
dtaas config reconcile --fix
```

**unexpected** services are never touched by `--fix` removing something
that's actually running is a deliberate action use
`dtaas user delete` for those.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory to inspect |
| `--fix` | off | Reprovision missing/drifted registry users after reporting |

---

## 👥 User files

User management spans three files, each with a single owner, modelled on the
config/state split Terraform uses for `.tf` vs `terraform.tfstate`:

| File | Owner | Contents | Git |
|---|---|---|---|
| `dtaas.toml` `[[users]]` | Human, at install time | **Starting** users: one self-contained record per user (`username`, `email`, `groups`, `load_balance`) | Tracked hand-edited |
| `dtaas.users.registry.json` | CLI (`user add` / `delete` / `pause` / `stop` / `resume`) | **Additional** users: the same fields, plus `desired_status` (`running`/`paused`/`stopped`) | Tracked CLI-written, never hand-edited |
| `.dtaas.state.json` | CLI, at provisioning time | Observed runtime facts: container id, status, provisioned-at, config hash | Ignored runtime cache |

- **`dtaas.toml`** is written once by a human and never rewritten by the CLI,
  so a comment-bearing, reviewed config is never silently mutated.
- **`dtaas.users.registry.json`** is a database the CLI owns and mutates
  atomically (the way `useradd` owns `/etc/passwd`). Edit its users through
  `dtaas user add --file users.csv` / `delete` / `pause` / `stop` /
  `resume`, not by hand. `users.csv` copied by `dtaas config generate`
  is the human-editable bulk input that feeds `add`/`delete`. `desired_status`
  defaults to `running` for a user who has never been paused or stopped, and
  `user add`/`config reconcile --fix` skip starting any user whose
  `desired_status` is not `running`.
- **`.dtaas.state.json`** is a disposable cache of what is actually running,
  refreshed on every add/delete. It is git-ignored and safe to delete.

---

## ⚙️ Configuration Reference dtaas.toml

`dtaas.toml` is the single source of truth for all CLI commands. Generate a
blank template with `dtaas config generate`, fill it in, then confirm
it is valid with `dtaas config validate` before running any other command.

### Which sections does my deployment need?

Required ✅

Optional ○

Not-Used —

#### Server deployments

| Section | `localhost` | `insecure-server` | `secure-server` | `secure-server-gitlab` |
|---|:---:|:---:|:---:|:---:|
| `[common]` | ✅ | ✅ | ✅ | ✅ |
| `[common.security]` | — | — | ✅ | ✅ |
| `[common.resources]` | ○ | ○ | ○ | ○ |
| `[[users]]` | ✅ | ✅ | ✅ | ✅ |
| `[frontend]` | — | ✅ | ✅ | ✅ |
| `[localhost]` | ✅ | — | — | — |
| `[insecure-server]` | — | ✅ | — | — |
| `[secure-server]` | — | — | ✅ | — |
| `[secure-server-gitlab]` | — | — | — | ✅ |

#### Workspace deployments

| Section | `workspace-localhost` | `workspace-secure-server` |
|---|:---:|:---:|
| `[common]` | ✅ | ✅ |
| `[common.security]` | — | ✅ |
| `[common.resources]` | ○ | ○ |
| `[[users]]` | ✅ | ✅ |
| `[workspace-localhost]` | ✅ | — |
| `[workspace-secure-server]` | — | ✅ |

### Annotated dtaas.toml

The full file below shows every possible key with inline comments. Copy it
as a starting point and delete sections that do not apply to your deployment
type (see matrix above).

```toml
# ── Common settings (all deployment types) ────────────────────────────────────
[common]
# Public hostname of the server. Use "localhost" for local deployments,
# a fully-qualified domain name (e.g. "dtaas.example.com") for servers,
# or a bare IP address.
server-dns = "dtaas.example.com"

# Absolute path to the DTaaS installation directory on the deployment host.
# Must exist before running validate.
path = "/opt/dtaas"

# ── TLS settings (required for: secure-server, secure-server-gitlab,
#                               workspace-secure-server) ──────────────────────
[common.security]
tls = true

# Absolute path to the directory containing fullchain.pem and privkey.pem.
# Used by deployment generate (seeds certs/) and platform update --certs.
certs-src = "/etc/letsencrypt/live/dtaas.example.com"

# ── Per-user container resource limits (optional, all types) ──────────────────
[common.resources]
# Enforce the limits below. Set to false to add users without any caps.
# The 4 fields are then optional and ignored. Defaults to true when omitted.
set_limits = true
cpus       = 4        # CPU cores; may be fractional, e.g. 0.5
mem_limit  = "4G"     # memory limit unit required: G, m, k …
pids_limit = 4960     # maximum number of processes per container (integer)
shm_size   = "512m"   # shared memory unit required

# ── Starting users (all deployment types) ─────────────────────────────────────
# One self-contained [[users]] block per user, hand-edited once at install
# time. Presence in this file is the desired state there are no add/delete
# lists. Additional users added later with `dtaas user add` live in the
# CLI-owned dtaas.users.registry.json instead.
# Usernames must match GitLab accounts and be unique across the array.
#
# email enables traefik-forward-auth routing rules automatically;
# groups/load_balance carry per-user tags. password is optional (used by
# future GitLab-provisioning onboarding); avoid committing a real secret
# here; prefer supplying it at runtime instead.
[[users]]
username     = "alice"
email        = "alice@example.com"
groups       = ["default", "dtaas"]
load_balance = true

[[users]]
username     = "bob"
email        = "bob@example.com"
groups       = ["default", "dtaas"]
load_balance = false

# ── React web client OAuth app (insecure-server, secure-server,
#                                secure-server-gitlab) ────────────────────────
# This is a SEPARATE OAuth application from the traefik-forward-auth app
# configured in [insecure-server] / [secure-server] below.
# Redirect URI: https://<server-dns>/signin-oidc
[frontend]
react-app-client-id = "dtaas-client"
react-app-oauth-url = "https://gitlab.example.com"

# ── localhost deployment (dev / demo only) ────────────────────────────────────
[localhost]
default-user   = "alice"
client-id      = "dtaas-local"
auth-authority = "https://dex.example.com"

# ── insecure-server deployment (HTTP, demo only not internet-facing) ─────────
# GitLab OAuth app for traefik-forward-auth.
# Redirect URI: http://<server-dns>/_oauth
# Scopes: openid profile read_user   Type: Confidential
[insecure-server]
oauth-url           = "https://gitlab.example.com"
oauth-client-id     = "abc123"
oauth-client-secret = "s3cr3t"
oauth-secret        = "random-signing-string"   # random; used to sign session cookies

# ── secure-server deployment (HTTPS/TLS production-ready) ───────────────────
# Same GitLab OAuth app as insecure-server, with Redirect URI using https.
[secure-server]
oauth-url           = "https://gitlab.example.com"
oauth-client-id     = "abc123"
oauth-client-secret = "s3cr3t"
oauth-secret        = "random-signing-string"

# ── secure-server-gitlab deployment (bundled GitLab production-ready) ───────
# oauth-url is omitted; it is derived from the bundled GitLab service.
[secure-server-gitlab]
oauth-client-id     = "abc123"
oauth-client-secret = "s3cr3t"
oauth-secret        = "random-signing-string"

# ── workspace-localhost deployment (Dex on localhost dev / demo only) ───────
[workspace-localhost]
default-user   = "alice"
client-id      = "workspace-local"
auth-authority = "http://localhost:5556/dex"

# ── workspace-secure-server deployment (Keycloak production-ready) ──────────
[workspace-secure-server]
keycloak-admin          = "admin"
keycloak-admin-password = "change-me"
keycloak-realm          = "dtaas"
keycloak-issuer-url     = "https://keycloak.example.com/realms/dtaas"
keycloak-client-id      = "workspace"
keycloak-client-secret  = "s3cr3t"
oauth-secret            = "random-signing-string"
client-id               = "dtaas-frontend"
auth-authority          = "https://keycloak.example.com/realms/dtaas"
```

