Metadata-Version: 2.4
Name: cf-remote
Version: 0.9.6
Summary: Tooling to deploy CFEngine (and much more)
Home-page: https://github.com/cfengine/cf-remote
Author: Northern.tech, Inc.
Author-email: contact@northern.tech
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Requires-Python: >=3.5
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: apache-libcloud>=3.3.1
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# cf-remote

`cf-remote` is a tool to deploy CFEngine.
It works by contacting remote hosts with SSH and using `ssh` / `scp` to copy files and run commands.
Commands for provisioning hosts in the cloud (AWS or GCP) are also available.

## Requirements

- cf-remote requires python 3.6 or greater.
- SSH must be configured in such a way that cf-remote can login without a password.
- The account cf-remote logs in as must be root or be able to `sudo`. Passwordless sudo is not required, see [Switching user on the remote hosts](#switching-user-on-the-remote-hosts).
- An sftp server for transferring files on UNIX hosts. e.g. openssh-sftp-server for debian-based distributions.

## Installation

Install with [pipx](https://pipx.pypa.io/):

```bash
pipx install cf-remote
```

Or pip3:

```bash
pip3 install cf-remote
```

Or pip:

```bash
pip install cf-remote
```

## Examples

### See information about remote host

The `info` command can be used to check basic information about a system.
The --hosts/-H option accepts [user@]hostname[:port] for the hostname.
In the case that hostname is an ipv6 address use literal square brackets as described in RFC-3986 (https://www.ietf.org/rfc/rfc3986.txt)

e.g. user@[FEDC:BA98:7654:3210:FEDC:BA98:7654:3210]:8022

```
$ cf-remote info -H 34.241.203.218

ubuntu@34.241.203.218
OS            : ubuntu (debian)
Architecture  : x86_64
CFEngine      : 3.12.1
Policy server : 172.31.42.192
Binaries      : dpkg, apt
```

(You must have ssh access).

### Installing and bootstrapping CFEngine Enterprise Hub

The `install` command can automatically download and install packages as well as bootstrap both hubs and clients.

```
$ cf-remote install --hub 34.247.181.100 --bootstrap 172.31.44.146 --demo

ubuntu@34.247.181.100
OS            : ubuntu (debian)
Architecture  : x86_64
CFEngine      : Not installed
Policy server : None
Binaries      : dpkg, apt

Package already downloaded: '/Users/olehermanse/.cfengine/cf-remote/packages/cfengine-nova-hub_3.12.1-1_amd64.deb'
Copying: '/Users/olehermanse/.cfengine/cf-remote/packages/cfengine-nova-hub_3.12.1-1_amd64.deb' to '34.247.181.100'
Installing: 'cfengine-nova-hub_3.12.1-1_amd64.deb' on '34.247.181.100'
CFEngine 3.12.1 was successfully installed on '34.247.181.100'
Bootstrapping: '34.247.181.100' -> '172.31.44.146'
Bootstrap successful: '34.247.181.100' -> '172.31.44.146'
Transferring def.json to hub: '34.247.181.100'
Copying: '/Users/olehermanse/.cfengine/cf-remote/json/def.json' to '34.247.181.100'
Triggering an agent run on: '34.247.181.100'
Disabling password change on hub: '34.247.181.100'
Triggering an agent run on: '34.247.181.100'
Your demo hub is ready: https://34.247.181.100/ (Username: admin, Password: QxvTmKdLbRsWnp)
```

The username is always `admin`.
The password is randomly generated for each hub.
It is only shown in that last log message, so take note of it.

Note that this demo setup (`--demo`) is notoriously insecure.
It has open access controls.
Don't use it in a production environment.

### Spawning instances in AWS EC2

`cf-remote spawn` can create cloud instances on demand, for example in AWS EC2, but you'll have to add some credentials and settings:

```
$ cf-remote spawn --init-config
Config file /home/olehermanse/.cfengine/cf-remote/cloud_config.json created, please complete the configuration in it.
$ cat /home/olehermanse/.cfengine/cf-remote/cloud_config.json
{
  "aws": {
    "key": "TBD",
    "secret": "TBD",
    "key_pair": "TBD",
    "security_groups": [
      "TBD"
    ],
    "region": "OPTIONAL (DEFAULT: eu-west-1)"
  },
  "gcp": {
    "project_id": "TBD",
    "service_account_id": "TBD",
    "key_path": "TBD",
    "region": "OPTIONAL (DEFAULT: europe-west1-b)"
  }
}
```

You can skip the `gcp` values if you will only be using AWS. After filling out those 4, it should just work:

```
$ cf-remote spawn --count 1 --platform ubuntu-20-04-x64 --role hub --name hub
Spawning VMs....DONE
Waiting for VMs to get IP addresses..........DONE
Details about the spawned VMs can be found in /home/olehermanse/.cfengine/cf-remote/cloud_state.json
```

You can now install nightlies, and use the `--demo` to make testing easier (**Not** secure for production use).
Referring to the group names set by spawn, makes the commands a lot shorter and easier to script:

```
$ cf-remote --version master install --hub hub --bootstrap hub --demo

ubuntu@52.214.209.170
OS            : ubuntu (debian)
Architecture  : x86_64
CFEngine      : Not installed
Policy server : None
Binaries      : dpkg, apt

Downloading package: '/home/olehermanse/.cfengine/cf-remote/packages/cfengine-nova-hub_3.18.0a.a24173342~12762.ubuntu18_amd64.deb'
Copying: '/home/olehermanse/.cfengine/cf-remote/packages/cfengine-nova-hub_3.18.0a.a24173342~12762.ubuntu18_amd64.deb' to 'ubuntu@52.214.209.170'
Installing: 'cfengine-nova-hub_3.18.0a.a24173342~12762.ubuntu18_amd64.deb' on 'ubuntu@52.214.209.170'
CFEngine 3.18.0a.a24173342 (Enterprise) was successfully installed on 'ubuntu@52.214.209.170'
Bootstrapping: '52.214.209.170' -> '172.31.5.84'
Bootstrap successful: '52.214.209.170' -> '172.31.5.84'
Transferring def.json to hub: 'ubuntu@52.214.209.170'
Copying: '/home/olehermanse/.cfengine/cf-remote/json/def.json' to 'ubuntu@52.214.209.170'
Triggering an agent run on: '52.214.209.170'
Disabling password change on hub: 'ubuntu@52.214.209.170'
Triggering an agent run on: '52.214.209.170'
Your demo hub is ready: https://52.214.209.170/ (Username: admin, Password: hJmZqRtvBkNwdc)
```

Mission portal will be available at that IP, using the username and password from the last log message.
The password is randomly generated, so it differs from the one above.

When you are done, you can decommission your spawned instance(s) using:

```
$ cf-remote destroy --all
Destroying all hosts
```

### Deploying a version of masterfiles you're working on locally

The `deploy` command allows you to deploy your local checkout of masterfiles, to test policy while working on it:

```
$ cf-remote deploy --hub hub ~/code/northern.tech/cfengine/masterfiles

ubuntu@18.202.238.128
OS            : ubuntu (debian)
Architecture  : x86_64
CFEngine      : 3.18.0a.a24173342 (Enterprise)
Policy server : None
Binaries      : dpkg, apt

Copying: '/home/olehermanse/.cfengine/cf-remote/masterfiles.tgz' to 'ubuntu@18.202.238.128'
Running: 'systemctl stop cfengine3 && rm -rf /var/cfengine/masterfiles && mv masterfiles /var/cfengine/masterfiles && systemctl start cfengine3 && cf-agent -Kf update.cf && cf-agent -K'
$
```

### Specify an SSH key

If you have more than one key in `~/.ssh` you may need to specify which key `cf-remote` is to use.

```
$ export CF_REMOTE_SSH_KEY="~/.ssh/id_rsa.pub"
```

### Switching user on the remote hosts

Most of what `cf-remote` does needs root, so unless it logs in as root it runs commands through `sudo`.
If `sudo` asks for a password, use `--ask-pass` (`-K`) and `cf-remote` prompts for it once and uses it for all the hosts in the run:

```
$ cf-remote --ask-pass install --clients ubuntu@10.0.0.5
Password for switching user:
```

The password is written to the standard input of the `ssh` process, so it is never part of a command line and doesn't show up in the process list, in the shell history on the target host, or in the output of `--log-level DEBUG`.
It is only sent to hosts where switching user actually asks for a password.

Where there is nobody to answer a prompt, such as in a script or a CI job, put the password on the first line of a file and point `--password-file` at it:

```
$ cf-remote --password-file ~/.cf-remote-password install --clients ubuntu@10.0.0.5
```

`cf-remote` refuses to read the file if others can read it, the same way `ssh` refuses to use a private key with too generous permissions, so `chmod 600` it first.

Use `--switch-user-command` if `sudo` is not what you want to switch user with:

```
$ cf-remote --ask-pass --switch-user-command "doas /bin/sh -c" info -H bsd-host
```

The command to run is appended as a single quoted argument.
The default is `sudo -n bash -c`, or `sudo -S -p '' bash -c` with `--ask-pass`, since `sudo` only reads the password from standard input when it is given `-S`.
`-n` in the first is because there is no terminal to prompt on, so a `sudo` that wants a password should say so instead of trying to ask; it is left out of the second because it means never prompt, and `sudo` then refuses the password rather than reading it.

Whichever command is used, it is run with `LC_ALL=C`.
`cf-remote` recognizes "this needs a password you didn't give me" by what the command said, and `sudo` says it in the caller's language on the distributions that ship its translations, which `ssh` carries over by default.
`sudo` keeps `LC_ALL`, so the command being run is left in the C locale as well; commands run without switching user are not.

A password can only reach a command that reads it from standard input, which in practice means `sudo -S` and the tools that copy its interface, such as `dzdo -S`.
`doas` and `su` read from a terminal instead, so they work with `--switch-user-command` where they need no password, but cannot be given one by `cf-remote`.

### Working on the local host

`cf-remote` can work on the local host when the target host is `localhost`. In this case, it executes commands locally without connecting over SSH.

```
$ cf-remote info -H localhost

ubuntu@localhost
OS            : ubuntu (debian)
Architecture  : x86_64
CFEngine      : 3.12.1
Policy server : 172.31.42.192
Binaries      : dpkg, apt
```

When performing actions locally, `cf-remote` may require your password to run commands with `sudo`:

```
$ cf-remote install --clients localhost
ubuntu@localhost
OS            : debian
Architecture  : x86_64
CFEngine      : Not installed
Policy server :
Binaries      : dpkg, apt
Installing: '/home/ubuntu/.cfengine/cf-remote/packages/cfengine-nova_3.15.3-1.debian10_amd64.deb' on 'localhost'
[sudo] password for ubuntu:
CFEngine 3.15.3 (Enterprise) was successfully installed on 'localhost'
```

## Contribute

Feel free to open pull requests to expand this documentation, add features or fix problems.
You can also pick up an existing task or file an issue in [our bug tracker](https://northerntech.atlassian.net/issues/?filter=10068).

## Development

This project uses [uv](https://docs.astral.sh/uv/) for managing the virtual environment, dependencies, building, etc.
To set up a virtual environment with all dependencies and run all formatters, linters, and tests, use:

```
$ make check
```

To install `cf-remote` so that it reflects any changes in this source directory use:

```
$ pipx install --force --editable .
```

## cloud_data.py tips

In order to find AWS images for a particular owner to work on cloud_data.py name_pattern list the names for an owner with the following `aws` command:

aws ec2 describe-images --region us-east-2 --owners 801119661308 --query 'Images[*].[Name]' --output text
