DEBOPS.ELASTIC_CO(5)

NAME
debops.elastic_co - Configure Elastic APT repositories

SYNOPSIS

debops service/elastic_co [--limit group,host,...] [--diff] [--check] [--tags tag1,tag2,...] [--skip-tags tag1,tag2,...] [<ansible-playbook options>] ...

DESCRIPTION

The debops.elastic_co Ansible role can be used to configure APT repositories maintained by the Elastic <https://www.elastic.co/about> company on Debian and Ubuntu hosts. The APT repositories are used to distribute elasticsearch, logstash, kibana, filebeat, metricbeat, packetbeat and heartbeat APT packages. The role allows only for installation of packages, additional configuration and management of the installed software is performed by other Ansible roles.

GETTING STARTED

  General role usage

The role will configure an APT repository provided by the Elastic company to allow secure installation of various software packages developed by said company. The GPG key used by the repository is downloaded from a OpenPGP keyserver to ensure authenticity and verification by a third party.

The role itself does not install any APT packages by default. The APT package selection can be performed either using Ansible inventory, or through role dependent variables.

In an example application role, create a YAML list variable that contains APT packages you want to install:

application__elastic_co__dependent_packages:
  - 'elasticsearch'
  - 'kibana'

Then, in either the playbook, or in role dependencies, use that variable with the debops.elastic_co role to specify what APT packages to install:

roles:
  - role: debops.elastic_co
    elastic_co__dependent_packages:
      - '{{ application__elastic_co__dependent_packages }}'

The role will install selected packages automatically after configuring the repository. When this mechanism is used, the Ansible local facts that contain application versions will be correctly updated by the role.

  Ansible local facts

The debops.elastic_co role provides a set of Ansible local facts which can be used by other roles. In the ansible_local.elastic_co.packages YAML dictionary you can find a YAML dictionary which contains all APT package names recognized by the role as keys, and their installed version as values. You can use these facts to for example provide an accurate version number on your role to check against version_compare() Ansible filter:

application__version: '{{ ansible_local.elastic_co.packages.elasticsearch
                          if (ansible_local.elastic_co.packages|d() and
                              ansible_local.elastic_co.packages.elasticsearch|d())
                          else "0.0.0" }}'

You can also use the ansible_local.elastic_co.packages fact to check if a given Elastic application is installed:

application__es_present: '{{ True
                             if (ansible_local.elastic_co.packages|d() and
                                 "elasticsearch" in ansible_local.elastic_co.packages.keys())
                             else False }}'

  Heartbeat package name conflict

The Elastic APT repositories provide the heartbeat package, which installs the Heartbeat <https://www.elastic.co/products/beats/heartbeat> uptime monitoring application.

The Debian Archive contains the heartbeat <https://packages.debian.org/search?keywords=heartbeat> package which provides the heartbeat service which is a part of the Linux High-Availability Stack <http://www.linux-ha.org/wiki/Main_Page>.

This creates a conflict in the APT package manager database. The Elastic developers are aware of the issue <https://github.com/elastic/beats/issues/3765> and are considering a number of options to resolve it. In the meantime, a proposed solution is to use APT pinning to change APT preferences <https://github.com/elastic/beats/issues/3765#issuecomment-289924787> so that the heartbeat package from the Elastic APT repository is installed instead of the Debian Archive version (which is a completely different application).

The research <https://serverfault.com/questions/260152/> performed by one of the cluster software users suggests that the corosync <https://corosync.github.io/corosync/> service is a preferred replacement for the heartbeat service from the HA Stack. The Debian HA Cluster HOWTO <https://wiki.debian.org/Debian-HA/ClustersFromScratch> uses the corosync service as the messaging layer as well.

Due to the above facts the decision was made that the debops.elastic_co role will configure the APT preferences of the hosts it's executed on to prefer the heartbeat package from the Elastic APT repositories, using APT preferences. The example Ansible playbook contains the necessary code which uses the debops.apt_preferences role to perform this task. In a case where you wish to not configure this override, you can disable it by setting the elastic_co__heartbeat_override variable to False. This will affect the list of package versions included in the Ansible local facts.

  Example inventory

To enable configuration of the Elastic APT repositories, you need to add a host to the [debops_service_elastic_co] Ansible inventory group:

[debops_service_elastic_co]
hostname

The role will be automatically used by other Ansible roles that manage Elastic software, therefore the above step is not strictly necessary. Refer to the documentation of these roles for more details.

  Example playbook

If you are using this role without DebOps, here's an example Ansible playbook that uses the debops.elastic_co role:

---
- name: Manage Elastic APT repositories
  collections: [ 'debops.debops', 'debops.roles01',
                 'debops.roles02', 'debops.roles03' ]
  hosts: [ 'debops_service_elastic_co' ]
  become: True
  environment: '{{ inventory__environment | d({})
                   | combine(inventory__group_environment | d({}))
                   | combine(inventory__host_environment  | d({})) }}'
  roles:
    - role: keyring
      tags: [ 'role::keyring', 'skip::keyring', 'role::elastic_co' ]
      keyring__dependent_apt_keys:
        - '{{ elastic_co__keyring__dependent_apt_keys }}'
    - role: apt_preferences
      tags: [ 'role::apt_preferences', 'skip::apt_preferences' ]
      apt_preferences__dependent_list:
        - '{{ elastic_co__apt_preferences__dependent_list }}'
    - role: elastic_co
      tags: [ 'role::elastic_co', 'skip::elastic_co' ]

AUTHOR
Maciej Delmanowski

COPYRIGHT
2014-2022, Maciej Delmanowski, Nick Janetakis, Robin Schneider and others