DEBOPS.RABBITMQ_SERVER(5)

NAME
debops.rabbitmq_server - Manage RabbitMQ service using Ansible

SYNOPSIS

debops service/rabbitmq_server [--limit group,host,...] [--diff] [--check] [--tags tag1,tag2,...] [--skip-tags tag1,tag2,...] [<ansible-playbook options>] ...

DESCRIPTION

RabbitMQ <https://www.rabbitmq.com/> is an Open Source message broker which supports AMQP <https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol>, STOMP <https://en.wikipedia.org/wiki/Streaming_Text_Oriented_Messaging_Protocol> and MQTT <https://en.wikipedia.org/wiki/MQTT> protocols.

This Ansible role can be used to install and manage RabbitMQ service on Debian/Ubuntu hosts.

GETTING STARTED

  Shared Erlang cookie

The role configures the same Erlang cookie (password) on all hosts managed in a given environment with an idea that all of the hosts are joined in the same RabbitMQ cluster. If you want to do things differently, change the rabbitmq_server__erlang_cookie_password as needed.

  Erlang 19.x from 'jessie-backports' on Debian Jessie

On Debian Jessie hosts, the role will configure an APT preference for backported Erlang 19.x packages from Debian Stretch. They provide better Elliptic Curve Cryptography (ECC) support and allow deactivation of TLS client-initiated protocol renegotiation, which mitigates potential DoS attacks.

  Encrypted client connections

The role will check if the debops.pki and debops.dhparam Ansible roles configured their environment on a host, and will automatically enable or disable support for encrypted AMQP connections. Plaintext connections will be available if encryption is disabled.

  RabbitMQ clustering

By default the debops.rabbitmq_server role configures RabbitMQ service in a standalone mode, without external access through the firewall. To allow for clustering, you need to define IP addresses and/or CIDR subnets, which will be allowed to connect to the epmd (Erlang Port Mapper Daemon) and einc (Erlang Inter-Process Communication) TCP ports. To do that, set the variable below in the Ansible inventory:

---
# Allow for cluster communication
rabbitmq_server__cluster_allow: [ '192.0.2.0/24' ]

After that, re-run the role to apply changes to the firewall configuration.

At the moment role does not create clusters automatically. To create a cluster manually using three hosts (host1, host2, host3) with host1 being the main cluster node, login to the other hosts and using the root account, run the commands:

rabbitmqctl stop_app
rabbitmqctl join_cluster rabbit@host1
rabbitmqctl start_app

You can check the RabbitMQ cluster status by running the command:

rabbitmqctl cluster_status

See the RabbitMQ Clustering Guide <https://www.rabbitmq.com/clustering.html> for more details.

  Inter-node communication is not encrypted

Erlang supports encrypting communication between nodes (processes on the same or other hosts) using TLS, which RabbitMQ can use to secure traffic between hosts <https://www.rabbitmq.com/clustering-ssl.html>. However one downside is that when inter-node traffic is encrypted, Erlang uses dynamic random ports <https://groups.google.com/forum/#!msg/rabbitmq-users/rJaJWctOYKQ/q5nP2Cb-5k0J> for communication, which might interfere with the host's firewall. Therefore by default debops.rabbitmq_server role does not configure encrypted inter-node communication. You should consider alternative means of securing the traffic between hosts, for example a separate VLAN or use of a VPN connection.

  Example inventory

To configure RabbitMQ on a host, it should be added to the [debops_service_rabbitmq_server] Ansible inventory group:

[debops_service_rabbitmq_server]
hostname

  Example playbook

If you are using this role without DebOps, here's an example Ansible playbook that uses the debops.rabbitmq_server role:

---
- name: Manage RabbitMQ service
  collections: [ 'debops.debops', 'debops.roles01',
                 'debops.roles02', 'debops.roles03' ]
  hosts: [ 'debops_service_rabbitmq_server' ]
  become: True
  environment: '{{ inventory__environment | d({})
                   | combine(inventory__group_environment | d({}))
                   | combine(inventory__host_environment  | d({})) }}'
  pre_tasks:
    - name: Prepare rabbitmq_server environment
      import_role:
        name: 'rabbitmq_server'
        tasks_from: 'main_env'
      tags: [ 'role::rabbitmq_server', 'role::secret', 'role::rabbitmq_server:config' ]
  roles:
    - role: secret
      tags: [ 'role::secret', 'role::rabbitmq_server', 'role::rabbitmq_server:config' ]
      secret__directories:
        - '{{ rabbitmq_server__secret__directories }}'
    - role: apt_preferences
      tags: [ 'role::apt_preferences', 'skip::apt_preferences' ]
      apt_preferences__dependent_list:
        - '{{ rabbitmq_server__apt_preferences__dependent_list }}'
    - role: etc_services
      tags: [ 'role::etc_services', 'skip::etc_services' ]
      etc_services__dependent_list:
        - '{{ rabbitmq_server__etc_services__dependent_list }}'
    - role: ferm
      tags: [ 'role::ferm', 'skip::ferm' ]
      ferm__dependent_rules:
        - '{{ rabbitmq_server__ferm__dependent_rules }}'
    - role: rabbitmq_server
      tags: [ 'role::rabbitmq_server', 'skip::rabbitmq_server' ]

USAGE AS A ROLE DEPENDENCY

The debops.rabbitmq_server role can be used as a dependency by other Ansible roles to manage RabbitMQ main configuration file idempotently. Configuration options from multiple roles can be merged together and included in the configuration file, or removed conditionally.

  Dependent role variables

The role exposes three default variables that can be used by other Ansible roles as dependent variables:

rabbitmq_server__dependent_role
    Required. Name of the role that uses the debops.rabbitmq_server as a dependency. This will be used to store the configuration in its own YAML dictionary. The selected name shouldn't be changed, otherwise configuration will be desynchronized.

rabbitmq_server__dependent_config
    Required. List of the RabbitMQ configuration options defined in the same format as the main configuration. See rabbitmq_server__config for more details.

rabbitmq_server__dependent_state
    Optional. If not specified or present, the configuration will be included in the /etc/rabbitmq/rabbitmq.config configuration file and stored as Ansible local fact. if absent, the configuration will be removed from the generated configuration file.

  Dependent configuration storage and retrieval

The dependent configuration from other roles is stored in the secret/ directory on the Ansible Controller (see debops.secret for more details) in a JSON file, with each role configuration in a separate dictionary. The debops.rabbitmq_server role reads this file when Ansible local facts indicate that the RabbitMQ service is installed, otherwise a new empty file is created. This ensures that the stale configuration is not present on a new or re-installed host.

The YAML dictionaries from different roles are be merged with the main configuration in the rabbitmq_server__combined_config variable that is used to generate the final configuration. The merge order of the different rabbitmq_server__*_config variables allows to further affect the dependent configuration through Ansible inventory if necessary, therefore the Ansible roles that use this method don't need to provide additional variables for this purpose themselves.

  Example role default variables

---
# This is a set of default variables in an example 'application' role that uses
# dependent variables to pass configuration to 'debops.rabbitmq_server' role.
# State of the application deployment
application__deploy_state: 'present'
# RabbitMQ configuration defined by the application
application__rabbitmq_server__dependent_config:
  - name: 'application_name'
    options:
      - name: 'config_first_option'
        value: 'value1'
      - config_second_option: 'value2'

  Example role playbook

---
# This is a playbook for an example 'application' role which uses
# 'debops.rabbitmq_server' as a dependency and passes its own set of
# configuration options to it.
- name: Manage application
  collections: [ 'debops.debops' ]
  hosts: [ 'debops_service_rabbitmq_application' ]
  become: True
  environment: '{{ inventory__environment | d({})
                   | combine(inventory__group_environment | d({}))
                   | combine(inventory__host_environment  | d({})) }}'
  pre_tasks:
    - name: Prepare rabbitmq_server environment
      import_role:
        name: 'rabbitmq_server'
        tasks_from: 'main_env'
      tags: [ 'role::rabbitmq_server', 'role::secret', 'role::rabbitmq_server:config' ]
  roles:
    - role: secret
      tags: [ 'role::secret', 'role::rabbitmq_server', 'role::rabbitmq_server:config' ]
      secret__directories:
        - '{{ rabbitmq_server__secret__directories }}'
    - role: rabbitmq_server
      tags: [ 'role::rabbitmq_server' ]
      rabbitmq_server__dependent_role: 'application'
      rabbitmq_server__dependent_state: '{{ application__deploy_state }}'
      rabbitmq_server__dependent_config:
        - '{{ application__rabbitmq_server__dependent_config }}'
    - role: application
      tags: [ 'role::application' ]

DEFAULT VARIABLE DETAILS

Some of the debops.rabbitmq_server default variables have more extensive configuration than simple strings or lists, here you can find documentation and examples for them.

  rabbitmq_server__config

The rabbitmq_server__*_config variables describe contents of the /etc/rabbitmq/rabbitmq.config configuration file. Each entry in the rabbitmq_server__*_config variables is a YAML dictionary with specific parameters:

name
    Required. The name of an Erlang application to configure. Each application can contain a set of configuration options. Configuration options from multiple applications with the same name parameter are merged together.

state
    Optional. If not specified or present, a given application configuration will be included in the finished configuration file.

    If absent, a given application configuration will be removed from the configuration file.

    If ignore, a given application entry is not evaluated by the configuration template. This can be used to conditionally enable or disable configuration sections.

comment
    Optional. A string or YAML text block which will be added as a comment to the configuration section.

weight
    Optional. A positive or negative number which will be used to affect the position of a given Erlang application within the configuration file. The higher the number, the more a given application section "weighs", and therefore it will be placed lower in the finished configuration file. If not specified, 0 is used by default.

options
    A YAML list of configuration options for a given Erlang application. See RabbitMQ configuration options for more details.

  Examples

---
# Create a basic set of Erlang applications used by RabbitMQ, based on the
# example configuration file:
rabbitmq_server__config:
  - name: 'rabbit'
    weight: 1
  - name: 'rabbitmq_management'
    comment: |
      RabbitMQ Management Plugin
      See https://www.rabbitmq.com/management.html for details
    options: []
    weight: 2
  - name: 'rabbitmq_management_agent'
    weight: 3

  RabbitMQ configuration options

RabbitMQ is written in the Erlang <https://en.wikipedia.org/wiki/Erlang_(programming_language)> programming language, which is also used for its configuration. YAML, used by Ansible, does not provide enough data types to directly map them to the Erlang data types <https://erlang.org/doc/reference_manual/data_types.html> used in the RabbitMQ configuration file, therefore the configuration used by debops.rabbitmq_server focuses on description of the desired data types and conditional activation of the configuration sections. This means that simple values like strings, numbers, lists are mapped directly, however more complex configuration needs to be written in Erlang using YAML text blocks. The role tries to detect the value type automatically, but in some cases you might need to use the extended YAML dictionary syntax described below.

The role does not provide original configuration variables due to the issues with template generation (commented out options are not supported). You can find a reference RabbitMQ configuration file after the service installation, in the /usr/share/doc/rabbitmq-server/rabbitmq.config.example.gz file. An example rabbitmq.config file <https://github.com/rabbitmq/rabbitmq-server/blob/master/docs/rabbitmq.config.example> is also available online.

RabbitMQ configuration options are included in the options parameter of an Erlang application section (see rabbitmq_server__config for more details). The options parameter is a YAML list, each entry is a YAML dictionary. The dictionary keys are used as option names, and dictionary values are used as option values. You can specify simple options this way:

---
# Example of a set of simple RabbitMQ options
rabbitmq_server__config:
  - name: 'rabbit'
    options:
      # String
      - example_option: 'value'
      # Simple list
      - tcp_listeners: [ 5672 ]
      # Boolean value
      - reverse_dns_lookups: True
      # Numbers
      - vm_memory_high_watermark: 0.4
      # Raw Erlang code (note absence of }, at the end)
      - tcp_listeners: |
          [{"127.0.0.1", 5672},
           {"::1",       5672}]

If a given dictionary contains a name parameter, the configuration template will switch to a more verbose option interpretation, using known parameters:

name
    The name of a given configuration option. Multiple entries with the same name are merged together, with the latter ones takim precedence over the former.

value
    Required. A value to set for a given option. The value can be an YAML string, a list, number, boolean.

    YAML text block is used to indicate a raw Erlang code which should be used as a value. The raw Erlang code should not end with any flow control Erlang characters (} or },), they will be added automatically by the role.

type
    Optional. Specify the type of a given value to use. If the type parameter is not specified, the template will try to select one based on the YAML value type. Supported value types:

•   string: a quoted string, selected automatically if a YAML string is used as the value;

•   list: a list of values, selected automatically if a YAML list is used as the value;

•   number: an unquoted number, selected automatically if a YAML number or float is used as the value;

•   boolean: a boolean true/false value, selected automatically if a YAML boolean is used as the value;

•   bit-string: a bit string <https://erlang.org/doc/reference_manual/data_types.html#bit-strings-and-binaries> value with special quotation marks. Only YAML strings are supported at this time;

•   bit-list: a list of bit-strings with special quotation marks. Only YAML strings are supported at this time. if the value type is set as bit-string and a YAML list is set, the role should change to a bit-list type automatically;

•   raw: a raw Erlang expression, inserted in the finished configuration file as-is. The Erlang code should not end with Erlang flow control characters } or },, they will be added automatically by the role. if the value is specified using a YAML text block, the raw type should be selected automatically, based on the number of lines used in the value;

option
    Optional. If specified, the configuration option will use this value for the name instead of name.

state
    Optional. If not specified or present, a given option be included in the finished configuration file.

    If absent, a given option will be removed from the configuration file.

    If ignore, a given option entry is not evaluated by the configuration template. This can be used to conditionally enable or disable configuration options.

comment
    Optional. A string or YAML text block which will be added as a comment to the configuration option.

weight
    Optional. A positive or negative number which will be used to affect the position of a given option within the configuration file. The higher the number, the more a given option "weighs", and therefore it will be placed lower in the finished configuration file. If not specified, 0 is used by default.

  Examples

---
# Example of a set of verbose RabbitMQ options
rabbitmq_server__config:
  - name: 'rabbit'
    options:
      # String
      - name: 'example_option'
        value: 'value'
        type: 'string'
      # Simple list
      - name: 'tcp_listeners'
        value: [ 5672 ]
        type: 'list'
      # Boolean value
      - name: 'reverse_dns_lookups'
        value: True
      # Numbers
      - name: 'vm_memroy_high_watermark'
        value: 0.4
      # Bit-string (result: '<<"bit-string">>')
      - name: 'bit_option'
        value: 'bit-value'
        type: 'bit-string'
      # Bit-list (result: '[<<".*">>, <<".*">>, <<".*">>]')
      - name: 'default_permissions'
        value: [ '.*', '.*', '.*' ]
        type: 'bit-list'
      # Raw Erlang code (note absence of }, at the end)
      - name: 'tcp_listeners'
        value: |
          [{"127.0.0.1", 5672},
           {"::1",       5672}]
        type: 'raw'

  rabbitmq_server__plugins

The rabbitmq_server__*_plugins lists can be used to enable or disable RabbitMQ plugins conditionally. You can find the available plugins on a givem host by running the command:

rabbitmq-plugins list

Each list entry is either a RabbitMQ plugin name, or a YAML dictionary with specific parameters:

name
    The name of a RabbitMQ plugin to manage.

state
    Optional. If not defined or present, the plugin will be enabled. If absent, the plugin will be disabled.

prefix
    Optional. Custom install prefix to a Rabbit.

  Examples

Enable the RabbitMQ Management Console agent:

rabbitmq_server__plugins:
  - 'rabbitmq_management_agent'

  rabbitmq_server__accounts

The rabbitmq_server__*_accounts list variables can be used to manage RabbitMQ user accounts. Each list entry is a YAML dictionary with specific parameters. The parameter names are the same as the rabbitmq_user Ansible module. Some more common parameters:

user or name
    The name of a given user account.

state
    Optional. If not specified or present, the user account will be created. If absent, the user account will be removed.

password
    Optional. Plaintext password of a given user account. If not specified, the role will generate a random password and store it in the secret/rabbitmq_server/accounts/ directory on the Ansible Controller. See debops.secret Ansible role for more details.

tags
    Optional. A string or a YAML list of tags <https://www.rabbitmq.com/management.html> assigned to a given account. Possible choices: management, policymaker, monitoring, administrator.

vhost
    Optional. Name of the virtual host to which a given set of permissions should apply. If not specified, / vhost is used by default.

configure_priv, read_priv, write_priv
    Optional. A regular expression which defines what resources on a given virtual host the user can configure, read from or write to. By default the ^$ regexp is used which means no permissions are given to any resources on a virtual host.

  Examples

Create an administrator account and a regular user account:

rabbitmq_server__accounts:
  - name: 'admin_account'
    vhost: '/'
    tags: [ 'administrator' ]
    configure_priv: '.*'
    read_priv: '.*'
    write_priv: '.*'
  - name: 'user_account'
    vhost: '/'
    read_priv: '.*'
    write_priv: '.*'

  rabbitmq_server__vhosts

The rabbitmq_server__*_vhosts list variables can be used to manage RabbitMQ virtual hosts. Each list entry is a YAML dictionary with specific parameters. The parameter names are the same as the rabbitmq_vhost Ansible module. Some more common parameters:

name
    The name of a given virtual host. If not specified, the whole list entry will be used as the name (see examples).

state
    Optional. If not specified or present, the virtual host will be created. If absent, the virtual host will be removed.

tracing
    Optional. Enable message tracing in a given virtual host.

  Examples

Create a set of virtual hosts:

rabbitmq_server__vhosts:
  - 'vhost1'
  - 'vhost2'
  - name: 'vhost3'
    state: 'absent'

  rabbitmq_server__parameters

The rabbitmq_server__*_parameters list variables can be used to manage RabbitMQ parameters <https://www.rabbitmq.com/parameters.html>. Each list entry is a YAML dictionary with specific parameters. The parameter names are the same as the rabbitmq_parameter Ansible module. Some more common parameters:

component
    Required. Name of the component of which the parameter is being set.

name
    Required. The name of a given RabbitMQ parameter being set.

value
    The value of a given parameter in a JSON format. The values are usually quoted using single quotes and contain double-quotes.

vhost
    Optional. Specify the RabbitMQ virtual host to which a given parameter applies.

state
    Optional. If not specified or present, the parameter will be created. If absent, the parameter will be removed.

  Examples

Define a RabbitMQ parameter:

rabbitmq_server__parameters:
  - component: 'federation'
    name: 'local-username'
    value: '"guest"'

  rabbitmq_server__policies

The rabbitmq_server__*_policies list variables can be used to manage RabbitMQ policies <https://www.rabbitmq.com/parameters.html>. Each list entry is a YAML dictionary with specific parameters. The parameter names are the same as the rabbitmq_policy Ansible module. Some more common parameters:

name
    Required. The name of a given RabbitMQ policy.

pattern
    Required. A regexp pattern of RabbitMQ queue names to which a given policy applies.

tags
    Required. An YAML dictionary with key/value parameters that describe the policy. Relevant documentation can be found in the RabbitMQ Management Console, Admin section, Policies.

vhost
    Optional. Specify the RabbitMQ virtual host to which a given policy applies.

apply_to
    Optional. The resource type to which a given policy applies to. Supported choices: all, exchanges, queues. If not specified, all is used by default.

state
    Optional. If not specified or present, the policy will be created. If absent, the policy will be removed.

priority
    Optional. The numerical priority of a given policy, used for sorting.

  Examples

Create a set of RabbitMQ policies:

rabbitmq_server__policies:
  - name: 'HA'
    pattern: '.*'
    tags:
      'ha-mode': 'all'

AUTHOR
Maciej Delmanowski

COPYRIGHT
2014-2022, Maciej Delmanowski, Nick Janetakis, Robin Schneider and others