DEBOPS.REDIS_SENTINEL(5)

NAME
debops.redis_sentinel - Manage Redis Sentinel instances using Ansible

SYNOPSIS

debops service/redis_sentinel [--limit group,host,...] [--diff] [--check] [--tags tag1,tag2,...] [--skip-tags tag1,tag2,...] [<ansible-playbook options>] ...

DESCRIPTION

Redis <https://redis.io/> is an in-memory key/value store, usable as a persistent database, cache or a message broker. Redis Sentinel <https://redis.io/topics/sentinel> manages the failover and high availability of a Redis cluster.

The debops.redis_sentinel Ansible role can be used to install and manage multiple Redis Sentinel instances on Debian/Ubuntu hosts.

GETTING STARTED

  Default configuration

At the initial deployment, the role will disable the default Redis Sentinel instance configuration defined by the distribution packages. Next, a new, systemd-based instance will be created with the configuration like TCP ports and UNIX socket the same as the default Redis Sentinel setup. This should allow easy creation of additional Redis Sentinel instances when necessary.

Keep in mind that once the sentinel.conf configuration file is generated, the role will not modify it. You should design the required Redis Sentinel setup in a development environment, and then deploy it in production. To reconfigure an instance from scratch, you can remove it by setting its state to absent, and then re-create it again.

  Access control and autorization

The debops.redis_server role configures Redis Server instances with a randomly generated password, the same for all instances in the same domain. The password is stored in the secret/ directory on the Ansible Controller (see debops.secret role for details).

The debops.redis_sentinel role will use the same password retrieved from the secret/ directory for the monitor configuration.

Redis password is also exposed in the Ansible local facts, so that other Ansible roles can use it to configure access to Redis by other applications. Run the /etc/ansible/facts.d/redis_sentinel.fact script on the remote host to see the local fact structure and contents.

  Example inventory

To enable Redis Sentinel configuration on a host, it needs to be added to a specific Ansible inventory group:

[debops_service_redis_sentinel]
hostname

By default Redis listens only for local connections on the loopback network interface. If you want to set up a cluster of Redis instances on different hosts that talk to each other, you should configure the default instance to bind to all network interfaces, as well as open the TCP ports in the firewall:

# Listen to TCP connections on all interfaces
redis_sentinel__bind: [ '0.0.0.0', '::' ]
# Allow connections to Redis from specific subnets
redis_sentinel__allow: [ '192.0.2.0/24', '2001:db8::/32' ]

You should also tell the default monitor to use the public IP address of a host instead of localhost, this address is shared between Sentinel instances - you should ensure that each instance sees has the same "view" of the Redis cluster:

redis_sentinel__monitors:
  - name: 'redis-ha'
    host: '{{ ansible_fqdn }}'

  Example playbook

If you are using this role without DebOps, here's an example Ansible playbook that uses the debops.redis_sentinel role:

---
- name: Manage Redis Sentinel service
  collections: [ 'debops.debops', 'debops.roles01',
                 'debops.roles02', 'debops.roles03' ]
  hosts: [ 'debops_service_redis_sentinel' ]
  become: True
  environment: '{{ inventory__environment | d({})
                   | combine(inventory__group_environment | d({}))
                   | combine(inventory__host_environment  | d({})) }}'
  pre_tasks:
    - name: Prepare redis_sentinel environment
      import_role:
        name: 'redis_sentinel'
        tasks_from: 'main_env'
      tags: [ 'role::redis_sentinel', 'role::ferm' ]
  roles:
    - role: apt_preferences
      tags: [ 'role::apt_preferences', 'skip::apt_preferences' ]
      apt_preferences__dependent_list:
        - '{{ redis_sentinel__apt_preferences__dependent_list }}'
    - role: etc_services
      tags: [ 'role::etc_services', 'skip::etc_services' ]
      etc_services__dependent_list:
        - '{{ redis_sentinel__etc_services__dependent_list }}'
    - role: ferm
      tags: [ 'role::ferm', 'skip::ferm' ]
      ferm__dependent_rules:
        - '{{ redis_sentinel__ferm__dependent_rules }}'
    - role: python
      tags: [ 'role::python', 'skip::python', 'role::redis_sentinel' ]
      python__dependent_packages3:
        - '{{ redis_sentinel__python__dependent_packages3 }}'
      python__dependent_packages2:
        - '{{ redis_sentinel__python__dependent_packages2 }}'
    - role: redis_sentinel
      tags: [ 'role::redis_sentinel', 'skip::redis_sentinel' ]

  Ansible tags

You can use Ansible --tags or --skip-tags parameters to limit what tasks are performed during Ansible run. This can be used after a host was first configured to speed up playbook execution, when you are sure that most of the configuration is already in the desired state.

Available role tags:

role::redis_sentinel
    Main role tag, should be used in the playbook to execute all of the role tasks as well as role dependencies.

  Other resources

List of other useful resources related to the debops.redis_sentinel Ansible role:

•   Official Redis Sentinel documentation <https://redis.io/topics/sentinel>

•   Example Redis Sentinel config file <http://download.redis.io/redis-stable/sentinel.conf>

REDIS SENTINEL CONFIGURATION PIPELINE

The default Redis Sentinel installation in Debian Jessie and Debian Stretch supports only 1 instance of Redis per host. The pacakges in Debian Buster and the stretch-backports repository support multiple instances by using a single /etc/redis/sentinel-<instance>.conf configuration file per a systemd instance. However, due to the Redis modifying its own configuration file on the fly, using a single sentinel.conf configuration file does not work well with an Ansible-based approach to configuration.

The solution to this problem implemented in debops.redis_sentinel role is usage of a separate /etc/redis/sentinel-<instance>/ directory for each Redis Sentinel instance. This allows usage of multiple configuration files and even scripts for each Redis Sentinel instance, with configuration applied dynamically at runtime. The sentinel.conf configuration file is generated by Ansible at the instance initialization and not touched after that, since it is modified directly by Redis. This ensures idempotency and allows Ansible and Redis to work together.

  Configuration variables

The debops.redis_sentinel Ansible role exposes a set of default variables that can be used to define and modify Redis configuration per instance. Configuration defined in each one is merged together in the redis_sentinel__combined_configuration using a special filter plugin. Multiple configuration entries defined in the format of the redis_sentinel__configuration parameters are merged together, therefore there's no need to copy everything to the Ansible inventory.

The variables are merged in the following order:

•   the redis_sentinel__default_base_options and the redis_sentinel__base_options hold the default parameters applied to all of the Redis Sentinel instances on a particular host. These variables can be used to override options applied to all instances when needed.

•   the redis_sentinel__default_instances and the all/group/host variant of the same variable are used to generate configuration for each instance, which is then put in the configuration pipeline via the redis_sentinel__default_configuration variable. Each instance will include the base options defined for all instances, and per-instance configuration like port, UNIX socket path, optional systemd overrides, etc.

    Additionally, each instance checks the redis_sentinel__default_monitors variable and the all/group/host variant of the same variable to include any monitors defined in them. Monitors that are confined to a particular Redis Sentinel instance are not included in others.

•   the redis_sentinel__default_configuration and the all/group/host variants include the actual configuration used by the role to generate the Redis Sentinel configuration files, systemd service configuration. The variables are joined together in the redis_sentinel__combined_configuration variable which is used in varius role tasks and templates. These variables can be used to override per-instance configuration if needed.

  Configuration file structure

The generated configuration file structure contains the following files:

/etc/redis
├── sentinel-main/
│   ├── notify.d/
│   ├── reconfig.d/
│   ├── notify.sh*
│   ├── reconfig.sh*
│   └── sentinel.conf
├── sentinel-second/
│   ├── notify.d/
│   ├── reconfig.d/
│   ├── notify.sh*
│   ├── reconfig.sh*
│   └── sentinel.conf
├── sentinel-third/
│   ├── notify.d/
│   ├── reconfig.d/
│   ├── notify.sh*
│   ├── reconfig.sh*
│   └── sentinel.conf
└── sentinel.conf

The sentinel.conf file in each subdirectory is generated by Ansible at instance initialization. It will not be touched by Ansible later on, because Redis Sentinel modifies it directly.

The reconfig.sh and notify.sh scripts are hooks for the client-reconfig-script and notification-script parameters. They will run all scripts in their corresponding directories via the run-parts command. You can put custom scripts in these subdirectories to perform actions on certain Redis Sentinel events; see the Sentinel documentation for more details.

DEFAULT VARIABLE DETAILS

Some of debops.redis_sentinel default variables have more extensive configuration than simple strings or lists, here you can find documentation and examples for them.

  redis_sentinel__instances

The role can manage multiple Redis Sentinel instances on a single host via the redis_sentinel__*_instances default variables. Each variable is a list of YAML dictionaries, each dictionary defines an instance of Redis Sentinel managed by systemd unit template.

Configuration specified in the instance YAML dictionary is parsed by the role and used to generate the final configuration which is then used to manage the Redis Sentinel instances (see Redis Sentinel configuration pipeline).

Multiple dictionaries with the same name parameter will be merged together; this can be used to override previously defined instance configuration without copying everything to the Ansible inventory.

  Examples

Define multiple Redis Sentinel instances:

---
# Configure all Redis Sentinel instances to listen for network connections
redis_sentinel__bind: [ '0.0.0.0', '::' ]
# Allow connections from specific subnets to all Redis Sentinel instances
redis_sentinel__allow: [ '192.0.2.0/24', '2001:db8::/32' ]
# Define minimal parameters for additional Redis Sentinel instances with
# autogenerated configuration
redis_sentinel__instances:
  - name: 'second'
    port: '6380'
  - name: 'third'
    port: '6381'

Modify existing instance configuration:

---
# Change the default bind of the specific Redis Sentinel instance 'main' to
# listen on all network interfaces and configure it to monitor a distributed
# Redis setup
redis_sentinel__instances:
  - name: 'main'
    bind: [ '0.0.0.0', '::' ]
redis_sentinel__monitors:
  - name: 'redis-ha'
    host: '{{ ansible_fqdn }}'

  Syntax

Each entry can contain specific parameters:

name
    Required. The name of a given Redis Sentinel instance. This parameter is used as an anchor for merging of multiple YAML dictionaries that specify Redis Sentinel instances together.

    The instance name main is significant and used in Ansible local fact script to denote the "default" Redis Sentinel instance if none is specified.

port
    Required. The TCP port on which a given instance listens for network connections. Only ports defined in the instance list will be included in the automatically managed firewall configuration.

state
    Optional. If not specified or present, a given Redis Sentinel instance will be created or managed by the role. If absent, a given instance will be removed by the role. If ignore, a given instance entry will not be included in the configuration.

pidfile
    Optional. Absolute path to a PID file of a given Redis Sentinel instance. If not specified, the role will generate one based on the instance name.

unixsocket
    Optional. Absolute path to an UNIX socket file of a given Redis Sentinel instance. If not specified, the role will generate one based on the instance name.

bind
    Optional. A string or a YAML list of IP addresses to which a given Redis Sentinel instance should bind to to listen for network connections. If not specified, the instance will bind on the IP addresses specified in the redis_sentinel__bind variable, by default localhost.

logfile
    Optional. Absolute path to a log file of a given Redis Sentinel instance. If not specified, the role will generate one based on the instance name.

syslog_ident
    Optional. A short string that identifies a given Redis Sentinel instance in the syslog stream. If not specified, the role will generate one based on the instance name.

systemd_override
    Optional. An YAML text block that contains systemd unit configuration entries. This can be used to override the configuration of a Redis Sentinel instance managed by systemd.

    Other configuration options for a given Redis Sentinel instance should be specified in the redis_sentinel__*_configuration variables. Some of the instance parameters like port are used in other parts of the role and should be overridden only on the list of instances.

  redis_sentinel__monitors

Redis Sentinel uses "monitors" to track the state of Redis Server instances. The monitors can be defined using the redis_sentinel__*_monitors default variables; each variable is a list of YAML dictionaries. Multiple entries with the same name parameter are combined together, you don't need to copy the entire entry to the Ansible inventory to modify it.

The Redis Sentinel configuration files are generated only once at the initialization of a given instance. After that, Redis Sentinel modifies these files directly, therefore Ansible will not try to change them as long as they are present. You should prepare an adequate monitor configuration beforehand.

  Examples

Define additional monitors for all Redis Sentinel instances:

---
# Enable additional monitors for non-default Redis Server instances.
redis_sentinel__monitors:
  - name: 'redis-ha-second'
    host: 'redis.example.org'
    port: '6380'
    quorum: 2
    'failover-timeout': 180000
    'down-after-miliseconds': 30000
  - name: 'redis-ha-third'
    host: 'redis.example.org'
    port: '6381'
    quorum: 2

  Syntax

Each Redis Sentinel monitor entry can be defined using specific parameters:

name
    Required. The name of a given monitor. This parameter is used as an anchor to combine multiple entries together.

host
    Required. FQDN or IP address of the Redis Server master which will be monitored.

port
    Required. The TCP port on which a given Redis Server master listens for connections.

quorum
    Required. Number of Redis Sentinel instances that are expected to be in agreement about the state of Redis Server that is being monitored. It's usually number of Redis Sentinel instances / 2 + 1. This parameter is currently not computed automatically.

password
    Optional. If not specified, the role will set the value of the redis_sentinel__auth_password as the password used by a given monitor to access the Redis Server, as the auth-pass option.

    If False, the password is not set automatically for a given monitor.

instance
    Optional. Name of the Redis Sentinel instance in which to define a given monitor. If not specified, the monitor will be defined in all instances on a given host.

state
    Optional. If not specified or present, a given monitor will be defined by the role. If absent, a given monitor will not be defined in the initial configuration file. If ignore, a given monitor entry will not be included in the configuration and will be ignored by the role.

notification-script, client-reconfig-script
    These parameters are automatically configured to point to the custom scripts inside of a given Redis Sentinel instance configuration directory. You can place custom scripts in the corresponding notify.d/ and reconfig.d/ subdirectories, they will be executed by Redis Sentinel using the run-parts command.

    All other parameters specified in a given monitor entry will be added as-is in the Redis Sentinel configuration file.

  redis_sentinel__configuration

The redis_sentinel__*_configuration variables define the configuration of the Redis Sentinel instances. A Redis Sentinel instance consists of a set of configuration files in /etc/redis/sentinel-<instance>/ subdirectory, as well as systemd service template configuration. See Redis Sentinel configuration pipeline for more details.

The Redis Sentinel configuration files are generated only once at the initialization of a given instance. After that, Redis Sentinel modifies these files directly, therefore Ansible will not try to change them as long as they are present.

  Examples

Define additional instance configuration:

---
# Copyright (C) 2018 Maciej Delmanowski <drybjed@gmail.com>
# Copyright (C) 2018 DebOps <https://debops.org/>
# SPDX-License-Identifier: GPL-3.0-only
# Modify the configuration of a generated Sentinel instance
redis_sentinel__configuration:
  - name: 'main'
    options:
      # Allow modification of the reconfigure and notify script paths at
      # runtime. Warning, this might result in a security issue.
      - 'deny-scripts-reconfig': False
      # Change the syslog identifier of a particular Sentinel instance. This
      # parameter needs to be specified with an empty prefix to not prepend the
      # 'sentinel ' string.
      - name: 'syslog-ident'
        value: 'sentinel-main-instance'
        prefix: ''
      # Change the reconfiguration script of the default monitor
      - name: 'client-reconfig-script redis-ha'
        value: '/tmp/sentinel-reconfig.sh'

  Syntax

Each variable contains a list of YAML dictionaries, each dictionary defines a Redis Sentinel instance using specific parameters:

name
    Required. Name of a given Redis Sentinel instance, should be a short alphanumeric string. This parameter is used as an anchor to merge multiple instance entries together.

port
    Required. The TCP port on which a given instance listens for network connections.

state
    Optional. If not specified or present, a given Redis Sentinel instance will be created and/or managed on a given host. If absent, a given Redis Sentinel instance will be stopped and its configuration will be removed. If ignore, a given configuration entry will be ignored by the role during the Ansible run; this can be used to conditionally enable or disable instance options if needed.

systemd_override
    Optional. An YAML text block that contains systemd unit configuration entries. This can be used to override the configuration of a Redis Sentinel instance managed by systemd.

options
    A list of configuration options for the Redis Sentinel instance. The options lists from multiple instance configuration entries are merged together. Each element of the options list is a YAML dictionary with specific parameters:

name
    Redis Sentinel parameter name. Parameter names containing hypens should be quoted to avoid any issues with YAML parsing.

value
    Redis Sentinel parameter value. It can be a number, a string or a list of strings. If a list is used, by default the configuration file will contain multiple parameters with the same name and values specified on separate lines. you can also use the Python True and False values to represent booleans.

state
    Optional. If not specified or present, a given parameter will be present in the generated configuration. If absent, a given parameter will be removed from the configuration.

prefix
    Optional. Specify a custom prefix for a given parameter. By default, Redis Sentinel configuration options are specified with the sentinel string, space as a separator needs to be included as well; some of the configuration options related to Redis daemon don't contain the prefix. If you need to add an option without the prefix, set this parameter to an empty string.

separator
    Optional, boolean. Add an empty line before a given option. This is a cosmetic parameter, which allows for better readability of the generated configuration file.

multiple
    Optional, boolean. If False, and the parameter is not dynamic, and it's a list, the values will be concatenated into one string, separated by spaces. This is required by some of the Redis Server configuration options, for example bind.

    If the name and value parameters are not present, each key of the YAML dictionary will be interpreted as a separate Redis Sentinel parameter. This can be used as a shorthand to define Redis Sentinel parameters, but for more complicated parameters (dynamic, with custom requirements), you should use the expanded form explained above.

AUTHOR
Maciej Delmanowski

COPYRIGHT
2014-2022, Maciej Delmanowski, Nick Janetakis, Robin Schneider and others