DEBOPS.REDIS_SERVER(5)

NAME
debops.redis_server - Manage Redis Server instances using Ansible

SYNOPSIS

debops service/redis_server [--limit group,host,...] [--diff] [--check] [--tags tag1,tag2,...] [--skip-tags tag1,tag2,...] [<ansible-playbook options>] ...

DESCRIPTION

Redis <https://redis.io/> is an in-memory key/value store, usable as a persistent database, cache or a message broker.

The debops.redis_server Ansible role can be used to install and manage Redis on Debian/Ubuntu hosts. Role supports management of multiple Redis instances on a single host and is designed to cope with modifications done to Redis configuration files at runtime.

GETTING STARTED

  Default configuration

At the initial deployment, the role will disable the default Redis instance configuration defined by the distribution packages. Next, a new, systemd-based instance will be created with the configuration like TCP ports and UNIX socket the same as the default Redis Server setup. This should allow easy creation of additional Redis instances when necessary.

  Access control and autorization

The debops.redis_server role configures Redis Server instances with a randomly generated password, the same for all instances in the same domain. The password is stored in the secret/ directory on the Ansible Controller (see debops.secret role for details).

The Redis password can be accessed on the Redis hosts using the redis-password script. Only the users that have read access to the redis.conf configuration files will be able to use it; the role sets up an auxiliary redis-auth UNIX system group to allow members of this group access to the configuration. The script accepts an instance name as an argument; if not specified the main instance will be checked and the password will be retrieved from the configuration file.

To access the default Redis Server instance via the redis-cli interface, you can use the command:

redis-cli -a $(redis-password)

Redis password is also exposed in the Ansible local facts, so that other Ansible roles can use it to configure access to Redis by other applications. Run the /etc/ansible/facts.d/redis_server.fact script on the remote host to see the local fact structure and contents.

  Example inventory

To enable Redis Server configuration on a host, it needs to be added to a specific Ansible inventory group:

[debops_service_redis_server]
hostname

By default Redis listens only for local connections on the loopback network interface. If you want to set up a cluster of Redis instances on different hosts that talk to each other, you should configure the default instance to bind to all network interfaces, as well as open the TCP ports in the firewall:

# Listen to TCP connections on all interfaces
redis_server__bind: [ '0.0.0.0', '::' ]
# Allow connections to Redis from specific subnets
redis_server__allow: [ '192.0.2.0/24', '2001:db8::/32' ]

  Example playbook

If you are using this role without DebOps, here's an example Ansible playbook that uses the debops.redis_server role:

---
- name: Manage Redis server
  collections: [ 'debops.debops', 'debops.roles01',
                 'debops.roles02', 'debops.roles03' ]
  hosts: [ 'debops_service_redis_server' ]
  become: True
  environment: '{{ inventory__environment | d({})
                   | combine(inventory__group_environment | d({}))
                   | combine(inventory__host_environment  | d({})) }}'
  pre_tasks:
    - name: Prepare sysfs environment
      import_role:
        name: 'sysfs'
        tasks_from: 'main_env'
      tags: [ 'role::sysfs', 'role::secret' ]
    - name: Prepare redis_server environment
      import_role:
        name: 'redis_server'
        tasks_from: 'main_env'
      tags: [ 'role::redis_server', 'role::ferm' ]
  roles:
    - role: secret
      tags: [ 'role::secret', 'role::sysfs' ]
      secret__directories:
        - '{{ sysfs__secret__directories | d([]) }}'
    - role: apt_preferences
      tags: [ 'role::apt_preferences', 'skip::apt_preferences' ]
      apt_preferences__dependent_list:
        - '{{ redis_server__apt_preferences__dependent_list }}'
    - role: etc_services
      tags: [ 'role::etc_services', 'skip::etc_services' ]
      etc_services__dependent_list:
        - '{{ redis_server__etc_services__dependent_list }}'
    - role: ferm
      tags: [ 'role::ferm', 'skip::ferm' ]
      ferm__dependent_rules:
        - '{{ redis_server__ferm__dependent_rules }}'
    - role: sysctl
      tags: [ 'role::sysctl', 'skip::sysctl' ]
      sysctl__dependent_parameters:
        - '{{ redis_server__sysctl__dependent_parameters }}'
    - role: sysfs
      tags: [ 'role::sysfs', 'skip::sysfs' ]
      sysfs__dependent_attributes:
        - '{{ redis_server__sysfs__dependent_attributes }}'
    - role: python
      tags: [ 'role::python', 'skip::python', 'role::redis_server' ]
      python__dependent_packages3:
        - '{{ redis_server__python__dependent_packages3 }}'
      python__dependent_packages2:
        - '{{ redis_server__python__dependent_packages2 }}'
    - role: redis_server
      tags: [ 'role::redis_server', 'skip::redis_server' ]

  Ansible tags

You can use Ansible --tags or --skip-tags parameters to limit what tasks are performed during Ansible run. This can be used after a host was first configured to speed up playbook execution, when you are sure that most of the configuration is already in the desired state.

Available role tags:

role::redis_server
    Main role tag, should be used in the playbook to execute all of the role tasks as well as role dependencies.

  Other resources

List of other useful resources related to the debops.redis_server Ansible role:

•   Official Redis documentation <https://redis.io/documentation>

•   Redis configuration file format <https://redis.io/topics/config>

REDIS SERVER CONFIGURATION PIPELINE

The default Redis Server installation in Debian Jessie and Debian Stretch supports only 1 instance of Redis per host. The pacakges in Debian Buster and the stretch-backports repository support multiple instances by using a single /etc/redis/redis-<instance>.conf configuration file per a systemd instance. However, due to the Redis modifying its own configuration file on the fly, using a single redis.conf configuration file does not work well with an Ansible-based approach to configuration.

The solution to this problem implemented in debops.redis_server role is usage of a separate /etc/redis/<instance>/ directory for each Redis Server instance. This allows usage of multiple configuration files and even scripts for each Redis Server instance, with configuration applied dynamically at runtime. The redis.conf configuration file is not touched directly by Ansible, apart from ensuring that additional configuration file with Ansible-generated parameters is included at the end. This ensures idempotency and allows Ansible and Redis to work together.

  Configuration variables

The debops.redis_server Ansible role exposes a set of default variables that can be used to define and modify Redis configuration per instance. Configuration defined in each one is merged together in the redis_server__combined_configuration using a special filter plugin. Multiple configuration entries defined in the format of the redis_server__configuration parameters are merged together, therefore there's no need to copy everything to the Ansible inventory.

The variables are merged in the following order:

•   the redis_server__default_base_options and the redis_server__base_options hold the default parameters applied to all of the Redis Server instances on a particular host. These variables can be used to override options applied to all instances when needed.

•   the redis_server__default_instances and the all/group/host variant of the same variable are used to generate configuration for each instance, which is then put in the configuration pipeline via the redis_server__default_configuration variable. Each instance will include the base options defined for all instances, and per-instance configuration like port, UNIX socket path, optional systemd overrides, etc.

•   the redis_server__default_configuration and the all/group/host variants include the actual configuration used by the role to generate the Redis Server configuration files, systemd service configuration. The variables are joined together in the redis_server__combined_configuration variable which is used in varius role tasks and templates. These variables can be used to override per-instance configuration if needed.

  Configuration file structure

The generated configuration file structure contains the following files:

/etc/redis
├── main/
│   ├── ansible-redis-dynamic.conf*
│   ├── ansible-redis-static.conf
│   └── redis.conf
├── second/
│   ├── ansible-redis-dynamic.conf*
│   ├── ansible-redis-static.conf
│   └── redis.conf
├── third/
│   ├── ansible-redis-dynamic.conf*
│   ├── ansible-redis-static.conf
│   └── redis.conf
└── redis.conf

The ansible-redis-static.conf files contain static configuration options for each Redis Server instance. If any options there change, a given instance is restarted.

The ansible-redis-dynamic.conf files are Bash scripts which apply Redis Server configuration dynamically at runtime, using the CONFIG SET commands via the redis-cli interface. The CONFIG REWRITE command is then executed so that Redis can update its own configuration file; this way the dynamic configuration is preserved between restarts.

The redis.conf configuration files are copies of the original /etc/redis/redis.conf configuration file created when each instance is initialized. The role assumes that Redis modifies these files dynamically and doesn't touch them directly, apart from ensuring that an include line for the ansible-redis-static.conf is present and near the end of the file.

DEFAULT VARIABLE DETAILS

Some of debops.redis_server default variables have more extensive configuration than simple strings or lists, here you can find documentation and examples for them.

  redis_server__instances

The role can manage multiple Redis Server instances on a single host via the redis_server__*_instances default variables. Each variable is a list of YAML dictionaries, each dictionary defines an instance of Redis managed by systemd unit template.

Configuration specified in the instance YAML dictionary is parsed by the role and used to generate the final configuration which is then used to manage the Redis instances (see Redis Server configuration pipeline).

Multiple dictionaries with the same name parameter will be merged together; this can be used to override previously defined instance configuration without copying everything to the Ansible inventory.

  Examples

Define multiple Redis Server instances:

---
# Configure all Redis instances to listen for network connections
redis_server__bind: [ '0.0.0.0', '::' ]
# Allow connections from specific subnets to all Redis instances
redis_server__allow: [ '192.0.2.0/24', '2001:db8::/32' ]
# Define minimal parameters for additional Redis Server instances with
# autogenerated configuration
redis_server__instances:
  - name: 'second'
    port: '6380'
  - name: 'third'
    port: '6381'

Modify existing instance configuration:

---
# Change the default bind of the specific Redis Server instance 'main' to
# listen on all network interfaces and configure it as a slave to another Redis
# server
redis_server__instances:
  - name: 'main'
    bind: [ '0.0.0.0', '::' ]
    master_host: 'redis.example.org'
    master_port: '6379'

  Syntax

Each entry can contain specific parameters:

name
    Required. The name of a given Redis instance. This parameter is used as an anchor for merging of multiple YAML dictionaries that specify Redis instances together.

    The instance name main is significant and used in Ansible local fact script and the password script to denote the "default" Redis instance if none is specified.

port
    Required. The TCP port on which a given instance listens for network connections. Only ports defined in the instance list will be included in the automatically managed firewall configuration.

state
    Optional. If not specified or present, a given Redis instance will be created or managed by the role. If absent, a given instance will be removed by the role. If ignore, a given instance entry will not be included in the configuration.

pidfile
    Optional. Absolute path to a PID file of a given Redis instance. If not specified, the role will generate one based on the instance name.

unixsocket
    Optional. Absolute path to an UNIX socket file of a given Redis instance. If not specified, the role will generate one based on the instance name.

bind
    Optional. A string or a YAML list of IP addresses to which a given Redis instance should bind to to listen for network connections. If not specified, the instance will bind on the IP addresses specified in the redis_server__bind variable, by default localhost.

dbfilename
    Optional. Name of the Redis database file which will contain the persisten storage, stored in the /var/lib/redis/ directory. If not specified, the role will generate the name based on the instance name.

logfile
    Optional. Absolute path to a log file of a given Redis instance. If not specified, the role will generate one based on the instance name.

syslog_ident
    Optional. A short string that identifies a given Redis instance in the syslog stream. If not specified, the role will generate one based on the instance name.

requirepass
    Optional. Plaintext password which will be required by Redis to allow certain operations. If not specified, the value of the redis_server__auth_password will be used automatically.

systemd_override
    Optional. An YAML text block that contains systemd unit configuration entries. This can be used to override the configuration of a Redis instance managed by systemd.

master_host and master_port
    Optional. The FQDN address of the host with the Redis master instance, and its TCP port. If these parameters are set, a given Redis instance will be configured as a slave of the specified Redis master on the initial configuration, but not subsequent ones.

    Other configuration options for a given Redis instance should be specified in the redis_server__*_configuration variables. Some of the instance parameters like port are used in other parts of the role and should be overridden only on the list of instances.

  redis_server__configuration

The redis_server__*_configuration variables define the configuration of the Redis Server instances. A Redis Server instance consists of a set of configuration files in /etc/redis/<instance>/ subdirectory, as well as systemd service template configuration. See Redis Server configuration pipeline for more details.

  Examples

Define additional instance configuration:

---
# Disable certain Redis commands in a specific Redis Server instance. This
# cannot be done in the instance parameters and needs to be done on the
# configuration level.
redis_server__configuration:
  - name: 'main'
    options:
      - appendonly: False
      - 'auto-aof-rewrite-percentage': 100
      - 'auto-aof-rewrite-min-size': '64mb'
      - name: 'rename-command'
        value:
          - 'FLUSHDB ""'
          - 'FLUSHALL ""'
          - 'KEYS ""'
          - 'CONFIG ""'
          - 'PEXPIRE ""'
          - 'DEL ""'
          - 'CONFIG ""'
          - 'SHUTDOWN ""'
          - 'BGREWRITEAOF ""'
          - 'BGSAVE ""'
          - 'SAVE ""'
          - 'SPOP ""'
          - 'SREM ""'
          - 'RENAME ""'
          - 'DEBUG ""'

To replace options which are lists you have to reset them first:

---
# Overwrite default snapshoting configuration.
redis_server__configuration:
  - name: 'main'
    options:
      - name: 'save'
        value: ''
        dynamic: True
      - name: 'save'
        value:
          - '1200 1'
        dynamic: True

You can also remove some items from the list:

---
# Remove single save command and add another one.
redis_server__configuration:
  - name: 'main'
    options:
      - name: 'save'
        value:
          - '90 1000'
          - name: '60 10000'
            state: 'absent'
        dynamic: True

  Syntax

Each variable contains a list of YAML dictionaries, each dictionary defines a Redis Server instance using specific parameters:

name
    Required. Name of a given Redis Server instance, should be a short alphanumeric string. This parameter is used as an anchor to merge multiple instance entries together.

port
    Required. The TCP port on which a given instance listens for network connections.

state
    Optional. If not specified or present, a given Redis Server instance will be created and/or managed on a given host. If absent, a given Redis Server instance will be stopped and its configuration will be removed. If ignore, a given configuration entry will be ignored by the role during the Ansible run; this can be used to conditionally enable or disable instance options if needed.

requirepass
    Optional. Plaintext password which will be required by Redis to allow certain operations.

systemd_override
    Optional. An YAML text block that contains systemd unit configuration entries. This can be used to override the configuration of a Redis instance managed by systemd.

master_host and master_port
    Optional. The FQDN address of the host with the Redis master instance, and its TCP port. If these parameters are set, a given Redis instance will be configured as a slave of the specified Redis master on the initial configuration, but not subsequent ones.

options
    A list of configuration options for the Redis Server instance. The options lists from multiple instance configuration entries are merged together. Each element of the options list is a YAML dictionary with specific parameters:

name
    Redis Server parameter name. Parameter names containing hypens should be quoted to avoid any issues with YAML parsing.

value
    Redis Server parameter value. It can be a number, a string or a list of strings. If a list is used, by default the configuration file will contain multiple parameters with the same name and values specified on separate lines. you can also use the Python True and False values to represent booleans.

state
    Optional. If not specified or present, a given parameter will be present in the generated configuration. If absent, a given parameter will be removed from the configuration.

dynamic
    Optional, bollean. If True, a given parameter will be marked as a "dynamic" Redis configuration and will not be included in the static configuration file. Instead, it will be applied dynamically via a configuration script during deployment.

multiple
    Optional, boolean. If False, and the parameter is not dynamic, and it's a list, the values will be concatenated into one string, separated by spaces. This is required by some of the Redis Server configuration options, for example bind.

    If the name and value parameters are not present, each key of the YAML dictionary will be interpreted as a separate Redis Server parameter. This can be used as a shorthand to define Redis Server parameters, but for more complicated parameters (dynamic, with custom requirements), you should use the expanded form explained above.

AUTHOR
Maciej Delmanowski

COPYRIGHT
2014-2022, Maciej Delmanowski, Nick Janetakis, Robin Schneider and others