DEBOPS.MINIO(5)

NAME
debops.minio - Install MinIO, Amazon S3 Compatible Object Storage

SYNOPSIS

debops service/minio [--limit group,host,...] [--diff] [--check] [--tags tag1,tag2,...] [--skip-tags tag1,tag2,...] [<ansible-playbook options>] ...

DESCRIPTION

MinIO <https://min.io/> is an Open Source Amazon Simple Storage Service (S3) compatible object storage service.

The debops.minio Ansible role can be used to deploy and configure MinIO in various scenarions, either as a single node, single tenant service, or a multi node and/or multi tenant distributed storage service. The role relies on other DebOps roles to install MinIO binary from upstream URL or build it from source code, configure firewall access, provide TLS support and configure web server access to the service. See the Getting Started page for more details about the deployment stack.

The MinIO Client <https://docs.min.io/docs/minio-client-complete-guide> application is used to interface with MinIO and perform various administrative tasks, including extended configuration of the service. It can be installed on a host using the debops.mcli Ansible role.

GETTING STARTED

  Default installation and upgrades

The role will install the latest MinIO release available by default. After that, the selected release will be "locked" via the Ansible local facts, and normal execution of the debops.minio role will not upgrade an existing installation.

To perform an upgrade, you can use the following command:

debops service/minio -l <host|group> -t role::golang \
                     -e 'minio__upstream_upgrade=true'

This command will execute the debops.golang Ansible role in the context of the service/minio playbook and perform the upgrade of MinIO binary if a new release is found. The existing MinIO instances will be restarted afterwards.

If you want to perform upgrades automatically when the role is executed when they are available, you can set the minio__upstream_upgrade variable to True in the Ansible inventory.

The previously downloaded MinIO releases are currently not removed automatically, therefore make sure that you clean up the ~_golang/go/src/ directory managed by the debops.golang role from time to time.

  Dependent role usage

The debops.minio role is focused on configuration of the MinIO service, and relies on other DebOps roles for the actual installation and configuration of additional services:

•   The debops.golang role is used to download and verify MinIO binary, or clone the MinIO source code and build the binary locally.

•   The debops.keyring role is used by the debops.golang role to fetch and import the MinIO GPG signing key used to verify the MinIO binaries or source code.

•   The debops.pki role (not included in the playbook) is used to provide the PKI infrastructure and X.509 certificates required for TLS connections to MinIO instances.

•   The debops.sysctl and debops.sysfs roles are used to configure kernel parameters in the /proc/ and /sys/ directories related to MinIO.

•   The debops.nginx role is used to configure the web server access to MinIO service over HTTPS and redirects the subdomain queries to the correct MinIO instances configured on the host.

•   The debops.ferm role is used to open the firewall access to MinIO instances via their TCP ports.

    See the provided Ansible playbook for the order of the roles and usage of the dependent role variables. Without the mentioned DebOps roles and services managed by them, debops.minio role alone will not deploy the MinIO service correctly.

  Example inventory

To configure MinIO service on a host, it needs to be added to the [debops_service_minio] Ansible inventory group:

[debops_service_minio]
hostname

See the MinIO deployment guide more examples of distributed MinIO deployments.

  Example playbook

If you are using this role without DebOps, here's an example Ansible playbook that uses the debops.minio role:

---
- name: Manage MinIO service
  collections: [ 'debops.debops', 'debops.roles01',
                 'debops.roles02', 'debops.roles03' ]
  hosts: [ 'debops_service_minio' ]
  become: True
  environment: '{{ inventory__environment | d({})
                   | combine(inventory__group_environment | d({}))
                   | combine(inventory__host_environment  | d({})) }}'
  pre_tasks:
    - name: Prepare minio environment
      import_role:
        name: 'minio'
        tasks_from: 'main_env'
      tags: [ 'role::minio', 'role::etc_services', 'role::ferm',
              'role::keyring', 'role::golang', 'role::nginx' ]
    - name: Prepare sysfs environment
      import_role:
        name: 'sysfs'
        tasks_from: 'main_env'
      tags: [ 'role::sysfs', 'role::secret' ]
  roles:
    - role: keyring
      tags: [ 'role::keyring', 'skip::keyring', 'role::golang' ]
      keyring__dependent_gpg_user: '{{ golang__keyring__dependent_gpg_user }}'
      keyring__dependent_gpg_keys:
        - '{{ nginx__keyring__dependent_apt_keys }}'
        - '{{ golang__keyring__dependent_gpg_keys }}'
      golang__dependent_packages:
        - '{{ minio__golang__dependent_packages }}'
    - role: secret
      tags: [ 'role::secret', 'role::sysfs' ]
      secret__directories:
        - '{{ sysfs__secret__directories | d([]) }}'
    - role: apt_preferences
      tags: [ 'role::apt_preferences', 'skip::apt_preferences' ]
      apt_preferences__dependent_list:
        - '{{ golang__apt_preferences__dependent_list }}'
        - '{{ nginx__apt_preferences__dependent_list }}'
    - role: etc_services
      tags: [ 'role::etc_services', 'skip::etc_services' ]
      etc_services__dependent_list:
        - '{{ minio__etc_services__dependent_list }}'
    - role: ferm
      tags: [ 'role::ferm', 'skip::ferm' ]
      ferm__dependent_rules:
        - '{{ minio__ferm__dependent_rules }}'
        - '{{ nginx__ferm__dependent_rules }}'
    - role: sysctl
      tags: [ 'role::sysctl', 'skip::sysctl' ]
      sysctl__dependent_parameters:
        - '{{ minio__sysctl__dependent_parameters }}'
    - role: sysfs
      tags: [ 'role::sysfs', 'skip::sysfs' ]
      sysfs__dependent_attributes:
        - '{{ minio__sysfs__dependent_attributes }}'
    - role: python
      tags: [ 'role::python', 'skip::python' ]
      python__dependent_packages3:
        - '{{ nginx__python__dependent_packages3 }}'
      python__dependent_packages2:
        - '{{ nginx__python__dependent_packages2 }}'
    - role: nginx
      tags: [ 'role::nginx', 'skip::nginx' ]
      nginx__dependent_upstreams:
        - '{{ minio__nginx__dependent_upstreams }}'
      nginx__dependent_servers:
        - '{{ minio__nginx__dependent_servers }}'
    - role: golang
      tags: [ 'role::golang', 'skip::golang' ]
      golang__dependent_packages:
        - '{{ minio__golang__dependent_packages }}'
    - role: minio
      tags: [ 'role::minio', 'skip::minio' ]

  Ansible tags

You can use Ansible --tags or --skip-tags parameters to limit what tasks are performed during Ansible run. This can be used after a host was first configured to speed up playbook execution, when you are sure that most of the configuration is already in the desired state.

Available role tags:

role::minio
    Main role tag, should be used in the playbook to execute all of the role tasks as well as role dependencies.

  Other resources

List of other useful resources related to the debops.minio Ansible role:

•   MinIO documentation <https://docs.min.io/>

MINIO DEPLOYMENT GUIDE

MinIO can be deployed in different ways depending on the desired configuration. You should refer to the MinIO documentation <https://docs.min.io/> for various deployment examples. This page focuses on explaining how to deploy MinIO in multi-tenant environment <https://docs.min.io/docs/multi-tenant-minio-deployment-guide.html> using the debops.minio Ansible role to provide more complex examples of the role usage.

WARNING: Once deployed, the structure of the MinIO cluster cannot be changed (new disks/hosts cannot be added/removed from the cluster <https://github.com/minio/minio/issues/4364>). It's best to prepare the desired configuration in a development environment before deploying it in production.

  PKI infrastructure

MinIO supports encrypted connections using TLS and X.509 certificates - when this mode is enabled, unencrypted HTTP connections are disabled, therefore communication with upstream MinIO services through the nginx proxy has to be done over HTTPS. The TLS protocol also enforces checking the Host: HTTP header against the currently enabled X.509 certificates - any connections to hosts or IP addresses not in the X.509 certificates will be denied.

The debops.minio role uses the PKI infrastructure maintained by the debops.pki role when available. The default PKI deployment configures an internal Certificate Authority which is trusted by all hosts in the cluster; the host certificates contain wildcard addresses for the domain part as well as host subdomains, which simplifies the internal certificate management. However, if you plan to use public X.509 certificates for MinIO services directly, you need to ensure that the certificates use the correct FQDNs for each host in the cluster. The debops.minio role currently does not support using IP addresses for connections, this feature can be implemented if there's a demand for it.

Since connections from the outside to the MinIO cluster via the nginx proxy can be handled by a separate set of certificates, use of the internal CA and the domain PKI realm for MinIO service is currently recommended.

  Single tenant, multiple nodes

The default main MinIO instance is configured for a single tenant on multiple, separate hosts with its access and secret keys stored in the secret/minio/distributed/main/ files on the Ansible Controller (see debops.secret role documentation for details). This configuration allows easy scaling of storage by setting up additional hosts with MinIO service installed on each one. The access and secret keys will be the same, therefore your application(s) can use the same credentials to access the storage on different nodes. An example inventory with 2 MinIO hosts:

# ansible/inventory/hosts
# Configure Ansible inventory groups
[debops_all_hosts]
server1    ansible_host=server1.example.org
server2    ansible_host=server2.example.org
[debops_service_minio]
server1
server2

The MinIO instances will be reachable directly via these addresses:

•   https://server1.example.org:9000/

•   https://server2.example.org:9000/

    The nginx HTTP proxy configured by debops.minio role will publish the MinIO instances on these addresses:

•   https://server1.example.org/

•   https://server2.example.org/

    You can combine separate MinIO instances in a federated mode <https://docs.min.io/docs/minio-federation-quickstart-guide.html> to make host lookups via DNS easier, however this configuration is currently out of scope for the debops.minio role.

  Single tenant, single node

If you want to configure separate tenants on each MinIO host, for example by separating tenants using LXC containers and frontend HTTP proxy, you can easily change the main MinIO instance to standalone configuration by setting in the inventory:

# ansible/inventory/hosts
# Configure Ansible inventory groups
[debops_all_hosts]
tenant1    ansible_host=tenant1.example.org
tenant2    ansible_host=tenant2.example.org
tenant3    ansible_host=tenant3.example.org
[debops_service_minio]
tenant1
tenant2
tenant3

# ansible/inventory/group_vars/all/minio.yml
# Override configuration for 'main' instance
minio__instances:
  - name: 'main'
    standalone: True

With this configuration, each MinIO main instance on a separate host gets its own set of access and secret keys stored in the secret/minio/standalone/<host>/main/ directory on the Ansible Controller.

The MinIO instances will be reachable directly via these addresses:

•   https://tenant1.example.org:9000/

•   https://tenant2.example.org:9000/

•   https://tenant3.example.org:9000/

    The nginx HTTP proxy configured by debops.minio role will publish the MinIO instances on these addresses:

•   https://tenant1.example.org/

•   https://tenant2.example.org/

•   https://tenant3.example.org/

    The DNS records and the X.509 certificates may contain wildcard addresses (*.tenant1.example.org, etc.) to allow access to buckets via subdomains in addition to access via subdirectories; for example https://bucket.tenant1.example.org will redirect to https://tenant1.example.org/bucket/.

  Standalone deployment

In a standalone deployment example <https://docs.min.io/docs/multi-tenant-minio-deployment-guide.html#standalone-deployment>, we will configure MinIO with three tenants on a single MinIO host, once with a single disk drive, and once with multiple disk drives. In this example, the main MinIO cluster will be removed for consistency.

Each MinIO tenant instance will be accessible over a separate TCP port. The nginx proxy configured by the debops.minio role will also allow connections to each MinIO instance based on its name parameter as a subdomain of the main DNS domain of the host. For that to work reliably, X.509 certificates used by the debops.nginx role need to include the relevant FQDN addresses.

The host configuration in the Ansible inventory:

# ansible/inventory/hosts
# Configure Ansible inventory groups
[debops_all_hosts]
server    ansible_host=server.example.org
[debops_service_minio]
server

The MinIO instances will be reachable directly via these addresses:

•   https://server.example.org:9001/

•   https://server.example.org:9002/

•   https://server.example.org:9003/

    The nginx HTTP proxy configured by debops.minio role will publish the MinIO instances on these addresses:

•   https://tenant1.example.org/

•   https://tenant2.example.org/

•   https://tenant3.example.org/

    Note that the proxied URLs are based on the MinIO instance names instead of the host names. The DNS configuration which directs the above FQDNs to the server.example.org host has to be performed separately.

  Multiple tenants on a single drive

In this set up there's a single host with large disk drive mounted at /data mount point (mounting can be configured by the debops.mount Ansible role). Since the default is to configure the MinIO instance volumes at /srv/minio/ directory, we override that using the minio__volumes_dir variable. The role will configure each MinIO instance to use a subdirectory in the /data directory.

# ansible/inventory/host_vars/server/minio.yml
# Override default MinIO volumes path
minio__volumes_dir: '/data'
# Ensure that data directory is accessible by the 'minio' UNIX account
minio__host_volumes:
  - '/data'
# Configure MinIO instances
minio__host_instances:
  - name: 'main'
    state: 'absent'
  - name: 'tenant1'
    port: 9001
    console_port: 19001
  - name: 'tenant2'
    port: 9002
    console_port: 19002
  - name: 'tenant3'
    port: 9003
    console_port: 19003

  Multiple tenants on multiple drives

In this case the storage server has 4 disk drives mounted at /disk{1,4}/ directories. Here we have to specify each volume directly for each tenant, so that the data is distributed among the disk drives.

# ansible/inventory/host_vars/server/minio.yml
# Ensure that data directories are accessible by the 'minio' UNIX account
minio__host_volumes:
  - '/disk1/data'
  - '/disk2/data'
  - '/disk3/data'
  - '/disk4/data'
# Configure MinIO instances
minio__host_instances:
  - name: 'main'
    state: 'absent'
  - name: 'tenant1'
    port: 9001
    console_port: 19001
    volumes:
      - '/disk1/data/tenant1'
      - '/disk2/data/tenant1'
      - '/disk3/data/tenant1'
      - '/disk4/data/tenant1'
  - name: 'tenant2'
    port: 9002
    console_port: 19002
    volumes:
      - '/disk1/data/tenant2'
      - '/disk2/data/tenant2'
      - '/disk3/data/tenant2'
      - '/disk4/data/tenant2'
  - name: 'tenant3'
    port: 9003
    console_port: 19003
    volumes:
      - '/disk1/data/tenant3'
      - '/disk2/data/tenant3'
      - '/disk3/data/tenant3'
      - '/disk4/data/tenant3'

  Distributed deployment

The distributed MinIO deployment <https://docs.min.io/docs/multi-tenant-minio-deployment-guide.html#distributed-deployment> uses multiple hosts to distribute the data across a number of devices to improve resiliency. The minimum amount of hosts required by MinIO is 4, maximum is 32.

In this example, we will use 4 hosts with single disk each, mounted at the /data directory. The connection between MinIO instances will be done over TLS, connecting to the TCP ports directly. The nginx proxies on each host will be configured to direct the traffic to the local MinIO instance, in which case the tenant\\{1,4\\}.example.org DNS records should point to all server\\{1,4\\}.example.org hosts in a round-robin fashion.

An example Ansible inventory (note that the configuration is set at the [minio_cluster1] group level, not the host level):

# ansible/inventory/hosts
# Configure Ansible inventory groups
[debops_all_hosts]
server1    ansible_host=server1.example.org
server2    ansible_host=server2.example.org
server3    ansible_host=server3.example.org
server4    ansible_host=server4.example.org
[minio_cluster1]
server1
server2
server3
server4
[debops_service_minio:children]
minio_cluster1

The MinIO tenant1 instance will be reachable directly via these addresses:

•   https://server1.example.org:9001/

•   https://server2.example.org:9001/

•   https://server2.example.org:9001/

•   https://server4.example.org:9001/

    You can reach other MinIO instances in the same way by changing the destination TCP port.

    The nginx HTTP proxy configured by debops.minio role will publish the MinIO instances on these addresses:

•   https://tenant1.example.org/

•   https://tenant2.example.org/

•   https://tenant3.example.org/

    The DNS configuration which directs the above FQDNs to the underlying hosts has to be performed separately. You should use a round-robin DNS records, where each tenantX.example.org record points to all servers in the cluster.

    The configuration for the entire cluster is defined on the Ansible inventory group level, in this case [minio_cluster1] group. There can be multiple clusters defined in the Ansible inventory, just make sure that the MinIO-related variables don't overlap between groups.

    # ansible/inventory/group_vars/minio_cluster1/minio.yml
    # Ensure that data directory is accessible by the 'minio' UNIX account
    minio__group_volumes:
      - '/data'
    # Configure MinIO instances
    minio__group_instances:
      - name: 'main'
        state: 'absent'
      - name: 'tenant1'
        port: 9001
        console_port: 19001
        fqdn: 'tenant1.example.org'
        volumes:
          - 'https://server1.example.org:9001/data/tenant1'
          - 'https://server2.example.org:9001/data/tenant1'
          - 'https://server3.example.org:9001/data/tenant1'
          - 'https://server4.example.org:9001/data/tenant1'
      - name: 'tenant2'
        port: 9002
        console_port: 19002
        fqdn: 'tenant2.example.org'
        volumes:
          - 'https://server1.example.org:9002/data/tenant2'
          - 'https://server2.example.org:9002/data/tenant2'
          - 'https://server3.example.org:9002/data/tenant2'
          - 'https://server4.example.org:9002/data/tenant2'
      - name: 'tenant3'
        port: 9003
        console_port: 19003
        fqdn: 'tenant3.example.org'
        volumes:
          - 'https://server1.example.org:9003/data/tenant3'
          - 'https://server2.example.org:9003/data/tenant3'
          - 'https://server3.example.org:9003/data/tenant3'
          - 'https://server4.example.org:9003/data/tenant3'

DEFAULT VARIABLE DETAILS

Some of debops.minio default variables have more extensive configuration than simple strings or lists, here you can find documentation and examples for them.

  minio__instances

The minio__*_instances variable define the MinIO service instances, managed by systemd. Each instance can be accessed over its TCP port, additionally for each instance a corresponding debops.nginx configuration is generated that allows access to the instance over HTTP via a subdomain based on its name.

  Examples

By default the main MinIO instance uses shared set of credentials to allow multiple hosts with the same "tenant". With the configuration below, each host will have separate set of credentials, and therefore will be owned by a separate "tenant":

minio__instances:
  - name: 'main'
    standalone: True

Set an environment variable for a given MinIO instance, for example to set the instance region (variable names are converted to uppercase automatically):

minio__instances:
  - name: 'main'
    environment:
      minio_region: 'us-east-1'

Create additional instances for new tenants:

minio__instances:
  - name: 'tenant1'
    port: 9001
    console_port: 19001
  - name: 'tenant2'
    port: 9002
    console_port: 19002

Configure a MinIO instance as a NAS gateway <https://docs.min.io/docs/minio-gateway-for-nas.html>, with a custom volume mounted from a remote storage server elsewhere:

minio__volumes:
  - '/shared/nasvol'
minio__instances:
  - name: 'nas-gw'
    port: 9001
    console_port: 19001
    type: 'gateway'
    minio_options: 'nas'
    volumes: [ '/shared/nasvol' ]

You can find more example configurations in the MinIO deployment guide documentation page.

  Syntax

The variables are a list, each instance is defined as a YAML dictionary with specific parameters:

name
    Required. The name of a MinIO instance, used in various file paths. Should be a short, alphanumeric string without spaces. Configuration entries with the same name parameter are merged together in order of appearance.

    By default the name parameter is used as the subdomain of the DNS domain defined in the minio__domain variable, on which a given MinIO instance can be reached over HTTP, configured in the nginx service. This can be overridden using the fqdn parameter.

port
    Required. The TCP port on which a given MinIO instance listens for connections. Usually the port numbers start from 9000 up.

console_port
    Required. The static TCP port on which a given MinIO instance listens for connections to the embedded MinIO Console. If not set, MinIO will select a random console port on each startup. By convention, console port is the API port + 10000, so for example 19000.

server_url
    Optional. Specify the URL the MinIO Console should use for connecting to the MinIO Server. If not specified, https://{{ minio__fqdn }}/ will be used automatically.

state
    Optional. If not defined or present, a given MinIO instance and all related configuration will be created on a host. If absent, a MinIO instance and related configuration will be removed from the host (data is left intact). If ignore, a given configuration entry will not be evaluated during role execution.

bind
    Optional. A string that defines the IP address on which a given MinIO instance should listen for connections, for example localhost or 192.0.2.1. If not defined, MinIO will listen for connections on all available interfaces.

console_bind
    Optional. A string that defines the IP address on which a given MinIO instance should listen for connections to the embedded MinIO Console, for example localhost or 192.0.2.1. If not defined, MinIO will listen for connections on all available interfaces.

allow
    Optional. A list of IP addresses or CIDR subnets which are allowed to connect to a given MinIO instance over its TCP port, managed by the firewall. If not specified, connections from anywhere are allowed.

fqdn
    Optional. A Fully Qualified Domain Name on which a given MinIO instance can be reached, defined in the nginx configuration. If not specified, a FQDN will be generated automatically, based on the instance name parameter and the DNS domain defined in the minio__domain variable.

domain / domains
    Optional. A string or a list with additional DNS domain for which a given MinIO instance supports using subdomains as "bucket" names. The nginx service will be configured to pass requests on subdomains of these DNS domains to a given MinIO instance.

comment
    Optional. A string or YAML text block with comments for a given MinIO instance, included in the generated /etc/minio/<name> configuration file.

type
    Optional. If not specified or server, the MinIO instance is started in the "server" mode, normal operation. If gateway, the MinIO instance is started in the "gateway" mode.

standalone
    Optional, boolean. If not specified or False, the MinIO instance is configured in a "distributed" mode, with the access and secret keys shared between instances with the same name on different host nodes. When True, a MinIO instance is configured in a "standalone" mode, with each instance with the same name using different access and secret keys on different host nodes.

volumes
    Optional. A string or a list with MinIO "volumes" that store the data. This can be either an absolute path to a local filesystem directory, or a https:// URL to a MinIO instance with absolute path to a filesystem directory, for example https://disk.example.org:9000/srv/minio/disk. See MinIO deployment guide for more relevant examples.

    If not specified, a given MinIO instance will use a subdirectory based on its name parameter in the local filesystem minio__volumes_dir directory, by default /srv/minio/.

    If the value is set to False boolean, the $MINIO_VOLUMES environment variable is not set and the volumes are not defined on the command line. This might be needed in certain configuration scenarios.

minio_options
    Optional. A string with additional minio binary options for a given MinIO instance. The --address option is generated automatically by the role and should not be specified here.

root_user
    Optional. A string which defines the MinIO instance "root" account, should be an alphanumeric string. If not specified, the role will generate a randomized account name and store it in the secret/minio/ directory on the Ansible Controller, exact location depending on the instance deployment type (distributed or standalone). See debops.secret for more details about the secret/ directory.

root_password
    Optional. A string which defines the MinIO instance "root" account password, should be an randomized string. If not specified, the role will generate a randomized password and store it in the secret/minio/ directory on the Ansible Controller, exact location depending on the instance deployment type (distributed or standalone). See debops.secret for more details about the secret/ directory.

browser
    Optional, boolean. If not specified or True, the MinIO web interface is enabled on a given MinIO instance. Setting this parameter to False disables the web interface access.

environment
    Optional. YAML dictionary with key-value pairs that define additional environment variables for a given MinIO instance, stored in the /etc/minio/* configuration files. Variable names are automatically converted to uppercase. Values can be either strings or YAML lists which will be concatenated using commas.

AUTHOR
Maciej Delmanowski

COPYRIGHT
2014-2022, Maciej Delmanowski, Nick Janetakis, Robin Schneider and others