Andrew Bonney caf2442fd4 Add additional commented RabbitMQ policy to manage segment sizes
The default RabbitMQ segment sizes of 500MB mean you may need to
acquire many days or weeks worth of messages in each fanout/stream
queue before the x-max-age setting takes effect. Subject to the
number of streams in use across the deployment, this could consume
many tens of GB of disk space.

This example shows an additional policy which has been tested in
order to combat this in the absence of suitable per-stream settings
in oslo.messaging. The downside is that this must be applied before
any services create their streams.

Change-Id: I69699f99a88b0ba8c8ee283ecbd3b4ed0c98a373
2025-01-22 08:29:55 +00:00

76 lines
2.7 KiB
YAML

---
# Copyright 2017, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
## Rabbit options
rabbitmq_host_group: "rabbitmq_all"
rabbitmq_port: "{{ (rabbitmq_use_ssl | bool) | ternary(5671, 5672) }}"
rabbitmq_use_ssl: True
rabbitmq_servers: >-
{{
groups[rabbitmq_host_group]
| map('extract', hostvars, 'management_address')
| list | join(',')
}}
# If you do not want to use quorum queues, set rabbitmq_queue_replication to False
rabbitmq_queue_replication: True
rabbitmq_policies:
- name: CQv2
pattern: '.*'
priority: 0
tags:
queue-version: 2
state: >-
{{
((oslomsg_rabbit_quorum_queues | default(True) or not rabbitmq_queue_replication) and rabbitmq_install_method | default('') != 'distro'
) | ternary('present', 'absent')
}}
# The following is an example of an additional policy which applies to fanout/stream queues only
# By default, each stream uses RabbitMQ's 500MB segment size, and no messages will be discarded
# until that size is reached. This may result in undesirable disk usage.
# If using this policy, it must be applied BEFORE any stream queues are created.
# See also https://bugs.launchpad.net/oslo.messaging/+bug/2089845 and https://www.rabbitmq.com/docs/streams#retention
# - name: CQv2F
# pattern: '^.*_fanout'
# priority: 1
# tags:
# queue-version: 2
# stream-max-segment-size-bytes: 1000000
# state: >-
# {{
# ((oslomsg_rabbit_quorum_queues | default(True) or not rabbitmq_queue_replication) and rabbitmq_install_method | default('') != 'distro'
# ) | ternary('present', 'absent')
# }}
## Galera options
galera_client_package_state: "{{ package_state }}"
galera_address: "{{ internal_lb_vip_address }}"
galera_root_user: "admin"
# Disable SSL for CentOS distro path due to https://bugzilla.redhat.com/show_bug.cgi?id=2074555
galera_use_ssl: "{{ not (ansible_facts['os_family'] | lower == 'redhat' and install_method == 'distro') }}"
## Memcached options
memcached_port: 11211
memcached_servers: >-
{{
(groups['memcached'] | map('extract', hostvars, 'management_address') | list)
| map('regex_replace', '(.+)' ,'\1:' ~ memcached_port)
| list
| join(',')
}}