2764844ee2
Backport note: This patch has been updated to retain the existing behaviour by default. A temporary variable, rabbitmq_remove_ha_all_policy, has been added which may be set to true in order to remove the ha-all policy. In order to support changing the policy without upgrading, the the ha-all policy is removed on deploys, in addition to upgrades. When OpenStack is deployed with Kolla-Ansible, by default there are no durable queues or exchanges created by the OpenStack services in RabbitMQ. In Rabbit terminology, not being durable is referred to as `transient`, and this means that the queue is generally held in memory. Whether OpenStack services create durable or transient queues is traditionally controlled by the Oslo Notification config option: `amqp_durable_queues`. In Kolla-Ansible, this remains set to the default of `False` in all services. The only `durable` objects are the `amq*` exchanges which are internal to RabbitMQ. More recently, Oslo Notification has introduced support for Quorum queues [7]. These are a successor to durable classic queues, however it isn't yet clear if they are a good fit for OpenStack in general [8]. For clustered RabbitMQ deployments, Kolla-Ansible configures all queues as `replicated` [1]. Replication occurs over all nodes in the cluster. RabbitMQ refers to this as 'mirroring of classic queues'. In summary, this means that a multi-node Kolla-Ansible deployment will end up with a large number of transient, mirrored queues and exchanges. However, the RabbitMQ documentation warns against this, stating that 'For replicated queues, the only reasonable option is to use durable queues: [2]`. This is discussed further in the following bug report: [3]. Whilst we could try enabling the `amqp_durable_queues` option for each service (this is suggested in [4]), there are a number of complexities with this approach, not limited to: 1) RabbitMQ is planning to remove classic queue mirroring in favor of 'Quorum queues' in a forthcoming release [5]. 2) Durable queues will be written to disk, which may cause performance problems at scale. Note that this includes Quorum queues which are always durable. 3) Potential for race conditions and other complexity discussed recently on the mailing list under: `[ops] [kolla] RabbitMQ High Availability` The remaining option, proposed here, is to use classic non-mirrored queues everywhere, and rely on services to recover if the node hosting a queue or exchange they are using fails. There is some discussion of this approach in [6]. The downside of potential message loss needs to be weighed against the real upsides of increasing the performance of RabbitMQ, and moving to a configuration which is officially supported and hopefully more stable. In the future, we can then consider promoting specific queues to quorum queues, in cases where message loss can result in failure states which are hard to recover from. [1] https://www.rabbitmq.com/ha.html [2] https://www.rabbitmq.com/queues.html [3] https://github.com/rabbitmq/rabbitmq-server/issues/2045 [4] https://wiki.openstack.org/wiki/Large_Scale_Configuration_Rabbit [5] https://blog.rabbitmq.com/posts/2021/08/4.0-deprecation-announcements/ [6] https://fuel-ccp.readthedocs.io/en/latest/design/ref_arch_1000_nodes.html#replication [7] https://bugs.launchpad.net/oslo.messaging/+bug/1942933 [8] https://www.rabbitmq.com/quorum-queues.html#use-cases Partial-Bug: #1954925 Change-Id: I91d0e23b22319cf3fdb7603f5401d24e3b76a56e (cherry picked from commit |
||
|---|---|---|
| ansible | ||
| contrib | ||
| deploy-guide/source | ||
| doc | ||
| etc/kolla | ||
| kolla_ansible | ||
| releasenotes | ||
| roles | ||
| specs | ||
| tests | ||
| tools | ||
| zuul.d | ||
| .ansible-lint | ||
| .gitignore | ||
| .gitreview | ||
| .stestr.conf | ||
| .yamllint | ||
| CONTRIBUTING.rst | ||
| LICENSE | ||
| README.rst | ||
| bindep.txt | ||
| requirements.txt | ||
| setup.cfg | ||
| setup.py | ||
| test-requirements.txt | ||
| tox.ini | ||
README.rst
Kolla-Ansible
The Kolla-Ansible is a deliverable project separated from Kolla project.
Kolla-Ansible deploys OpenStack services and infrastructure components in Docker containers.
Kolla's mission statement is:
To provide production-ready containers and deployment tools for operating
OpenStack clouds.Kolla is highly opinionated out of the box, but allows for complete customization. This permits operators with little experience to deploy OpenStack quickly and as experience grows modify the OpenStack configuration to suit the operator's exact requirements.
Getting Started
Learn about Kolla-Ansible by reading the documentation online Kolla-Ansible.
Get started by reading the Developer Quickstart.
OpenStack services
Kolla-Ansible deploys containers for the following OpenStack projects:
- Aodh
- Barbican
- Bifrost
- Blazar
- Ceilometer
- Cinder
- CloudKitty
- Cyborg
- Designate
- Freezer
- Glance
- Heat
- Horizon
- Ironic
- Karbor
- Keystone
- Kuryr
- Magnum
- Manila
- Masakari
- Mistral
- Monasca
- Murano
- Neutron
- Nova
- Octavia
- Panko
- Qinling
- Rally
- Sahara
- Searchlight
- Senlin
- Solum
- Swift
- Tacker
- Tempest
- Trove
- Vitrage
- Vmtp
- Watcher
- Zun
Infrastructure components
Kolla-Ansible deploys containers for the following infrastructure components:
- Collectd, Telegraf, InfluxDB, Prometheus, and Grafana for performance monitoring.
- Elasticsearch and Kibana to search, analyze, and visualize log messages.
- Etcd a distributed reliable key-value store.
- Fluentd as an open source data collector for unified logging layer.
- Gnocchi A time-series storage database.
- HAProxy and Keepalived for high availability of services and their endpoints.
- MariaDB and Galera Cluster for highly available MySQL databases.
- Memcached a distributed memory object caching system.
- Open vSwitch and Linuxbridge backends for Neutron.
- RabbitMQ as a messaging backend for communication between services.
- Redis an in-memory data structure store.
- Zookeeper an open-source server which enables highly reliable distributed coordination.
Directories
ansible- Contains Ansible playbooks to deploy OpenStack services and infrastructure components in Docker containers.contrib- Contains demos scenarios for Heat, Magnum and Tacker and a development environment for Vagrantdoc- Contains documentation.etc- Contains a reference etc directory structure which requires configuration of a small number of configuration variables to achieve a working All-in-One (AIO) deployment.kolla_ansible- Contains password generation script.releasenotes- Contains releasenote of all features added in Kolla-Ansible.specs- Contains the Kolla-Ansible communities key arguments about architectural shifts in the code base.tests- Contains functional testing tools.tools- Contains tools for interacting with Kolla-Ansible.zuul.d- Contains project gate job definitions.
Getting Involved
Need a feature? Find a bug? Let us know! Contributions are much appreciated and should follow the standard Gerrit workflow.
- We communicate using the #openstack-kolla irc channel.
- File bugs, blueprints, track releases, etc on Launchpad.
- Attend weekly meetings.
- Contribute code.
Contributors
Check out who's contributing code and contributing reviews.
Notices
Docker and the Docker logo are trademarks or registered trademarks of Docker, Inc. in the United States and/or other countries. Docker, Inc. and other parties may also have trademark rights in other terms used herein.