openstack/puppet-tripleo
Code Issues Proposed changes

Fix stack update with rabbitmq containers

Browse Source 
In change I44865af3d5eb2d37eb648ac7227277e86c8fbc54 we
add support to change rabbitmq password on update.
This breaks when using containers in a number of scenarios:
- FFU because at this stage rabbitmq can be down on the node
  and the call to rabbitmq_user will trigger a rabbitmqctl list_user
  call which will eventually time out.
- Controller replacement procedure because on the newly replaced
  controller rabbitmq will not be up yet and the rabbitmq_user call
  will timeout just like during FFU.
- Upgrades from Pike to Queens upgrades we seem to be hitting the
  same issue as FFU

The exact error that we will get on the non bootstrap nodes is the
following:
=ERROR REPORT==== 3-Mar-2018::05:37:39 ===** Connection attempt from
disallowed node 'rabbitmq-cli-67@controller-1' **

The reason for this is that on non bootstrap node we do not enforce
the Exec['rabbitmq-ready'] -> Rabbitmq_user<||> collector, because
we do not want to enforce it there (think controller replacement)

Let's make sure we enforce the Rabbitmq_user class only on bootstrap
nodes, since in HA deployments the users get replicated by the cluster anyway.

Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>
Co-Authored-By: John Eckersberg <jeckersb@redhat.com>
Closes-Bug: #1753949

Change-Id: I483fe61f09fa2c3034d2b3d8ffa1ca53feefe6af
This commit is contained in:
Michele Baldessari
2018-03-05 16:17:11 +01:00
parent 50f2bf278c
commit 2abe91fe86
1 changed files with 15 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
manifests/profile/base/rabbitmq.pp
View File

@@ -160,9 +160,20 @@ class tripleo::profile::base::rabbitmq (
}
}
if $step >= 2 {
if $::hostname == downcase($bootstrap_node) {
$rabbitmq_bootstrapnode = true
} else {
$rabbitmq_bootstrapnode = false
}
if $rabbitmq_bootstrapnode and $step >= 2 {
# In case of HA, starting of rabbitmq-server is managed by pacemaker, because of which, a dependency
# to Service['rabbitmq-server'] will not work. Sticking with UPDATE action.
# When need to enforce the rabbitmq user inside a bootstrap node check for two reasons:
# a) on HA the users get replicated by the cluster anyway
# b) in the pacemaker profiles for rabbitmq we have an Exec['rabbitmq-ready'] -> Rabbitmq_User<||> collector
# which is applied only on the bootstrap node (because enforcing the readiness on all nodes can be problematic
# in situations like controller replacement)
if $stack_action == 'UPDATE' {
# Required for changing password on update scenario. Password will be changed only when
# called explicity, if the rabbitmq service is already running.
@@ -171,14 +182,9 @@ class tripleo::profile::base::rabbitmq (
admin => true,
}
}
if hiera('veritas_hyperscale_controller_enabled', false) {
include ::veritas_hyperscale::hs_rabbitmq
}
}
if $::hostname == downcase($bootstrap_node) {
$rabbitmq_bootstrapnode = true
} else {
$rabbitmq_bootstrapnode = false
}
if $rabbitmq_bootstrapnode and $step >= 2 and hiera('veritas_hyperscale_controller_enabled', false) {
include ::veritas_hyperscale::hs_rabbitmq
}
}