Add configurable monitor timeouts for ovn dbs

Under pressure, the default monitor timeout value of 20 seconds is not
enough to prevent unnecessary failovers of the ovn-dbs pacemaker resource.
While spawning a few VMs in the same time this could lead to unnecessary
movements of master DB, then re-connections of ovn-controllers (slaves are
read-only), further peaks of load on DBs, and at the end it could lead to
snowball effect. Now this value can be configurable by
OVNDBSPacemakerTimeout which will configure
tripleo::profile::pacemaker::ovn_dbs_bundle (default is set to 60s)

Depends-On: https://review.opendev.org/#/c/692123/
Signed-off-by: Kamil Sambor <ksambor@redhat.com>

Change-Id: I1afb5f2ef31ec61b3b224e5e1672fb9f12bcb110
(cherry picked from commit ad1ef91aa4)
(cherry picked from commit ca335af7cb)
(cherry picked from commit a429b0cf0b)
(cherry picked from commit 298788bc78)
This commit is contained in:
Kamil Sambor 2019-10-17 16:06:26 +02:00
parent f3c2cc05f9
commit f1ac643ee1
2 changed files with 16 additions and 0 deletions

View File

@ -54,6 +54,10 @@ parameters:
description: >
Setting this to a unique value will re-run any deployment tasks which
perform configuration on a Heat stack-update.
OVNDBSPacemakerTimeout:
description: timeout for monitor of ovn dbs resource in seconds
type: number
default: 60
conditions:
puppet_debug_enabled: {get_param: ConfigDebug}
@ -92,6 +96,7 @@ outputs:
- 'pcmklatest'
- tripleo::profile::pacemaker::ovn_dbs_bundle::nb_db_port: {get_param: OVNNorthboundServerPort}
- tripleo::profile::pacemaker::ovn_dbs_bundle::sb_db_port: {get_param: OVNSouthboundServerPort}
- tripleo::profile::pacemaker::ovn_dbs_bundle::dbs_timeout: {get_param: OVNDBSPacemakerTimeout}
logging_source: {get_attr: [OVNDbsBase, role_data, logging_source]}
logging_groups: {get_attr: [OVNDbsBase, role_data, logging_groups]}
service_config_settings: {get_attr: [OVNDbsBase, role_data, service_config_settings]}

View File

@ -0,0 +1,11 @@
---
features:
- |
Under pressure, the default monitor timeout value of 20 seconds is not
enough to prevent unnecessary failovers of the ovn-dbs pacemaker resource.
While spawning a few VMs in the same time this could lead to unnecessary
movements of master DB, then re-connections of ovn-controllers (slaves are
read-only), further peaks of load on DBs, and at the end it could lead to
snowball effect. Now this value can be configurable by
OVNDBSPacemakerTimeout which will configure
tripleo::profile::pacemaker::ovn_dbs_bundle (default is set to 60s).