Browse Source

Update mariadb-server suspect_timeout to default value to align

with garbd's suspect_timeout

In openstack-helm-infra, it launch evs.suspect_timeout=PT30S
for mariadb-server in configmap, mariadb-etc. This setting is
for three mariadb-server pod deployment, every mariadb-server
with same setting suspect_timeout=30s. But after change to two
mariadb-server and one garbd arbitrator. Setting in configmap
mariadb-etc evs.suspect_timeout=PT30S, only takes effect for 2
mariadb-server, for garbd arbitrator, it use galera default
setting evs.suspect_timeout=PT5S. If mariadb-server-1 exit
abnormal, after 5s, garbd arbitrator suspects mariadb-server-1
is dead, but as not reach 30s, mariadb-server-0 thinks mariadb-server-1
is not dead. In this state, quorum fail, garbd arbitrator and
mariadb-server-0 both set to none primary component, service
down.
For fix solution, set value.conf.data.config_override to override
wsrep_provider_option in mariadb helm chart, which makes garbd
arbitrator and mariadb-server launch with same setting for
"evs.suspect_timeout=PT5S", default value. By this way, mariadb
server recovery time will also improve. To update setting for
"evs.suspect_timeout", it should both update override for mariadb
and garbd helm chart.

Setting for "gmcast.listen_addr=tcp://0.0.0.0:<port>", takes
effect for both ipv4 and ipv6. So keeps such setting.

Reference link for wsrep option and galera cluster quorum
https://mariadb.com/kb/en/wsrep_provider_options/
https://galeracluster.com/library/documentation/weighted-quorum.html

Closes-Bug: 1888546

Change-Id: I06983cf0d91d4d9aa88f352e64b1e6571b816ec6
Signed-off-by: Martin, Chen <haochuan.z.chen@intel.com>
changes/86/744486/6
Martin, Chen 2 years ago
parent
commit
2f664927c4
  1. 3
      openstack-helm-infra/centos/openstack-helm-infra.spec
  2. 59
      openstack-helm-infra/files/0013-enable-Values.conf.database.config_override-for-mari.patch
  3. 11
      python-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/helm/mariadb.py
  4. 4
      stx-openstack-helm/stx-openstack-helm/manifests/manifest.yaml

3
openstack-helm-infra/centos/openstack-helm-infra.spec

@ -27,6 +27,7 @@ Patch09: 0009-Enable-override-of-mariadb-server-probe-parameters.patch
Patch10: 0010-Mariadb-use-utf8_general_ci-collation-as-default.patch
Patch11: 0011-Add-mariadb-database-config-override-to-support-ipv6.patch
Patch12: 0012-Add-tolerations-to-rabbitmq-chart.patch
Patch13: 0013-enable-Values.conf.database.config_override-for-mari.patch
BuildRequires: helm
BuildRequires: chartmuseum
@ -48,7 +49,7 @@ Openstack Helm Infra charts
%patch10 -p1
%patch11 -p1
%patch12 -p1
%patch13 -p1
%build
# Host a server for the charts

59
openstack-helm-infra/files/0013-enable-Values.conf.database.config_override-for-mari.patch

@ -0,0 +1,59 @@
From c75c04ea1c2250f16c1aabe57a2d4e94d986a90e Mon Sep 17 00:00:00 2001
From: "Chen, Haochuan Z" <haochuan.z.chen@intel.com>
Date: Wed, 5 Aug 2020 03:08:14 +0000
Subject: [PATCH 13/13] enable Values.conf.database.config_override for mariadb
option wsrep_provider_options override
---
mariadb/templates/configmap-etc.yaml | 5 ++---
mariadb/templates/statefulset.yaml | 8 +++++++-
2 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/mariadb/templates/configmap-etc.yaml b/mariadb/templates/configmap-etc.yaml
index 53072f4..7bdfd8a 100644
--- a/mariadb/templates/configmap-etc.yaml
+++ b/mariadb/templates/configmap-etc.yaml
@@ -26,14 +26,13 @@ data:
{{- include "helm-toolkit.snippets.values_template_renderer" (dict "envAll" $envAll "template" ( index $envAll.Values.conf.database "00_base" ) "key" "00-base.cnf" ) | indent 2 }}
{{- if $envAll.Values.conf.database.config_override }}
{{- include "helm-toolkit.snippets.values_template_renderer" (dict "envAll" $envAll "template" ( index $envAll.Values.conf.database "config_override" ) "key" "20-override.cnf" ) | indent 2 }}
-{{- else }}
+{{- end }}
# A user override for conf.database.config_override might break the IPv6 system overrides, need including below ipv6
# config for ipv6 case in user override as well.
{{- if $envAll.Values.manifests.config_ipv6 }}
- 20-override.cnf: |
+ 30-override.cnf: |
[mysqld]
bind_address=::
{{- end }}
-{{- end }}
{{- include "helm-toolkit.snippets.values_template_renderer" (dict "envAll" $envAll "template" ( index $envAll.Values.conf.database "99_force" ) "key" "99-force.cnf" ) | indent 2 }}
{{- end }}
diff --git a/mariadb/templates/statefulset.yaml b/mariadb/templates/statefulset.yaml
index 3ef7383..88439e2 100644
--- a/mariadb/templates/statefulset.yaml
+++ b/mariadb/templates/statefulset.yaml
@@ -216,12 +216,18 @@ spec:
mountPath: /etc/mysql/conf.d/00-base.cnf
subPath: 00-base.cnf
readOnly: true
- {{- if or (.Values.conf.database.config_override) (.Values.manifests.config_ipv6) }}
+ {{- if .Values.conf.database.config_override }}
- name: mariadb-etc
mountPath: /etc/mysql/conf.d/20-override.cnf
subPath: 20-override.cnf
readOnly: true
{{- end }}
+ {{- if .Values.manifests.config_ipv6 }}
+ - name: mariadb-etc
+ mountPath: /etc/mysql/conf.d/30-override.cnf
+ subPath: 30-override.cnf
+ readOnly: true
+ {{- end }}
- name: mariadb-etc
mountPath: /etc/mysql/conf.d/99-force.cnf
subPath: 99-force.cnf
--
1.8.3.1

11
python-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/helm/mariadb.py

@ -7,6 +7,7 @@
from k8sapp_openstack.common import constants as app_constants
from k8sapp_openstack.helm import openstack
from sysinv.common import utils as cutils
from sysinv.common import exception
from sysinv.helm import common
@ -35,6 +36,16 @@ class MariadbHelm(openstack.OpenstackBaseHelm):
}
}
if not cutils.is_std_system(self.dbapi):
config_override = {
'conf': {
'database': {
'config_override': ''
}
}
}
overrides[common.HELM_NS_OPENSTACK].update(config_override)
if namespace in self.SUPPORTED_NAMESPACES:
return overrides[namespace]
elif namespace:

4
stx-openstack-helm/stx-openstack-helm/manifests/manifest.yaml

@ -186,6 +186,10 @@ data:
conf:
ingress_conf:
worker-processes: '4'
database:
config_override: |
[mysqld]
wsrep_provider_options="gmcast.listen_addr=tcp://0.0.0.0:{{ tuple "oslo_db" "direct" "wsrep" . | include "helm-toolkit.endpoints.endpoint_port_lookup" }}"
source:
type: tar
location: http://172.17.0.1/helm_charts/starlingx/mariadb-0.1.0.tgz

Loading…
Cancel
Save