Browse Source

Fix series-upgrade issue where seeded file goes missing

During resume on a non-leader unit, the 'seeded' file can go missing
when the package syncs over to the /var/lib/mysql or
/var/lib/percona-xtradb-cluster directories (vivid+).  It's not really
clear why it doesn't do this every time (i.e. not every non-leader unit
fails), but this fix ensures that if the unit *is* seeded prior to the
series-upgrade, then it stays seeded after the series upgrade.

The related zaza-openstack-tests change [1] is about fixing the
series-upgrade test.

Note there is no trusty-mitaka test here as the charm doesn't support
trusty.  The last version of percona-cluster in the charm store that
supports trusty is rev. 276

[1]: https://github.com/openstack-charmers/zaza-openstack-tests/pull/406

Change-Id: I628be1c24081d7e0e150e5064c5fa4ab694632e9
Closes-bug: #1868326
changes/22/749922/4
Alex Kavanagh 6 months ago
parent
commit
1b813d1fd9
4 changed files with 39 additions and 3 deletions
  1. +20
    -3
      hooks/percona_utils.py
  2. +1
    -0
      tests/bundles/overlays/xenial-queens-ha-series-upgrade.yaml.j2
  3. +15
    -0
      tests/bundles/xenial-queens-ha-series-upgrade.yaml
  4. +3
    -0
      tests/tests.yaml

+ 20
- 3
hooks/percona_utils.py View File

@ -682,11 +682,23 @@ def cluster_in_sync():
return False
def charm_check_func():
def charm_check_func(ensure_seeded=False):
"""Custom function to assess the status of the current unit
@returns (status, message) - tuple of strings if an issue
:param ensure_seeded: if set, then ensure that the seeded file exists.
This is to work around bug: #1868326 where the resume action to
manage_payload_services causes the seeded file to be removed under
certain circumstances. This is one of the only places where we can
actually re-assert it prior to the resume failing, as this function
gets a chance to run after management_payload_services() but prior to
the actual
:type ensure_seeded: bool
:returns: (status, message)
:rtype: Tuple[str, str]
"""
if ensure_seeded and not seeded():
log("'seeded' file is missing but should exists; putting it back.")
mark_seeded()
if is_unit_upgrading_set():
# Avoid looping through attempting to determine cluster_in_sync
return ("blocked", "Unit upgrading.")
@ -830,9 +842,14 @@ def assess_status_func(configs):
@param configs: a templating.OSConfigRenderer() object
@return f() -> None : a function that assesses the unit's workload status
"""
# BUG: 1868326 - the resume action on the mysql service can wipe out the
# seeded file; if it is seeded, then we put it back. This function is
# called as part of the resume_unit() call and is one of the only places to
# actually do and fix this check.
is_seeded = seeded()
return make_assess_status_func(
configs, REQUIRED_INTERFACES,
charm_func=lambda _: charm_check_func(),
charm_func=lambda _: charm_check_func(ensure_seeded=is_seeded),
services=services(), ports=None)


+ 1
- 0
tests/bundles/overlays/xenial-queens-ha-series-upgrade.yaml.j2 View File

@ -0,0 +1 @@
ha.yaml

+ 15
- 0
tests/bundles/xenial-queens-ha-series-upgrade.yaml View File

@ -0,0 +1,15 @@
series: xenial
relations:
- - keystone:shared-db
- percona-cluster:shared-db
applications:
percona-cluster:
series: xenial
charm: ../../../percona-cluster
num_units: 1
keystone:
charm: cs:~openstack-charmers-next/keystone
num_units: 1
options:
openstack-origin: cloud:xenial-queens
token-expiration: 60

+ 3
- 0
tests/tests.yaml View File

@ -7,6 +7,8 @@ tests:
- xenial_model:
- zaza.openstack.charm_tests.mysql.tests.PerconaClusterCharmTests
- zaza.openstack.charm_tests.mysql.tests.PerconaClusterScaleTests
- xenial_series_upgrade_model:
- zaza.openstack.charm_tests.series_upgrade.tests.XenialBionicSeriesUpgrade
configure:
- zaza.model.block_until_all_units_idle
target_deploy_status:
@ -16,3 +18,4 @@ gate_bundles:
smoke_bundles:
- bionic_model: bionic-ha
dev_bundles:
- xenial_series_upgrade_model: xenial-queens-ha-series-upgrade

Loading…
Cancel
Save