openstack/charm-swift-storage
Code Issues Proposed changes

Swift Global Cluster

Browse Source 
This patchset adds a support for Swift Global Cluster feature as
described at:

https://docs.openstack.org/swift/latest/overview_global_cluster.html

It allows specifying 'region' config option as part of the deployment.
Theregion is reflected later on in the Swift rings.

Change-Id: I11b6c7802e5bfbd61b06e4d11c65804a165781b6
Partial-Bug: 1815879
Needed-By: I406445493e2226aa5ae40a09c9053ac8633a46e9
This commit is contained in:
Tytus Kurek 2019-02-21 12:04:00 +00:00 committed by Liam Young
parent 60dd2f0189
commit fc261a5adc
17 changed files with 378 additions and 94 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
README.md
View File

@ -8,14 +8,12 @@ those are also managed by Juju. For Swift to function, you'll also need to
deploy an additional swift-proxy using the cs:precise/swift-proxy charm.
For more information about Swift and its architecture, visit the official
project website https://docs.openstack.org/developer/swift.
[Swift project website][swift-project].
This charm was developed to support deploying multiple version of Swift on
Ubuntu Precise 12.04, as they relate to the release series of OpenStack. That
is, OpenStack Essex corresponds to Swift 1.4.8 while OpenStack Folsom shipped
1.7.4. This charm can be used to deploy either (and future) versions of Swift
onto an Ubuntu Precise 12.04, making use of the Ubuntu Cloud Archive when
needed.
This charm is intended to track each LTS release of Ubuntu Server, as well as
newer OpenStack releases via the Ubuntu Cloud Archive as supported by each
Ubuntu LTS version. Non-LTS (interim release) Ubuntu server versions are
enabled in the charms strictly for development and testing purposes.
Usage
-----
@ -32,6 +30,13 @@ If the swift-proxy charm is configured for manual zone assignment (recommended),
the 'zone' option should be set for each swift-storage service being deployed.
See the swift-proxy README for more information about zone assignment.
**Region assignment**
If the swift-proxy charm is configured with the Swift Global Cluster feature,
the 'region' option should be set for each swift-storage service being deployed.
See the [swift-proxy charm README][swift-proxy-charm-readme] for more information
about the Swift Global Cluster feature.
**Storage**
Swift storage nodes require access to local storage and filesystem. The charm
@ -43,7 +48,8 @@ device(s) to use. Options include:
block devices should be listed as a space-separated list of device nodes.
- a path to a local file on the filesystem with the size appended after a pipe,
eg "/etc/swift/storagedev1.img|5G". This will be created if it does not
exist and be mapped to a loopback device. Good for development and testing.
exist and be mapped to a loopback device. Intended strictly for development
and testing.
- "guess" can be used to tell the charm to do its best to find a local devices
to use. *EXPERIMENTAL*
@ -52,7 +58,10 @@ will each be formatted as XFS file system and mounted at /srv/node/$devname.
**Installation repository**
The 'openstack-origin' setting allows Swift to be installed from installation
repositories and can be used to setup access to the Ubuntu Cloud Archive
to support installing Swift versions more recent than what is shipped with
Ubuntu 12.04 (1.4.8). For more information, see config.yaml.
The 'openstack-origin' setting allows Swift to be installed from repositories
such as the Ubuntu Cloud Archive, which enables installation of Swift versions
more recent than what shipped with the Ubuntu LTS release. For more
information, see config.yaml.
[swift-proxy-charm-readme]: https://opendev.org/openstack/charm-swift-proxy/src/branch/master/README.md
[swift-project]: https://docs.openstack.org/developer/swift

34
config.yaml
View File

@ -5,25 +5,18 @@ options:
description: |
Repository from which to install. May be one of the following:
distro (default), ppa:somecustom/ppa, a deb url sources entry,
or a supported Cloud Archive release pocket.
or a supported Ubuntu Cloud Archive release pocket.
Supported Cloud Archive sources include:
Supported Ubuntu Cloud Archive sources include:
cloud:<series>-<openstack-release>
cloud:<series>-<openstack-release>/updates
cloud:<series>-<openstack-release>/staging
cloud:<series>-<openstack-release>/proposed
For series=Precise we support cloud archives for openstack-release:
* icehouse
For series=Trusty we support cloud archives for openstack-release:
* juno
* kilo
* ...
NOTE: updating this setting to a source that is known to provide
a later version of OpenStack will trigger a software upgrade.
a later version of OpenStack will trigger a software upgrade unless
the action-managed-upgrade configuration option is in play.
block-device:
default: sdb
type: string
@ -210,3 +203,22 @@ options:
default: 512
type: int
description: XFS inode size to use for block devices.
storage-region:
default: 1
type: int
description: |
Swift storage region to request membership. Relevant only when the
swift-proxy charm has been configured with the Swift Global
Cluster feature.
object-server-port-rep:
default: 6010
type: int
description: Listening port of the swift-object-replicator server.
container-server-port-rep:
default: 6011
type: int
description: Listening port of the swift-container-replicator server.
account-server-port-rep:
default: 6012
type: int
description: Listening port of the swift-account-replicator server.

19
hooks/swift_storage_hooks.py
View File

@ -39,7 +39,7 @@ _add_path(_root)
from lib.swift_storage_utils import (
RESTART_MAP,
restart_map,
SWIFT_SVCS,
determine_packages,
do_openstack_upgrade,
@ -57,6 +57,7 @@ from lib.swift_storage_utils import (
VERSION_PACKAGE,
setup_ufw,
revoke_access,
enable_replication,
)
from lib.misc_utils import pause_aware_restart_on_change
@ -65,6 +66,7 @@ from charmhelpers.core.hookenv import (
Hooks, UnregisteredHookError,
config,
log,
network_get_primary_address,
relation_get,
relation_ids,
relation_set,
@ -214,7 +216,7 @@ def install():
@hooks.hook('config-changed')
@pause_aware_restart_on_change(RESTART_MAP)
@pause_aware_restart_on_change(restart_map())
@harden()
def config_changed():
if config('enable-firewall'):
@ -286,7 +288,16 @@ def swift_storage_relation_joined(rid=None):
'container_port': config('container-server-port'),
'account_port': config('account-server-port'),
}
if enable_replication():
replication_ip = network_get_primary_address('replication')
cluster_ip = network_get_primary_address('cluster')
rel_settings.update({
'ip_rep': replication_ip,
'ip_cls': cluster_ip,
'region': config('storage-region'),
'object_port_rep': config('object-server-port-rep'),
'container_port_rep': config('container-server-port-rep'),
'account_port_rep': config('account-server-port-rep')})
db = kv()
devs = db.get('prepared-devices', [])
devs = [os.path.basename(d) for d in devs]
@ -300,7 +311,7 @@ def swift_storage_relation_joined(rid=None):
@hooks.hook('swift-storage-relation-changed')
@pause_aware_restart_on_change(RESTART_MAP)
@pause_aware_restart_on_change(restart_map())
def swift_storage_relation_changed():
setup_ufw()
rings_url = relation_get('rings_url')

13
lib/swift_storage_context.py
View File

@ -17,6 +17,11 @@ from charmhelpers.contrib.network.ip import (
get_ipv6_addr,
)
from charmhelpers.core.host import (
lsb_release,
CompareHostReleases,
)
class SwiftStorageContext(OSContextGenerator):
interfaces = ['swift-storage']
@ -83,8 +88,11 @@ class SwiftStorageServerContext(OSContextGenerator):
ctxt = {
'local_ip': unit_private_ip(),
'account_server_port': config('account-server-port'),
'account_server_port_rep': config('account-server-port-rep'),
'container_server_port': config('container-server-port'),
'container_server_port_rep': config('container-server-port-rep'),
'object_server_port': config('object-server-port'),
'object_server_port_rep': config('object-server-port-rep'),
'object_server_threads_per_disk': config(
'object-server-threads-per-disk'),
'account_max_connections': config('account-max-connections'),
@ -97,4 +105,9 @@ class SwiftStorageServerContext(OSContextGenerator):
'statsd_port': config('statsd-port'),
'statsd_sample_rate': config('statsd-sample-rate'),
}
ubuntu_release = lsb_release()['DISTRIB_CODENAME'].lower()
if CompareHostReleases(ubuntu_release) > "trusty":
ctxt['standalone_replicator'] = True
else:
ctxt['standalone_replicator'] = False
return ctxt

65
lib/swift_storage_utils.py
View File

@ -136,32 +136,58 @@ REQUIRED_INTERFACES = {
}
ACCOUNT_SVCS = [
'swift-account', 'swift-account-auditor',
'swift-account-reaper', 'swift-account-replicator'
'swift-account', 'swift-account-auditor', 'swift-account-reaper'
]
ACCOUNT_SVCS_REP = [
'swift-account-replicator'
]
CONTAINER_SVCS = [
'swift-container', 'swift-container-auditor',
'swift-container-updater', 'swift-container-replicator',
'swift-container', 'swift-container-auditor', 'swift-container-updater',
'swift-container-sync'
]
OBJECT_SVCS = [
'swift-object', 'swift-object-auditor',
'swift-object-updater', 'swift-object-replicator'
CONTAINER_SVCS_REP = [
'swift-container-replicator'
]
SWIFT_SVCS = ACCOUNT_SVCS + CONTAINER_SVCS + OBJECT_SVCS
OBJECT_SVCS = [
'swift-object', 'swift-object-auditor', 'swift-object-updater'
]
OBJECT_SVCS_REP = [
'swift-object-replicator'
]
SWIFT_SVCS = (
ACCOUNT_SVCS + ACCOUNT_SVCS_REP + CONTAINER_SVCS + CONTAINER_SVCS_REP +
OBJECT_SVCS + OBJECT_SVCS_REP
)
RESTART_MAP = {
'/etc/rsync-juju.d/050-swift-storage.conf': ['rsync'],
'/etc/swift/account-server.conf': ACCOUNT_SVCS,
'/etc/swift/account-server/replicator.conf': ACCOUNT_SVCS_REP,
'/etc/swift/container-server.conf': CONTAINER_SVCS,
'/etc/swift/container-server/replicator.conf': CONTAINER_SVCS_REP,
'/etc/swift/object-server.conf': OBJECT_SVCS,
'/etc/swift/swift.conf': ACCOUNT_SVCS + CONTAINER_SVCS + OBJECT_SVCS
'/etc/swift/object-server/replicator.conf': OBJECT_SVCS_REP,
'/etc/swift/swift.conf': SWIFT_SVCS
}
LEGACY_RESTART_MAP = {
'/etc/rsync-juju.d/050-swift-storage.conf': ['rsync'],
'/etc/swift/account-server.conf': ACCOUNT_SVCS + ACCOUNT_SVCS_REP,
'/etc/swift/container-server.conf': CONTAINER_SVCS + CONTAINER_SVCS_REP,
'/etc/swift/object-server.conf': OBJECT_SVCS + OBJECT_SVCS_REP,
'/etc/swift/swift.conf': SWIFT_SVCS
}
SWIFT_CONF_DIR = '/etc/swift'
SWIFT_ACCOUNT_CONF_DIR = os.path.join(SWIFT_CONF_DIR, 'account-server')
SWIFT_CONTAINER_CONF_DIR = os.path.join(SWIFT_CONF_DIR, 'container-server')
SWIFT_OBJECT_CONF_DIR = os.path.join(SWIFT_CONF_DIR, 'object-server')
SWIFT_RING_EXT = 'ring.gz'
FIRST = 1
@ -184,6 +210,9 @@ def ensure_swift_directories():
'''
dirs = [
SWIFT_CONF_DIR,
SWIFT_ACCOUNT_CONF_DIR,
SWIFT_CONTAINER_CONF_DIR,
SWIFT_OBJECT_CONF_DIR,
'/var/cache/swift',
'/srv/node',
]
@ -195,6 +224,18 @@ def vaultlocker_installed():
return len(filter_installed_packages(['vaultlocker'])) == 0
def enable_replication():
ubuntu_release = lsb_release()['DISTRIB_CODENAME'].lower()
return CompareHostReleases(ubuntu_release) > "trusty"
def restart_map():
if enable_replication():
return RESTART_MAP
else:
return LEGACY_RESTART_MAP
def register_configs():
release = get_os_codename_package('swift', fatal=False) or 'essex'
configs = templating.OSConfigRenderer(templates_dir=TEMPLATES,
@ -217,6 +258,12 @@ def register_configs():
configs.register('/etc/swift/%s-server.conf' % server, contexts)
if enable_replication():
configs.register(
'/etc/swift/{svc}-server/{svc}-server-replicator.conf'.format(
svc=server),
contexts)
return configs

3
metadata.yaml
View File

@ -23,6 +23,9 @@ series:
- eoan
- disco
- trusty
extra-bindings:
replication:
cluster:
provides:
nrpe-external-master:
interface: nrpe-external-master

33
templates/050-swift-storage.conf
View File

@ -9,6 +9,17 @@ lock file = /var/lock/account.lock
hosts allow = {{ allowed_hosts }}
{% endif %}
[account-rep]
uid = swift
gid = swift
max connections = {{ account_max_connections }}
path = /srv/node/
read only = false
lock file = /var/lock/account-rep.lock
{% if allowed_hosts -%}
hosts allow = {{ allowed_hosts }}
{% endif %}
[container]
uid = swift
gid = swift
@ -20,6 +31,17 @@ lock file = /var/lock/container.lock
hosts allow = {{ allowed_hosts }}
{% endif %}
[container-rep]
uid = swift
gid = swift
max connections = {{ container_max_connections }}
path = /srv/node/
read only = false
lock file = /var/lock/container-rep.lock
{% if allowed_hosts -%}
hosts allow = {{ allowed_hosts }}
{% endif %}
[object]
uid = swift
gid = swift
@ -30,3 +52,14 @@ lock file = /var/lock/object.lock
{% if allowed_hosts -%}
hosts allow = {{ allowed_hosts }}
{% endif %}
[object-rep]
uid = swift
gid = swift
max connections = {{ object_max_connections }}
path = /srv/node/
read only = false
lock file = /var/lock/object-rep.lock
{% if allowed_hosts -%}
hosts allow = {{ allowed_hosts }}
{% endif %}

23
templates/account-server-replicator.conf Normal file
View File

@ -0,0 +1,23 @@
[DEFAULT]
bind_ip = {{ bind_host }}
bind_port = {{ account_server_port_rep }}
workers = {{ workers }}
{% if statsd_host %}
log_statsd_host = {{ statsd_host }}
log_statsd_port = {{ statsd_port }}
log_statsd_default_sample_rate = {{ statsd_sample_rate }}
{% endif %}
[pipeline:main]
pipeline = recon account-server
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
[app:account-server]
use = egg:swift#account
replication_server = true
[account-replicator]

2
templates/account-server.conf
View File

@ -19,7 +19,9 @@ recon_cache_path = /var/cache/swift
[app:account-server]
use = egg:swift#account
{% if not standalone_replicator %}
[account-replicator]
{% endif %}
[account-auditor]

24
templates/container-server-replicator.conf Normal file
View File

@ -0,0 +1,24 @@
[DEFAULT]
bind_ip = {{ bind_host }}
bind_port = {{ container_server_port_rep }}
workers = {{ workers }}
{% if statsd_host %}
log_statsd_host = {{ statsd_host }}
log_statsd_port = {{ statsd_port }}
log_statsd_default_sample_rate = {{ statsd_sample_rate }}
{% endif %}
[pipeline:main]
pipeline = recon container-server
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
[app:container-server]
use = egg:swift#container
allow_versions = true
replication_server = true
[container-replicator]

2
templates/container-server.conf
View File

@ -20,7 +20,9 @@ recon_cache_path = /var/cache/swift
use = egg:swift#container
allow_versions = true
{% if not standalone_replicator %}
[container-replicator]
{% endif %}
[container-updater]

26
templates/object-server-replicator.conf Normal file
View File

@ -0,0 +1,26 @@
[DEFAULT]
bind_ip = {{ bind_host }}
bind_port = {{ object_server_port_rep }}
workers = {{ workers }}
{% if statsd_host %}
log_statsd_host = {{ statsd_host }}
log_statsd_port = {{ statsd_port }}
log_statsd_default_sample_rate = {{ statsd_sample_rate }}
{% endif %}
[pipeline:main]
pipeline = recon object-server
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
[app:object-server]
use = egg:swift#object
threads_per_disk = {{ object_server_threads_per_disk }}
replication_server = true
[object-replicator]
concurrency = {{ object_replicator_concurrency }}
rsync_timeout = {{ object_rsync_timeout }}

8
templates/object-server.conf
View File

@ -20,12 +20,16 @@ recon_cache_path = /var/cache/swift
use = egg:swift#object
threads_per_disk = {{ object_server_threads_per_disk }}
{% if standalone_replicator %}
[object-reconstructor]
{% else %}
[object-replicator]
concurrency = {{ object_replicator_concurrency }}
rsync_timeout = {{ object_rsync_timeout }}
[object-updater]
[object-sync]
{% endif %}
[object-auditor]
[object-sync]
[object-updater]

7
unit_tests/test_swift_storage_context.py
View File

@ -84,8 +84,11 @@ class SwiftStorageContextTests(CharmTestCase):
def test_swift_storage_server_context(self):
self.unit_private_ip.return_value = '10.0.0.5'
self.test_config.set('account-server-port', '500')
self.test_config.set('account-server-port-rep', '510')
self.test_config.set('object-server-port', '501')
self.test_config.set('object-server-port-rep', '511')
self.test_config.set('container-server-port', '502')
self.test_config.set('container-server-port-rep', '512')
self.test_config.set('object-server-threads-per-disk', '3')
self.test_config.set('object-replicator-concurrency', '3')
self.test_config.set('account-max-connections', '10')
@ -96,8 +99,11 @@ class SwiftStorageContextTests(CharmTestCase):
result = ctxt()
ex = {
'container_server_port': '502',
'container_server_port_rep': '512',
'object_server_port': '501',
'object_server_port_rep': '511',
'account_server_port': '500',
'account_server_port_rep': '510',
'local_ip': '10.0.0.5',
'object_server_threads_per_disk': '3',
'object_replicator_concurrency': '3',
@ -105,6 +111,7 @@ class SwiftStorageContextTests(CharmTestCase):
'container_max_connections': '10',
'object_max_connections': '10',
'object_rsync_timeout': '950',
'standalone_replicator': True,
'statsd_host': '',
'statsd_port': 3125,
'statsd_sample_rate': 1.0

20
unit_tests/test_swift_storage_relations.py
View File

@ -35,6 +35,7 @@ TO_PATCH = [
'Hooks',
'config',
'log',
'network_get_primary_address',
'relation_set',
'relation_ids',
'relation_get',
@ -101,7 +102,9 @@ class SwiftStorageRelationsTests(CharmTestCase):
self.add_to_updatedb_prunepath.assert_called_with("/srv/node")
@patch.object(hooks, 'add_ufw_gre_rule', lambda *args: None)
def test_install_hook(self):
@patch('lib.swift_storage_utils.swift_init')
def test_install_hook(self, mock_swift_init):
mock_swift_init.return_value = 0
self.test_config.set('openstack-origin', 'cloud:precise-havana')
hooks.install()
self.configure_installation_source.assert_called_with(
@ -198,6 +201,7 @@ class SwiftStorageRelationsTests(CharmTestCase):
kvstore.__enter__.return_value = kvstore
kvstore.get.return_value = None
self.test_kv.set('prepared-devices', ['/dev/vdb'])
self.network_get_primary_address.return_value = "10.10.10.2"
# py3 is very picky, and log is only patched in
# hooks.swift_storage_hooks
@ -209,12 +213,18 @@ class SwiftStorageRelationsTests(CharmTestCase):
mock_rel_set.assert_called_with(
relation_id=None,
relation_settings={
"device": 'vdb',
"object_port": 6000,
"account_port": 6002,
"ip_cls": "10.10.10.2",
"ip_rep": "10.10.10.2",
"private-address": "10.10.10.2",
"region": 1,
"zone": 1,
"device": 'vdb',
"account_port": 6002,
"account_port_rep": 6012,
"container_port": 6001,
"private-address": "10.10.10.2"
"container_port_rep": 6011,
"object_port": 6000,
"object_port_rep": 6010
}
)

88
unit_tests/test_swift_storage_utils.py
View File

@ -32,6 +32,7 @@ TO_PATCH = [
'mkdir',
'mount',
'check_call',
'check_output',
'call',
'ensure_block_device',
'clean_storage',
@ -124,6 +125,13 @@ class SwiftStorageUtilsTests(CharmTestCase):
swift_utils.ensure_swift_directories()
ex_dirs = [
call('/etc/swift', owner='swift', group='swift'),
call('/etc/swift/account-server',
owner='swift',
group='swift'),
call('/etc/swift/container-server',
owner='swift',
group='swift'),
call('/etc/swift/object-server', owner='swift', group='swift'),
call('/var/cache/swift', owner='swift', group='swift'),
call('/srv/node', owner='swift', group='swift')
]
@ -435,13 +443,29 @@ class SwiftStorageUtilsTests(CharmTestCase):
'DISTRIB_DESCRIPTION': 'Ubuntu 14.04'}
swift_utils.assert_charm_supports_ipv6()
def test_enable_replication(self):
self.lsb_release.return_value = {'DISTRIB_ID': 'Ubuntu',
'DISTRIB_RELEASE': '14.04',
'DISTRIB_CODENAME': 'trusty',
'DISTRIB_DESCRIPTION': 'Ubuntu 14.04'}
self.assertFalse(swift_utils.enable_replication())
self.lsb_release.return_value = {'DISTRIB_ID': 'Ubuntu',
'DISTRIB_RELEASE': '18.04',
'DISTRIB_CODENAME': 'bionic',
'DISTRIB_DESCRIPTION': 'Ubuntu 18.04'}
self.assertTrue(swift_utils.enable_replication())
@patch.object(swift_utils, 'enable_replication')
@patch('charmhelpers.contrib.openstack.templating.OSConfigRenderer')
def test_register_configs_pre_install(self, renderer):
def test_register_configs_pre_install(self, renderer, enable_replication):
enable_replication.return_value = True
self.get_os_codename_package.return_value = None
swift_utils.register_configs()
renderer.assert_called_with(templates_dir=swift_utils.TEMPLATES,
openstack_release='essex')
@patch.object(swift_utils, 'vaultlocker_installed')
@patch.object(swift_utils, 'enable_replication')
@patch.object(swift_utils, 'filter_installed_packages')
@patch('charmhelpers.contrib.openstack.context.WorkerConfigContext')
@patch('charmhelpers.contrib.openstack.context.BindHostContext')
@ -452,7 +476,11 @@ class SwiftStorageUtilsTests(CharmTestCase):
def test_register_configs_post_install(self, renderer,
swift, rsync, server,
bind_context, worker_context,
filter_installed_packages):
filter_installed_packages,
enable_replication,
vaultlocker_installed):
vaultlocker_installed.return_value = True
enable_replication.return_value = True
filter_installed_packages.return_value = []
swift.return_value = 'swift_context'
rsync.return_value = 'rsync_context'
@ -471,20 +499,50 @@ class SwiftStorageUtilsTests(CharmTestCase):
call('/etc/swift/swift.conf', ['swift_server_context']),
call('/etc/rsync-juju.d/050-swift-storage.conf',
['rsync_context', 'swift_context']),
call('/etc/swift/account-server.conf', ['swift_context',
'bind_host_context',
'worker_context',
'vl_context']),
call('/etc/swift/object-server.conf', ['swift_context',
'bind_host_context',
'worker_context',
'vl_context']),
call('/etc/swift/container-server.conf', ['swift_context',
'bind_host_context',
'worker_context',
'vl_context'])
call(
'/etc/swift/account-server.conf',
[
'swift_context',
'bind_host_context',
'worker_context',
'vl_context']),
call(
'/etc/swift/container-server.conf',
[
'swift_context',
'bind_host_context',
'worker_context',
'vl_context']),
call(
'/etc/swift/object-server.conf',
[
'swift_context',
'bind_host_context',
'worker_context',
'vl_context']),
call(
'/etc/swift/account-server/account-server-replicator.conf',
[
'swift_context',
'bind_host_context',
'worker_context',
'vl_context']),
call('/etc/swift/container-server/container-server-replicator.conf',
[
'swift_context',
'bind_host_context',
'worker_context',
'vl_context']),
call(
'/etc/swift/object-server/object-server-replicator.conf',
[
'swift_context',
'bind_host_context',
'worker_context',
'vl_context'])
]
self.assertEqual(ex, configs.register.call_args_list)
self.assertEqual(sorted(ex), sorted(configs.register.call_args_list))
@patch.object(swift_utils, 'remove_old_packages')
def test_do_upgrade_queens(self, mock_remove_old_packages):

70
unit_tests/test_templates.py
View File

@ -27,6 +27,7 @@ class StorageServerTemplateTestCase(unittest.TestCase):
self,
os_release,
server,
suffix,
mock_log):
if not server:
@ -35,53 +36,52 @@ class StorageServerTemplateTestCase(unittest.TestCase):
loader = get_loader('./templates', os_release)
env = Environment(loader=loader)
return env.get_template('{}-server.conf'.format(server))
return env.get_template('{}-server{}.conf'.format(server, suffix))
def get_templates(self, os_releases):
templates = []
for release in os_releases:
for server in ('object', 'container', 'account'):
for suffix in ('', '-replicator'):
templates.append(self.get_template_for_release_and_server(
release,
server,
suffix))
return templates
def test_os_release_not_in_templates(self):
"""Regression test for bug 1251551.
The os_release is no longer provided as context to the templates.
"""
for release in ('essex', 'grizzly', 'havana', 'icehouse'):
for server in ('object', 'container', 'account'):
template = self.get_template_for_release_and_server(
release,
server)
with open(template.filename, 'r') as template_orig:
self.assertNotIn(
'os_release', template_orig.read(),
"The template '{}' contains os_release which is "
"no longer provided in the context.".format(
template.filename))
for template in self.get_templates(('essex', 'grizzly', 'havana',
'icehouse')):
with open(template.filename, 'r') as template_orig:
self.assertNotIn(
'os_release', template_orig.read(),
"The template '{}' contains os_release which is "
"no longer provided in the context.".format(
template.filename))
def test_config_renders_for_all_releases_and_servers(self):
"""The configs render without syntax error."""
for release in ('essex', 'grizzly', 'havana', 'icehouse'):
for server in ('object', 'container', 'account'):
template = self.get_template_for_release_and_server(
release,
server)
result = template.render()
self.assertTrue(result.startswith("[DEFAULT]"))
for template in self.get_templates(('essex', 'grizzly', 'havana',
'icehouse')):
result = template.render()
self.assertTrue(result.startswith("[DEFAULT]"))
def test_statsd_config_for_all_releases_and_servers(self):
"""The configs contain statsd settings if statsd-host is set."""
for release in ('grizzly', 'havana', 'icehouse', 'mitaka'):
for server in ('object', 'container', 'account'):
template = self.get_template_for_release_and_server(
release,
server)
for template in self.get_templates(('grizzly', 'havana', 'icehouse',
'mitaka')):
result = template.render(statsd_host='127.0.0.1')
result = template.render(statsd_host='127.0.0.1')
self.assertIn("log_statsd_host", result)
self.assertIn("log_statsd_port", result)
self.assertIn("log_statsd_default_sample_rate", result)
self.assertIn("log_statsd_host", result)
self.assertIn("log_statsd_port", result)
self.assertIn("log_statsd_default_sample_rate", result)
result = template.render()
result = template.render()
self.assertNotIn("log_statsd_host", result)
self.assertNotIn("log_statsd_port", result)
self.assertNotIn("log_statsd_default_sample_rate", result)
self.assertNotIn("log_statsd_host", result)
self.assertNotIn("log_statsd_port", result)
self.assertNotIn("log_statsd_default_sample_rate", result)