Swift Global Cluster

This patchset adds a support for Swift Global Cluster feature as
described at:

https://docs.openstack.org/swift/latest/overview_global_cluster.html

It allows specifying 'region' config option as part of the deployment.
Theregion is reflected later on in the Swift rings.

Change-Id: I11b6c7802e5bfbd61b06e4d11c65804a165781b6
Partial-Bug: 1815879
Needed-By: I406445493e2226aa5ae40a09c9053ac8633a46e9

README.md

@@ -8,14 +8,12 @@ those are also managed by Juju.  For Swift to function, you'll also need to
 deploy an additional swift-proxy using the cs:precise/swift-proxy charm.
 
 For more information about Swift and its architecture, visit the official
-project website https://docs.openstack.org/developer/swift.
+[Swift project website][swift-project].
 
-This charm was developed to support deploying multiple version of Swift on
+This charm is intended to track each LTS release of Ubuntu Server, as well as
-Ubuntu Precise 12.04, as they relate to the release series of OpenStack.  That
+newer OpenStack releases via the Ubuntu Cloud Archive as supported by each
-is, OpenStack Essex corresponds to Swift 1.4.8 while OpenStack Folsom shipped
+Ubuntu LTS version.  Non-LTS (interim release) Ubuntu server versions are
-1.7.4.  This charm can be used to deploy either (and future) versions of Swift
+enabled in the charms strictly for development and testing purposes.
-onto an Ubuntu Precise 12.04, making use of the Ubuntu Cloud Archive when
-needed.
 
 Usage
 -----
@@ -32,6 +30,13 @@ If the swift-proxy charm is configured for manual zone assignment (recommended),
 the 'zone' option should be set for each swift-storage service being deployed.
 See the swift-proxy README for more information about zone assignment.
 
+**Region assignment**
+
+If the swift-proxy charm is configured with the Swift Global Cluster feature,
+the 'region' option should be set for each swift-storage service being deployed.
+See the [swift-proxy charm README][swift-proxy-charm-readme] for more information
+about the Swift Global Cluster feature.
+
 **Storage**
 
 Swift storage nodes require access to local storage and filesystem.  The charm
@@ -43,7 +48,8 @@ device(s) to use.  Options include:
    block devices should be listed as a space-separated list of device nodes.
  - a path to a local file on the filesystem with the size appended after a pipe,
    eg "/etc/swift/storagedev1.img|5G".  This will be created if it does not
-   exist and be mapped to a loopback device. Good for development and testing.
+   exist and be mapped to a loopback device. Intended strictly for development
+   and testing.
  - "guess" can be used to tell the charm to do its best to find a local devices
    to use. *EXPERIMENTAL*
 
@@ -52,7 +58,10 @@ will each be formatted as XFS file system and mounted at /srv/node/$devname.
 
 **Installation repository**
 
-The 'openstack-origin' setting allows Swift to be installed from installation
+The 'openstack-origin' setting allows Swift to be installed from repositories
-repositories and can be used to setup access to the Ubuntu Cloud Archive
+such as the Ubuntu Cloud Archive, which enables installation of Swift versions
-to support installing Swift versions more recent than what is shipped with
+more recent than what shipped with the Ubuntu LTS release.  For more
-Ubuntu 12.04 (1.4.8).  For more information, see config.yaml.
+information, see config.yaml.
+
+[swift-proxy-charm-readme]: https://opendev.org/openstack/charm-swift-proxy/src/branch/master/README.md
+[swift-project]: https://docs.openstack.org/developer/swift

config.yaml

@@ -5,25 +5,18 @@ options:
     description: |
       Repository from which to install.  May be one of the following:
       distro (default), ppa:somecustom/ppa, a deb url sources entry,
-      or a supported Cloud Archive release pocket.
+      or a supported Ubuntu Cloud Archive release pocket.
 
-      Supported Cloud Archive sources include:
+      Supported Ubuntu Cloud Archive sources include:
 
       cloud:<series>-<openstack-release>
       cloud:<series>-<openstack-release>/updates
       cloud:<series>-<openstack-release>/staging
       cloud:<series>-<openstack-release>/proposed
-
+      
-      For series=Precise we support cloud archives for openstack-release:
-         * icehouse
-
-      For series=Trusty we support cloud archives for openstack-release:
-         * juno
-         * kilo
-         * ...
-
       NOTE: updating this setting to a source that is known to provide
-      a later version of OpenStack will trigger a software upgrade.
+      a later version of OpenStack will trigger a software upgrade unless
+      the action-managed-upgrade configuration option is in play.
   block-device:
     default: sdb
     type: string
@@ -210,3 +203,22 @@ options:
     default: 512
     type: int
     description: XFS inode size to use for block devices.
+  storage-region:
+    default: 1
+    type: int
+    description: |
+      Swift storage region to request membership. Relevant only when the
+      swift-proxy charm has been configured with the Swift Global
+      Cluster feature.
+  object-server-port-rep:
+    default: 6010
+    type: int
+    description: Listening port of the swift-object-replicator server.
+  container-server-port-rep:
+    default: 6011
+    type: int
+    description: Listening port of the swift-container-replicator server.
+  account-server-port-rep:
+    default: 6012
+    type: int
+    description: Listening port of the swift-account-replicator server.

hooks/swift_storage_hooks.py

@@ -39,7 +39,7 @@ _add_path(_root)
 
 
 from lib.swift_storage_utils import (
-    RESTART_MAP,
+    restart_map,
     SWIFT_SVCS,
     determine_packages,
     do_openstack_upgrade,
@@ -57,6 +57,7 @@ from lib.swift_storage_utils import (
     VERSION_PACKAGE,
     setup_ufw,
     revoke_access,
+    enable_replication,
 )
 
 from lib.misc_utils import pause_aware_restart_on_change
@@ -65,6 +66,7 @@ from charmhelpers.core.hookenv import (
     Hooks, UnregisteredHookError,
     config,
     log,
+    network_get_primary_address,
     relation_get,
     relation_ids,
     relation_set,
@@ -214,7 +216,7 @@ def install():
 
 
 @hooks.hook('config-changed')
-@pause_aware_restart_on_change(RESTART_MAP)
+@pause_aware_restart_on_change(restart_map())
 @harden()
 def config_changed():
     if config('enable-firewall'):
@@ -286,7 +288,16 @@ def swift_storage_relation_joined(rid=None):
         'container_port': config('container-server-port'),
         'account_port': config('account-server-port'),
     }
-
+    if enable_replication():
+        replication_ip = network_get_primary_address('replication')
+        cluster_ip = network_get_primary_address('cluster')
+        rel_settings.update({
+            'ip_rep': replication_ip,
+            'ip_cls': cluster_ip,
+            'region': config('storage-region'),
+            'object_port_rep': config('object-server-port-rep'),
+            'container_port_rep': config('container-server-port-rep'),
+            'account_port_rep': config('account-server-port-rep')})
     db = kv()
     devs = db.get('prepared-devices', [])
     devs = [os.path.basename(d) for d in devs]
@@ -300,7 +311,7 @@ def swift_storage_relation_joined(rid=None):
 
 
 @hooks.hook('swift-storage-relation-changed')
-@pause_aware_restart_on_change(RESTART_MAP)
+@pause_aware_restart_on_change(restart_map())
 def swift_storage_relation_changed():
     setup_ufw()
     rings_url = relation_get('rings_url')

lib/swift_storage_context.py

@@ -17,6 +17,11 @@ from charmhelpers.contrib.network.ip import (
     get_ipv6_addr,
 )
 
+from charmhelpers.core.host import (
+    lsb_release,
+    CompareHostReleases,
+)
+
 
 class SwiftStorageContext(OSContextGenerator):
     interfaces = ['swift-storage']
@@ -83,8 +88,11 @@ class SwiftStorageServerContext(OSContextGenerator):
         ctxt = {
             'local_ip': unit_private_ip(),
             'account_server_port': config('account-server-port'),
+            'account_server_port_rep': config('account-server-port-rep'),
             'container_server_port': config('container-server-port'),
+            'container_server_port_rep': config('container-server-port-rep'),
             'object_server_port': config('object-server-port'),
+            'object_server_port_rep': config('object-server-port-rep'),
             'object_server_threads_per_disk': config(
                 'object-server-threads-per-disk'),
             'account_max_connections': config('account-max-connections'),
@@ -97,4 +105,9 @@ class SwiftStorageServerContext(OSContextGenerator):
             'statsd_port': config('statsd-port'),
             'statsd_sample_rate': config('statsd-sample-rate'),
         }
+        ubuntu_release = lsb_release()['DISTRIB_CODENAME'].lower()
+        if CompareHostReleases(ubuntu_release) > "trusty":
+            ctxt['standalone_replicator'] = True
+        else:
+            ctxt['standalone_replicator'] = False
         return ctxt

lib/swift_storage_utils.py

@@ -136,32 +136,58 @@ REQUIRED_INTERFACES = {
 }
 
 ACCOUNT_SVCS = [
-    'swift-account', 'swift-account-auditor',
+    'swift-account', 'swift-account-auditor', 'swift-account-reaper'
-    'swift-account-reaper', 'swift-account-replicator'
+]
+
+ACCOUNT_SVCS_REP = [
+    'swift-account-replicator'
 ]
 
 CONTAINER_SVCS = [
-    'swift-container', 'swift-container-auditor',
+    'swift-container', 'swift-container-auditor', 'swift-container-updater',
-    'swift-container-updater', 'swift-container-replicator',
     'swift-container-sync'
 ]
 
-OBJECT_SVCS = [
+CONTAINER_SVCS_REP = [
-    'swift-object', 'swift-object-auditor',
+    'swift-container-replicator'
-    'swift-object-updater', 'swift-object-replicator'
 ]
 
-SWIFT_SVCS = ACCOUNT_SVCS + CONTAINER_SVCS + OBJECT_SVCS
+OBJECT_SVCS = [
+    'swift-object', 'swift-object-auditor', 'swift-object-updater'
+]
+
+OBJECT_SVCS_REP = [
+    'swift-object-replicator'
+]
+
+SWIFT_SVCS = (
+    ACCOUNT_SVCS + ACCOUNT_SVCS_REP + CONTAINER_SVCS + CONTAINER_SVCS_REP +
+    OBJECT_SVCS + OBJECT_SVCS_REP
+)
 
 RESTART_MAP = {
     '/etc/rsync-juju.d/050-swift-storage.conf': ['rsync'],
     '/etc/swift/account-server.conf': ACCOUNT_SVCS,
+    '/etc/swift/account-server/replicator.conf': ACCOUNT_SVCS_REP,
     '/etc/swift/container-server.conf': CONTAINER_SVCS,
+    '/etc/swift/container-server/replicator.conf': CONTAINER_SVCS_REP,
     '/etc/swift/object-server.conf': OBJECT_SVCS,
-    '/etc/swift/swift.conf': ACCOUNT_SVCS + CONTAINER_SVCS + OBJECT_SVCS
+    '/etc/swift/object-server/replicator.conf': OBJECT_SVCS_REP,
+    '/etc/swift/swift.conf': SWIFT_SVCS
+}
+
+LEGACY_RESTART_MAP = {
+    '/etc/rsync-juju.d/050-swift-storage.conf': ['rsync'],
+    '/etc/swift/account-server.conf': ACCOUNT_SVCS + ACCOUNT_SVCS_REP,
+    '/etc/swift/container-server.conf': CONTAINER_SVCS + CONTAINER_SVCS_REP,
+    '/etc/swift/object-server.conf': OBJECT_SVCS + OBJECT_SVCS_REP,
+    '/etc/swift/swift.conf': SWIFT_SVCS
 }
 
 SWIFT_CONF_DIR = '/etc/swift'
+SWIFT_ACCOUNT_CONF_DIR = os.path.join(SWIFT_CONF_DIR, 'account-server')
+SWIFT_CONTAINER_CONF_DIR = os.path.join(SWIFT_CONF_DIR, 'container-server')
+SWIFT_OBJECT_CONF_DIR = os.path.join(SWIFT_CONF_DIR, 'object-server')
 SWIFT_RING_EXT = 'ring.gz'
 
 FIRST = 1
@@ -184,6 +210,9 @@ def ensure_swift_directories():
     '''
     dirs = [
         SWIFT_CONF_DIR,
+        SWIFT_ACCOUNT_CONF_DIR,
+        SWIFT_CONTAINER_CONF_DIR,
+        SWIFT_OBJECT_CONF_DIR,
         '/var/cache/swift',
         '/srv/node',
     ]
@@ -195,6 +224,18 @@ def vaultlocker_installed():
     return len(filter_installed_packages(['vaultlocker'])) == 0
 
 
+def enable_replication():
+    ubuntu_release = lsb_release()['DISTRIB_CODENAME'].lower()
+    return CompareHostReleases(ubuntu_release) > "trusty"
+
+
+def restart_map():
+    if enable_replication():
+        return RESTART_MAP
+    else:
+        return LEGACY_RESTART_MAP
+
+
 def register_configs():
     release = get_os_codename_package('swift', fatal=False) or 'essex'
     configs = templating.OSConfigRenderer(templates_dir=TEMPLATES,
@@ -217,6 +258,12 @@ def register_configs():
 
         configs.register('/etc/swift/%s-server.conf' % server, contexts)
 
+        if enable_replication():
+            configs.register(
+                '/etc/swift/{svc}-server/{svc}-server-replicator.conf'.format(
+                    svc=server),
+                contexts)
+
     return configs
 
 

metadata.yaml

@@ -23,6 +23,9 @@ series:
   - eoan
   - disco
   - trusty
+extra-bindings:
+  replication:
+  cluster:
 provides:
   nrpe-external-master:
     interface: nrpe-external-master

templates/050-swift-storage.conf

@@ -9,6 +9,17 @@ lock file = /var/lock/account.lock
 hosts allow = {{ allowed_hosts }}
 {% endif %}
 
+[account-rep]
+uid = swift
+gid = swift
+max connections = {{ account_max_connections }}
+path = /srv/node/
+read only = false
+lock file = /var/lock/account-rep.lock
+{% if allowed_hosts -%}
+hosts allow = {{ allowed_hosts }}
+{% endif %}
+
 [container]
 uid = swift
 gid = swift
@@ -20,6 +31,17 @@ lock file = /var/lock/container.lock
 hosts allow = {{ allowed_hosts }}
 {% endif %}
 
+[container-rep]
+uid = swift
+gid = swift
+max connections = {{ container_max_connections }}
+path = /srv/node/
+read only = false
+lock file = /var/lock/container-rep.lock
+{% if allowed_hosts -%}
+hosts allow = {{ allowed_hosts }}
+{% endif %}
+
 [object]
 uid = swift
 gid = swift
@@ -30,3 +52,14 @@ lock file = /var/lock/object.lock
 {% if allowed_hosts -%}
 hosts allow = {{ allowed_hosts }}
 {% endif %}
+
+[object-rep]
+uid = swift
+gid = swift
+max connections = {{ object_max_connections }}
+path = /srv/node/
+read only = false
+lock file = /var/lock/object-rep.lock
+{% if allowed_hosts -%}
+hosts allow = {{ allowed_hosts }}
+{% endif %}

templates/account-server-replicator.conf

@@ -0,0 +1,23 @@
+[DEFAULT]
+bind_ip = {{ bind_host }}
+bind_port = {{ account_server_port_rep }}
+workers = {{ workers }}
+
+{% if statsd_host %}
+log_statsd_host = {{ statsd_host }}
+log_statsd_port = {{ statsd_port }}
+log_statsd_default_sample_rate = {{ statsd_sample_rate }}
+
+{% endif %}
+[pipeline:main]
+pipeline = recon account-server
+
+[filter:recon]
+use = egg:swift#recon
+recon_cache_path = /var/cache/swift
+
+[app:account-server]
+use = egg:swift#account
+replication_server = true
+
+[account-replicator]

templates/account-server.conf

@@ -19,7 +19,9 @@ recon_cache_path = /var/cache/swift
 [app:account-server]
 use = egg:swift#account
 
+{% if not standalone_replicator %}
 [account-replicator]
+{% endif %}
 
 [account-auditor]
 

templates/container-server-replicator.conf

@@ -0,0 +1,24 @@
+[DEFAULT]
+bind_ip = {{ bind_host }}
+bind_port = {{ container_server_port_rep }}
+workers = {{ workers }}
+
+{% if statsd_host %}
+log_statsd_host = {{ statsd_host }}
+log_statsd_port = {{ statsd_port }}
+log_statsd_default_sample_rate = {{ statsd_sample_rate }}
+
+{% endif %}
+[pipeline:main]
+pipeline = recon container-server
+
+[filter:recon]
+use = egg:swift#recon
+recon_cache_path = /var/cache/swift
+
+[app:container-server]
+use = egg:swift#container
+allow_versions = true
+replication_server = true
+
+[container-replicator]

templates/container-server.conf

@@ -20,7 +20,9 @@ recon_cache_path = /var/cache/swift
 use = egg:swift#container
 allow_versions = true
 
+{% if not standalone_replicator %}
 [container-replicator]
+{% endif %}
 
 [container-updater]
 

templates/object-server-replicator.conf

@@ -0,0 +1,26 @@
+[DEFAULT]
+bind_ip = {{ bind_host }}
+bind_port = {{ object_server_port_rep }}
+workers = {{ workers }}
+
+{% if statsd_host %}
+log_statsd_host = {{ statsd_host }}
+log_statsd_port = {{ statsd_port }}
+log_statsd_default_sample_rate = {{ statsd_sample_rate }}
+
+{% endif %}
+[pipeline:main]
+pipeline = recon object-server
+
+[filter:recon]
+use = egg:swift#recon
+recon_cache_path = /var/cache/swift
+
+[app:object-server]
+use = egg:swift#object
+threads_per_disk = {{ object_server_threads_per_disk }}
+replication_server = true
+
+[object-replicator]
+concurrency = {{ object_replicator_concurrency }}
+rsync_timeout = {{ object_rsync_timeout }}

templates/object-server.conf

@@ -20,12 +20,16 @@ recon_cache_path = /var/cache/swift
 use = egg:swift#object
 threads_per_disk = {{ object_server_threads_per_disk }}
 
+{% if standalone_replicator %}
+[object-reconstructor]
+{% else %}
 [object-replicator]
 concurrency = {{ object_replicator_concurrency }}
 rsync_timeout = {{ object_rsync_timeout }}
 
-[object-updater]
+[object-sync]
+{% endif %}
 
 [object-auditor]
 
-[object-sync]
+[object-updater]

unit_tests/test_swift_storage_context.py

@@ -84,8 +84,11 @@ class SwiftStorageContextTests(CharmTestCase):
     def test_swift_storage_server_context(self):
         self.unit_private_ip.return_value = '10.0.0.5'
         self.test_config.set('account-server-port', '500')
+        self.test_config.set('account-server-port-rep', '510')
         self.test_config.set('object-server-port', '501')
+        self.test_config.set('object-server-port-rep', '511')
         self.test_config.set('container-server-port', '502')
+        self.test_config.set('container-server-port-rep', '512')
         self.test_config.set('object-server-threads-per-disk', '3')
         self.test_config.set('object-replicator-concurrency', '3')
         self.test_config.set('account-max-connections', '10')
@@ -96,8 +99,11 @@ class SwiftStorageContextTests(CharmTestCase):
         result = ctxt()
         ex = {
             'container_server_port': '502',
+            'container_server_port_rep': '512',
             'object_server_port': '501',
+            'object_server_port_rep': '511',
             'account_server_port': '500',
+            'account_server_port_rep': '510',
             'local_ip': '10.0.0.5',
             'object_server_threads_per_disk': '3',
             'object_replicator_concurrency': '3',
@@ -105,6 +111,7 @@ class SwiftStorageContextTests(CharmTestCase):
             'container_max_connections': '10',
             'object_max_connections': '10',
             'object_rsync_timeout': '950',
+            'standalone_replicator': True,
             'statsd_host': '',
             'statsd_port': 3125,
             'statsd_sample_rate': 1.0

unit_tests/test_swift_storage_relations.py

@@ -35,6 +35,7 @@ TO_PATCH = [
     'Hooks',
     'config',
     'log',
+    'network_get_primary_address',
     'relation_set',
     'relation_ids',
     'relation_get',
@@ -101,7 +102,9 @@ class SwiftStorageRelationsTests(CharmTestCase):
         self.add_to_updatedb_prunepath.assert_called_with("/srv/node")
 
     @patch.object(hooks, 'add_ufw_gre_rule', lambda *args: None)
-    def test_install_hook(self):
+    @patch('lib.swift_storage_utils.swift_init')
+    def test_install_hook(self, mock_swift_init):
+        mock_swift_init.return_value = 0
         self.test_config.set('openstack-origin', 'cloud:precise-havana')
         hooks.install()
         self.configure_installation_source.assert_called_with(
@@ -198,6 +201,7 @@ class SwiftStorageRelationsTests(CharmTestCase):
         kvstore.__enter__.return_value = kvstore
         kvstore.get.return_value = None
         self.test_kv.set('prepared-devices', ['/dev/vdb'])
+        self.network_get_primary_address.return_value = "10.10.10.2"
 
         # py3 is very picky, and log is only patched in
         # hooks.swift_storage_hooks
@@ -209,12 +213,18 @@ class SwiftStorageRelationsTests(CharmTestCase):
         mock_rel_set.assert_called_with(
             relation_id=None,
             relation_settings={
-                "device": 'vdb',
+                "ip_cls": "10.10.10.2",
-                "object_port": 6000,
+                "ip_rep": "10.10.10.2",
-                "account_port": 6002,
+                "private-address": "10.10.10.2",
+                "region": 1,
                 "zone": 1,
+                "device": 'vdb',
+                "account_port": 6002,
+                "account_port_rep": 6012,
                 "container_port": 6001,
-                "private-address": "10.10.10.2"
+                "container_port_rep": 6011,
+                "object_port": 6000,
+                "object_port_rep": 6010
             }
         )
 

unit_tests/test_swift_storage_utils.py

@@ -32,6 +32,7 @@ TO_PATCH = [
     'mkdir',
     'mount',
     'check_call',
+    'check_output',
     'call',
     'ensure_block_device',
     'clean_storage',
@@ -124,6 +125,13 @@ class SwiftStorageUtilsTests(CharmTestCase):
             swift_utils.ensure_swift_directories()
             ex_dirs = [
                 call('/etc/swift', owner='swift', group='swift'),
+                call('/etc/swift/account-server',
+                     owner='swift',
+                     group='swift'),
+                call('/etc/swift/container-server',
+                     owner='swift',
+                     group='swift'),
+                call('/etc/swift/object-server', owner='swift', group='swift'),
                 call('/var/cache/swift', owner='swift', group='swift'),
                 call('/srv/node', owner='swift', group='swift')
             ]
@@ -435,13 +443,29 @@ class SwiftStorageUtilsTests(CharmTestCase):
                                          'DISTRIB_DESCRIPTION': 'Ubuntu 14.04'}
         swift_utils.assert_charm_supports_ipv6()
 
+    def test_enable_replication(self):
+        self.lsb_release.return_value = {'DISTRIB_ID': 'Ubuntu',
+                                         'DISTRIB_RELEASE': '14.04',
+                                         'DISTRIB_CODENAME': 'trusty',
+                                         'DISTRIB_DESCRIPTION': 'Ubuntu 14.04'}
+        self.assertFalse(swift_utils.enable_replication())
+        self.lsb_release.return_value = {'DISTRIB_ID': 'Ubuntu',
+                                         'DISTRIB_RELEASE': '18.04',
+                                         'DISTRIB_CODENAME': 'bionic',
+                                         'DISTRIB_DESCRIPTION': 'Ubuntu 18.04'}
+        self.assertTrue(swift_utils.enable_replication())
+
+    @patch.object(swift_utils, 'enable_replication')
     @patch('charmhelpers.contrib.openstack.templating.OSConfigRenderer')
-    def test_register_configs_pre_install(self, renderer):
+    def test_register_configs_pre_install(self, renderer, enable_replication):
+        enable_replication.return_value = True
         self.get_os_codename_package.return_value = None
         swift_utils.register_configs()
         renderer.assert_called_with(templates_dir=swift_utils.TEMPLATES,
                                     openstack_release='essex')
 
+    @patch.object(swift_utils, 'vaultlocker_installed')
+    @patch.object(swift_utils, 'enable_replication')
     @patch.object(swift_utils, 'filter_installed_packages')
     @patch('charmhelpers.contrib.openstack.context.WorkerConfigContext')
     @patch('charmhelpers.contrib.openstack.context.BindHostContext')
@@ -452,7 +476,11 @@ class SwiftStorageUtilsTests(CharmTestCase):
     def test_register_configs_post_install(self, renderer,
                                            swift, rsync, server,
                                            bind_context, worker_context,
-                                           filter_installed_packages):
+                                           filter_installed_packages,
+                                           enable_replication,
+                                           vaultlocker_installed):
+        vaultlocker_installed.return_value = True
+        enable_replication.return_value = True
         filter_installed_packages.return_value = []
         swift.return_value = 'swift_context'
         rsync.return_value = 'rsync_context'
@@ -471,20 +499,50 @@ class SwiftStorageUtilsTests(CharmTestCase):
             call('/etc/swift/swift.conf', ['swift_server_context']),
             call('/etc/rsync-juju.d/050-swift-storage.conf',
                  ['rsync_context', 'swift_context']),
-            call('/etc/swift/account-server.conf', ['swift_context',
+            call(
-                                                    'bind_host_context',
+                '/etc/swift/account-server.conf',
-                                                    'worker_context',
+                [
-                                                    'vl_context']),
+                    'swift_context',
-            call('/etc/swift/object-server.conf', ['swift_context',
+                    'bind_host_context',
-                                                   'bind_host_context',
+                    'worker_context',
-                                                   'worker_context',
+                    'vl_context']),
-                                                   'vl_context']),
+            call(
-            call('/etc/swift/container-server.conf', ['swift_context',
+                '/etc/swift/container-server.conf',
-                                                      'bind_host_context',
+                [
-                                                      'worker_context',
+                    'swift_context',
-                                                      'vl_context'])
+                    'bind_host_context',
+                    'worker_context',
+                    'vl_context']),
+            call(
+                '/etc/swift/object-server.conf',
+                [
+                    'swift_context',
+                    'bind_host_context',
+                    'worker_context',
+                    'vl_context']),
+            call(
+                '/etc/swift/account-server/account-server-replicator.conf',
+                [
+                    'swift_context',
+                    'bind_host_context',
+                    'worker_context',
+                    'vl_context']),
+            call('/etc/swift/container-server/container-server-replicator.conf',
+                [
+                    'swift_context',
+                    'bind_host_context',
+                    'worker_context',
+                    'vl_context']),
+            call(
+                '/etc/swift/object-server/object-server-replicator.conf',
+                [
+                    'swift_context',
+                    'bind_host_context',
+                    'worker_context',
+                    'vl_context'])
         ]
-        self.assertEqual(ex, configs.register.call_args_list)
+        self.assertEqual(sorted(ex), sorted(configs.register.call_args_list))
+
 
     @patch.object(swift_utils, 'remove_old_packages')
     def test_do_upgrade_queens(self, mock_remove_old_packages):

unit_tests/test_templates.py

@@ -27,6 +27,7 @@ class StorageServerTemplateTestCase(unittest.TestCase):
             self,
             os_release,
             server,
+            suffix,
             mock_log):
 
         if not server:
@@ -35,53 +36,52 @@ class StorageServerTemplateTestCase(unittest.TestCase):
         loader = get_loader('./templates', os_release)
         env = Environment(loader=loader)
 
-        return env.get_template('{}-server.conf'.format(server))
+        return env.get_template('{}-server{}.conf'.format(server, suffix))
+
+    def get_templates(self, os_releases):
+        templates = []
+        for release in os_releases:
+            for server in ('object', 'container', 'account'):
+                for suffix in ('', '-replicator'):
+                    templates.append(self.get_template_for_release_and_server(
+                        release,
+                        server,
+                        suffix))
+        return templates
 
     def test_os_release_not_in_templates(self):
         """Regression test for bug 1251551.
 
         The os_release is no longer provided as context to the templates.
         """
-        for release in ('essex', 'grizzly', 'havana', 'icehouse'):
+        for template in self.get_templates(('essex', 'grizzly', 'havana',
-            for server in ('object', 'container', 'account'):
+                                            'icehouse')):
-                template = self.get_template_for_release_and_server(
+            with open(template.filename, 'r') as template_orig:
-                    release,
+                self.assertNotIn(
-                    server)
+                    'os_release', template_orig.read(),
-                with open(template.filename, 'r') as template_orig:
+                    "The template '{}' contains os_release which is "
-                    self.assertNotIn(
+                    "no longer provided in the context.".format(
-                        'os_release', template_orig.read(),
+                        template.filename))
-                        "The template '{}' contains os_release which is "
-                        "no longer provided in the context.".format(
-                            template.filename))
 
     def test_config_renders_for_all_releases_and_servers(self):
         """The configs render without syntax error."""
-        for release in ('essex', 'grizzly', 'havana', 'icehouse'):
+        for template in self.get_templates(('essex', 'grizzly', 'havana',
-            for server in ('object', 'container', 'account'):
+                                            'icehouse')):
-                template = self.get_template_for_release_and_server(
+            result = template.render()
-                    release,
+            self.assertTrue(result.startswith("[DEFAULT]"))
-                    server)
-
-                result = template.render()
-
-                self.assertTrue(result.startswith("[DEFAULT]"))
 
     def test_statsd_config_for_all_releases_and_servers(self):
         """The configs contain statsd settings if statsd-host is set."""
-        for release in ('grizzly', 'havana', 'icehouse', 'mitaka'):
+        for template in self.get_templates(('grizzly', 'havana', 'icehouse',
-            for server in ('object', 'container', 'account'):
+                                            'mitaka')):
-                template = self.get_template_for_release_and_server(
+            result = template.render(statsd_host='127.0.0.1')
-                    release,
-                    server)
 
-                result = template.render(statsd_host='127.0.0.1')
+            self.assertIn("log_statsd_host", result)
+            self.assertIn("log_statsd_port", result)
+            self.assertIn("log_statsd_default_sample_rate", result)
 
-                self.assertIn("log_statsd_host", result)
+            result = template.render()
-                self.assertIn("log_statsd_port", result)
-                self.assertIn("log_statsd_default_sample_rate", result)
 
-                result = template.render()
+            self.assertNotIn("log_statsd_host", result)
-
+            self.assertNotIn("log_statsd_port", result)
-                self.assertNotIn("log_statsd_host", result)
+            self.assertNotIn("log_statsd_default_sample_rate", result)
-                self.assertNotIn("log_statsd_port", result)
-                self.assertNotIn("log_statsd_default_sample_rate", result)