Support Ansible collections

This change adds support for installing Ansible collections via
requirements.yml in Kayobe or Kayobe config.

Story: 2008391
Task: 41315

Change-Id: I764ff019a18266b593add7ab80ee095d7d07a869

doc/source/contributor/development.rst

@@ -38,25 +38,27 @@ in `etc/kayobe/*.yml
 <https://opendev.org/openstack/kayobe/src/branch/master/etc/kayobe/>`__.
 A number of custom Jinja filters exist in `ansible/filter_plugins/*.py
 <https://opendev.org/openstack/kayobe/src/branch/master/ansible/filter_plugins>`__.
-Kayobe depends on roles hosted on Ansible Galaxy, and these and their version
+Kayobe depends on roles and collections hosted on Ansible Galaxy, and these and
-requirements are defined in `requirements.yml
+their version requirements are defined in `requirements.yml
 <https://opendev.org/openstack/kayobe/src/branch/master/requirements.yml>`__.
 
 Ansible Galaxy
 ==============
 
-Kayobe uses a number of Ansible roles hosted on Ansible Galaxy. The role
+Kayobe uses a number of Ansible roles and collections hosted on Ansible Galaxy.
-dependencies are tracked in ``requirements.yml``, and specify required
+The role dependencies are tracked in ``requirements.yml``, and specify required
-versions. The process for changing a Galaxy role is as follows:
+versions. The process for changing a Galaxy role or collection is as follows:
 
-#. If required, develop changes for the role. This may be done outside of
+#. If required, develop changes for the role or collection. This may be done
-   Kayobe, or by modifying the role in place during development. If upstream
+   outside of Kayobe, or by modifying the code in place during development. If
-   changes to the role have already been made, this step can be skipped.
+   upstream changes to the code have already been made, this step can be
-#. Commit changes to the role, typically via a Github pull request.
+   skipped.
-#. Request that a tagged release of the role be made, or make one if you have
+#. Commit changes to the role or collection, typically via a Github pull
-   the necessary privileges.
+   request.
-#. Ensure that automatic imports are configured for the role using e.g. a
+#. Request that a tagged release of the role or collection be made, or make one
-   TravisCI webhook notification, or perform a manual import of the role on
+   if you have the necessary privileges.
-   Ansible Galaxy.
+#. Ensure that automatic imports are configured for the repository using e.g. a
+   webhook notification, or perform a manual import of the role on Ansible
+   Galaxy.
 #. Modify the version in ``requirements.yml`` to match the new release of the
-   role.
+   role or collection.

doc/source/custom-ansible-playbooks.rst

@@ -75,14 +75,16 @@ These symlinks can even be committed to the kayobe-config Git repository.
 Ansible Galaxy
 --------------
 
-Ansible Galaxy provides a means for sharing Ansible roles.  Kayobe
+Ansible Galaxy provides a means for sharing Ansible roles and collections.
-configuration may provide a Galaxy requirements file that defines roles to be
+Kayobe configuration may provide a Galaxy requirements file that defines roles
-installed from Galaxy.  These roles may then be used by custom playbooks.
+and collections to be installed from Galaxy.  These roles and collections may
+then be used by custom playbooks.
 
-Galaxy role dependencies may be defined in
+Galaxy dependencies may be defined in
-``$KAYOBE_CONFIG_PATH/ansible/requirements.yml``.  These roles will be
+``$KAYOBE_CONFIG_PATH/ansible/requirements.yml``.  These roles and collections
-installed in ``$KAYOBE_CONFIG_PATH/ansible/roles/`` when bootstrapping the
+will be installed in ``$KAYOBE_CONFIG_PATH/ansible/roles/`` and
-Ansible control host::
+``$KAYOBE_CONFIG_PATH/ansible/collections`` when bootstrapping the Ansible
+control host::
 
     (kayobe) $ kayobe control host bootstrap
 
@@ -90,8 +92,8 @@ And updated when upgrading the Ansible control host::
 
     (kayobe) $ kayobe control host upgrade
 
-Example
+Example: roles
-=======
+==============
 
 The following example adds a ``foo.yml`` playbook to a set of kayobe
 configuration.  The playbook uses a Galaxy role, ``bar.baz``.
@@ -116,7 +118,8 @@ Here is the playbook, ``ansible/foo.yml``::
 Here is the Galaxy requirements file, ``ansible/requirements.yml``::
 
     ---
-    - bar.baz
+    roles:
+      - bar.baz
 
 We should first install the Galaxy role dependencies, to download the
 ``bar.baz`` role::
@@ -127,6 +130,45 @@ Then, to run the ``foo.yml`` playbook::
 
     (kayobe) $ kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/foo.yml
 
+Example: collections
+====================
+
+The following example adds a ``foo.yml`` playbook to a set of kayobe
+configuration.  The playbook uses a role from a Galaxy collection,
+``bar.baz.qux``.
+
+Here is the kayobe configuration repository structure::
+
+    etc/kayobe/
+        ansible/
+            collections/
+            foo.yml
+            requirements.yml
+        bifrost.yml
+    ...
+
+Here is the playbook, ``ansible/foo.yml``::
+
+    ---
+    - hosts: controllers
+      roles:
+        - name: bar.baz.qux
+
+Here is the Galaxy requirements file, ``ansible/requirements.yml``::
+
+    ---
+    collections:
+      - bar.baz
+
+We should first install the Galaxy dependencies, to download the ``bar.baz``
+collection::
+
+    (kayobe) $ kayobe control host bootstrap
+
+Then, to run the ``foo.yml`` playbook::
+
+    (kayobe) $ kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/foo.yml
+
 Hooks
 =====
 

kayobe/ansible.py

@@ -291,7 +291,7 @@ def config_dump(parsed_args, host=None, hosts=None, var_name=None,
 def install_galaxy_roles(parsed_args, force=False):
     """Install Ansible Galaxy role dependencies.
 
-    Installs dependencies specified in kayobe, and if present, in kayobe
+    Installs role dependencies specified in kayobe, and if present, in kayobe
     configuration.
 
     :param parsed_args: Parsed command line arguments.
@@ -300,7 +300,7 @@ def install_galaxy_roles(parsed_args, force=False):
     LOG.info("Installing galaxy role dependencies from kayobe")
     requirements = utils.get_data_files_path("requirements.yml")
     roles_destination = utils.get_data_files_path('ansible', 'roles')
-    utils.galaxy_install(requirements, roles_destination, force=force)
+    utils.galaxy_role_install(requirements, roles_destination, force=force)
 
     # Check for requirements in kayobe configuration.
     kc_reqs_path = os.path.join(parsed_args.config_path,
@@ -323,7 +323,49 @@ def install_galaxy_roles(parsed_args, force=False):
                                   (parsed_args.config_path, str(e)))
 
     # Install roles from kayobe-config.
-    utils.galaxy_install(kc_reqs_path, kc_roles_path, force=force)
+    utils.galaxy_role_install(kc_reqs_path, kc_roles_path, force=force)
+
+
+def install_galaxy_collections(parsed_args, force=False):
+    """Install Ansible Galaxy collection dependencies.
+
+    Installs collection dependencies specified in kayobe, and if present, in
+    kayobe configuration.
+
+    :param parsed_args: Parsed command line arguments.
+    :param force: Whether to force reinstallation of roles.
+    """
+    LOG.info("Installing galaxy collection dependencies from kayobe")
+    requirements = utils.get_data_files_path("requirements.yml")
+    collections_destination = utils.get_data_files_path('ansible',
+                                                        'collections')
+    utils.galaxy_collection_install(requirements, collections_destination,
+                                    force=force)
+
+    # Check for requirements in kayobe configuration.
+    kc_reqs_path = os.path.join(parsed_args.config_path,
+                                "ansible", "requirements.yml")
+    if not utils.is_readable_file(kc_reqs_path)["result"]:
+        LOG.info("Not installing galaxy collection dependencies from kayobe "
+                 "config - requirements.yml not present")
+        return
+
+    LOG.info("Installing galaxy collection dependencies from kayobe config")
+    # Ensure a collections directory exists in kayobe-config.
+    kc_collections_path = os.path.join(parsed_args.config_path,
+                                       "ansible", "collections")
+    try:
+        os.makedirs(kc_collections_path)
+    except OSError as e:
+        if e.errno != errno.EEXIST:
+            raise exception.Error("Failed to create directory "
+                                  "ansible/collections/ "
+                                  "in kayobe configuration at %s: %s" %
+                                  (parsed_args.config_path, str(e)))
+
+    # Install collections from kayobe-config.
+    utils.galaxy_collection_install(kc_reqs_path, kc_collections_path,
+                                    force=force)
 
 
 def prune_galaxy_roles(parsed_args):

kayobe/cli/commands.py

@@ -232,6 +232,7 @@ class ControlHostBootstrap(KayobeAnsibleMixin, KollaAnsibleMixin, VaultMixin,
     def take_action(self, parsed_args):
         self.app.LOG.debug("Bootstrapping Kayobe Ansible control host")
         ansible.install_galaxy_roles(parsed_args)
+        ansible.install_galaxy_collections(parsed_args)
         playbooks = _build_playbook_list("bootstrap")
         self.run_kayobe_playbooks(parsed_args, playbooks, ignore_limit=True)
 
@@ -271,8 +272,9 @@ class ControlHostUpgrade(KayobeAnsibleMixin, VaultMixin, Command):
         # Remove roles that are no longer used. Do this before installing new
         # ones, just in case a custom role dependency includes any.
         ansible.prune_galaxy_roles(parsed_args)
-        # Use force to upgrade roles.
+        # Use force to upgrade roles and collections.
         ansible.install_galaxy_roles(parsed_args, force=True)
+        ansible.install_galaxy_collections(parsed_args, force=True)
         playbooks = _build_playbook_list("bootstrap")
         self.run_kayobe_playbooks(parsed_args, playbooks, ignore_limit=True)
         playbooks = _build_playbook_list("kolla-ansible")

kayobe/tests/unit/cli/test_commands.py

@@ -35,18 +35,21 @@ class TestApp(cliff.app.App):
 class TestCase(unittest.TestCase):
 
     @mock.patch.object(ansible, "install_galaxy_roles", autospec=True)
+    @mock.patch.object(ansible, "install_galaxy_collections", autospec=True)
     @mock.patch.object(ansible, "passwords_yml_exists", autospec=True)
     @mock.patch.object(commands.KayobeAnsibleMixin,
                        "run_kayobe_playbooks")
     def test_control_host_bootstrap(self, mock_run, mock_passwords,
-                                    mock_install):
+                                    mock_install_collections,
+                                    mock_install_roles):
         mock_passwords.return_value = False
         command = commands.ControlHostBootstrap(TestApp(), [])
         parser = command.get_parser("test")
         parsed_args = parser.parse_args([])
         result = command.run(parsed_args)
         self.assertEqual(0, result)
-        mock_install.assert_called_once_with(parsed_args)
+        mock_install_roles.assert_called_once_with(parsed_args)
+        mock_install_collections.assert_called_once_with(parsed_args)
         expected_calls = [
             mock.call(
                 mock.ANY,
@@ -63,20 +66,23 @@ class TestCase(unittest.TestCase):
         self.assertEqual(expected_calls, mock_run.call_args_list)
 
     @mock.patch.object(ansible, "install_galaxy_roles", autospec=True)
+    @mock.patch.object(ansible, "install_galaxy_collections", autospec=True)
     @mock.patch.object(ansible, "passwords_yml_exists", autospec=True)
     @mock.patch.object(commands.KayobeAnsibleMixin,
                        "run_kayobe_playbooks")
     @mock.patch.object(commands.KollaAnsibleMixin,
                        "run_kolla_ansible_overcloud")
     def test_control_host_bootstrap_with_passwords(
-            self, mock_kolla_run, mock_run, mock_passwords, mock_install):
+            self, mock_kolla_run, mock_run, mock_passwords,
+            mock_install_collections, mock_install_roles):
         mock_passwords.return_value = True
         command = commands.ControlHostBootstrap(TestApp(), [])
         parser = command.get_parser("test")
         parsed_args = parser.parse_args([])
         result = command.run(parsed_args)
         self.assertEqual(0, result)
-        mock_install.assert_called_once_with(parsed_args)
+        mock_install_roles.assert_called_once_with(parsed_args)
+        mock_install_collections.assert_called_once_with(parsed_args)
         expected_calls = [
             mock.call(
                 mock.ANY,
@@ -106,16 +112,21 @@ class TestCase(unittest.TestCase):
         self.assertEqual(expected_calls, mock_kolla_run.call_args_list)
 
     @mock.patch.object(ansible, "install_galaxy_roles", autospec=True)
+    @mock.patch.object(ansible, "install_galaxy_collections", autospec=True)
     @mock.patch.object(ansible, "prune_galaxy_roles", autospec=True)
     @mock.patch.object(commands.KayobeAnsibleMixin,
                        "run_kayobe_playbooks")
-    def test_control_host_upgrade(self, mock_run, mock_prune, mock_install):
+    def test_control_host_upgrade(self, mock_run, mock_prune,
+                                  mock_install_roles,
+                                  mock_install_collections):
         command = commands.ControlHostUpgrade(TestApp(), [])
         parser = command.get_parser("test")
         parsed_args = parser.parse_args([])
         result = command.run(parsed_args)
         self.assertEqual(0, result)
-        mock_install.assert_called_once_with(parsed_args, force=True)
+        mock_install_roles.assert_called_once_with(parsed_args, force=True)
+        mock_install_collections.assert_called_once_with(parsed_args,
+                                                         force=True)
         mock_prune.assert_called_once_with(parsed_args)
         expected_calls = [
             mock.call(

kayobe/tests/unit/test_ansible.py

@@ -434,7 +434,7 @@ class TestCase(unittest.TestCase):
             mock.call(os.path.join(dump_dir, "host2.yml")),
         ])
 
-    @mock.patch.object(utils, 'galaxy_install', autospec=True)
+    @mock.patch.object(utils, 'galaxy_role_install', autospec=True)
     @mock.patch.object(utils, 'is_readable_file', autospec=True)
     @mock.patch.object(os, 'makedirs', autospec=True)
     def test_install_galaxy_roles(self, mock_mkdirs, mock_is_readable,
@@ -453,7 +453,7 @@ class TestCase(unittest.TestCase):
             "/etc/kayobe/ansible/requirements.yml")
         self.assertFalse(mock_mkdirs.called)
 
-    @mock.patch.object(utils, 'galaxy_install', autospec=True)
+    @mock.patch.object(utils, 'galaxy_role_install', autospec=True)
     @mock.patch.object(utils, 'is_readable_file', autospec=True)
     @mock.patch.object(os, 'makedirs', autospec=True)
     def test_install_galaxy_roles_with_kayobe_config(
@@ -476,7 +476,7 @@ class TestCase(unittest.TestCase):
             "/etc/kayobe/ansible/requirements.yml")
         mock_mkdirs.assert_called_once_with("/etc/kayobe/ansible/roles")
 
-    @mock.patch.object(utils, 'galaxy_install', autospec=True)
+    @mock.patch.object(utils, 'galaxy_role_install', autospec=True)
     @mock.patch.object(utils, 'is_readable_file', autospec=True)
     @mock.patch.object(os, 'makedirs', autospec=True)
     def test_install_galaxy_roles_with_kayobe_config_forced(
@@ -499,7 +499,7 @@ class TestCase(unittest.TestCase):
             "/etc/kayobe/ansible/requirements.yml")
         mock_mkdirs.assert_called_once_with("/etc/kayobe/ansible/roles")
 
-    @mock.patch.object(utils, 'galaxy_install', autospec=True)
+    @mock.patch.object(utils, 'galaxy_role_install', autospec=True)
     @mock.patch.object(utils, 'is_readable_file', autospec=True)
     @mock.patch.object(os, 'makedirs', autospec=True)
     def test_install_galaxy_roles_with_kayobe_config_mkdirs_failure(
@@ -520,6 +520,92 @@ class TestCase(unittest.TestCase):
             "/etc/kayobe/ansible/requirements.yml")
         mock_mkdirs.assert_called_once_with("/etc/kayobe/ansible/roles")
 
+    @mock.patch.object(utils, 'galaxy_collection_install', autospec=True)
+    @mock.patch.object(utils, 'is_readable_file', autospec=True)
+    @mock.patch.object(os, 'makedirs', autospec=True)
+    def test_install_galaxy_collections(self, mock_mkdirs, mock_is_readable,
+                                        mock_install):
+        parser = argparse.ArgumentParser()
+        ansible.add_args(parser)
+        parsed_args = parser.parse_args([])
+        mock_is_readable.return_value = {"result": False}
+
+        ansible.install_galaxy_collections(parsed_args)
+
+        mock_install.assert_called_once_with(utils.get_data_files_path(
+            "requirements.yml"), utils.get_data_files_path(
+            "ansible", "collections"), force=False)
+        mock_is_readable.assert_called_once_with(
+            "/etc/kayobe/ansible/requirements.yml")
+        self.assertFalse(mock_mkdirs.called)
+
+    @mock.patch.object(utils, 'galaxy_collection_install', autospec=True)
+    @mock.patch.object(utils, 'is_readable_file', autospec=True)
+    @mock.patch.object(os, 'makedirs', autospec=True)
+    def test_install_galaxy_collections_with_kayobe_config(
+            self, mock_mkdirs, mock_is_readable, mock_install):
+        parser = argparse.ArgumentParser()
+        ansible.add_args(parser)
+        parsed_args = parser.parse_args([])
+        mock_is_readable.return_value = {"result": True}
+
+        ansible.install_galaxy_collections(parsed_args)
+
+        expected_calls = [
+            mock.call(utils.get_data_files_path("requirements.yml"),
+                      utils.get_data_files_path("ansible", "collections"),
+                      force=False),
+            mock.call("/etc/kayobe/ansible/requirements.yml",
+                      "/etc/kayobe/ansible/collections", force=False)]
+        self.assertEqual(expected_calls, mock_install.call_args_list)
+        mock_is_readable.assert_called_once_with(
+            "/etc/kayobe/ansible/requirements.yml")
+        mock_mkdirs.assert_called_once_with("/etc/kayobe/ansible/collections")
+
+    @mock.patch.object(utils, 'galaxy_collection_install', autospec=True)
+    @mock.patch.object(utils, 'is_readable_file', autospec=True)
+    @mock.patch.object(os, 'makedirs', autospec=True)
+    def test_install_galaxy_collections_with_kayobe_config_forced(
+            self, mock_mkdirs, mock_is_readable, mock_install):
+        parser = argparse.ArgumentParser()
+        ansible.add_args(parser)
+        parsed_args = parser.parse_args([])
+        mock_is_readable.return_value = {"result": True}
+
+        ansible.install_galaxy_collections(parsed_args, force=True)
+
+        expected_calls = [
+            mock.call(utils.get_data_files_path("requirements.yml"),
+                      utils.get_data_files_path("ansible", "collections"),
+                      force=True),
+            mock.call("/etc/kayobe/ansible/requirements.yml",
+                      "/etc/kayobe/ansible/collections", force=True)]
+        self.assertEqual(expected_calls, mock_install.call_args_list)
+        mock_is_readable.assert_called_once_with(
+            "/etc/kayobe/ansible/requirements.yml")
+        mock_mkdirs.assert_called_once_with("/etc/kayobe/ansible/collections")
+
+    @mock.patch.object(utils, 'galaxy_collection_install', autospec=True)
+    @mock.patch.object(utils, 'is_readable_file', autospec=True)
+    @mock.patch.object(os, 'makedirs', autospec=True)
+    def test_install_galaxy_collections_with_kayobe_config_mkdirs_failure(
+            self, mock_mkdirs, mock_is_readable, mock_install):
+        parser = argparse.ArgumentParser()
+        ansible.add_args(parser)
+        parsed_args = parser.parse_args([])
+        mock_is_readable.return_value = {"result": True}
+        mock_mkdirs.side_effect = OSError(errno.EPERM)
+
+        self.assertRaises(exception.Error,
+                          ansible.install_galaxy_collections, parsed_args)
+
+        mock_install.assert_called_once_with(
+            utils.get_data_files_path("requirements.yml"),
+            utils.get_data_files_path("ansible", "collections"), force=False)
+        mock_is_readable.assert_called_once_with(
+            "/etc/kayobe/ansible/requirements.yml")
+        mock_mkdirs.assert_called_once_with("/etc/kayobe/ansible/collections")
+
     @mock.patch.object(utils, 'galaxy_remove', autospec=True)
     def test_prune_galaxy_roles(self, mock_remove):
         parser = argparse.ArgumentParser()

kayobe/tests/unit/test_utils.py

@@ -26,23 +26,72 @@ from kayobe import utils
 class TestCase(unittest.TestCase):
 
     @mock.patch.object(utils, "run_command")
-    def test_galaxy_install(self, mock_run):
+    def test_galaxy_role_install(self, mock_run):
-        utils.galaxy_install("/path/to/role/file", "/path/to/roles")
+        utils.galaxy_role_install("/path/to/role/file", "/path/to/roles")
-        mock_run.assert_called_once_with(["ansible-galaxy", "install",
+        mock_run.assert_called_once_with(["ansible-galaxy", "role", "install",
                                           "--roles-path", "/path/to/roles",
                                           "--role-file", "/path/to/role/file"])
 
     @mock.patch.object(utils, "run_command")
-    def test_galaxy_install_failure(self, mock_run):
+    def test_galaxy_role_install_failure(self, mock_run):
         mock_run.side_effect = subprocess.CalledProcessError(1, "command")
         self.assertRaises(SystemExit,
-                          utils.galaxy_install, "/path/to/role/file",
+                          utils.galaxy_role_install, "/path/to/role/file",
                           "/path/to/roles")
 
+    @mock.patch.object(utils, "run_command")
+    @mock.patch.object(utils, "read_yaml_file")
+    def test_galaxy_collection_install(self, mock_read, mock_run):
+        mock_read.return_value = {"collections": []}
+        utils.galaxy_collection_install("/path/to/collection/file",
+                                        "/path/to/collections")
+        mock_run.assert_called_once_with(["ansible-galaxy", "collection",
+                                          "install", "--collections-path",
+                                          "/path/to/collections",
+                                          "--requirements-file",
+                                          "/path/to/collection/file"])
+
+    @mock.patch.object(utils, "run_command")
+    @mock.patch.object(utils, "read_yaml_file")
+    def test_galaxy_collection_install_failure(self, mock_read, mock_run):
+        mock_read.return_value = {"collections": []}
+        mock_run.side_effect = subprocess.CalledProcessError(1, "command")
+        self.assertRaises(SystemExit,
+                          utils.galaxy_collection_install,
+                          "/path/to/collection/file", "/path/to/collections")
+
+    @mock.patch.object(utils, "run_command")
+    @mock.patch.object(utils, "read_file")
+    def test_galaxy_collection_read_failure(self, mock_read, mock_run):
+        mock_read.side_effect = IOError
+        self.assertRaises(SystemExit,
+                          utils.galaxy_collection_install,
+                          "/path/to/collection/file", "/path/to/collections")
+
+    @mock.patch.object(utils, "run_command")
+    @mock.patch.object(utils, "read_yaml_file")
+    def test_galaxy_collection_no_collections(self, mock_read, mock_run):
+        mock_read.return_value = {"roles": []}
+        utils.galaxy_collection_install("/path/to/collection/file",
+                                        "/path/to/collections")
+        mock_run.assert_called_once_with(["ansible-galaxy", "collection",
+                                          "install", "--collections-path",
+                                          "/path/to/collections",
+                                          "--requirements-file",
+                                          "/path/to/collection/file"])
+
+    @mock.patch.object(utils, "run_command")
+    @mock.patch.object(utils, "read_yaml_file")
+    def test_galaxy_collection_legacy_format(self, mock_read, mock_run):
+        mock_read.return_value = []
+        utils.galaxy_collection_install("/path/to/collection/file",
+                                        "/path/to/collections")
+        self.assertFalse(mock_run.called)
+
     @mock.patch.object(utils, "run_command")
     def test_galaxy_remove(self, mock_run):
         utils.galaxy_remove(["role1", "role2"], "/path/to/roles")
-        mock_run.assert_called_once_with(["ansible-galaxy", "remove",
+        mock_run.assert_called_once_with(["ansible-galaxy", "role", "remove",
                                           "--roles-path", "/path/to/roles",
                                           "role1", "role2"])
 
@@ -50,7 +99,7 @@ class TestCase(unittest.TestCase):
     def test_galaxy_remove_failure(self, mock_run):
         mock_run.side_effect = subprocess.CalledProcessError(1, "command")
         self.assertRaises(SystemExit,
-                          utils.galaxy_install, ["role1", "role2"],
+                          utils.galaxy_remove, ["role1", "role2"],
                           "/path/to/roles")
 
     @mock.patch.object(utils, "read_file")

kayobe/utils.py

@@ -72,9 +72,9 @@ def _get_base_path():
     return os.path.join(os.path.realpath(__file__), "..")
 
 
-def galaxy_install(role_file, roles_path, force=False):
+def galaxy_role_install(role_file, roles_path, force=False):
     """Install Ansible roles via Ansible Galaxy."""
-    cmd = ["ansible-galaxy", "install"]
+    cmd = ["ansible-galaxy", "role", "install"]
     cmd += ["--roles-path", roles_path]
     cmd += ["--role-file", role_file]
     if force:
@@ -87,10 +87,29 @@ def galaxy_install(role_file, roles_path, force=False):
         sys.exit(e.returncode)
 
 
+def galaxy_collection_install(requirements_file, collections_path,
+                              force=False):
+    requirements = read_yaml_file(requirements_file)
+    if not isinstance(requirements, dict):
+        # Handle legacy role list format, which causes the command to fail.
+        return
+    cmd = ["ansible-galaxy", "collection", "install"]
+    cmd += ["--collections-path", collections_path]
+    cmd += ["--requirements-file", requirements_file]
+    if force:
+        cmd += ["--force"]
+    try:
+        run_command(cmd)
+    except subprocess.CalledProcessError as e:
+        LOG.error("Failed to install Ansible collections from %s via Ansible "
+                  "Galaxy: returncode %d", requirements_file, e.returncode)
+        sys.exit(e.returncode)
+
+
 def galaxy_remove(roles_to_remove, roles_path):
 
     """Remove Ansible roles via Ansible Galaxy."""
-    cmd = ["ansible-galaxy", "remove"]
+    cmd = ["ansible-galaxy", "role", "remove"]
     cmd += ["--roles-path", roles_path]
     cmd += roles_to_remove
     try:

releasenotes/notes/collections-b1b9a017c843dc1c.yaml

@@ -0,0 +1,5 @@
+---
+features:
+  - |
+    Adds support for installing Ansible collections. See `story 2008391
+    <https://storyboard.openstack.org/#!/story/2008391>`__ for details.

requirements.yml

@@ -1,46 +1,47 @@
 ---
-- src: ahuffman.resolv
+roles:
-  version: 1.3.1
+  - src: ahuffman.resolv
-- src: stackhpc.systemd_networkd
+    version: 1.3.1
-  version: v1.0.1
+  - src: stackhpc.systemd_networkd
-- src: jriguera.configdrive
+    version: v1.0.1
-  # There are no versioned releases of this role.
+  - src: jriguera.configdrive
-  version: e12d38378ae127c9c61d170fa4ba4729f2c5f2ad
+    # There are no versioned releases of this role.
-- src: MichaelRigart.interfaces
+    version: e12d38378ae127c9c61d170fa4ba4729f2c5f2ad
-  version: v1.12.0
+  - src: MichaelRigart.interfaces
-- src: mrlesmithjr.chrony
+    version: v1.12.0
-  version: v0.1.1
+  - src: mrlesmithjr.chrony
-- src: mrlesmithjr.manage-lvm
+    version: v0.1.1
-  version: v0.2.2
+  - src: mrlesmithjr.manage-lvm
-- src: mrlesmithjr.mdadm
+    version: v0.2.2
-  version: v0.1.1
+  - src: mrlesmithjr.mdadm
-- src: singleplatform-eng.users
+    version: v0.1.1
-  version: v1.2.5
+  - src: singleplatform-eng.users
-- src: stackhpc.dell-powerconnect-switch
+    version: v1.2.5
-  version: v1.1.0
+  - src: stackhpc.dell-powerconnect-switch
-- src: stackhpc.drac
+    version: v1.1.0
-  version: 1.1.5
+  - src: stackhpc.drac
-- src: stackhpc.drac-facts
+    version: 1.1.5
-  version: 1.0.0
+  - src: stackhpc.drac-facts
-- src: stackhpc.grafana-conf
+    version: 1.0.0
-  version: 1.1.1
+  - src: stackhpc.grafana-conf
-- src: stackhpc.libvirt-host
+    version: 1.1.1
-  version: v1.8.3
+  - src: stackhpc.libvirt-host
-- src: stackhpc.libvirt-vm
+    version: v1.8.3
-  version: v1.14.2
+  - src: stackhpc.libvirt-vm
-- src: stackhpc.luks
+    version: v1.14.2
-  version: 0.4.1
+  - src: stackhpc.luks
-- src: stackhpc.mellanox-switch
+    version: 0.4.1
-  version: v1.0.0
+  - src: stackhpc.mellanox-switch
-- src: stackhpc.os-images
+    version: v1.0.0
-  version: v1.10.7
+  - src: stackhpc.os-images
-- src: stackhpc.os-ironic-state
+    version: v1.10.7
-  version: v1.3.1
+  - src: stackhpc.os-ironic-state
-- src: stackhpc.os-networks
+    version: v1.3.1
-  version: v1.5.3
+  - src: stackhpc.os-networks
-- src: stackhpc.os-openstackclient
+    version: v1.5.3
-  version: v1.4.1
+  - src: stackhpc.os-openstackclient
-- src: stackhpc.os_openstacksdk
+    version: v1.4.1
-  version: v1.0.1
+  - src: stackhpc.os_openstacksdk
-- src: stackhpc.timezone
+    version: v1.0.1
-  version: 1.2.1
+  - src: stackhpc.timezone
+    version: 1.2.1