Browse Source

Add tripleo_dnf_stream Ansible module.

One of the functionalities the dnf Ansible module misses is the single
disabling and disabling of DNF modules [0]. Currently only installation and
removal are supported.

The tripleo_dnf_stream module is based on the dnf.py Ansible core module [1]
(so it will be simpler to contribute with this functionality to upstream
Ansible core) and covers the enabling and disabling of one or multiple
MODULE:STREAM.

This functionality is required when performing a minor update, as we
need to make sure that a set of streams are enabled to provide with
the right packages [2]. In this situation, the installation of the
whole module is not really required as a full upgrade will run after.

The module uses the options name (MODULE:STREAM) and state:[disabled or
enabled]. Example:
    - name: Enable the container-tools:3.0
      tripleo_dnf_stream:
        - name: container-tools:3.0
          state: enabled

[0]: https://github.com/ansible/ansible/issues/64852
[1]: https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/dnf.py
[2]: https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/tripleo-packages/tripleo-packages-baremetal-puppet.yaml#L378-L381

Change-Id: I992935e1c478591f3c8af5044df8a625af9eba85
(cherry picked from commit 5212d407d9)
(cherry picked from commit c2d083b03c)
(cherry picked from commit 9959ab3938)
(cherry picked from commit 88fe77140b)
(cherry picked from commit 1d38695d01)
changes/78/819178/4
Jose Luis Franco Arza 8 months ago committed by Sagi Shnaidman
parent
commit
54e953f9d4
  1. 264
      tripleo_ansible/ansible_plugins/modules/tripleo_dnf_stream.py
  2. 53
      tripleo_ansible/ansible_plugins/tests/molecule/tripleo_dnf_stream/molecule.yml
  3. 140
      tripleo_ansible/ansible_plugins/tests/molecule/tripleo_dnf_stream/playbook.yml
  4. 26
      tripleo_ansible/ansible_plugins/tests/molecule/tripleo_dnf_stream/prepare.yml

264
tripleo_ansible/ansible_plugins/modules/tripleo_dnf_stream.py

@ -0,0 +1,264 @@
# Copyright 2021 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
DOCUMENTATION = '''
---
module: tripleo_dnf_stream
short_description: Enable or disable a set of DNF stream modules if available.
description:
- "Enables or disables one or more I(dnf) module streams. If no stream is being
specified, the default stream will be enabled/disabled."
options:
name:
description:
- "A module name to enable or disable, like C(container-tools:3.0).
If no stream or profile is specified then the defaults will be enabled
To handle multiple I(dnf) modules this parameter can accept a comma
separated string or a list of module names with their streams.
Passing the profile in this parameter won't have any impact as the
module only enables or disables the stream, it doesn't install/uninstall
packages."
required: true
type: list
elements: str
state:
description:
- "Whether to enable or disable a module. After the task is executed only
the module will change, there is no packages synchronization performed.
To do so, please check the I(dnf) Ansible module."
default: 'enabled'
required: false
type: str
choices: ['enabled', 'disabled']
author:
- Jose Luis Franco Arza (@jfrancoa)
'''
EXAMPLES = '''
- hosts: dbservers
tasks:
- name: Enable container-tools:3.0 stream module
tripleo_dnf_stream:
name: container-tools:3.0
state: enabled
- name: Disable container-tools:3.0 stream module
tripleo_dnf_stream:
name: container-tools:3.0
state: disabled
- name: Enable nginx, php:7.4 and python36:36
tripleo_dnf_stream:
name:
- nginx
- php:7.4
- python36:3.6
- name: Update packages
dnf:
name: *
state: latest
'''
import sys
try:
import dnf
import dnf.cli
import dnf.const
import dnf.exceptions
import dnf.subject
import dnf.util
HAS_DNF = True
except ImportError:
HAS_DNF = False
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils._text import to_native
from yaml import safe_load as yaml_safe_load
class DnfModule():
"""
DNF Ansible module back-end implementation
"""
def __init__(self, module):
self.module = module
self.name = self.module.params['name']
self.state = self.module.params['state']
self._ensure_dnf()
try:
dnf.base.WITH_MODULES
except AttributeError:
self.module.fail_json(
msg="DNF modules are not supported.",
results=[],
)
def _ensure_dnf(self):
if not HAS_DNF:
self.module.fail_json(
msg="Could not import the dnf python module using {0} ({1}). "
"Please install `python3-dnf` package or ensure you have specified the "
"correct ansible_python_interpreter.".format(sys.executable, sys.version.replace('\n', '')),
results=[],
)
def _base(self):
"""Return a fully configured dnf Base object."""
base = dnf.Base()
base.read_all_repos()
base.fill_sack()
try:
# this method has been supported in dnf-4.2.17-6 or later
# https://bugzilla.redhat.com/show_bug.cgi?id=1788212
base.setup_loggers()
except AttributeError:
pass
try:
base.init_plugins()
base.pre_configure_plugins()
except AttributeError:
pass # older versions of dnf didn't require this and don't have these methods
try:
base.configure_plugins()
except AttributeError:
pass # older versions of dnf didn't require this and don't have these methods
return base
def _is_module_available(self, module_spec):
module_spec = module_spec.strip()
module_list, nsv = self.module_base._get_modules(module_spec)
if nsv:
return True, nsv
else:
return False, None
def _is_module_enabled(self, module_nsv):
enabled_streams = self.base._moduleContainer.getEnabledStream(module_nsv.name)
if enabled_streams:
if module_nsv.stream:
if module_nsv.stream in enabled_streams:
return True # The provided stream was found
else:
return False # The provided stream was not found
else:
return True # No stream provided, but module found
def ensure(self):
response = {
'msg': "",
'changed': False,
'results': [],
'rc': 0
}
# Accumulate failures. Package management modules install what they can
# and fail with a message about what they can't.
failure_response = {
'msg': "",
'failures': [],
'results': [],
'rc': 1
}
if self.state == 'enabled':
for module in self.name:
try:
module_found, nsv = self._is_module_available(module)
if module_found:
if self._is_module_enabled(nsv):
response['results'].append("Module {0} already enabled.".format(module))
self.module_base.enable([module])
else:
failure_response['failures'].append("Module {0} is not available in the system.".format(module))
except dnf.exceptions.MarkingErrors as e:
failure_response['failures'].append(' '.join((module, to_native(e))))
else:
# state = 'disabled'
for module in self.name:
try:
module_found, nsv = self._is_module_available(module)
if module_found:
if not self._is_module_enabled(nsv):
response['results'].append("Module {0} already disabled.".format(module))
self.module_base.disable([module])
self.module_base.reset([module])
else:
# If the module is not available move on
response['results'].append("Module {0} is not available in the system".format(module))
except dnf.exceptions.MarkingErrors as e:
failure_response['failures'].append(' '.join((module, to_native(e))))
try:
if failure_response['failures']:
failure_response['msg'] = 'Failed to manage some of the specified modules'
self.module.fail_json(**failure_response)
# Perform the transaction if no failures found
self.base.do_transaction()
self.module.exit_json(**response)
except dnf.exceptions.Error as e:
failure_response['msg'] = "Unknown Error occured: {0}".format(to_native(e))
self.module.fail_json(**failure_response)
response['changed'] = True
def run(self):
"""The main function."""
# Note: base takes a long time to run so we want to check for failure
# before running it.
if not dnf.util.am_i_root():
self.module.fail_json(
msg="This command has to be run under the root user.",
results=[],
)
self.base = self._base()
self.module_base = dnf.module.module_base.ModuleBase(self.base)
self.ensure()
def main():
module = AnsibleModule(
argument_spec=yaml_safe_load(DOCUMENTATION)['options'],
supports_check_mode=False,
)
module_implementation = DnfModule(module)
try:
module_implementation.run()
except dnf.exceptions.RepoError as de:
module.fail_json(
msg="Failed to synchronize repodata: {0}".format(to_native(de)),
rc=1,
results=[],
changed=False
)
if __name__ == '__main__':
main()

53
tripleo_ansible/ansible_plugins/tests/molecule/tripleo_dnf_stream/molecule.yml

@ -0,0 +1,53 @@
---
driver:
name: podman
log: true
platforms:
- name: ubi8
hostname: ubi8
image: ubi8/ubi-init
registry:
url: registry.access.redhat.com
dockerfile: Dockerfile
pkg_extras: python*setuptools
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
- /etc/ci/mirror_info.sh:/etc/ci/mirror_info.sh:ro
- /etc/pki/rpm-gpg:/etc/pki/rpm-gpg
- /etc/dnf/vars:/etc/dnf/vars
privileged: true
environment: &env
http_proxy: "{{ lookup('env', 'http_proxy') }}"
https_proxy: "{{ lookup('env', 'https_proxy') }}"
ulimits: &ulimit
- host
provisioner:
name: ansible
inventory:
hosts:
all:
hosts:
ubi8:
ansible_python_interpreter: /usr/bin/python3
log: true
env:
ANSIBLE_STDOUT_CALLBACK: yaml
ANSIBLE_ROLES_PATH: "${ANSIBLE_ROLES_PATH:-/usr/share/ansible/roles}:${HOME}/zuul-jobs/roles"
ANSIBLE_LIBRARY: "${ANSIBLE_LIBRARY:-/usr/share/ansible/plugins/modules}"
ANSIBLE_FILTER_PLUGINS: "${ANSIBLE_FILTER_PLUGINS:-/usr/share/ansible/plugins/filter}"
scenario:
name: tripleo_dnf_stream
test_sequence:
- destroy
- create
- prepare
- converge
- verify
- destroy
verifier:
name: testinfra

140
tripleo_ansible/ansible_plugins/tests/molecule/tripleo_dnf_stream/playbook.yml

@ -0,0 +1,140 @@
---
- name: Converge
hosts: all
become: true
tasks:
- name: List available modules at test start for debugging purposes
command: dnf module list
register: module_list
- debug:
msg: "{{ module_list.stdout_lines }}"
- debug:
msg: |
"================================================================
PREPARE: enable maven:3.5
================================================================"
- name: Make sure the module is removed before starting
command: dnf module -C -y remove maven:3.5
- name: Disable the module
command: dnf module -C -y reset maven:3.5
- debug:
msg: |
"================================================================
START: enable maven:3.5
================================================================"
- name: Enable maven:3.5 module
tripleo_dnf_stream:
name: "maven:3.5"
state: enabled
- debug:
msg: |
"================================================================
VERIFY: enable maven:3.5
================================================================"
- name: Ensure the module got enabled
shell: "set -o pipefail && dnf module -C -y list --enabled | grep 'maven\\s*3.5'"
failed_when: false
register: check_module
- name: Fail if module not found enabled
fail:
msg: Module maven:3.5 not found
when: check_module.rc != 0
- debug:
msg: |
"================================================================
PREPARE: change php:7.2 to php:7.3
================================================================"
- name: Make sure the module is enabled before starting
command: dnf module -C -y reset php
- name: Enable the module nginx (php has dependencies on nginx) and php
command: "dnf module -y install {{ item }}"
loop:
- "nginx"
- "php:7.2"
- debug:
msg: |
"================================================================
START: change php:7.2 to php:7.3
================================================================"
- name: Enable php:7.3 module
tripleo_dnf_stream:
name: "php:7.3"
state: enabled
- debug:
msg: |
"================================================================
VERIFY: change php:7.2 to php:7.3
================================================================"
- name: Ensure the module got enabled
shell: "set -o pipefail && dnf module -C -y list --enabled | grep 'php\\s*7.3'"
failed_when: false
register: check_module
- name: Fail if module not found enabled
fail:
msg: Module php:7.3 not found
when: check_module.rc != 0
- debug:
msg: |
"================================================================
PREPARE: enable and disable multiple streams
================================================================"
- name: Make sure the module is disabled before starting
command: "dnf module -C -y remove nodejs:12 javapackages-runtime:201801"
- name: Disable the module
command: "dnf module -C -y reset nodejs javapackages-runtime"
- debug:
msg: |
"================================================================
START 1: enable multiple streams
================================================================"
- name: Enable nodejs:12 and javapackages-runtime:201801 module
tripleo_dnf_stream:
name:
- "nodejs:12"
- "javapackages-runtime:201801"
state: enabled
- debug:
msg: |
"================================================================
VERIFY 1: enable multiple streams
================================================================"
- name: Ensure the module got enabled
shell: "set -o pipefail && dnf module -C -y list --enabled | grep '{{ item.split(\":\")[0] }}\\s*{{ item.split(\":\")[1] }}'"
failed_when: false
register: check_module
loop:
- "nodejs:12"
- "javapackages-runtime:201801"
- name: Fail if module not found enabled
fail:
msg: "Module {{ item.item }} not found"
when: item.rc != 0
loop: "{{ check_module.results }}"
- debug:
msg: |
"================================================================
START 2: disable multiple streams
================================================================"
- name: Disable all enabled modules
tripleo_dnf_stream:
name:
- "nodejs:12"
- "javapackages-runtime:201801"
state: disabled
- debug:
msg: |
"================================================================
VERIFY 2: disable multiple streams
================================================================"
- name: Ensure all modules got disabled
shell: "set -o pipefail && dnf module -C -y list --enabled | grep '{{ item.split(\":\")[0] }}\\s*{{ item.split(\":\")[1] }}'"
failed_when: false
register: check_module
loop:
- "nodejs:12"
- "javapackages-runtime:201801"
- name: Fail if module found enabled
fail:
msg: "Module {{ item.item }} found enabled when it shouldn't"
when: item.rc == 0
loop: "{{ check_module.results }}"

26
tripleo_ansible/ansible_plugins/tests/molecule/tripleo_dnf_stream/prepare.yml

@ -0,0 +1,26 @@
---
# Copyright 2019 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
- name: Prepare
hosts: all
roles:
- role: test_deps
tasks:
- debug:
msg: |
"================================================================
STARTING TEST tripleo_dnf_stream
================================================================"
Loading…
Cancel
Save