openstack/validations-libs
Code Issues Proposed changes

Add new CLI sub command to create community validations

Browse Source 
Presently, the operator(s) can only execute the official and supported
validations coming from tripleo-validations and validations-common.
Community validations enable a sysadmin to create and execute
validations unique to their environment.

This patch introduces the new Command Line Interface sub command to
create a new community validation skeleton. First, this latter will
check if there is an existing role or a playbook either in the community
validations catalog or the official validations catalog.  And it will
create an Ansible role (with ansible-galaxy[1]) and a playbook in the
~/community-validations directory.

By default, the community validations feature is enabled but may be
disabled by setting [DEFAULT].enable_community_validations to ``False``
in the validation configuration file.

Example:

[stack@localhost]$ validation init my-new-validation
Validation config file found: /etc/validation.cfg
New role created successfully in /home/stack/community-validations/roles/my_new_validation
New playbook created successfully in /home/stack/community-validations/playbooks/my-new-validation.yaml

For a full demo of this new CLI sub command, please take a look at this
asciinema[2].

[1] - https://docs.ansible.com/ansible/latest/cli/ansible-galaxy.html
[2] - https://asciinema.org/a/445105

Change-Id: I8fb16e3456696187d4a9d3820740a7639a96e315
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
This commit is contained in:
Gael Chamoulaud (Strider) 2021-10-20 11:01:13 +02:00
parent 7d416acbe8
commit 1bbf282356
No known key found for this signature in database
GPG Key ID: 4119D0305C651D66
13 changed files with 1129 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

146
README.rst
View File

@ -86,4 +86,150 @@ If no hosts key is provided for a given validation, it will be considered as ``h
The ``reason`` and ``lp`` key are for tracking and documentation purposes,
the framework won't use those keys.
Community Validations
=====================
Community Validations enable a sysadmin to create and execute validations unique
to their environment through the ``validation`` CLI.
The Community Validations will be created and stored in an unique, standardized
and known place, called ``'community-validations/'``, in the home directory of the
non-root user which is running the CLI.
.. note::
The Community Validations are enabled by default. If you want to disable
them, please set ``[DEFAULT].enable_community_validations`` to ``False`` in the
validation configuration file located by default in ``/etc/validation.cfg``
The first level of the mandatory structure will be the following (assuming the
operator uses the ``pennywise`` user):
.. code-block:: console
/home/pennywise/community-validations
├── library
├── lookup_plugins
├── playbooks
└── roles
.. note::
The ``community-validations`` directory and its sub directories will be
created at the first CLI use and will be checked everytime a new community
validation will be created through the CLI.
How To Create A New Community Validation
----------------------------------------
.. code-block:: console
[pennywise@localhost]$ validation init my-new-validation
Validation config file found: /etc/validation.cfg
New role created successfully in /home/pennywise/community-validations/roles/my_new_validation
New playbook created successfully in /home/pennywise/community-validations/playbooks/my-new-validation.yaml
The ``community-validations/`` directory should have been created in the home
directory of the ``pennywise`` user.
.. code-block:: console
[pennywise@localhost ~]$ cd && tree community-validations/
community-validations/
├── library
├── lookup_plugins
├── playbooks
│   └── my-new-validation.yaml
└── roles
└── my_new_validation
├── defaults
│   └── main.yml
├── files
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── README.md
├── tasks
│   └── main.yml
├── templates
├── tests
│   ├── inventory
│   └── test.yml
└── vars
└── main.yml
13 directories, 9 files
Your new community validation should also be available when listing all the
validations available on your system.
.. code-block:: console
[pennywise@localhost ~]$ validation list
Validation config file found: /etc/validation.cfg
+-------------------------------+--------------------------------+--------------------------------+-----------------------------------+---------------+
| ID | Name | Groups | Categories | Products |
+-------------------------------+--------------------------------+--------------------------------+-----------------------------------+---------------+
| 512e | Advanced Format 512e Support | ['prep', 'pre-deployment'] | ['storage', 'disk', 'system'] | ['common'] |
| check-cpu | Verify if the server fits the | ['prep', 'backup-and-restore', | ['system', 'cpu', 'core', 'os'] | ['common'] |
| | CPU core requirements | 'pre-introspection'] | | |
| check-disk-space-pre-upgrade | Verify server fits the disk | ['pre-upgrade'] | ['system', 'disk', 'upgrade'] | ['common'] |
| | space requirements to perform | | | |
| | an upgrade | | | |
| check-disk-space | Verify server fits the disk | ['prep', 'pre-introspection'] | ['system', 'disk', 'upgrade'] | ['common'] |
| | space requirements | | | |
| check-ftype | XFS ftype check | ['pre-upgrade'] | ['storage', 'xfs', 'disk'] | ['common'] |
| check-latest-packages-version | Check if latest version of | ['pre-upgrade'] | ['packages', 'rpm', 'upgrade'] | ['common'] |
| | packages is installed | | | |
| check-ram | Verify the server fits the RAM | ['prep', 'pre-introspection', | ['system', 'ram', 'memory', 'os'] | ['common'] |
| | requirements | 'pre-upgrade'] | | |
| check-selinux-mode | SELinux Enforcing Mode Check | ['prep', 'pre-introspection'] | ['security', 'selinux'] | ['common'] |
| dns | Verify DNS | ['pre-deployment'] | ['networking', 'dns'] | ['common'] |
| no-op | NO-OP validation | ['no-op'] | ['noop', 'dummy', 'test'] | ['common'] |
| ntp | Verify all deployed servers | ['post-deployment'] | ['networking', 'time', 'os'] | ['common'] |
| | have their clock synchronised | | | |
| service-status | Ensure services state | ['prep', 'backup-and-restore', | ['systemd', 'container', | ['common'] |
| | | 'pre-deployment', 'pre- | 'docker', 'podman'] | |
| | | upgrade', 'post-deployment', | | |
| | | 'post-upgrade'] | | |
| validate-selinux | validate-selinux | ['backup-and-restore', 'pre- | ['security', 'selinux', 'audit'] | ['common'] |
| | | deployment', 'post- | | |
| | | deployment', 'pre-upgrade', | | |
| | | 'post-upgrade'] | | |
| my-new-validation | Brief and general description | ['prep', 'pre-deployment'] | ['networking', 'security', 'os', | ['community'] |
| | of the validation | | 'system'] | |
+-------------------------------+--------------------------------+--------------------------------+-----------------------------------+---------------+
To get only the list of your community validations, you can filter by products:
.. code-block:: console
[pennywise@localhost]$ validation list --product community
Validation config file found: /etc/validation.cfg
+-------------------+------------------------------------------+----------------------------+------------------------------------------+---------------+
| ID | Name | Groups | Categories | Products |
+-------------------+------------------------------------------+----------------------------+------------------------------------------+---------------+
| my-new-validation | Brief and general description of the | ['prep', 'pre-deployment'] | ['networking', 'security', 'os', | ['community'] |
| | validation | | 'system'] | |
+-------------------+------------------------------------------+----------------------------+------------------------------------------+---------------+
How To Develop Your New Community Validation
--------------------------------------------
As you can see above, the ``validation init`` CLI sub command has generated a
new Ansible role by using `ansible-galaxy
<https://docs.ansible.com/ansible/latest/cli/ansible-galaxy.html>`_
and a new Ansible playbook in the ``community-validations/`` directory.
.. warning::
The community validations won't be supported at all. We won't be responsible
as well for potential use of malignant code in their validations. Only the
creation of a community validation structure through the new Validation CLI sub
command will be supported.
You are now able to implement your own validation by editing the generated
playbook and adding your ansible tasks in the associated role.
For people not familiar with how to write a validation, get started with this
`documentation <https://docs.openstack.org/tripleo-validations/latest/contributing/developer_guide.html#writing-validations>`_.
.. _Apache_license: http://www.apache.org/licenses/LICENSE-2.0

1
setup.cfg
View File

@ -52,3 +52,4 @@ validation.cli:
run = validations_libs.cli.run:Run
history_list = validations_libs.cli.history:ListHistory
history_get = validations_libs.cli.history:GetHistory
init = validations_libs.cli.community:CommunityValidationInit

4
validation.cfg
View File

@ -6,6 +6,10 @@
# Location where the Validation playbooks are stored.
validation_dir = /usr/share/ansible/validation-playbooks
# Whether to enable the creation and running of Community Validations
# (boolean value)
enable_community_validations = True
# Path where the framework is supposed to write logs and results.
# Note: this should not be a relative path.
# By default the framework log in $HOME/validations.

97
validations_libs/cli/community.py Normal file
View File

@ -0,0 +1,97 @@
#!/usr/bin/env python
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import logging
from validations_libs import constants, utils
from validations_libs.cli.base import BaseCommand
from validations_libs.community.init_validation import \
CommunityValidation as com_val
LOG = logging.getLogger(__name__)
class CommunityValidationInit(BaseCommand):
"""Initialize Community Validation Skeleton"""
def get_parser(self, parser):
"""Argument parser for Community Validation Init"""
parser = super(CommunityValidationInit, self).get_parser(parser)
parser.add_argument(
'validation_name',
metavar="<validation_name>",
type=str,
help=(
"The name of the Community Validation:\n"
"Validation name is limited to contain only lowercase "
"alphanumeric characters, plus '_' or '-' and starts "
"with an alpha character. \n"
"Ex: my-val, my_val2. \n"
"This will generate an Ansible role and a playbook in "
f"{constants.COMMUNITY_VALIDATIONS_BASEDIR}. "
"Note that the structure of this directory will be created at "
"the first use."
)
)
if self.app:
# Merge config and CLI args:
return self.base.set_argument_parser(parser)
return parser
def take_action(self, parsed_args):
"""Take Community Validation Action"""
co_validation = com_val(parsed_args.validation_name)
if co_validation.is_community_validations_enabled(self.base.config):
LOG.debug(
(
"Checking the presence of the community validations "
f"{constants.COMMUNITY_VALIDATIONS_BASEDIR} directory..."
)
)
utils.check_community_validations_dir()
if co_validation.is_role_exists():
raise RuntimeError(
(
f"An Ansible role called {co_validation.role_name} "
"already exist in: \n"
f" - {constants.COMMUNITY_ROLES_DIR}\n"
f" - {constants.ANSIBLE_ROLES_DIR}"
)
)
if co_validation.is_playbook_exists():
raise RuntimeError(
(
f"An Ansible playbook called {co_validation.playbook_name} "
"already exist in: \n"
f" - {constants.COMMUNITY_PLAYBOOKS_DIR}\n"
f" - {constants.ANSIBLE_VALIDATION_DIR}"
)
)
co_validation.execute()
else:
raise RuntimeError(
"The Community Validations are disabled:\n"
"To enable them, set [DEFAULT].enable_community_validations "
"to 'True' in the configuration file."
)

15
validations_libs/community/__init__.py Normal file
View File

@ -0,0 +1,15 @@
#!/usr/bin/env python
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

207
validations_libs/community/init_validation.py Normal file
View File

@ -0,0 +1,207 @@
#!/usr/bin/env python
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
import logging
import re
from pathlib import Path
from validations_libs import constants, utils
LOG = logging.getLogger(__name__)
class CommunityValidation:
"""Init Community Validation Role and Playbook Command Class
Initialize a new community role using ansible-galaxy and create a playboook
from a template.
"""
def __init__(self, validation_name):
"""Construct Role and Playbook."""
self._validation_name = validation_name
def execute(self):
"""Execute the actions necessary to create a new community validation
Check if the role name is compliant with Ansible specification
Initializing the new role using ansible-galaxy
Creating the validation playbook from a template on disk
:rtype: ``NoneType``
"""
if not self.is_role_name_compliant:
raise RuntimeError(
"Role Name are limited to contain only lowercase "
"alphanumeric characters, plus '_', '-' and start with an "
"alpha character."
)
cmd = ['ansible-galaxy', 'init', '-v',
'--offline', self.role_name,
'--init-path', self.role_basedir]
result = utils.run_command_and_log(LOG, cmd)
if result != 0:
raise RuntimeError(
(
f"Ansible Galaxy failed to create the role "
f"{self.role_name}, returned {result}."
)
)
LOG.info(f"New role created successfully in {self.role_dir_path}")
try:
self.create_playbook()
except (PermissionError, OSError) as error:
raise RuntimeError(
(
f"Exception {error} encountered while trying to write "
f"the community validation playbook file {self.playbook_path}."
)
)
LOG.info(f"New playbook created successfully in {self.playbook_path}")
def create_playbook(self):
"""Create the playbook for the new community validation"""
playbook = constants.COMMUNITY_PLAYBOOK_TEMPLATE.format(self.role_name)
with open(self.playbook_path, 'w') as playbook_file:
playbook_file.write(playbook)
def is_role_exists(self):
"""New role existence check
This class method checks if the new role name is already existing
in the official validations catalog and in the current community
validations directory.
First, it gets the list of the role names available in
``constants.ANSIBLE_ROLES_DIR``. If there is a match in at least one
of the directories, it returns ``True``, otherwise ``False``.
:rtype: ``Boolean``
"""
non_community_roles = [
Path(x).name
for x in Path(constants.ANSIBLE_ROLES_DIR).iterdir()
if x.is_dir()
]
return Path(self.role_dir_path).exists() or \
self.role_name in non_community_roles
def is_playbook_exists(self):
"""New playbook existence check
This class method checks if the new playbook file is already existing
in the official validations catalog and in the current community
validations directory.
First, it gets the list of the playbooks yaml file available in
``constants.ANSIBLE_VALIDATIONS_DIR``. If there is a match in at least
one of the directories, it returns ``True``, otherwise ``False``.
:rtype: ``Boolean``
"""
non_community_playbooks = [
Path(x).name
for x in Path(constants.ANSIBLE_VALIDATION_DIR).iterdir()
if x.is_file()
]
return Path(self.playbook_path).exists() or \
self.playbook_name in non_community_playbooks
def is_community_validations_enabled(self, base_config):
"""Checks if the community validations are enabled in the config file
:param base_config: Contents of the configuration file
:type base_config: ``Dict``
:rtype: ``Boolean``
"""
config = base_config
default_conf = (config.get('default', {})
if isinstance(config, dict) else {})
return default_conf.get('enable_community_validations', True)
@property
def role_name(self):
"""Returns the community validation role name
:rtype: ``str``
"""
if re.match(r'^[a-z][a-z0-9_-]+$', self._validation_name) and \
'-' in self._validation_name:
return self._validation_name.replace('-', '_')
return self._validation_name
@property
def role_basedir(self):
"""Returns the absolute path of the community validations roles
:rtype: ``pathlib.PosixPath``
"""
return constants.COMMUNITY_ROLES_DIR
@property
def role_dir_path(self):
"""Returns the community validation role directory name
:rtype: ``pathlib.PosixPath``
"""
return Path.joinpath(self.role_basedir, self.role_name)
@property
def is_role_name_compliant(self):
"""Check if the role name is compliant with Ansible Rules
Roles Name are limited to contain only lowercase
alphanumeric characters, plus '_' and start with an
alpha character.
:rtype: ``Boolean``
"""
if not re.match(r'^[a-z][a-z0-9_]+$', self.role_name):
return False
return True
@property
def playbook_name(self):
"""Return the new playbook name with the yaml extension
:rtype: ``str``
"""
return self._validation_name.replace('_', '-') + ".yaml"
@property
def playbook_basedir(self):
"""Returns the absolute path of the community playbooks directory
:rtype: ``pathlib.PosixPath``
"""
return constants.COMMUNITY_PLAYBOOKS_DIR
@property
def playbook_path(self):
"""Returns the absolute path of the new community playbook yaml file
:rtype: ``pathlib.PosixPath``
"""
return Path.joinpath(self.playbook_basedir, self.playbook_name)

84
validations_libs/constants.py
View File

@ -22,12 +22,17 @@ or as a fallback, when custom locations fail.
import os
from pathlib import Path
DEFAULT_VALIDATIONS_BASEDIR = '/usr/share/ansible'
ANSIBLE_VALIDATION_DIR = os.path.join(
DEFAULT_VALIDATIONS_BASEDIR,
'validation-playbooks')
ANSIBLE_ROLES_DIR = Path.joinpath(Path(DEFAULT_VALIDATIONS_BASEDIR),
'roles')
VALIDATION_GROUPS_INFO = os.path.join(
DEFAULT_VALIDATIONS_BASEDIR,
'groups.yaml')
@ -42,3 +47,82 @@ VALIDATION_ANSIBLE_ARTIFACT_PATH = os.path.join(
ANSIBLE_RUNNER_CONFIG_PARAMETERS = ['verbosity', 'extravars', 'fact_cache',
'fact_cache_type', 'inventory', 'playbook',
'project_dir', 'quiet', 'rotate_artifacts']
# Community Validations paths
COMMUNITY_VALIDATIONS_BASEDIR = Path.home().joinpath('community-validations')
COMMUNITY_ROLES_DIR = Path.joinpath(COMMUNITY_VALIDATIONS_BASEDIR, 'roles')
COMMUNITY_PLAYBOOKS_DIR = Path.joinpath(
COMMUNITY_VALIDATIONS_BASEDIR, 'playbooks')
COMMUNITY_LIBRARY_DIR = Path.joinpath(
COMMUNITY_VALIDATIONS_BASEDIR, 'library')
COMMUNITY_LOOKUP_DIR = Path.joinpath(
COMMUNITY_VALIDATIONS_BASEDIR, 'lookup_plugins')
COMMUNITY_VALIDATIONS_SUBDIR = [COMMUNITY_ROLES_DIR,
COMMUNITY_PLAYBOOKS_DIR,
COMMUNITY_LIBRARY_DIR,
COMMUNITY_LOOKUP_DIR]
COMMUNITY_PLAYBOOK_TEMPLATE = \
"""---
# This playbook has been generated by the `validation init` CLI.
#
# As shown here in this template, the validation playbook requires three
# top-level directive:
# ``hosts``, ``vars -> metadata`` and ``roles``.
#
# ``hosts``: specifies which nodes to run the validation on. The options can
# be ``all`` (run on all nodes), or you could use the hosts defined
# in the inventory.
# ``vars``: this section serves for storing variables that are going to be
# available to the Ansible playbook. The validations API uses the
# ``metadata`` section to read each validation's name and description
# These values are then reported by the API.
#
# The validations can be grouped together by specyfying a ``groups`` metadata.
# Groups function similar to tags and a validation can thus be part of many
# groups. To get a full list of the groups available and their description,
# please run the following command on your Ansible Controller host:
#
# $ validation show group
#
# The validations can also be categorized by technical domain and acan belong to
# one or multiple ``categories``. For example, if your validation checks some
# networking related configuration, you may want to put ``networking`` as a
# category. Note that this section is open and you are free to categorize your
# validations as you like.
#
# The ``products`` section refers to the product on which you would like to run
# the validation. It's another way to categorized your community validations.
# Note that, by default, ``community`` is set in the ``products`` section to
# help you list your validations by filtering by products:
#
# $ validation list --product community
#
- hosts: hostname
gather_facts: false
vars:
metadata:
name: Brief and general description of the validation
description: |
The complete description of this validation should be here
# GROUPS:
# Run ``validation show group`` to get the list of groups
# :type group: `list`
# If you don't want to add groups for your validation, just
# set an empty list to the groups key
groups: []
# CATEGORIES:
# :type group: `list`
# If you don't want to categorize your validation, just
# set an empty list to the categories key
categories: []
products:
- community
roles:
- {}
"""

92
validations_libs/tests/cli/test_community.py Normal file
View File

@ -0,0 +1,92 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
try:
from unittest import mock
except ImportError:
import mock
from validations_libs.cli import community
from validations_libs.cli import base
from validations_libs.community.init_validation import \
CommunityValidation as cv
from validations_libs.tests import fakes
from validations_libs.tests.cli.fakes import BaseCommand
class TestCommunityValidationInit(BaseCommand):
def setUp(self):
super(TestCommunityValidationInit, self).setUp()
self.cmd = community.CommunityValidationInit(self.app, None)
self.base = base.Base()
@mock.patch(
'validations_libs.community.init_validation.CommunityValidation.execute')
@mock.patch(
'validations_libs.community.init_validation.CommunityValidation.is_playbook_exists',
return_value=False)
@mock.patch(
'validations_libs.community.init_validation.CommunityValidation.is_role_exists',
return_value=False)
def test_validation_init(self,
mock_role_exists,
mock_play_exists,
mock_execute):
args = self._set_args(['my_new_community_val'])
verifylist = [('validation_name', 'my_new_community_val')]
parsed_args = self.check_parser(self.cmd, args, verifylist)
self.cmd.take_action(parsed_args)
@mock.patch(
'validations_libs.community.init_validation.CommunityValidation.is_community_validations_enabled',
return_value=False)
def test_validation_init_with_com_val_disabled(self, mock_config):
args = self._set_args(['my_new_community_val'])
verifylist = [('validation_name', 'my_new_community_val')]
parsed_args = self.check_parser(self.cmd, args, verifylist)
self.assertRaises(RuntimeError, self.cmd.take_action,
parsed_args)
@mock.patch(
'validations_libs.community.init_validation.CommunityValidation.is_role_exists',
return_value=True)
@mock.patch(
'validations_libs.community.init_validation.CommunityValidation.is_playbook_exists',
return_value=False)
def test_validation_init_with_role_existing(self, mock_playbook_exists,
mock_role_exists):
args = self._set_args(['my_new_community_val'])
verifylist = [('validation_name', 'my_new_community_val')]
parsed_args = self.check_parser(self.cmd, args, verifylist)
self.assertRaises(RuntimeError, self.cmd.take_action,
parsed_args)
@mock.patch(
'validations_libs.community.init_validation.CommunityValidation.is_role_exists',
return_value=False)
@mock.patch(
'validations_libs.community.init_validation.CommunityValidation.is_playbook_exists',
return_value=True)
def test_validation_with_playbook_existing(self, mock_playbook_exists,
mock_role_exists):
args = self._set_args(['my_new_community_val'])
verifylist = [('validation_name', 'my_new_community_val')]
parsed_args = self.check_parser(self.cmd, args, verifylist)
self.assertRaises(RuntimeError, self.cmd.take_action,
parsed_args)

14
validations_libs/tests/community/__init__.py Normal file
View File

@ -0,0 +1,14 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#

172
validations_libs/tests/community/test_init_validation.py Normal file
View File

@ -0,0 +1,172 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
try:
from unittest import mock
except ImportError:
import mock
from pathlib import PosixPath
from unittest import TestCase
from validations_libs import constants
from validations_libs.community.init_validation import \
CommunityValidation as cv
from validations_libs.tests import fakes
class TestCommunityValidation(TestCase):
def setUp(self):
super(TestCommunityValidation, self).setUp()
def test_role_name_underscored(self):
validation_name = "my_new_validation"
co_val = cv(validation_name)
role_name = co_val.role_name
self.assertEqual(role_name, validation_name)
def test_role_name_with_underscores_and_dashes(self):
validation_name = "my_new-validation"
co_val = cv(validation_name)
self.assertEqual(co_val.role_name, "my_new_validation")
def test_role_name_with_dashes_only(self):
validation_name = "my-new-validation"
co_val = cv(validation_name)
self.assertEqual(co_val.role_name,
"my_new_validation")
def test_role_name_compliant(self):
validation_name = "my_new_validation"
co_val = cv(validation_name)
self.assertTrue(co_val.is_role_name_compliant)
def test_role_name_not_compliant(self):
validation_name = "123_my_new-validation"
co_val = cv(validation_name)
self.assertFalse(co_val.is_role_name_compliant)
def test_role_basedir(self):
validation_name = "my_new-validation"
co_val = cv(validation_name)
self.assertEqual(co_val.role_basedir,
constants.COMMUNITY_ROLES_DIR)
def test_playbook_name_with_underscores(self):
validation_name = "my_new_validation"
co_val = cv(validation_name)
self.assertEqual(co_val.playbook_name,
"my-new-validation.yaml")
def test_playbook_name_with_underscores_and_dashes(self):
validation_name = "my_new-validation"
co_val = cv(validation_name)
self.assertEqual(co_val.playbook_name,
"my-new-validation.yaml")
def test_playbook_basedir(self):
validation_name = "my_new-validation"
co_val = cv(validation_name)
self.assertEqual(co_val.playbook_basedir,
constants.COMMUNITY_PLAYBOOKS_DIR)
@mock.patch('pathlib.Path.iterdir',
return_value=fakes.FAKE_ROLES_ITERDIR1)
@mock.patch('pathlib.Path.is_dir')
@mock.patch('pathlib.Path.exists', return_value=False)
def test_role_already_exists(self,
mock_path_exists,
mock_path_is_dir,
mock_path_iterdir):
validation_name = "my-val"
co_val = cv(validation_name)
self.assertTrue(co_val.is_role_exists())
@mock.patch('pathlib.Path.iterdir',
return_value=fakes.FAKE_ROLES_ITERDIR2)
@mock.patch('pathlib.Path.is_dir')
@mock.patch('pathlib.Path.exists', return_value=False)
def test_role_not_exists(self,
mock_path_exists,
mock_path_is_dir,
mock_path_iterdir):
validation_name = "my-val"
co_val = cv(validation_name)
self.assertFalse(co_val.is_role_exists())
@mock.patch('pathlib.Path.iterdir',
return_value=fakes.FAKE_PLAYBOOKS_ITERDIR1)
@mock.patch('pathlib.Path.is_file')
@mock.patch('pathlib.Path.exists', return_value=True)
def test_playbook_already_exists(self,
mock_path_exists,
mock_path_is_file,
mock_path_iterdir):
validation_name = "my_val"
co_val = cv(validation_name)
self.assertTrue(co_val.is_playbook_exists())
@mock.patch('pathlib.Path.iterdir',
return_value=fakes.FAKE_PLAYBOOKS_ITERDIR2)
@mock.patch('pathlib.Path.is_file')
@mock.patch('pathlib.Path.exists', return_value=False)
def test_playbook_not_exists(self,
mock_path_exists,
mock_path_is_file,
mock_path_iterdir):
validation_name = "my_val"
co_val = cv(validation_name)
self.assertFalse(co_val.is_playbook_exists())
def test_execute_with_role_name_not_compliant(self):
validation_name = "3_my-val"
co_val = cv(validation_name)
self.assertRaises(RuntimeError, co_val.execute)
@mock.patch('validations_libs.utils.run_command_and_log',
return_value=0)
@mock.patch('validations_libs.community.init_validation.CommunityValidation.role_basedir',
return_value=PosixPath("/foo/bar/roles"))
@mock.patch('validations_libs.community.init_validation.LOG',
autospec=True)
def test_exec_new_role_with_galaxy(self,
mock_log,
mock_role_basedir,
mock_run):
validation_name = "my_val"
cmd = ['ansible-galaxy', 'init', '-v',
'--offline', validation_name,
'--init-path', mock_role_basedir]
co_val = cv(validation_name)
co_val.execute()
mock_run.assert_called_once_with(mock_log, cmd)
@mock.patch('validations_libs.utils.run_command_and_log',
return_value=1)
@mock.patch('validations_libs.community.init_validation.CommunityValidation.role_basedir',
return_value=PosixPath("/foo/bar/roles"))
@mock.patch('validations_libs.community.init_validation.LOG',
autospec=True)
def test_exec_new_role_with_galaxy_and_error(self,
mock_log,
mock_role_basedir,
mock_run):
validation_name = "my_val"
cmd = ['ansible-galaxy', 'init', '-v',
'--offline', validation_name,
'--init-path', mock_role_basedir]
co_val = cv(validation_name)
self.assertRaises(RuntimeError, co_val.execute)

50
validations_libs/tests/fakes.py
View File

@ -13,6 +13,7 @@
# under the License.
#
from pathlib import PosixPath
from validations_libs import constants
VALIDATIONS_LIST = [{
@ -319,11 +320,20 @@ FAKE_FAILED_RUN = [{'Duration': '0:00:01.761',
FAKE_VALIDATIONS_PATH = '/usr/share/ansible/validation-playbooks'
DEFAULT_CONFIG = {'validation_dir': '/usr/share/ansible/validation-playbooks',
'enable_community_validations': True,
'ansible_base_dir': '/usr/share/ansible/',
'output_log': 'output.log',
'history_limit': 15,
'fit_width': True}
CONFIG_WITH_COMMUNITY_VAL_DISABLED = {
'validation_dir': '/usr/share/ansible/validation-playbooks',
'enable_community_validations': False,
'ansible_base_dir': '/usr/share/ansible/',
'output_log': 'output.log',
'history_limit': 15,
'fit_width': True}
WRONG_HISTORY_CONFIG = {'default': {'history_limit': 0}}
ANSIBLE_RUNNER_CONFIG = {'verbosity': 5,
@ -335,6 +345,46 @@ ANSIBLE_ENVIRONNMENT_CONFIG = {'ANSIBLE_CALLBACK_WHITELIST':
'profile_tasks',
'ANSIBLE_STDOUT_CALLBACK': 'validation_stdout'}
COVAL_SUBDIR = [PosixPath("/foo/bar/community-validations/roles"),
PosixPath("/foo/bar/community-validations/playbooks"),
PosixPath("/foo/bar/community-validations/library"),
PosixPath("/foo/bar/community-validations/lookup_plugins")]
COVAL_MISSING_SUBDIR = [PosixPath("/foo/bar/community-validations/roles"),
PosixPath("/foo/bar/community-validations/playbooks")]
FAKE_COVAL_ITERDIR1 = iter(COVAL_SUBDIR)
FAKE_COVAL_MISSING_SUBDIR_ITERDIR1 = iter(COVAL_MISSING_SUBDIR)
FAKE_ROLES_ITERDIR1 = iter([PosixPath("/u/s/a/roles/role_1"),
PosixPath("/u/s/a/roles/role_2"),
PosixPath("/u/s/a/roles/role_3"),
PosixPath("/u/s/a/roles/role_4"),
PosixPath("/u/s/a/roles/role_5"),
PosixPath("/u/s/a/roles/my_val")])
FAKE_ROLES_ITERDIR2 = iter([PosixPath("/u/s/a/roles/role_1"),
PosixPath("/u/s/a/roles/role_2"),
PosixPath("/u/s/a/roles/role_3"),
PosixPath("/u/s/a/roles/role_4"),
PosixPath("/u/s/a/roles/role_5"),
PosixPath("/u/s/a/roles/role_6")])
FAKE_PLAYBOOKS_ITERDIR1 = iter([PosixPath("/u/s/a/plays/play_1.yaml"),
PosixPath("/u/s/a/plays/play_2.yaml"),
PosixPath("/u/s/a/plays/play_3.yaml"),
PosixPath("/u/s/a/plays/play_4.yaml"),
PosixPath("/u/s/a/plays/play_5.yaml"),
PosixPath("/u/s/a/plays/my-val.yaml")])
FAKE_PLAYBOOKS_ITERDIR2 = iter([PosixPath("/u/s/a/plays/play_1.yaml"),
PosixPath("/u/s/a/plays/play_2.yaml"),
PosixPath("/u/s/a/plays/play_3.yaml"),
PosixPath("/u/s/a/plays/play_4.yaml"),
PosixPath("/u/s/a/plays/play_5.yaml"),
PosixPath("/u/s/a/plays/play_6.yaml")])
def fake_ansible_runner_run_return(status='successful', rc=0):
return status, rc

134
validations_libs/tests/test_utils.py
View File

@ -13,11 +13,16 @@
# under the License.
#
import logging
import os
import subprocess
try:
from unittest import mock
except ImportError:
import mock
from pathlib import PosixPath
from unittest import TestCase
from validations_libs import utils, constants
@ -426,3 +431,132 @@ class TestUtils(TestCase):
self.assertEqual(
results['ansible_environment']['ANSIBLE_STDOUT_CALLBACK'],
fakes.ANSIBLE_ENVIRONNMENT_CONFIG['ANSIBLE_STDOUT_CALLBACK'])
@mock.patch('validations_libs.utils.LOG', autospec=True)
@mock.patch('pathlib.Path.exists',
return_value=False)
@mock.patch('pathlib.Path.is_dir',
return_value=False)
@mock.patch('pathlib.Path.iterdir',
return_value=iter([]))
@mock.patch('pathlib.Path.mkdir')
def test_check_creation_community_validations_dir(self, mock_mkdir,
mock_iterdir,
mock_isdir,
mock_exists,
mock_log):
basedir = PosixPath('/foo/bar/community-validations')
subdir = fakes.COVAL_SUBDIR
result = utils.check_community_validations_dir(basedir, subdir)
self.assertEqual(result,
[PosixPath('/foo/bar/community-validations'),
PosixPath("/foo/bar/community-validations/roles"),
PosixPath("/foo/bar/community-validations/playbooks"),
PosixPath("/foo/bar/community-validations/library"),
PosixPath("/foo/bar/community-validations/lookup_plugins")]
)
@mock.patch('validations_libs.utils.LOG', autospec=True)
@mock.patch('pathlib.Path.is_dir', return_value=True)
@mock.patch('pathlib.Path.exists', return_value=True)
@mock.patch('pathlib.Path.iterdir',
return_value=fakes.FAKE_COVAL_MISSING_SUBDIR_ITERDIR1)
@mock.patch('pathlib.Path.mkdir')
def test_check_community_validations_dir_with_missing_subdir(self,
mock_mkdir,
mock_iterdir,
mock_exists,
mock_isdir,
mock_log):
basedir = PosixPath('/foo/bar/community-validations')
subdir = fakes.COVAL_SUBDIR
result = utils.check_community_validations_dir(basedir, subdir)
self.assertEqual(result,
[PosixPath('/foo/bar/community-validations/library'),
PosixPath('/foo/bar/community-validations/lookup_plugins')])
class TestRunCommandAndLog(TestCase):
def setUp(self):
self.mock_logger = mock.Mock(spec=logging.Logger)
self.mock_process = mock.Mock()
self.mock_process.stdout.readline.side_effect = ['foo\n', 'bar\n']
self.mock_process.wait.side_effect = [0]
self.mock_process.returncode = 0
mock_sub = mock.patch('subprocess.Popen',
return_value=self.mock_process)
self.mock_popen = mock_sub.start()
self.addCleanup(mock_sub.stop)
self.cmd = ['exit', '0']
self.e_cmd = ['exit', '1']
self.log_calls = [mock.call('foo'),
mock.call('bar')]
def test_success_default(self):
retcode = utils.run_command_and_log(self.mock_logger, self.cmd)
self.mock_popen.assert_called_once_with(self.cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
shell=False,
cwd=None, env=None)
self.assertEqual(retcode, 0)
self.mock_logger.debug.assert_has_calls(self.log_calls,
any_order=False)
@mock.patch('subprocess.Popen')
def test_error_subprocess(self, mock_popen):
mock_process = mock.Mock()
mock_process.stdout.readline.side_effect = ['Error\n']
mock_process.wait.side_effect = [1]
mock_process.returncode = 1
mock_popen.return_value = mock_process
retcode = utils.run_command_and_log(self.mock_logger, self.e_cmd)
mock_popen.assert_called_once_with(self.e_cmd, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
shell=False, cwd=None,
env=None)
self.assertEqual(retcode, 1)
self.mock_logger.debug.assert_called_once_with('Error')
def test_success_env(self):
test_env = os.environ.copy()
retcode = utils.run_command_and_log(self.mock_logger, self.cmd,
env=test_env)
self.mock_popen.assert_called_once_with(self.cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
shell=False,
cwd=None, env=test_env)
self.assertEqual(retcode, 0)
self.mock_logger.debug.assert_has_calls(self.log_calls,
any_order=False)
def test_success_cwd(self):
test_cwd = '/usr/local/bin'
retcode = utils.run_command_and_log(self.mock_logger, self.cmd,
cwd=test_cwd)
self.mock_popen.assert_called_once_with(self.cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
shell=False,
cwd=test_cwd, env=None)
self.assertEqual(retcode, 0)
self.mock_logger.debug.assert_has_calls(self.log_calls,
any_order=False)
def test_success_no_retcode(self):
run = utils.run_command_and_log(self.mock_logger, self.cmd,
retcode_only=False)
self.mock_popen.assert_called_once_with(self.cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
shell=False,
cwd=None, env=None)
self.assertEqual(run, self.mock_process)
self.mock_logger.debug.assert_not_called()

114
validations_libs/utils.py
View File

@ -20,10 +20,11 @@ import logging
import os
import site
import six
import sys
import subprocess
import uuid
from os.path import join
from pathlib import Path
from validations_libs import constants
from validations_libs.group import Group
from validations_libs.validation import Validation
@ -583,3 +584,114 @@ def find_config_file(config_file_name='validation.cfg'):
if _check_path(current_path):
return current_path
return current_path
def run_command_and_log(log, cmd, cwd=None,
env=None, retcode_only=True):
"""Run command and log output
:param log: Logger instance for logging
:type log: `Logger`
:param cmd: Command to run in list form
:type cmd: ``List``
:param cwd: Current working directory for execution
:type cmd: ``String``
:param env: Modified environment for command run
:type env: ``List``
:param retcode_only: Returns only retcode instead or proc object
:type retcdode_only: ``Boolean``
"""
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, shell=False,
cwd=cwd, env=env)
if retcode_only:
while True:
try:
line = proc.stdout.readline()
except StopIteration:
break
if line != b'':
if isinstance(line, bytes):
line = line.decode('utf-8')
log.debug(line.rstrip())
else:
break
proc.stdout.close()
return proc.wait()
return proc
def check_community_validations_dir(
basedir=constants.COMMUNITY_VALIDATIONS_BASEDIR,
subdirs=constants.COMMUNITY_VALIDATIONS_SUBDIR):
"""Check presence of the community validations directory structure
The community validations are stored and located in:
.. code-block:: console
/home/<username>/community-validations
├── library
├── lookup_plugins
├── playbooks
└── roles
This function checks for the presence of the community-validations directory
in the $HOME of the user running the validation CLI. If the primary
directory doesn't exist, this function will create it and will check if the
four subdirectories are present and will create them otherwise.
:param basedir: Absolute path of the community validations
:type basedir: ``pathlib.PosixPath``
:param subdirs: List of Absolute path of the community validations subdirs
:type subdirs: ``list`` of ``pathlib.PosixPath``
:rtype: ``NoneType``
"""
recreated_comval_dir = []
def create_subdir(subdir):
for _dir in subdir:
LOG.debug(
f"Missing {Path(_dir).name} directory in {basedir}:"
)
Path.mkdir(_dir)
recreated_comval_dir.append(_dir)
LOG.debug(
f"└── {_dir} directory created successfully..."
)
if Path(basedir).exists and Path(basedir).is_dir():
_subdirectories = [x for x in basedir.iterdir() if x.is_dir()]
missing_dirs = [
_dir for _dir in subdirs
if _dir not in _subdirectories
]
create_subdir(missing_dirs)
else:
LOG.debug(
f"The community validations {basedir} directory is not present:"
)
Path.mkdir(basedir)
recreated_comval_dir.append(basedir)
LOG.debug(f"└── {basedir} directory created...")
create_subdir(subdirs)
LOG.debug(
(
f"The {basedir} directory and its required subtree are present "
f"and correct:\n"
f"{basedir}/\n"
"├── library OK\n"
"├── lookup_plugins OK\n"
"├── playbooks OK\n"
"└── roles OK\n"
)
)
return recreated_comval_dir