RETIRED, A collection of python libraries for the Validation Framework
Go to file
Gael Chamoulaud (Strider) 1bbf282356
Add new CLI sub command to create community validations
Presently, the operator(s) can only execute the official and supported
validations coming from tripleo-validations and validations-common.
Community validations enable a sysadmin to create and execute
validations unique to their environment.

This patch introduces the new Command Line Interface sub command to
create a new community validation skeleton. First, this latter will
check if there is an existing role or a playbook either in the community
validations catalog or the official validations catalog.  And it will
create an Ansible role (with ansible-galaxy[1]) and a playbook in the
~/community-validations directory.

By default, the community validations feature is enabled but may be
disabled by setting [DEFAULT].enable_community_validations to ``False``
in the validation configuration file.

Example:

[stack@localhost]$ validation init my-new-validation
Validation config file found: /etc/validation.cfg
New role created successfully in /home/stack/community-validations/roles/my_new_validation
New playbook created successfully in /home/stack/community-validations/playbooks/my-new-validation.yaml

For a full demo of this new CLI sub command, please take a look at this
asciinema[2].

[1] - https://docs.ansible.com/ansible/latest/cli/ansible-galaxy.html
[2] - https://asciinema.org/a/445105

Change-Id: I8fb16e3456696187d4a9d3820740a7639a96e315
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
2021-10-29 15:39:39 +02:00
doc Add validation config file mechanism 2021-08-20 08:12:16 +00:00
dockerfiles/localvalidations Docker image refinement and preparation for future development. 2021-02-12 12:48:00 +01:00
playbooks fix var name in logging statement 2021-07-07 07:51:37 +02:00
releasenotes Moving doc structure closer to that in validations-commons. 2021-02-25 13:11:34 +01:00
validations_libs Add new CLI sub command to create community validations 2021-10-29 15:39:39 +02:00
.coveragerc Make the measuring code coverage test working 2020-12-11 10:23:15 +01:00
.dockerignore Docker image refinement and preparation for future development. 2021-02-12 12:48:00 +01:00
.gitignore Make the measuring code coverage test working 2020-12-11 10:23:15 +01:00
.gitreview Improve the way we log on the file system 2020-03-11 17:07:56 +01:00
.pre-commit-config.yaml Bump pre-commit repositories 2021-05-05 15:56:39 +02:00
.reqcheck_override.yaml Reqcheck job for upstream validations-libs 2021-05-20 17:22:20 +02:00
.stestr.conf Make the measuring code coverage test working 2020-12-11 10:23:15 +01:00
.zuul.yaml Triggering the 'tripleo-ci-centos-8-undercloud-containers' 2021-07-30 13:40:58 +02:00
bindep.txt Add missing font for PDF generation 2021-07-26 23:08:29 +09:00
CONTRIBUTING.rst Moving doc structure closer to that in validations-commons. 2021-02-25 13:11:34 +01:00
Dockerfile Docker image refinement and preparation for future development. 2021-02-12 12:48:00 +01:00
LICENSE Initial commit 2020-02-28 10:42:18 +01:00
MANIFEST.in Adding the foundation files 2020-02-28 14:47:28 +01:00
README.rst Add new CLI sub command to create community validations 2021-10-29 15:39:39 +02:00
requirements.txt Filling license info for Ansible and Pyflakes 2021-06-15 11:56:54 +02:00
setup.cfg Add new CLI sub command to create community validations 2021-10-29 15:39:39 +02:00
setup.py Adding the foundation files 2020-02-28 14:47:28 +01:00
skiplist-example.yaml Expose skip list mechanism via the CLI 2021-10-06 13:45:08 +00:00
test-requirements.txt Filling license info for Ansible and Pyflakes 2021-06-15 11:56:54 +02:00
tox.ini Setting language for the python3.6 tox environment 2021-08-19 08:36:32 +00:00
Vagrantfile.centos improve vagrant functionality 2021-07-13 09:34:05 -04:00
Vagrantfile.ubuntu improve vagrant functionality 2021-07-13 09:34:05 -04:00
validation.cfg Add new CLI sub command to create community validations 2021-10-29 15:39:39 +02:00

validations-libs

image

A collection of python libraries for the Validation Framework

The validations will help detect issues early in the deployment process and prevent field engineers from wasting time on misconfiguration or hardware issues in their environments.

Development Environment Setup

Vagrantfiles for CentOS and Ubuntu have been provided for convenience; simply copy one into your desired location and rename to Vagrantfile, then run:

vagrant up

Once complete you will have a clean development environment ready to go for working with Validation Framework.

podman Quickstart

A Dockerfile is provided at the root of the Validations Library project in order to quickly set and hack the Validation Framework, on a equivalent of a single machine. Build the container from the Dockerfile by running:

podman build -t "vf:dockerfile" .

From the validations-libs repo directory.

Note

More complex images are available in the dockerfiles directory and require explicit specification of both build context and the Dockerfile.

Since the podman build uses code sourced from the buildah project to build container images. It is also possible to build an image using:

buildah bud -t "vf:dockerfile" .

Then you can run the container and start to run some builtin Validations:

podman run -ti vf:dockerfile /bin/bash

Then run validations:

validation.py run --validation check-ftype,512e --inventory /etc/ansible/hosts

Skip list

You can provide a file with a list of Validations to skip via the run command:

validation.py run --validation check-ftype,512e --inventory /etc/ansible/hosts --skiplist my-skip-list.yaml

This file should be formed as:

validation-name:
  hosts: targeted_hostname
  reason: reason to ignore the file
  lp: bug number

The framework will skip the validation against the hosts key. In order to skip the validation on every hosts, you can set all value such as:

hosts: all

If no hosts key is provided for a given validation, it will be considered as hosts: all.

Note

The reason and lp key are for tracking and documentation purposes, the framework won't use those keys.

Community Validations

Community Validations enable a sysadmin to create and execute validations unique to their environment through the validation CLI.

The Community Validations will be created and stored in an unique, standardized and known place, called 'community-validations/', in the home directory of the non-root user which is running the CLI.

Note

The Community Validations are enabled by default. If you want to disable them, please set [DEFAULT].enable_community_validations to False in the validation configuration file located by default in /etc/validation.cfg

The first level of the mandatory structure will be the following (assuming the operator uses the pennywise user):

/home/pennywise/community-validations
├── library
├── lookup_plugins
├── playbooks
└── roles

Note

The community-validations directory and its sub directories will be created at the first CLI use and will be checked everytime a new community validation will be created through the CLI.

How To Create A New Community Validation

[pennywise@localhost]$ validation init my-new-validation
Validation config file found: /etc/validation.cfg
New role created successfully in /home/pennywise/community-validations/roles/my_new_validation
New playbook created successfully in /home/pennywise/community-validations/playbooks/my-new-validation.yaml

The community-validations/ directory should have been created in the home directory of the pennywise user.

[pennywise@localhost ~]$ cd && tree community-validations/
community-validations/
├── library
├── lookup_plugins
├── playbooks
│   └── my-new-validation.yaml
└── roles
    └── my_new_validation
        ├── defaults
        │   └── main.yml
        ├── files
        ├── handlers
        │   └── main.yml
        ├── meta
        │   └── main.yml
        ├── README.md
        ├── tasks
        │   └── main.yml
        ├── templates
        ├── tests
        │   ├── inventory
        │   └── test.yml
        └── vars
            └── main.yml

13 directories, 9 files

Your new community validation should also be available when listing all the validations available on your system.

[pennywise@localhost ~]$ validation list
Validation config file found: /etc/validation.cfg
+-------------------------------+--------------------------------+--------------------------------+-----------------------------------+---------------+
| ID                            | Name                           | Groups                         | Categories                        | Products      |
+-------------------------------+--------------------------------+--------------------------------+-----------------------------------+---------------+
| 512e                          | Advanced Format 512e Support   | ['prep', 'pre-deployment']     | ['storage', 'disk', 'system']     | ['common']    |
| check-cpu                     | Verify if the server fits the  | ['prep', 'backup-and-restore', | ['system', 'cpu', 'core', 'os']   | ['common']    |
|                               | CPU core requirements          | 'pre-introspection']           |                                   |               |
| check-disk-space-pre-upgrade  | Verify server fits the disk    | ['pre-upgrade']                | ['system', 'disk', 'upgrade']     | ['common']    |
|                               | space requirements to perform  |                                |                                   |               |
|                               | an upgrade                     |                                |                                   |               |
| check-disk-space              | Verify server fits the disk    | ['prep', 'pre-introspection']  | ['system', 'disk', 'upgrade']     | ['common']    |
|                               | space requirements             |                                |                                   |               |
| check-ftype                   | XFS ftype check                | ['pre-upgrade']                | ['storage', 'xfs', 'disk']        | ['common']    |
| check-latest-packages-version | Check if latest version of     | ['pre-upgrade']                | ['packages', 'rpm', 'upgrade']    | ['common']    |
|                               | packages is installed          |                                |                                   |               |
| check-ram                     | Verify the server fits the RAM | ['prep', 'pre-introspection',  | ['system', 'ram', 'memory', 'os'] | ['common']    |
|                               | requirements                   | 'pre-upgrade']                 |                                   |               |
| check-selinux-mode            | SELinux Enforcing Mode Check   | ['prep', 'pre-introspection']  | ['security', 'selinux']           | ['common']    |
| dns                           | Verify DNS                     | ['pre-deployment']             | ['networking', 'dns']             | ['common']    |
| no-op                         | NO-OP validation               | ['no-op']                      | ['noop', 'dummy', 'test']         | ['common']    |
| ntp                           | Verify all deployed servers    | ['post-deployment']            | ['networking', 'time', 'os']      | ['common']    |
|                               | have their clock synchronised  |                                |                                   |               |
| service-status                | Ensure services state          | ['prep', 'backup-and-restore', | ['systemd', 'container',          | ['common']    |
|                               |                                | 'pre-deployment', 'pre-        | 'docker', 'podman']               |               |
|                               |                                | upgrade', 'post-deployment',   |                                   |               |
|                               |                                | 'post-upgrade']                |                                   |               |
| validate-selinux              | validate-selinux               | ['backup-and-restore', 'pre-   | ['security', 'selinux', 'audit']  | ['common']    |
|                               |                                | deployment', 'post-            |                                   |               |
|                               |                                | deployment', 'pre-upgrade',    |                                   |               |
|                               |                                | 'post-upgrade']                |                                   |               |
| my-new-validation             | Brief and general description  | ['prep', 'pre-deployment']     | ['networking', 'security', 'os',  | ['community'] |
|                               | of the validation              |                                | 'system']                         |               |
+-------------------------------+--------------------------------+--------------------------------+-----------------------------------+---------------+

To get only the list of your community validations, you can filter by products:

[pennywise@localhost]$ validation list --product community
Validation config file found: /etc/validation.cfg
+-------------------+------------------------------------------+----------------------------+------------------------------------------+---------------+
| ID                | Name                                     | Groups                     | Categories                               | Products      |
+-------------------+------------------------------------------+----------------------------+------------------------------------------+---------------+
| my-new-validation | Brief and general description of the     | ['prep', 'pre-deployment'] | ['networking', 'security', 'os',         | ['community'] |
|                   | validation                               |                            | 'system']                                |               |
+-------------------+------------------------------------------+----------------------------+------------------------------------------+---------------+

How To Develop Your New Community Validation

As you can see above, the validation init CLI sub command has generated a new Ansible role by using ansible-galaxy and a new Ansible playbook in the community-validations/ directory.

Warning

The community validations won't be supported at all. We won't be responsible as well for potential use of malignant code in their validations. Only the creation of a community validation structure through the new Validation CLI sub command will be supported.

You are now able to implement your own validation by editing the generated playbook and adding your ansible tasks in the associated role.

For people not familiar with how to write a validation, get started with this documentation.