Commit Graph

317 Commits

Author SHA1 Message Date
apetrich
68adc1fe16 Convert fstrings to .format to facilitate porting
Fstrings are a python 3.6 feature, in order to faciliate porting
to early architectures the fstrings were converted to .format

Change-Id: Ib23097c149ec8ff06cc70bb873b53ee45645ff15
2022-01-06 10:53:45 +01:00
Zuul
e875e1671b Merge "Use py3 as the default runtime for tox" 2021-11-29 19:14:09 +00:00
Zuul
3825b9c230 Merge "Mocking 'open' to improve test isolation" 2021-11-26 17:03:04 +00:00
Jiri Podivin
1eee81d906 Mocking 'open' to improve test isolation
As a result of a minor oversight, the test execution could
initiate a successful call to the 'open' and write in the storage.
This patch addresses the issue.

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: Iceb6df712b2ac0dc56aa86f465fd39cf308de2d7
2021-11-25 13:32:26 +01:00
Gael Chamoulaud (Strider)
ed33291a86
Disable spinner when running validation in non quiet mode
When using the custom callback feature through the run CLI subcommand,
by passing --extra-env-vars ANSIBLE_STDOUT_CALLBACK=default, the spinner
makes the output a bit ugly.

This patch disables the spinner in non quiet mode.

Change-Id: I9f92894fe446f4881e0234b1577e49f9e7091b7b
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
2021-11-24 15:55:16 +01:00
Zuul
1b94c03ee7 Merge "Add the community validation paths" 2021-11-18 19:39:23 +00:00
apetrich
5ba9c429d5 Add the community validation paths
The validation framework now respect and look for community
validation on the path. There config setting for
enable_community_validations is respected.

The community validations appear on validations list,
can be called by group, category, id and so on seamless
like any other validation.

Change-Id: I4a6230ac5433028c7297fa88f77728c87d60173d
Signed-off-by: Adriano Petrich (apetrich) <apetrich@redhat.com>
2021-11-18 09:55:30 +01:00
matbu
399d29059e Refactor set_argument_parser to fix shell regression
Openstack Client and Validation Shell is hitting a regression
due to set_argument_parser calling directly parse_known_args
from argparse.
This patch fix the regression and avoid the need to call
parse_known_args directly.

Closes-Bug: #1949596
Change-Id: Ic9a682e7c3d5431e8779064947bd379c00aba584
2021-11-17 00:02:58 +00:00
Gael Chamoulaud (Strider)
53b573230a
Enforce existing roles and playbooks checks
This patch enforces the roles and playbooks checks when creating a new
community validation by checking first if the ANSIBLE_ROLES_DIR and
ANSIBLE_VALIDATION_DIR are existing in the host.

It avoids getting error message like:
  - [Errno 2] No such file or directory: '/usr/share/ansible/roles'
when validations-common and tripleo-validations are not installed.

It also adjusts the unittests in order to avoid leaving leftovers such
as community-validations directory during the test execution.

Change-Id: I6facca59c2ca9db2b5b1e7905ad8c9be68fff0b4
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
2021-11-16 15:22:34 +01:00
Gael Chamoulaud (Strider)
1bbf282356
Add new CLI sub command to create community validations
Presently, the operator(s) can only execute the official and supported
validations coming from tripleo-validations and validations-common.
Community validations enable a sysadmin to create and execute
validations unique to their environment.

This patch introduces the new Command Line Interface sub command to
create a new community validation skeleton. First, this latter will
check if there is an existing role or a playbook either in the community
validations catalog or the official validations catalog.  And it will
create an Ansible role (with ansible-galaxy[1]) and a playbook in the
~/community-validations directory.

By default, the community validations feature is enabled but may be
disabled by setting [DEFAULT].enable_community_validations to ``False``
in the validation configuration file.

Example:

[stack@localhost]$ validation init my-new-validation
Validation config file found: /etc/validation.cfg
New role created successfully in /home/stack/community-validations/roles/my_new_validation
New playbook created successfully in /home/stack/community-validations/playbooks/my-new-validation.yaml

For a full demo of this new CLI sub command, please take a look at this
asciinema[2].

[1] - https://docs.ansible.com/ansible/latest/cli/ansible-galaxy.html
[2] - https://asciinema.org/a/445105

Change-Id: I8fb16e3456696187d4a9d3820740a7639a96e315
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
2021-10-29 15:39:39 +02:00
zhangboye
d61126be56
Use py3 as the default runtime for tox
Moving on py3 as the default runtime for tox to avoid to update this at each new cycle.

Change-Id: I356ba7cc702843fe702b3a50b48539d4887bacf4
2021-10-15 08:46:33 +02:00
Jiri Podivin
7d416acbe8 Default value of the constructor parameter
The default value of the validation_path parameter
is now handled properly, without unnecessary conditional.

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: I4cc16732ea9d98ec47c2dc616105944bdd6106f0
2021-10-08 14:18:19 +02:00
matbu
b37015ee1f Expose skip list mechanism via the CLI
The skip list feature was available only via the API, this patch
allow the user to pass a file with the list of the validations
which he wants to skip during the run, example:

check-ram: hosts: all
  reason: Wrong ram value
  lp: https://lp.fake.net
check-cpu: hosts: undercloud
  reason: Unstable validation
  lp: https://lp.fake.net
Change-Id: I04059f9339085e0dcef1f018cad1be511ee7d3c7
2021-10-06 13:45:08 +00:00
matbu
11488cd88d Add validation config file mechanism
Introduce validation config file.
The config default location of the config file will be stored
in /etc/validation.cfg

The variables precedence will be the following:
* user's cli args
* config file
* default interval values

Change-Id: I05c54a43bc0a03878793cca3f51e23f4a8b63a23
2021-08-20 08:12:16 +00:00
Jiri Podivin
c8480cbf80 Setting language for the python3.6 tox environment
By ensuring the locale is set to US utf-8 we can avoid
encoding errors.

Closes-Bug: #1940313

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: I585a433e7480c22d52f402c1661194968ec66307
2021-08-19 08:36:32 +00:00
Jiri Podivin
f1cd8eebc1 Validations show group filtering
The 'ValidationActions.group_information' method was refactored
to reflect new signature of the 'utils.parse_all_validations_on_disk' function.

Tests were adjusted accordingly.

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: Ic2ca556441169594792d7d5835ea057e9256a7a9
2021-08-09 16:06:12 +02:00
Jiri Podivin
07e484a861 Triggering the 'tripleo-ci-centos-8-undercloud-containers'
This job initiates checks defined in the 'tripleoclient.undercloud_preflight'
module, which use the 'validations_libs.validation_actions.ValidationActions' class.

Any issues, arising in the integration of the aformentioned modules,
should be detected during the job run.

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: I947f0bcc2d4a6152de41b1592dc51a3757f0a8af
2021-07-30 13:40:58 +02:00
Zuul
442b7071c9 Merge "reqcheck job is now voting" 2021-07-29 13:49:12 +00:00
Jiri Podivin
07558871d6 Unit test refactor
- redundant imports in test modules were removed
- reset color code management in test simplified

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: Iac5e68286782c5c2fbc185650c0bdf915fe959ef
2021-07-28 12:39:24 +02:00
Gael Chamoulaud (Strider)
e4b5dc3ea7 Add CLI auto generated documentation using Cliff
This patch adds the Cliff Sphinx Extension support for auto documenting
the `validation` Command Line Interface. It also standardizes the
`metavar` description for all the --group, --category and --product
arguments by using <group_id>, <category_id> and <product_id>.

Moreover, it removes usage example(s) in the arguments helpers which are often
duplicated with the `metavar`.

Change-Id: I3009337e8afadd705bbee5b89d9022c78093cdc8
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
2021-07-27 07:01:19 +00:00
Gael Chamoulaud (Strider)
c46c90394c Add Products metadata key management
This patch adds the management of the new `products` metadata key in the
validation playbooks. We can now filter the validations by their groups,
by their categories or by their products while listing or running them.

The `list` sub command has now a new --product argument. When filtering
by groups, by categories or by products (see the example below), the
`list` sub command will return all the validation playbooks belonging to
the 'prep' group OR all the validation playbooks belonging to the 'os'
and/or 'system' categories OR all the validation playbooks to the
'tripleo' product:

$ validation list -h
$ validation list --group prep --category os,system --product tripleo

The `run` sub command has also its new --product argument. Note that
this new argument is mutually exclusive with the --validation, --group
and --category arguments:

$ validation run -h
$ validation run --product tripleo --inventory /etc/ansible/hosts

The `show parameter` sub command has the same new argument which is also
mutually exclusive with the --validation, --group and the --category
arguments:

$ validation show parameter -h
$ validation show parameter --product tripleo

Change-Id: Ief13e506c2bee18da47b31f1c2d5c0dbb1ad1ecf
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
2021-07-27 07:01:05 +00:00
Gael Chamoulaud (Strider)
3928305329 Add Categories metadata key management
This patch adds the management of the new `categories` metadata key in
the validation playbooks. We can now filter the validations by their
groups and/or by their categories while listing or running them.

The `list` sub command has now a new --category argument. When filtering
by groups and by categories (see the example below), the `list` sub
command will return all the validation playbooks belonging to the prep
group OR all the validation playbooks belonging to the os and/or system
categories:

$ validation list -h
$ validation list --group prep --category os,system

The `run` sub command has also its new --category argument. Note that
this new argument is mutually exclusive with the --validation and
--group arguments:

$ validation run -h
$ validation run --category networking --inventory /etc/ansible/hosts

The `show parameter` sub command has the same new argument which is also
mutually exclusive with the --validation and --group arguments:

$ validation show parameter -h
$ validation show parameter --category os,system,hardware,ram

Change-Id: I4297f83355bdd209d21518fbadb17d1343fd4680
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
2021-07-27 07:00:47 +00:00
Zuul
ff7f8727d2 Merge "Add missing font for PDF generation" 2021-07-27 03:27:05 +00:00
Takashi Kajinami
8c7651e8a4 Add missing font for PDF generation
openstack-tox-docs job started to fail with the following error:

! LaTeX Error: File `tgtermes.sty' not found.

This patch adds the required font package to bindep doc profile.

Co-authored-by: Elod Illes <elod.illes@est.tech>
Closes-Bug: #1938044
Change-Id: I0758890b65212b064118d7f94c04825e4a04a394
2021-07-26 23:08:29 +09:00
Ronelle Landy
6f4a203f51 Build containers in gate - to match check
tripleo-ci-centos-8-standalone-validation-libs
should have build_container_images: true
in gate if that setting is run in check.

This review adds a vars anchor to match
vars in check and gate.

Change-Id: Iec14a28a2a4d0000c7b4c573760eaeb603004cbd
2021-07-23 14:58:58 -04:00
Zuul
9a594f70c9 Merge "Remove convert_data function" 2021-07-16 03:27:39 +00:00
Zuul
205d4ea9d0 Merge "fix var name in logging statement" 2021-07-14 22:33:01 +00:00
Zuul
bb15e057a7 Merge "Fix history list sub command when not using the default log dir" 2021-07-14 05:35:35 +00:00
Zuul
d1899b71ba Merge "add spinner to highlight progress" 2021-07-14 00:02:06 +00:00
Gael Chamoulaud (Strider)
2f3b09d41e
Remove convert_data function
This patch removes the convert_data function which has been introduced
with the first one and deprecated Command Line Interface. The
--validation and --group arguments were sending comma-separated strings
instead of list. The new Command Line Interface is now enforcing this by
using its own CommaList Parser Action and only accepting List of groups
or validation names.

This function is now useless and can be removed safely. Tests and usages
have been modified accordingly.

Change-Id: Ief9459e914ea73fbd59a85c07a55c26078e0126a
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
2021-07-13 22:14:19 +02:00
Zuul
1c4c819301 Merge "improve vagrant functionality" 2021-07-13 18:11:52 +00:00
David J Peacock
e11b458946 add spinner to highlight progress
Implementing a Spinner class and utilizing it around a run action.  This
will add a moving icon that animates in place until output is reached,
letting the operator know that things are happening.

The interactive TTY test is included to ensure that during some unknown
usage such as logging to disk this feature doesn't spew unnecessary
clutter.

Change-Id: Ieef256a5c12b238008e9250c0ee182d80a2b6dfb
2021-07-13 11:14:41 -04:00
David J Peacock
753e7f0ed7 improve vagrant functionality
this patch improves vagrant functionality by taking advantage of the
non-root shell provisioner; namely via the privileged: false flag

previously the vagrant dev environments were lacking utility in out of
the box experience

Change-Id: Iaabe918c7eee5ac18d1ffd393f59fda568088eb5
2021-07-13 09:34:05 -04:00
Jiri Podivin
b2a9d71361 Test cross contamination prevention
The existing tests were using shared fakes as return value
of certain mocks. Unfortunately, while perfectly fine in most cases,
certain methods change values passed in place.
This created a race condition, which caused fluctuation in the reported
coverage of the relevant modules.

The undesirable test behavior only becomes apparent when the tests
are run in a succession and their behavior is analyzed, for example
during the run of the recently implemented 'coverchange' job.

Final statistics of the coverage measurement, displaying the totals
for number of lines, lines without coverage, branches etc., display
following pattern. Note the third column.

TOTAL                        995    105    340     56    86%
TOTAL                        995    105    340     56    86%
TOTAL                        995    105    340     56    86%
TOTAL                        995    108    340     56    86%
TOTAL                        995    108    340     56    86%
TOTAL                        995    105    340     56    86%
TOTAL                        995    105    340     56    86%
TOTAL                        995    108    340     56    86%
TOTAL                        995    105    340     56    86%

Number of lines without coverage oscillates between '108' and '105' depending
on the order in which the tests were run. The culprit being one of the functions
of the 'validations_libs.cli.common' module, which edits data passed in place.

As the data processed are passed from patched object, and the data itself are
defined withing the relevant 'fakes' module, it emerges with inevitability,
that the tested code operates on data previously altered by the tests.

Furthermore the 'test_run_command_failed_validation' test was augmented
to place further assertion on the arguments passed.

Closes-Bug: #1935003

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: I2210fa1775691dd503eb882a24824eaf18d7bd3e
2021-07-08 11:04:10 +02:00
Jiri Podivin
27a20e59be reqcheck job is now voting
After several weeks being a nonvoting job, the reqcheck is moving to voting status.

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: Ie54a0d256529277a69a72334e45040070ec32dc5
2021-07-07 12:46:56 +02:00
Jiri Podivin
8a87da12bf fix var name in logging statement
The variable referenced in the logging statement
of the job was not the one on which the assertion was made.

As a result the message was confusing.

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: Ia7b9951d5c47c13f344fc1303604018f2420c457
2021-07-07 07:51:37 +02:00
Gael Chamoulaud (Strider)
bf95674c5f
Fix history list sub command when not using the default log dir
The 'history list' sub command is always getting the log files from the
default directory ('~/validations') even if the user decided to write
them in one other directory.

$ validation run --validation 512e \
             --validation-log-dir /var/log/validations  \
             --inventory /etc/ansible/hosts

$ validation -vvv --debug history list \
             --validation-log-dir /var/log/validations
...
FileNotFoundError: [Errno 2] No such file or directory: '/home/validations/validations'

This patch fixes this wrong behavior by passing the
--validation-log-dir argument value to the show_history actions.

Moreover, the ValidationActions class is taken validation_path and group
as parameters. Passing --validation-log-dir argument value is wrong and useless.

Change-Id: I9962d449ee507ee64d0c884199d02cd9ce786c9b
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
2021-07-06 15:18:18 +02:00
Zuul
85947ee8e0 Merge "Simplification of the log path handling by the ValidationLog class" 2021-07-05 19:28:21 +00:00
Zuul
9248716026 Merge "Coverage change job" 2021-07-02 21:07:09 +00:00
Zuul
ab14603a9a Merge "Handle backward compatibility for Cliff 2.16.0 (stable/train)" 2021-07-01 17:09:16 +00:00
Jiri Podivin
583fad78db Simplification of the log path handling by the ValidationLog class
Redundant checks were removed an branching reduced.
Previously de-facto function _get_content now behaves as
a method should and works with attributes.

Several string processsing snippets were factored out
and are now separate helper methods.

Added check for abs path.

Changed related test to provide absolute path.
Added new test case for absolute path checking.

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: I46df7b4befbef36a74b9138961650806c457fc13
2021-07-01 15:02:41 +00:00
Zuul
e70dbce0cc Merge "Remove content-provider to decrease CI time" 2021-07-01 14:39:56 +00:00
matbu
357697d0fb Handle backward compatibility for Cliff 2.16.0 (stable/train)
In stable/train Cliff version is 2.16.0 which force us
to pin validations-libs or to handle backward compat directly
in the code.
For long term solution and to avoid tagging, pinning and complex
release management this review fix the compatibility with Cliff 2.16.0

Change-Id: I6dec4c154f088e8cc6450d5d9cf2f9b407fdefdc
2021-07-01 07:47:22 +00:00
Jiri Podivin
56bb1a0680 Coverage change job
Zuul check for changes in unit test coverage.
With this running we can ensure that no more code is
merged without tests.

Checks both per file, and total coverage.

At this stage, I'm proposing is a non-voting job.
After a grace period, for example one sprint or two, it would become voting.

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: If2e61c9bbf59e91163357023d16620934dc7d3a0
2021-06-30 13:54:57 +02:00
matbu
334c8f3ff3 Remove content-provider to decrease CI time
Since we only use 1 TripleO standalone job, we dont need
to run the content provided job, but only build containers
in the standalone job.

Removing the content-provider job will decrease significatively
the duration of the zuul runs

Change-Id: Ie13c85f8d80e17077d1a5b591a171248ab0eed28
2021-06-30 10:38:38 +00:00
Zuul
8645aea875 Merge "Fix a regression in the ShowGroup sub-command" 2021-06-29 22:29:41 +00:00
Zuul
26bea93d28 Merge "Filling license info for Ansible and Pyflakes" 2021-06-23 15:10:37 +00:00
Jiri Podivin
9db02ebefd Fix a regression in the ShowGroup sub-command
The ShowGroup sub-command was not working anymore due
to a regression introduced by the new CLI next generation
commit[1].

This patch fixes the ShowGroup sub-command by addressing the
following points:
  * The ValidationActions.group_information method should take
    the absolute path of the groups.yaml file by default.
  * Using cliff.lister.Lister base class instead of
    cliff.show.ShowOne base class for the ShowGroup class.

     Resolves: rhbz#1972155
[1] - 5fad1a4d18

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: Ia5aa3e96e8b5a6634a709ce8677f90108c80dfd3
2021-06-22 14:35:33 +02:00
Zuul
d6ecf241d1 Merge "Fix VALIDATIONS_LOG_BASEDIR constant when $HOME is undefined" 2021-06-18 17:33:30 +00:00
Zuul
b575dfcf95 Merge "Module docstring for validation_libs.constants" 2021-06-18 16:32:36 +00:00