Introducing the oslo_config_validator role

This role is intended to leverage the `oslo-config-validator` on each one
of the configuration files found on a deployment. The goal is to quickly
catch erroneous configurations.

When called manually, it will also be possible to generate a report
returning all the differences between the current configuration and the
default configuration

One last thing, it will also validate the values of some specific
settings, based on the invalid_settings dict in the configuration.
Because of that, we moved some checks in nova_event_callback to the
oslo_config_validator role.

NOTE(dvd): Removing backup-and-restore group from this branch as it's
not available prior to Wallaby.

Depends-On: https://review.opendev.org/c/openstack/oslo.config/+/794816
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1940815
Resolves: rhbz#1940815
Change-Id: Id047fe378cf512b985c9d7478a991f11b280102b
(cherry picked from commit 4aaca32f19)
(cherry picked from commit fab6429d98)
(cherry picked from commit 727625fe95)
This commit is contained in:
David Vallee Delisle 2021-06-04 04:24:24 +00:00
parent d88808d063
commit 92c55031e6
19 changed files with 2450 additions and 0 deletions

View File

@ -0,0 +1,93 @@
=====================
oslo_config_validator
=====================
--------------
About the role
--------------
An Ansible role that will loop through all the containers on selected host, find the Openstack service configuration file
and leverage the [oslo-config-validator](https://docs.openstack.org/oslo.config/latest/cli/validator.html) utility to validate the current running configuration.
It's also possible to generate a report that contains all differences between the sample or default values with current running configuration.
Finally, it will also verify that the current running configuration doesn't contain any known invalid settings that might have been deprecated and removed in previous versions.
Exceptions
==========
Some services like ``cinder`` can have dynamic configuration sections. In ``cinder``'s case, this is for the storage backends. To perform validation on these dynamic sections, we need to generate a yaml formatted config sample with ``oslo-config-generator`` beforehand, append a new sample configuration for each storage backends, and validate against that newly generated configuration file by passing ``--opt-data`` to the ``oslo-config-validator`` command instead of using ``--namespaces``. Since generating a sample config adds some delay to the validation, this is not the default way of validating, we prefer to validate directly using ``--namespaces``.
NOTE: At the time of writing this role, ``oslo-config-generator`` has a bug [1] when generating yaml config files, most notably with ``cinder``. Since the inclusion of oslo.config patch can't be garanteed, the role will inject this patch [2] to the oslo.config code, inside the validation container. This code change is ephemeral for the time of the configuration file generation. The reason why we want to inject this patch is because it's possible that we run the validation on containers that were created before it was merged. This ensures a smooth validation across the board.
[1] https://bugs.launchpad.net/oslo.config/+bug/1928582
[2] https://review.opendev.org/c/openstack/oslo.config/+/790883
Requirements
============
This role needs to be run on an Undercloud with a deployed Overcloud.
Role Variables
==============
- oslo_config_validator_validation: Wether or not to run assertions on produced outputs. That also means that the role will fail if anything is output post-filtering. If this is enabled with the reporting, this will most likely trigger a failure unless executed against default configuration
- oslo_config_validator_report: Wether or not we compare the configuration files found with the default config
- oslo_config_validator_invalid_settings: When running validation, wether or not we should check for invalid settings. This adds to the time it takes to complete validation because of the way the validations_read_ini module works. This won't work without ``oslo_config_validator_validation`` enabled.
- oslo_config_validator_report_path: The folder used when generating the reports.
- oslo_config_validator_global_ignored_messages: List of regular expressions that will filter out messages globally, across all namespaces
- oslo_config_validator_namespaces_config: Specific namespace configurations. It contains namespace-specific ignored patterns as well as invalid settings configuration.
- oslo_config_validator_service_configs: Mapping of known Openstack services with their namespace configuration.
- oslo_config_validator_checked_services: List of services being validated.
Dependencies
============
- podman_container
- podman_container_info
- validations_read_ini
- https://review.opendev.org/c/openstack/oslo.config/+/790883
Example Reporting Playbook
==========================
.. code-block:: yaml
- hosts: all
vars:
- oslo_config_validator_report: true
- oslo_config_validator_validation: false
roles:
- { role: oslo_config_validator}
Example playbook to validate only one service
=============================================
.. code-block:: yaml
- hosts: all
vars:
- oslo_config_validator_checked_services:
- nova
roles:
- { role: oslo_config_validator}
License
=======
Apache
Author Information
==================
**Red Hat TripleO DFG:Compute Deployment Squad**
----------------
Full Description
----------------
.. ansibleautoplugin::
:role: roles/oslo_config_validator

View File

@ -0,0 +1,60 @@
---
- hosts: all
gather_facts: false
vars:
metadata:
name: Openstack services configuration validation
description: |
This role is intended to leverage the `oslo-config-validator` on each one
of the configuration files found on a deployment. The goal is to quickly
catch erroneous configurations.
When called manually, it will also be possible to generate a report
returning all the differences between the current configuration and the
default configuration
groups:
- post-deployment
- post-system-upgrade
- post-update
- post-upgrade
# Debug log level
oslo_config_validator_debug: false
# Comparison report with current settings and default settings
oslo_config_validator_report: false
# Returns all settings with possibly invalid values or simply inexistent
# oslo.config uses a typed system for possible values of each settings.
# if a setting is not typed correctly, or is non-existent, oslo-config-validator
# will trigger an error here.
oslo_config_validator_validation: true
# Returns all settings erroneous values
# Developpers have identified some settings that can't have certain values anymore.
# This will scan all the config files and return those settings with invalid values.
oslo_config_validator_invalid_settings: true
# This is a temporary folder used when generating reports.
# It will be created and deleted if necessary on all the nodes
oslo_config_validator_report_path: "/var/tmp/config_validator_report"
# Whether or not we should archive into a single file the report after creation
oslo_config_validator_report_archive: true
# This is the working folder when running validations. It will be bind mounted
# on the validation containers
# It will be created and deleted if necessary on all the nodes
oslo_config_validator_work_path: "/var/lib/tripleo-config/oslo_config_validator"
# List of services we want to validate the settings
oslo_config_validator_checked_services:
- nova
- cinder
- glance
- heat
- ironic
- placement
- neutron
- keystone
roles:
- oslo_config_validator

View File

@ -0,0 +1,387 @@
---
# Copyright 2020 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# All variables intended for modification should place placed in this file.
# All variables within this role should have a prefix of "oslo_config_validator"
oslo_config_validator_debug: false
# Comparison report with current settings and default settings
oslo_config_validator_report: false
# Returns all settings with possibly invalid values or simply inexistent
# oslo.config uses a typed system for possible values of each settings.
# if a setting is not typed correctly, or is non-existent, oslo-config-validator
# will trigger an error here.
oslo_config_validator_validation: true
# This is a temporary folder used when generating reports.
# It will be created and deleted if necessary on all the nodes
oslo_config_validator_report_path: "/var/tmp/config_validator_report"
# Whether or not we should archive into a single file the report after creation
oslo_config_validator_report_archive: true
# This is the working folder when running validations. It will be bind mounted
# on the validation containers
# It will be created and deleted if necessary on all the nodes
oslo_config_validator_work_path: "/var/lib/tripleo-config/oslo_config_validator"
# When running validation, whether or not we should check for invalid settings
# This adds to the time it takes to complete validation because of the way
# the validations_read_ini module works.
oslo_config_validator_invalid_settings: true
# These messages are globally ignored and will not trigger a validation failure
oslo_config_validator_global_ignored_messages:
- 'INFO:keyring.backend:Loading.*'
- 'WARNING:oslo_config.generator:normalizing group name.*'
- 'WARNING:stevedore.named:Could not load .*'
- 'WARNING:root:Deprecated opt .*'
- 'ERROR:root:quotas/quota_[^\s]+ {{ invalid_setting_regex }}'
- '.*Ignoring option because it is part of the excluded patterns. This can be changed with the.*'
# namespace configuration:
# We can define configuration for each namespaces.
# ignored_messages: list of regexes that, when returned from calling a validation using that
# specific namespace, will be ignore
# invalid_settings: list of settings that will trigger a failure if they are set to specific
# values. Separator is meant to split the setting value and checking each
# one of the values individually like nova filters. This is useful to
# validate if deprecated or removed option values are still in use.
# keys:
# - section: configuration section (ie: DEFAULT)
# - option: setting name (ie: debug)
# - separator: string delimiter that will be used to convert value to a list
# - value_list: List of strings that would be checked against.
# - operator: Can be either these values, default being "eq":
# - not: current value should not match exactly any element in value_list
# - lt: current value should be lesser than first element of value_list
# - gt: current value should be greater than first element of value_list
# - eq: current value, if defined, should be equal to one element in value_list
#
oslo_config_validator_namespaces_config:
- namespace: glance.store
ignored_messages:
- "ERROR:stevedore.extension:Could not load '(glance.store.)*s3(.Store)*': No module named 'boto3'"
# NOTE(dvd): openstack/puppet-glance: https://review.opendev.org/775633
# This setting was removed from puppet registry in Wallaby
# but it was removed from Glance a while ago.
- 'ERROR:root:DEFAULT/enable_v1_api {{ invalid_setting_regex }}'
- namespace: heat.common.config
ignored_messages:
# NOTE(dvd): openstack/heat fix: https://review.opendev.org/789680
# heat wasn't including its yaql and cache options by default. This is being worked
# on during the Xena cycle and should hopefully be backported down to train.
- 'ERROR:root:(yaql|(resource_finder_)*cache)/[^\s]+ {{ invalid_setting_regex }}'
- namespace: ironic
ignored_messages:
# NOTE(dvd): openstack/puppet-ironic: https://review.opendev.org/789592
# This setting was removed a while ago but puppet registry was updated only only recently
- 'ERROR:root:pxe/ipxe_enabled {{ invalid_setting_regex }}'
# NOTE(dvd): openstack/puppet-ironic: https://review.opendev.org/790526
# This setting isn't used in ironic since 2013, it was removed in Xena cycle.
- 'ERROR:root:conductor/max_time_interval {{ invalid_setting_regex }}'
- namespace: keystonemiddleware.auth_token
ignored_messages:
- >-
.*Ignoring missing option "(auth_url|username|password|(user|project)_(domain_)*name)" from group "keystone_authtoken" because the group is known to
have incomplete sample config data and thus cannot be validated properly.*
- namespace: neutron
invalid_settings:
- section: nova
option: tenant_name
operator: eq
value_list:
- service
- section: nova
option: project_name
operator: eq
value_list:
- service
- section: DEFAULT
option: notify_nova_on_port_data_changes
value_list:
- "True"
- section: DEFAULT
option: notify_nova_on_port_status_changes
value_list:
- "True"
ignored_messages:
# NOTE(dvd): openstack/neutron: https://review.opendev.org/789648
- 'ERROR:root:placement/[^\s]+ {{ invalid_setting_regex }}'
- namespace: cinder
invalid_settings:
- section: DEFAULT
option: enable_v3_api
value_list:
- "True"
ignored_messages:
- 'ERROR:root:DEFAULT/api_paste_config {{ invalid_setting_regex }}'
- namespace: nova.conf
invalid_settings:
- section: filter_scheduler
option: enabled_filters
separator: ","
value_list:
- ExactCoreFilter
- ExactRamFilter
- ExactDiskFilter
- CoreFilter
- RamFilter
- DiskFilter
- section: DEFAULT
option: vif_plugging_timeout
operator: lt
value_list:
- 300
- section: DEFAULT
option: vif_plugging_timeout
value_list:
- "True"
ignored_messages:
# NOTE(dvd): openstack/puppet-nova: https://review.opendev.org/789633
# This was erroneously categorized in the DEFAULT section but it should have been
# under the vif_plug_ovs section.
- 'ERROR:root:DEFAULT/ovsdb_connection {{ invalid_setting_regex }}'
# NOTE(dvd): openstack/os-vif: https://review.opendev.org/789645
# os_vif had no list_opts entrypoint before Xena.
- 'ERROR:root:vif_plug_ovs/ovsdb_connection {{ invalid_setting_regex }}'
# These settings are used by openstacksdk and not part of oslo_config_opts. They are not taken
# into account by oslo-config-(generator|validator)
- 'ERROR:root:(cinder|service_user)/region_name {{ invalid_setting_regex }}'
# Censoring password
- 'WARNING:root:neutron/metadata_proxy_shared_secret sample value is empty but input-file has'
# TODO(dvd): Needs to be investigated with TLSe
- 'ERROR:root:cache/tls_enabled not found'
# service configuration:
# Configuration for each openstack services is stored here
# config_files: List of config files with their specific namespaces
# default_namespaces: List of namespaces that should be checked against each files
# ignored_groups: List of groups that shouldn't be checked. This is passed as --exclude-group
# to the oslo-config-validator command.
# opt_data: Some sections are dynamically generated like cinder's backend sections. This will
# generate a custom opt_data using the content of template_sections as options for
# the list of items in index_key's values. This adds a lot of overhead to the parsing
# because we need to spawn an oslo-config-generator to pull out the default yaml
# config and build a new data structure from it.
oslo_config_validator_service_configs:
# https://opendev.org/openstack/nova/src/branch/master/etc/nova/nova-config-generator.conf
nova:
config_files:
- path: /etc/nova/nova.conf
namespaces: []
default_namespaces:
- nova.conf
- keystonemiddleware.auth_token
- oslo.log
- oslo.messaging
- oslo.policy
- oslo.privsep
- oslo.service.periodic_task
- oslo.service.service
- oslo.db
- oslo.db.concurrency
- oslo.cache
- oslo.middleware
- oslo.concurrency
- osprofiler
# https://opendev.org/openstack/cinder/src/branch/master/tools/config/cinder-config-generator.conf
cinder:
config_files:
- path: /etc/cinder/cinder.conf
namespaces: []
ignored_groups:
- nova
- service_user
opt_data:
- index_key:
section: DEFAULT
option: enabled_backends
separator: ","
template_section:
- backend_defaults
- backend
default_namespaces:
- cinder
- keystonemiddleware.auth_token
- oslo.log
- oslo.messaging
- oslo.policy
- oslo.privsep
- oslo.service.periodic_task
- oslo.service.service
- oslo.db
- oslo.db.concurrency
- oslo.middleware
- oslo.concurrency
- osprofiler
# Glance has multiple files
# https://opendev.org/openstack/glance/src/branch/master/etc/oslo-config-generator
glance:
ignored_groups:
- ref1
- default_backend
config_files:
- path: /etc/glance/glance-api.conf
namespaces: []
- path: /etc/glance/glance-cache.conf
namespaces: []
- path: /etc/glance/glance-image-import.conf
namespaces: []
- path: /etc/glance/glance-registry.conf
namespaces: []
- path: /etc/glance/glance-scrubber.conf
namespaces: []
- path: /etc/glance/glance-swift.conf
namespaces: []
default_namespaces:
- glance
- glance.api
- glance.store
- glance.multi_store
- keystonemiddleware.auth_token
- oslo.log
- oslo.messaging
- oslo.policy
- oslo.privsep
- oslo.service.periodic_task
- oslo.service.service
- oslo.db
- oslo.db.concurrency
- oslo.middleware.cors
- oslo.middleware.http_proxy_to_wsgi
# https://opendev.org/openstack/heat/src/branch/master/config-generator.conf
heat:
config_files:
- path: /etc/heat/heat.conf
namespaces: []
default_namespaces:
- heat.common.config
- heat.common.context
- heat.common.crypt
- heat.engine.clients.os.keystone.heat_keystoneclient
- heat.common.wsgi
- heat.engine.clients
- heat.engine.notification
- heat.engine.resources
- heat.api.aws.ec2token
- keystonemiddleware.auth_token
- oslo.messaging
- oslo.middleware
- oslo.db
- oslo.log
- oslo.policy
- oslo.service.service
- oslo.service.periodic_task
- oslo.service.sslutils
# https://opendev.org/openstack/ironic/src/branch/master/tools/config/ironic-config-generator.conf
ironic:
config_files:
- path: /etc/ironic/ironic.conf
namespaces: []
- path: /etc/ironic-inspector/inspector.conf
namespaces: []
default_namespaces:
- ironic
- ironic_lib.disk_utils
- ironic_lib.disk_partitioner
- ironic_lib.exception
- ironic_lib.mdns
- ironic_lib.metrics
- ironic_lib.metrics_statsd
- ironic_lib.utils
- oslo.db
- oslo.messaging
- oslo.middleware.cors
- oslo.middleware.healthcheck
- oslo.middleware.http_proxy_to_wsgi
- oslo.concurrency
- oslo.policy
- oslo.log
- oslo.reports
- oslo.service.service
- oslo.service.periodic_task
- oslo.service.sslutils
- osprofiler
- keystonemiddleware.auth_token
# https://opendev.org/openstack/placement/src/branch/master/etc/placement/config-generator.conf
placement:
config_files:
- path: /etc/placement/placement.conf
namespaces: []
default_namespaces:
- placement.conf
- keystonemiddleware.auth_token
- oslo.log
- oslo.middleware.cors
- oslo.policy
# https://opendev.org/openstack/neutron/src/branch/master/etc/oslo-config-generator/neutron.conf
neutron:
config_files:
- path: /etc/neutron/neutron.conf
namespaces: []
- path: /etc/neutron/plugins/ml2/ml2_conf.ini
namespaces: []
default_namespaces:
- neutron
- neutron.agent
- neutron.base.agent
- neutron.db
- neutron.extensions
- nova.auth
- ironic.auth
- placement.auth
- oslo.log
- oslo.db
- oslo.policy
- oslo.privsep
- oslo.concurrency
- oslo.messaging
- oslo.middleware.cors
- oslo.middleware.http_proxy_to_wsgi
- oslo.service.sslutils
- oslo.service.wsgi
- keystonemiddleware.auth_token
# https://opendev.org/openstack/keystone/src/branch/master/config-generator/keystone.conf
keystone:
config_files:
- path: /etc/keystone/keystone.conf
namespaces: []
default_namespaces:
- keystone
- oslo.cache
- oslo.log
- oslo.messaging
- oslo.policy
- oslo.db
- oslo.middleware
- oslo.service.sslutils
- osprofiler
# Default value for the list of services to check. Default is we check all the services
oslo_config_validator_checked_services: "{{ oslo_config_validator_service_configs.keys() | list }}"

View File

@ -0,0 +1,35 @@
---
# inherits tripleo-validations/.config/molecule/config.yml
# To override default values, please take a look at the config.yml.
provisioner:
name: ansible
playbooks:
prepare: ../../resources/playbooks/prepare.yml
converge: ../../resources/playbooks/converge.yml
inventory:
hosts:
all:
hosts:
ubi8:
ansible_python_interpreter: /usr/bin/python3
log: true
env:
ANSIBLE_STDOUT_CALLBACK: yaml
ANSIBLE_LIBRARY: "${ANSIBLE_LIBRARY:-/usr/share/ansible/plugins/modules}:../../resources/library"
MOLECULE_OCV:
service_name: nova_compute
config_file: etc/nova/nova.conf
config_folder: /var/lib/config-data/puppet-generated/nova_libvirt
# yamllint disable rule:line-length
validator_out: |
ERROR:root:DEFAULT/ovsdb_connection not found
ERROR:root:cinder/region_name not found
INFO:root:Ignoring missing option "auth_url" from group "keystone_authtoken" because the group is known to have incomplete sample config data and thus cannot be validated properly.
INFO:root:Ignoring missing option "username" from group "keystone_authtoken" because the group is known to have incomplete sample config data and thus cannot be validated properly.
INFO:root:Ignoring missing option "password" from group "keystone_authtoken" because the group is known to have incomplete sample config data and thus cannot be validated properly.
INFO:root:Ignoring missing option "user_domain_name" from group "keystone_authtoken" because the group is known to have incomplete sample config data and thus cannot be validated properly.
INFO:root:Ignoring missing option "project_name" from group "keystone_authtoken" because the group is known to have incomplete sample config data and thus cannot be validated properly.
INFO:root:Ignoring missing option "project_domain_name" from group "keystone_authtoken" because the group is known to have incomplete sample config data and thus cannot be validated properly.
ERROR:root:service_user/region_name not found
# yamllint enable rule:line-length

View File

@ -0,0 +1,25 @@
---
# inherits tripleo-validations/.config/molecule/config.yml
# To override default values, please take a look at the config.yml.
provisioner:
name: ansible
playbooks:
prepare: ../../resources/playbooks/prepare.yml
converge: ../../resources/playbooks/converge.yml
inventory:
hosts:
all:
hosts:
ubi8:
ansible_python_interpreter: /usr/bin/python3
log: true
env:
ANSIBLE_STDOUT_CALLBACK: yaml
ANSIBLE_LIBRARY: "${ANSIBLE_LIBRARY:-/usr/share/ansible/plugins/modules}:../../resources/library"
MOLECULE_OCV:
service_name: nova_compute
config_file: etc/nova/nova.conf
config_folder: /var/lib/config-data/puppet-generated/nova_libvirt
validator_out: |
ERROR:root:Houston we've got a problem

View File

@ -0,0 +1,903 @@
#!/usr/bin/python
# Copyright (c) 2019 OpenStack Foundation
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# flake8: noqa: E501
from __future__ import absolute_import, division, print_function
from yaml import safe_load as yaml_safe_load
__metaclass__ = type
DOCUMENTATION = r"""
module: mocked podman_container
author:
- "David Vallee Delisle (@dvd)"
version_added: '1.0.0'
short_description: mocked Manage podman containers
notes: []
description:
- mocked Start, stop, restart and manage Podman containers
requirements:
- podman
options:
name:
description:
- Name of the container
required: True
type: str
executable:
description:
- Path to C(podman) executable if it is not in the C($PATH) on the
machine running C(podman)
default: 'podman'
type: str
state:
description:
- I(absent) - A container matching the specified name will be stopped and
removed.
- I(present) - Asserts the existence of a container matching the name and
any provided configuration parameters. If no container matches the
name, a container will be created. If a container matches the name but
the provided configuration does not match, the container will be
updated, if it can be. If it cannot be updated, it will be removed and
re-created with the requested config. Image version will be taken into
account when comparing configuration. Use the recreate option to force
the re-creation of the matching container.
- I(started) - Asserts there is a running container matching the name and
any provided configuration. If no container matches the name, a
container will be created and started. Use recreate to always re-create
a matching container, even if it is running. Use force_restart to force
a matching container to be stopped and restarted.
- I(stopped) - Asserts that the container is first I(present), and then
if the container is running moves it to a stopped state.
- I(created) - Asserts that the container exists with given configuration.
If container doesn't exist, the module creates it and leaves it in
'created' state. If configuration doesn't match or 'recreate' option is
set, the container will be recreated
type: str
default: started
choices:
- absent
- present
- stopped
- started
- created
image:
description:
- Repository path (or image name) and tag used to create the container.
If an image is not found, the image will be pulled from the registry.
If no tag is included, C(latest) will be used.
- Can also be an image ID. If this is the case, the image is assumed to
be available locally.
type: str
annotation:
description:
- Add an annotation to the container. The format is key value, multiple
times.
type: dict
authfile:
description:
- Path of the authentication file. Default is
``${XDG_RUNTIME_DIR}/containers/auth.json``
(Not available for remote commands) You can also override the default
path of the authentication file by setting the ``REGISTRY_AUTH_FILE``
environment variable. ``export REGISTRY_AUTH_FILE=path``
type: path
blkio_weight:
description:
- Block IO weight (relative weight) accepts a weight value between 10 and
1000
type: int
blkio_weight_device:
description:
- Block IO weight (relative device weight, format DEVICE_NAME[:]WEIGHT).
type: dict
cap_add:
description:
- List of capabilities to add to the container.
type: list
elements: str
aliases:
- capabilities
cap_drop:
description:
- List of capabilities to drop from the container.
type: list
elements: str
cgroup_parent:
description:
- Path to cgroups under which the cgroup for the container will be
created.
If the path is not absolute, the path is considered to be relative to
the cgroups path of the init process. Cgroups will be created if they
do not already exist.
type: path
cgroupns:
description:
- Path to cgroups under which the cgroup for the container will be
created.
type: str
cgroups:
description:
- Determines whether the container will create CGroups.
Valid values are enabled and disabled, which the default being enabled.
The disabled option will force the container to not create CGroups,
and thus conflicts with CGroup options cgroupns and cgroup-parent.
type: str
choices:
- default
- disabled
cidfile:
description:
- Write the container ID to the file
type: path
cmd_args:
description:
- Any additional command options you want to pass to podman command,
cmd_args - ['--other-param', 'value']
Be aware module doesn't support idempotency if this is set.
type: list
elements: str
conmon_pidfile:
description:
- Write the pid of the conmon process to a file.
conmon runs in a separate process than Podman,
so this is necessary when using systemd to restart Podman containers.
type: path
command:
description:
- Override command of container. Can be a string or a list.
type: raw
cpu_period:
description:
- Limit the CPU real-time period in microseconds
type: int
cpu_rt_period:
description:
- Limit the CPU real-time period in microseconds.
Limit the container's Real Time CPU usage. This flag tell the kernel to
restrict the container's Real Time CPU usage to the period you specify.
type: int
cpu_rt_runtime:
description:
- Limit the CPU real-time runtime in microseconds.
This flag tells the kernel to limit the amount of time in a given CPU
period Real Time tasks may consume.
type: int
cpu_shares:
description:
- CPU shares (relative weight)
type: int
cpus:
description:
- Number of CPUs. The default is 0.0 which means no limit.
type: str
cpuset_cpus:
description:
- CPUs in which to allow execution (0-3, 0,1)
type: str
cpuset_mems:
description:
- Memory nodes (MEMs) in which to allow execution (0-3, 0,1). Only
effective on NUMA systems.
type: str
detach:
description:
- Run container in detach mode
type: bool
default: True
debug:
description:
- Return additional information which can be helpful for investigations.
type: bool
default: False
detach_keys:
description:
- Override the key sequence for detaching a container. Format is a single
character or ctrl-value
type: str
device:
description:
- Add a host device to the container.
The format is <device-on-host>[:<device-on-container>][:<permissions>]
(e.g. device /dev/sdc:/dev/xvdc:rwm)
type: list
elements: str
device_read_bps:
description:
- Limit read rate (bytes per second) from a device
(e.g. device-read-bps /dev/sda:1mb)
type: list
device_read_iops:
description:
- Limit read rate (IO per second) from a device
(e.g. device-read-iops /dev/sda:1000)
type: list
device_write_bps:
description:
- Limit write rate (bytes per second) to a device
(e.g. device-write-bps /dev/sda:1mb)
type: list
device_write_iops:
description:
- Limit write rate (IO per second) to a device
(e.g. device-write-iops /dev/sda:1000)
type: list
dns:
description:
- Set custom DNS servers
type: list
elements: str
aliases:
- dns_servers
dns_option:
description:
- Set custom DNS options
type: str
aliases:
- dns_opts
dns_search:
description:
- Set custom DNS search domains (Use dns_search with '' if you don't wish
to set the search domain)
type: str
aliases:
- dns_search_domains
entrypoint:
description:
- Overwrite the default ENTRYPOINT of the image
type: str
env:
description:
- Set environment variables.
This option allows you to specify arbitrary environment variables that
are available for the process that will be launched inside of the
container.
type: dict
env_file:
description:
- Read in a line delimited file of environment variables
type: path
env_host:
description:
- Use all current host environment variables in container.
Defaults to false.
type: bool
etc_hosts:
description:
- Dict of host-to-IP mappings, where each host name is a key in the
dictionary. Each host name will be added to the container's
``/etc/hosts`` file.
type: dict
aliases:
- add_hosts
expose:
description:
- Expose a port, or a range of ports (e.g. expose "3300-3310") to set up
port redirection on the host system.
type: list
elements: str
aliases:
- exposed
- exposed_ports
force_restart:
description:
- Force restart of container.
type: bool
default: False
aliases:
- restart
gidmap:
description:
- Run the container in a new user namespace using the supplied mapping.
type: str
group_add:
description:
- Add additional groups to run as
type: list
aliases:
- groups
healthcheck:
description:
- Set or alter a healthcheck command for a container.
type: str
healthcheck_interval:
description:
- Set an interval for the healthchecks
(a value of disable results in no automatic timer setup)
(default "30s")
type: str
healthcheck_retries:
description:
- The number of retries allowed before a healthcheck is considered to be
unhealthy. The default value is 3.
type: int
healthcheck_start_period:
description:
- The initialization time needed for a container to bootstrap.
The value can be expressed in time format like 2m3s. The default value
is 0s
type: str
healthcheck_timeout:
description:
- The maximum time allowed to complete the healthcheck before an interval
is considered failed. Like start-period, the value can be expressed in
a time format such as 1m22s. The default value is 30s
type: str
hostname:
description:
- Container host name. Sets the container host name that is available
inside the container.
type: str
http_proxy:
description:
- By default proxy environment variables are passed into the container if
set for the podman process. This can be disabled by setting the
http_proxy option to false. The environment variables passed in
include http_proxy, https_proxy, ftp_proxy, no_proxy, and also the
upper case versions of those.
Defaults to true
type: bool
image_volume:
description:
- Tells podman how to handle the builtin image volumes.
The options are bind, tmpfs, or ignore (default bind)
type: str
choices:
- 'bind'
- 'tmpfs'
- 'ignore'
image_strict:
description:
- Whether to compare images in idempotency by taking into account a full
name with registry and namespaces.
type: bool
default: False
init:
description:
- Run an init inside the container that forwards signals and reaps
processes. The default is false.
type: bool
init_path:
description:
- Path to the container-init binary.
type: str
interactive:
description:
- Keep STDIN open even if not attached. The default is false.
When set to true, keep stdin open even if not attached.
The default is false.
type: bool
ip:
description:
- Specify a static IP address for the container, for example
'10.88.64.128'.
Can only be used if no additional CNI networks to join were specified
via 'network:', and if the container is not joining another container's
network namespace via 'network container:<name|id>'.
The address must be within the default CNI network's pool
(default 10.88.0.0/16).
type: str
ipc:
description:
- Default is to create a private IPC namespace (POSIX SysV IPC) for the
container
type: str
aliases:
- ipc_mode
kernel_memory:
description:
- Kernel memory limit
(format <number>[<unit>], where unit = b, k, m or g)
Note - idempotency is supported for integers only.
type: str
label:
description:
- Add metadata to a container, pass dictionary of label names and values
aliases:
- labels
type: dict
label_file:
description:
- Read in a line delimited file of labels
type: str
log_driver:
description:
- Logging driver. Used to set the log driver for the container.
For example log_driver "k8s-file".
type: str
choices:
- k8s-file
- journald
- json-file
log_level:
description:
- Logging level for Podman. Log messages above specified level
("debug"|"info"|"warn"|"error"|"fatal"|"panic") (default "error")
type: str
choices:
- debug
- info
- warn
- error
- fatal
- panic
log_opt:
description:
- Logging driver specific options. Used to set the path to the container
log file. For example log_opt
"path=/var/log/container/mycontainer.json"
type: str
aliases:
- log_options
mac_address:
description:
- Specify a MAC address for the container, for example
'92:d0:c6:0a:29:33'.
Don't forget that it must be unique within one Ethernet network.
type: str
memory:
description:
- Memory limit (format 10k, where unit = b, k, m or g)
Note - idempotency is supported for integers only.
type: str
memory_reservation:
description:
- Memory soft limit (format 100m, where unit = b, k, m or g)
Note - idempotency is supported for integers only.
type: str
memory_swap:
description:
- A limit value equal to memory plus swap. Must be used with the -m
(--memory) flag.
The swap LIMIT should always be larger than -m (--memory) value.
By default, the swap LIMIT will be set to double the value of --memory
Note - idempotency is supported for integers only.
type: str
memory_swappiness:
description:
- Tune a container's memory swappiness behavior. Accepts an integer
between 0 and 100.
type: int
mount:
description:
- Attach a filesystem mount to the container. bind or tmpfs
For example mount
"type=bind,source=/path/on/host,destination=/path/in/container"
type: str
network:
description:
- Set the Network mode for the container
* bridge create a network stack on the default bridge
* none no networking
* container:<name|id> reuse another container's network stack
* host use the podman host network stack.
* <network-name>|<network-id> connect to a user-defined network
* ns:<path> path to a network namespace to join
* slirp4netns use slirp4netns to create a user network stack.
This is the default for rootless containers
type: list
elements: str
aliases:
- net
- network_mode
no_hosts:
description:
- Do not create /etc/hosts for the container
Default is false.
type: bool
oom_kill_disable:
description:
- Whether to disable OOM Killer for the container or not.
Default is false.
type: bool
oom_score_adj:
description:
- Tune the host's OOM preferences for containers (accepts -1000 to 1000)
type: int
pid:
description:
- Set the PID mode for the container
type: str
aliases:
- pid_mode
pids_limit:
description:
- Tune the container's PIDs limit. Set -1 to have unlimited PIDs for the
container.
type: str
pod:
description:
- Run container in an existing pod.
If you want podman to make the pod for you, preference the pod name
with "new:"
type: str
privileged:
description:
- Give extended privileges to this container. The default is false.
type: bool
publish:
description:
- Publish a container's port, or range of ports, to the host.
Format - ip:hostPort:containerPort | ip::containerPort |
hostPort:containerPort | containerPort
In case of only containerPort is set, the hostPort will chosen
randomly by Podman.
type: list
elements: str
aliases:
- ports
- published
- published_ports
publish_all:
description:
- Publish all exposed ports to random ports on the host interfaces. The
default is false.
type: bool
read_only:
description:
- Mount the container's root filesystem as read only. Default is false
type: bool
read_only_tmpfs:
description:
- If container is running in --read-only mode, then mount a read-write
tmpfs on /run, /tmp, and /var/tmp. The default is true
type: bool
recreate:
description:
- Use with present and started states to force the re-creation of an
existing container.
type: bool
default: False
restart_policy:
description:
- Restart policy to follow when containers exit.
Restart policy will not take effect if a container is stopped via the
podman kill or podman stop commands. Valid values are
* no - Do not restart containers on exit
* on-failure[:max_retries] - Restart containers when they exit with a
non-0 exit code, retrying indefinitely
or until the optional max_retries count is hit
* always - Restart containers when they exit, regardless of status,
retrying indefinitely
type: str
rm:
description:
- Automatically remove the container when it exits. The default is false.
type: bool
aliases:
- remove
- auto_remove
rootfs:
description:
- If true, the first argument refers to an exploded container on the file
system. The default is false.
type: bool
security_opt:
description:
- Security Options. For example security_opt "seccomp=unconfined"
type: list
elements: str
shm_size:
description:
- Size of /dev/shm. The format is <number><unit>. number must be greater
than 0.
Unit is optional and can be b (bytes), k (kilobytes), m(megabytes), or
g (gigabytes).
If you omit the unit, the system uses bytes. If you omit the size
entirely, the system uses 64m
type: str
sig_proxy:
description:
- Proxy signals sent to the podman run command to the container process.
SIGCHLD, SIGSTOP, and SIGKILL are not proxied. The default is true.
type: bool
stop_signal:
description:
- Signal to stop a container. Default is SIGTERM.
type: int
stop_timeout:
description:
- Timeout (in seconds) to stop a container. Default is 10.
type: int
subgidname:
description:
- Run the container in a new user namespace using the map with 'name' in
the /etc/subgid file.
type: str
subuidname:
description:
- Run the container in a new user namespace using the map with 'name' in
the /etc/subuid file.
type: str
sysctl:
description:
- Configure namespaced kernel parameters at runtime
type: dict
systemd:
description:
- Run container in systemd mode. The default is true.
type: bool
tmpfs:
description:
- Create a tmpfs mount. For example tmpfs
"/tmp" "rw,size=787448k,mode=1777"
type: dict
tty:
description:
- Allocate a pseudo-TTY. The default is false.
type: bool
uidmap:
description:
- Run the container in a new user namespace using the supplied mapping.
type: list
ulimit:
description:
- Ulimit options
type: list
aliases:
- ulimits
user:
description:
- Sets the username or UID used and optionally the groupname or GID for
the specified command.
type: str
userns:
description:
- Set the user namespace mode for the container.
It defaults to the PODMAN_USERNS environment variable.
An empty value means user namespaces are disabled.
type: str
aliases:
- userns_mode
uts:
description:
- Set the UTS mode for the container
type: str
volume:
description:
- Create a bind mount. If you specify, volume /HOST-DIR:/CONTAINER-DIR,
podman bind mounts /HOST-DIR in the host to /CONTAINER-DIR in the
podman container.
type: list
elements: str
aliases:
- volumes
volumes_from:
description:
- Mount volumes from the specified container(s).
type: list
elements: str
workdir:
description:
- Working directory inside the container.
The default working directory for running binaries within a container
is the root directory (/).
type: str
aliases:
- working_dir
"""
EXAMPLES = r"""
- name: Run container
podman_container:
name: container
image: quay.io/bitnami/wildfly
state: started
- name: Create a data container
podman_container:
name: mydata
image: busybox
volume:
- /tmp/data
- name: Re-create a redis container
podman_container:
name: myredis
image: redis
command: redis-server --appendonly yes
state: present
recreate: yes
expose:
- 6379
volumes_from:
- mydata
- name: Restart a container
podman_container:
name: myapplication
image: redis
state: started
restart: yes
etc_hosts:
other: "127.0.0.1"
restart_policy: "no"
device: "/dev/sda:/dev/xvda:rwm"
ports:
- "8080:9000"
- "127.0.0.1:8081:9001/udp"
env:
SECRET_KEY: "ssssh"
BOOLEAN_KEY: "yes"
- name: Container present
podman_container:
name: mycontainer
state: present
image: ubuntu:14.04
command: "sleep 1d"
- name: Stop a container
podman_container:
name: mycontainer
state: stopped
- name: Start 4 load-balanced containers
podman_container:
name: "container{{ item }}"
recreate: yes
image: someuser/anotherappimage
command: sleep 1d
with_sequence: count=4
- name: remove container
podman_container:
name: ohno
state: absent
- name: Writing output
podman_container:
name: myservice
image: busybox
log_options: path=/var/log/container/mycontainer.json
log_driver: k8s-file
"""
RETURN = r"""
container:
description:
- Facts representing the current state of the container. Matches the
podman inspection output.
- Note that facts are part of the registered vars since Ansible 2.8. For
compatibility reasons, the facts
are also accessible directly as C(podman_container). Note that the
returned fact will be removed in Ansible 2.12.
- Empty if C(state) is I(absent).
returned: always
type: dict
sample: '{
"AppArmorProfile": "",
"Args": [
"sh"
],
"BoundingCaps": [
"CAP_CHOWN",
...
],
"Config": {
"Annotations": {
"io.kubernetes.cri-o.ContainerType": "sandbox",
"io.kubernetes.cri-o.TTY": "false"
},
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [
"sh"
],
"Domainname": "",
"Entrypoint": "",
"Env": [
"PATH=/usr/sbin:/usr/bin:/sbin:/bin",
"TERM=xterm",
"HOSTNAME=",
"container=podman"
],
"Hostname": "",
"Image": "docker.io/library/busybox:latest",
"Labels": null,
"OpenStdin": false,
"StdinOnce": false,
"StopSignal": 15,
"Tty": false,
"User": {
"gid": 0,
"uid": 0
},
"Volumes": null,
"WorkingDir": "/"
},
"ConmonPidFile": "...",
"Created": "2019-06-17T19:13:09.873858307+03:00",
"Dependencies": [],
"Driver": "overlay",
"EffectiveCaps": [
"CAP_CHOWN",
...
],
"ExecIDs": [],
"ExitCommand": [
"/usr/bin/podman",
"--root",
...
],
"GraphDriver": {
...
},
"HostConfig": {
...
},
"HostnamePath": "...",
"HostsPath": "...",
"ID": "...",
"Image": "...",
"ImageName": "docker.io/library/busybox:latest",
"IsInfra": false,
"LogPath": "/tmp/container/mycontainer.json",
"MountLabel": "system_u:object_r:container_file_t:s0:c282,c782",
"Mounts": [
...
],
"Name": "myservice",
"Namespace": "",
"NetworkSettings": {
"Bridge": "",
...
},
"Path": "sh",
"ProcessLabel": "system_u:system_r:container_t:s0:c282,c782",
"ResolvConfPath": "...",
"RestartCount": 0,
"Rootfs": "",
"State": {
"Dead": false,
"Error": "",
"ExitCode": 0,
"FinishedAt": "2019-06-17T19:13:10.157518963+03:00",
"Healthcheck": {
"FailingStreak": 0,
"Log": null,
"Status": ""
},
"OOMKilled": false,
"OciVersion": "1.0.1-dev",
"Paused": false,
"Pid": 4083,
"Restarting": false,
"Running": false,
"StartedAt": "2019-06-17T19:13:10.152479729+03:00",
"Status": "exited"
},
"StaticDir": "..."
...
}'
"""
from ansible.module_utils.basic import AnsibleModule # noqa: F402
def main():
module = AnsibleModule(
argument_spec=yaml_safe_load(DOCUMENTATION)['options'],
supports_check_mode=True,
)
with open('/test.config.yml', 'r') as yaml_config:
test_config = yaml_safe_load(yaml_config)
returned = test_config.get('validator_out')
module.fail_json(msg=returned, stderr=returned, stdout=returned)
if __name__ == '__main__':
main()

View File

@ -0,0 +1,71 @@
# Copyright 2019 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from __future__ import absolute_import, division, print_function
__metaclass__ = type
DOCUMENTATION = r'''
module: mocked podman_container_info
author:
- David Vallee Delisle (@dvd)
short_description: Mocking gather facts about containers using podman
description:
- Mocking gather facts about containers using podman
'''
EXAMPLES = r"""
- name: Gather facts for all containers
podman_container_info:
"""
RETURN = r"""
containers:
description: Facts from all or specificed containers
returned: always
type: list
elements: dict
sample: {
"Id": "21d8b432eaec1b4eac2a21a78de524bdbb2f074d4ea43d3605b2b072ffe21878",
"State": {
"Status": "running",
"Running": true,
},
"Image": "0ece6dfb3015c221c8ad6d364dea7884ae3e24becd60e94b80d5361f4ed78f47",
"ImageName": "undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-nova-compute:16.1_20210430.1",
"Name": "nova_compute",
"Mounts": [],
}
"""
from ansible.module_utils.basic import AnsibleModule # noqa: F402
from yaml import safe_load as yaml_safe_load
def main():
module = AnsibleModule({}, supports_check_mode=True)
sample = yaml_safe_load(RETURN)['containers']['sample']
with open('/test.config.yml', 'r') as yaml_config:
test_config = yaml_safe_load(yaml_config)
config_folder = test_config.get('config_folder')
sample['Name'] = test_config.get('service_name')
sample['State']['Running'] = bool(test_config.get('service_running', True))
sample['Mounts'].append({'Type': 'bind', 'Source': config_folder})
module.exit_json(**{
"changed": False,
"containers": [sample],
})
if __name__ == '__main__':
main()

View File

@ -0,0 +1,35 @@
---
# Copyright 2020 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
- name: Converge
hosts: all
environment:
ANSIBLE_STDOUT_CALLBACK: yaml
ANSIBLE_LIBRARY: ../../resources/library
tasks:
- block:
- name: Include the oslo_config_validator role
include_role:
name: oslo_config_validator
rescue:
- fail:
msg: "Default test failed"
when: molecule_yml.scenario.name == "default"
- fail:
msg: "Scenario {{ molecule_yml.scenario.name }} was suppsoed to fail"
when:
- not validation_errors | count

View File

@ -0,0 +1,30 @@
---
- hosts: all
tasks:
- name: Save test config from environment variable
set_fact:
test_config: "{{ lookup('env', 'MOLECULE_OCV') }}"
- name: Print the test configuration
debug:
var: test_config
- name: Set config file fact
set_fact:
config_file: "{{ test_config.config_folder }}/{{ test_config.config_file }}"
- name: Saving test_config to file
copy:
content: "{{ test_config }}"
dest: "/test.config.yml"
- name: Creating mocked config folder
file:
path: "{{ config_file | dirname }}"
state: directory
recurse: true
- name: Creating mocked config file
file:
path: "{{ config_file }}"
state: touch

View File

@ -0,0 +1,166 @@
---
# Copyright 2021 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# These tasks are meant to reformat the data structure and merge
# them with the config files found on the nodes as well as running
# containers.
- name: Generating computed service config
set_fact:
service_config: >-
{% set config_list = [] %}{%
for svc in oslo_config_validator_checked_services %}{%
set ns = oslo_config_validator_service_configs[svc] %}{%
for cf in ns.config_files %}{%
set _ = config_list.append({
"path": cf.path,
"service": svc,
"ignored_groups": ns.ignored_groups | default([]),
"opt_data": ns.opt_data | default([]),
"namespaces": cf.namespaces | union(ns.default_namespaces) | unique
})
%}{%
endfor
%}{%
endfor
%}{{ config_list }}
- name: Printing computed service config
when:
- oslo_config_validator_debug | bool
debug:
var: service_config
- name: podman - Gather facts for all containers
become: true
podman_container_info:
register: containers_facts
- name: Generating config_location dict
set_fact:
config_locations: >-
{% set tmp_config_locations={} %}{%
for mount in container.Mounts | selectattr('Source', 'match', '/var/lib/config-data/puppet-generated/[^/]+$') | list
%}{%
if mount.Source not in config_locations
%}{%- set _ = tmp_config_locations.update({mount.Source: {"image": container.Image | default("") }}) -%}{%
endif %}{%
endfor
%}{{ config_locations | combine(tmp_config_locations) }}
loop: "{{ containers_facts.containers }}"
loop_control:
loop_var: container
label: "{{ container.Name }}"
when:
- container.State.Running
- container.Mounts | selectattr('Source', 'match', '/var/lib/config-data/puppet-generated/[^/]+$') | list | count
- name: Looking for possible config files
find:
recurse: true
depth: 4
excludes:
- ".*httpd.*"
- "[0-9]+.*"
- ".*wsgi"
paths: "{{ config_path.key }}"
patterns:
- ".*({{ oslo_config_validator_service_configs.keys() | list | join('|') }}).*\\.conf$"
use_regex: true
loop: "{{ dict(config_locations) | dict2items }}"
loop_control:
loop_var: config_path
label: "{{ config_path.key }}"
register: found_configs
- name: Printing found configs
when:
- oslo_config_validator_debug | bool
debug:
var: found_configs
- name: Building validations and invalidations data structures with real file names
when:
- result.matched
block:
- name: Building config validations dict
set_fact:
config_validations: >-
{% set tmp_config_locations = [] %}{%
for file in result.files
%}{%
set svc_config = service_config | selectattr('path', 'contains', file.path | basename) | list
%}{%
if svc_config | count %}{%
set _ = tmp_config_locations.append({
"path": file.path,
"image": result.config_path.value.image,
"ignored_groups": svc_config.0.ignored_groups,
"namespaces": svc_config.0.namespaces | list,
"opt_data": svc_config.0.opt_data,
"service": svc_config.0.service
})
%}{%
endif
%}{%
endfor
%}{{ config_validations | default([]) | union(tmp_config_locations) }}
loop: "{{ found_configs.results }}"
loop_control:
loop_var: result
label: "{{ result.config_path.key }}"
- name: Building config invalidations dict
set_fact:
config_invalidations: >-
{% set tmp_config_locations = [] %}{%
for file in result.files
%}{%
set svc_config = service_config | selectattr('path', 'contains', file.path | basename) | list
%}{%
if svc_config | count %}{%
for ns in svc_config[0].namespaces | list %}{%
set ns_config = oslo_config_validator_namespaces_config |
selectattr('namespace', 'equalto', ns) |
selectattr('invalid_settings', 'defined') | list %}{%
if ns_config | count %}{%
set _ = tmp_config_locations.append({
"path": file.path,
"ignored_groups": svc_config[0].ignored_groups,
"service": svc_config[0].service,
"invalid_settings": ns_config[0].invalid_settings
}) %}{%
endif %}{%
endfor %}{%
endif %}{%
endfor
%}{{ config_invalidations | default([]) | union(tmp_config_locations) }}
loop: "{{ found_configs.results }}"
loop_control:
loop_var: result
label: "{{ result.config_path.key }}"
- name: Printing config datastructure
when:
- oslo_config_validator_debug | bool
block:
- name: Config Validations
debug:
var: config_validations
- name: Config Invalidations
debug:
var: config_invalidations

View File

@ -0,0 +1,121 @@
---
# Copyright 2021 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# These tasks are wrapping around the oslo-config container creation.
# this shouldn't be necessary once this proposed change has landed
# and propagated in images
# https://review.opendev.org/c/openstack/oslo.config/+/790883
- name: Applying the most dirty patch in tripleo-validation history
when:
- "'oslo-config-generator' in oslo_command"
vars:
git_apply: >-
git apply --unsafe-paths --directory /usr/lib/python3.*/site-packages/oslo_config/
-p2 -C1 --recount /oslo_config_validation/oslo_config_generator.diff | cat
block:
- name: Generating namespace arguments
set_fact:
generator_patch: |
cat << EOF > /oslo_config_validation/oslo_config_generator.diff
diff --git a/oslo_config/generator.py b/oslo_config/generator.py
index 9845c81..5dd1044 100644
--- a/oslo_config/generator.py
+++ b/oslo_config/generator.py
@@ -52,6 +52,7 @@
import yaml
from oslo_config import cfg
+from oslo_i18n import _message
import stevedore.named # noqa
LOG = logging.getLogger(__name__)
@@ -723,6 +724,18 @@
return output_data
+def i18n_representer(dumper, data):
+ """oslo_i18n yaml representer
+
+ Returns a translated to the default locale string for yaml.safe_dump
+
+ :param dumper: a SafeDumper instance passed by yaml.safe_dump
+ :param data: a oslo_i18n._message.Message instance
+ """
+ serializedData = str(data.translation())
+ return dumper.represent_str(serializedData)
+
+
def _output_machine_readable(groups, output_file, conf):
"""Write a machine readable sample config file
@@ -736,6 +749,7 @@
"""
output_data = _generate_machine_readable_data(groups, conf)
if conf.format_ == 'yaml':
+ yaml.SafeDumper.add_representer(_message.Message, i18n_representer)
output_file.write(yaml.safe_dump(output_data,
default_flow_style=False))
else:
EOF
{{ git_apply }}
{{ oslo_command }}
- name: Sending over the patched script
changed_when: false
copy:
mode: 0777
content: "{{ generator_patch }}"
dest: "{{ oslo_config_validator_work_path }}/oslo_config_generator.sh"
- name: Replacing oslo_command with script
set_fact:
oslo_command: /oslo_config_validation/oslo_config_generator.sh
- name: Printing oslo-config command
when:
- oslo_config_validator_debug | bool
debug:
var: oslo_command
- name: Copying the config file to a temp path
changed_when: false
copy:
mode: 0666
remote_src: true
src: "{{ config_file.path }}"
dest: "{{ oslo_config_validator_work_path }}"
- name: Run oslo-config container
podman_container:
name: "{{ container_name }}"
image: "{{ config_file.image }}"
state: started
detach: false
rm: true
user: 0
mount: "type=bind,source={{ oslo_config_validator_work_path }},destination=/oslo_config_validation"
command: "{{ oslo_command }}"
register: container_run
failed_when:
- not container_run.stderr_lines | default([]) | count
- not container_run.stdout_lines | default([]) | count
- name: Printing container run output
when:
- oslo_config_validator_debug | bool
debug:
var: container_run

View File

@ -0,0 +1,76 @@
---
# Copyright 2021 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
- name: Getting possible invalid settings
become: true
validations_read_ini:
path: "{{ service.0.path }}"
section: "{{ service.1.section }}"
key: "{{ service.1.option }}"
loop: "{{ config_invalidations | subelements('invalid_settings') }}"
loop_control:
loop_var: service
label: "{{ service.1.section }}/{{ service.1.option }} in {{ service.0.path }}"
register: invalid_setting_validation
check_mode: false
- name: Printing invalid settings
when:
- oslo_config_validator_debug | bool
debug:
var: invalid_setting_validation
- name: Checking if settings are infact valid
when:
- setting.value
set_fact:
invalid_settings: >-
{% set errors = [] %}{%
set svc = setting.service.1 %}{%
set operator = svc.operator | default("not") %}{%
if 'separator' in svc %}{%
set setting_values = setting.value.split(svc.separator) | list %}{%
else %}{%
set setting_values = [setting.value] %}{%
endif %}{%
set setting_path = setting.service.0.path + ':' + svc.section + '/' + svc.option %}{%
for val in setting_values %}{%
if operator == "not" and val in svc.value_list %}{%
set _ = errors.append(setting_path +
' has an invalid value: ' + val +
' Forbidden values: ' + svc.value_list | join(',') ) %}{%
elif operator == "eq" and val not in svc.value_list %}{%
set _ = errors.append(setting_path +
' has an invalid value: ' + val +
' Should be one of these values: ' + svc.value_list | join(',') ) %}{%
elif operator == "lt" and val|int < svc.value_list.0 %}{%
set _ = errors.append(setting_path +
': Current value ' + val + ' is lesser than expected ' + svc.value_list.0 | string) %}{%
elif operator == "gt" and val|int > svc.value_list.0 %}{%
set _ = errors.append(setting_path +
': Current value ' + val + ' is greater than expected ' + svc.value_list.0 | string) %}{%
endif %}{%
endfor %}{{ invalid_settings | default([]) + errors }}
loop: "{{ invalid_setting_validation.results }}"
loop_control:
loop_var: setting
label: "{{ setting.service.1.section }}/{{ setting.service.1.option }} in {{ setting.service.0.path }}"
- name: Printing invalid settings
when:
- oslo_config_validator_debug | bool
debug:
var: invalid_settings

View File

@ -0,0 +1,72 @@
---
# Copyright 2021 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# tasks
- name: Include the config builder tasks
include_tasks: build_validation_config.yml
- name: Validate configuration
when:
- 'oslo_config_validator_validation | bool or
oslo_config_validator_report | bool'
include_tasks: validate_config.yml
- name: Invalidate configuration
when:
- oslo_config_validator_invalid_settings | bool
include_tasks: invalidate_config.yml
- name: Config validation reporting
when:
- oslo_config_validator_report | bool
include_tasks: report_generation.yml
- name: Config validation assertions
block:
- name: Verifying setting validation
assert:
that:
- not service.output | count
fail_msg: |
Config file {{ service.config_file }} for {{ service.service }} has returned validation errors:
{% for msg in service.output %}
{{ msg }}
{% endfor %}
loop: "{{ validation_output }}"
loop_control:
loop_var: service
label: "{{ service.config_file }}/{{ service.service }}"
register: validation_errors
ignore_errors: true
- name: Asserted failure
when: "'failed' in validation_errors or
(invalid_settings is defined and
invalid_settings | count)"
fail:
msg: |
Configuration validation failed for at least one service:
{%
if validation_errors.results | count %}{%
for service in validation_errors.results %}{%
if service.failed %}{{ service.msg }}{% endif %}{%
endfor %}{%
endif %}{%
if invalid_settings is defined and invalid_settings | count %}{%
for message in invalid_settings %}{{ message }}
{% endfor %}{%
endif %}

View File

@ -0,0 +1,66 @@
---
# Copyright 2021 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# These tasks are generating a new opt_data data structure based
# on the output of oslo-config-generator merged with the opt_data
# mapping in the config file settings. This has to be looped for
# each mapping.
- name: Get list value
become: true
validations_read_ini:
path: "{{ config_file.path }}"
section: "{{ option_data.index_key.section }}"
key: "{{ option_data.index_key.option }}"
register: opt_val
- name: Printing templatable values
when:
- oslo_config_validator_debug | bool
debug:
var: opt_val
- name: Building list of replaceable data
when:
- opt_val.value
- opt_data | length
set_fact:
new_sections: >-
{% set section_list = {} %}{%
set templates = [] %}{%
for section in option_data.template_section %}{%
set _ = templates.extend(opt_data.options[section].opts) %}{%
endfor %}{%
if 'separator' in option_data.index_key %}{%
set sections = opt_val.value.split(option_data.index_key.separator) | list %}{%
else %}{%
set sections = [opt_val.value] %}{%
endif %}{%
for val in sections %}{%
set _ = section_list.__setitem__(val, {"opts": templates }) %}{%
endfor %}{{ section_list }}
- name: Printing generated sections
when:
- oslo_config_validator_debug | bool
debug:
var: new_sections
- name: Adding new sections to opt_data
when:
- new_sections | default([]) | count
set_fact:
opt_data: "{{ opt_data | combine({'options': new_sections}, recursive=True) }}"

View File

@ -0,0 +1,90 @@
---
# Copyright 2021 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
- name: Set folder name fact
set_fact:
local_folder_name: "{{ oslo_config_validator_report_path }}/{{ inventory_hostname }}"
- name: Making sure local folder exists
changed_when: false
file:
path: "{{ local_folder_name }}"
state: directory
recurse: true
delegate_to: localhost
- name: Saving filtered outputs from validation to files
changed_when: false
copy:
content: "{{ service.output | join('\n') }}\n"
dest: "{{ local_folder_name }}/{{ service.service }}-{{ service.config_file | basename }}"
loop: "{{ validation_output }}"
loop_control:
loop_var: service
delegate_to: localhost
- name: Saving invalidation to files
changed_when: false
when:
- invalid_settings | default([]) | count
copy:
content: "{{ invalid_settings | join('\n') }}\n"
dest: "{{ local_folder_name }}/invalid_settings.log"
delegate_to: localhost
- name: Setting report generation default message
set_fact:
report_msg: "Reports are available in {{ oslo_config_validator_report_path }}"
- name: Archive report
when:
- oslo_config_validator_report_archive | bool
block:
- name: Zipping fetched files
changed_when: false
archive:
path:
- "{{ oslo_config_validator_report_path }}/*"
dest: "{{ oslo_config_validator_report_path }}.tar.xz"
remove: true
format: xz
run_once: true
delegate_to: localhost
delegate_facts: true
register: archive_out
- name: Delete local files
changed_when: false
file:
path: "{{ oslo_config_validator_report_path }}"
state: absent
run_once: true
delegate_to: localhost
- name: Setting report generation archived message
set_fact:
report_msg: |
Reports are archived in {{ oslo_config_validator_report_path }}.tar.xz and contains
{{ archive_out.archived | list | default([]) | count }} file(s)
- name: Report path
run_once: true
warn:
msg: "{{ report_msg }}"
- name: Role terminated after reports are ready.
meta: end_play

View File

@ -0,0 +1,68 @@
---
# Copyright 2021 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
- name: Making sure work folder is present
become: true
changed_when: false
file:
mode: 0777
path: "{{ oslo_config_validator_work_path }}"
state: directory
- name: Validate configuration
include_tasks: validation_command.yml
loop: "{{ config_validations }}"
loop_control:
loop_var: config_file
label: "{{ config_file.service }}: {{ config_file.path }}"
- name: Deleting work folder
become: true
changed_when: false
file:
path: "{{ oslo_config_validator_work_path }}"
state: absent
- name: Preparing and filtering output datastructure
when:
- validated_configs is defined
set_fact:
validation_output: >-
{% set ignored_patterns = oslo_config_validator_global_ignored_messages %}{%
for ns in oslo_config_validator_namespaces_config %}{%
if ns.namespace in out.config_file.namespaces %}{%
for ignored in ns.ignored_messages %}{%
set _ = ignored_patterns.append(ignored) %}{%
endfor %}{%
endif %}{%
endfor %}{%
set output = out.output.stderr_lines | union(out.output.stdout_lines) | unique | list | reject('match', ignored_patterns | join("|")) | list %}{%
set _ = validation_output.append({
"service": out.config_file.service,
"config_file": out.config_file.path,
"namespaces": out.config_file.namespaces,
"output": output,
}) %}{{ validation_output }}
loop: "{{ validated_configs }}"
loop_control:
loop_var: out
label: "{{ out.config_file.path }}"
- name: Printing validation output
when:
- oslo_config_validator_debug | bool
debug:
var: validation_output

View File

@ -0,0 +1,97 @@
---
# Copyright 2021 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
- name: Generating namespace arguments
set_fact:
namespace_args: "--namespace {{ config_file.namespaces | join(' --namespace ') }}"
container_name: "oslo-config-{{ config_file.service }}-{{ config_file.path | basename }}"
- name: Opt data required
when:
- config_file.opt_data | count
block:
- name: Generating oslo-config-generator command for opt_data
set_fact:
oslo_command: "oslo-config-generator --format yaml {{ namespace_args }}"
- name: Running container
args:
apply:
become: true
include_tasks: container_run.yml
- name: Saving opt_data datastructure
set_fact:
opt_data: "{{ container_run.stdout | from_yaml }}"
- name: Printing opt_data
when:
- oslo_config_validator_debug | bool
debug:
var: opt_data
- name: Generating the new config sections
include_tasks: opt_data_format.yml
loop: "{{ config_file.opt_data }}"
loop_control:
loop_var: option_data
label: >-
{{ option_data.index_key.section }} /
{{ option_data.index_key.option }}
- name: Saving new opt_data content
become: true
changed_when: false
copy:
content: "{{ opt_data | to_yaml }}"
dest: "{{ oslo_config_validator_work_path }}/opt_data.yaml"
mode: 0666
- name: Generate validation command
set_fact:
oslo_command: >-
oslo-config-validator --input-file /oslo_config_validation/{{ config_file.path | basename }}
{% if config_file.opt_data | count and opt_data | length %}
--opt-data /oslo_config_validation/opt_data.yaml
{% else %}
{{ namespace_args }}
{% endif %}
{% if oslo_config_validator_report | bool %} --check-defaults{% endif %}
{% if config_file.ignored_groups | count %}
--exclude-group {{ config_file.ignored_groups | join(' --exclude-group ') }}
{% endif %}
- name: Printing oslo-config-validator command
when:
- oslo_config_validator_debug | bool
debug:
var: oslo_command
- name: Running container
args:
apply:
become: true
include_tasks: container_run.yml
- name: Saving output to fact
set_fact:
validated_configs: "{{ validated_configs | default([]) + [{'output': container_run, 'config_file': config_file }] }}"
- name: Printing validated configs
when:
- oslo_config_validator_debug | bool
debug:
var: validated_configs

View File

@ -0,0 +1,44 @@
---
# Copyright 2020 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# While options found within the vars/ path can be overridden using extra
# vars, items within this path are considered part of the role and not
# intended to be modified.
# All variables within this role should have a prefix of "oslo_config_validator"
metadata:
name: Openstack services configuration validation
description: >
This role is intended to leverage the `oslo-config-validator` on each one
of the configuration files found on a deployment. The goal is to quickly
catch erroneous configurations.
When called manually, it will also be possible to generate a report
returning all the differences between the current configuration and the
default configuration
groups:
- backup-and-restore
- pre-upgrade
- post-deployment
- post-system-upgrade
- post-update
# Placeholder variables
config_locations: {}
config_validations: []
validation_output: []
invalid_setting_regex: "(is not part of the sample config|not found)"

View File

@ -329,6 +329,17 @@
vars:
tripleo_validations_role_name: openstack_endpoints
voting: false
- job:
files:
- ^roles/oslo_config_validator/.*
- ^tests/prepare-test-host.yml
- ^ci/playbooks/pre.yml
- ^ci/playbooks/run.yml
- ^molecule-requirements.txt
name: tripleo-validations-centos-8-molecule-oslo_config_validator
parent: tripleo-validations-centos-8-base
vars:
tripleo_validations_role_name: oslo_config_validator
- job:
files:
- ^roles/overcloud_service_status/.*