Commit Graph

32 Commits (master)

Author SHA1 Message Date
Sofer Athlan-Guyot 90c52db21e Ensure container's image get updated if their name stay the same.
In some environment the containers' tag isn't modified (think
"latest"). This is already supported during deployment so we re-use
the code during update.

We also make sure that the same mechanism is available for
ovn_controller update. It can be selectively disable if one pass:

    --skip-tags ovn_image

to the external-update run command.

We remove the ignore error as it's wrong indication that we are not
taking errors into account in those tasks. We also adjust the
delay as it's 3 seconds by default making the time we check longer
than the async tasks. Eventually, we add a new parameter for
controlling the downloading of the image
"OVNControllerImageUpdateTimeout" which is set to 10min by default.

Note that in order to pass CI, we need to modify the tht syntax test:
 - ensure containers.podman collection is installed;
 - ensure custom tripleo-ansible filter are found;
 - make sure tripleo_role_name is defined in the syntax test playbook.

Change-Id: I6d439367816d5f5968b8f890f74b7a2f74715f52
2022-09-27 12:20:54 +02:00
Rajesh Tailor 6861fb324b Fix typos in comments and parameter descriptions
This change fixes typos in comments, parameter descriptions
and ansible task names.

Change-Id: I82b67ca834077b66ebd71744face3bba0b43da2f
2022-08-03 17:07:11 +05:30
Takashi Kajinami b2a35f13d5 Remove 'u' prefix
In Python 3, u'...' and '...' are equivalent. Also, the prefix is no
longer printed.

Change-Id: I3de64b2526422322ad10231d9d682a326d4435ca
2022-05-18 11:13:54 +09:00
rabi 0da3db3b94 Install ansible.posix.debug stdout callback plugin
ansible.builtin.debug is redirected to ansible.posix.debug.
We need to install the collection explicitly.

This needs to be fixed for stdout callback plugin related changes
in ansible-runner >=2.2.0.

Also, increases ansible verbosity to track these issues better in
the future.

Closes-Bug: #1971703
Change-Id: Ibe883e19c25926ab302db741afae70821209a350
2022-05-06 09:55:52 +05:30
vcandapp dde9a258d8 Update dervice_pci script to handle pci address formats
When using nic paritioning feature, the VFs used
by host has to be excluded from passthrough
whitelist. Added support to the derive pci script
to support pci address as string or dict format.
Specifying product_id, vendor_id is also allowed.
Added test cases for the derive pci script.

Change-Id: I35b3fe177f08214d06e20334e08be9bfe06f16d1
2022-01-24 13:20:26 +00:00
Brendan Shephard b522254bc2 Remove six library
The six library was used to bridge the py2 > py3
gap. This library is longer required on branches
that do not support Python2.

Change-Id: I40cb90bc6bc058dcbf3659b97dbb489b53adb9d3
2021-10-06 07:01:42 +00:00
ramishra a263da138f Use merge strategy for EndpointMap
Now someone can override individual endpoints like
below in an environment:

  EndpointMap: merge

      protocol: http
      port: 9999

Closes-Bug: #1897592
Change-Id: Ibf2e0d183dd51421c4feb7467c3c01fb416d2965
2021-06-15 09:15:41 +05:30
Sorin Sbarnea a013f42e36 Enable ansible-lint
- enable run of ansible-lint, with a temporary set of excludes
- fixes two problems reported by ansible-linter

Change-Id: Ibbe23db8fd5ac1008109f50f514df96686b0fa19
Bug: #1921409
2021-03-30 09:18:15 +01:00
ramishra 7f195ff9a8 Remove DefaultPasswords interface
This was mainly there as an legacy interface which was
for internal use. Now that we pull the passwords from
the existing environment and don't use it, we can drop

Reduces a number of heat resources.

Change-Id: If83d0f3d72a229d737a45b2fd37507dc11a04649
2021-02-12 11:38:44 +05:30
Rajesh Tailor a672bedfc2 Fix typos
Change-Id: Ia9b0410d1ade1abc2d29d3634379b9128016d0e9
2020-09-16 15:45:12 +05:30
Jose Luis Franco Arza 8783ec9c45 Remove ffwd-upgrade leftovers from THT.
Now that the FFU process relies on the upgrade_tasks and deployment
tasts there is no need to keep the old fast_forward_upgrade_tasks.

This patch removes all the fast_forward_upgrade_tasks section from
the services, as well as from the common structures.

Change-Id: I39b8a846145fdc2fb3d0f6853df541c773ee455e
2020-07-23 15:33:25 +00:00
Hervé Beraud 11f84b6302 Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Change-Id: Iabd3e90a46fd087c8e780796e04fcc050c5277ab
2020-06-09 18:41:21 +02:00
Michele Baldessari b0e7008196 standalone/overcloud: enable the HA deployment by default
With the HA NG work having landed, the impact of pacemaker
is reduced and only very few core services are being managed
by pacemaker. Since the HA deployments work just fine
with a single node, it makes little sense to use the non-ha
deployment as default any longer (Also because downstream
we do the default to the HA deployment by default and this
keeps confusing users).

This patch does the following :

* Remove Keepalived services from all CI scenarios running it.
* Make sure all HA services deployed in CI run with Pacemaker.
* Remove non HA containers so Pacemaker can
  bootstrap the new containers safely.
* Before removing mysql container, create the clustercheck user and
  grant correct permissions to avoid bootstrap issues later when galera
  is spawned.
* Disable HA on the minor update job, it seems to not working fine if
  only one controller is deployed.

Change-Id: I0f61016df6a9f07971c5eab51cc9674a1458c66f
2020-04-09 21:27:02 +00:00
Emilien Macchi e9938bf49e Refactorize test_tht_ansible_syntax to welcome new roles/modules
tripleo-ansible isn't the only source of roles and modules, we also have
modules in ansible-pacemaker for example, and more. We need to make the
THT ansible syntax check able to also include these in the testing.

Change-Id: Ieb089d20fced84bf44980d35b88ba20964545b7f
2020-03-25 19:20:42 +00:00
Emilien Macchi a4de1eac9d mysql: refactor upgrade tasks to work without paunch
We are removing Paunch, and also Podman is the only container cli
supported when it comes to upgrades, so this patch simplifies a bit how
we deal with the upgrade of MySQL container.

This submission also adds the path for the ansible modules
installed from tripleo-ansible in the tht-check job.

Change-Id: I8b01ac05bae6544119769e50501bf705b22081f3
2020-01-14 14:55:22 +01:00
Carlos Camacho 0de387ee3c Ansible lint check in THT
Until now if there is a nit inside any
Ansible task part of tripleo-heat-templates we will
need to wait until the end-to-end test imports
the actual tasks to fail.

This submission adds the hability to run a syntax
check for any Ansible task part of any role
defined by default in the resource registry.

Also it will allow to reduce the CI CO2 footprint
by not executing unneeded jobs meant to fail.
Let's save the planet and some CI cycles.

Change-Id: I6eb7c873cc9a4b7d765e745dc260475bff7b268a
2019-10-30 04:56:05 -04:00
Takashi Kajinami f47dfe1059 Enforce pep8/pyflakes rule on python codes
This change makes sure that we apply pyflake8 checks on all python
codes to improve its readability.

Note that there are some rules applied for other OpenStack projects,
but not yet turned on, which should be enabled in the future.

Change-Id: Iaf0299983d3a3fe48e3beb8f47bd33c21deb4972
2019-09-05 15:40:46 +09:00
Zuul 47126b8431 Merge "Trim whitespace in multi-line env descriptions" 2019-04-16 05:28:25 +00:00
Ben Nemec 32d993bbf0 Trim whitespace in multi-line env descriptions
Indenting blank description lines just creates unnecessary trailing
whitespace in the output file.  Skip the indentation in this case.

Change-Id: Ib0fbd26b489f6d9cf262c1186d3b3e604d4c0c67
2018-11-02 11:53:25 -05:00
Ben Nemec 1e8577e858 Don't require files key in env-generator
We have environments that are exclusively resource_registry entries,
and we shouldn't have to include a dummy files: key to make that
work with the generator.

Change-Id: I8ffabb5e583aabbfa4f2f22a4418783c857118fa
2018-07-20 17:14:48 -05:00
Oliver Walsh fa66c327fc Do not generate apache/haproxy certs for invalid networks
Certs were being generated for all networks in service net map.
This was failing as we do not generate hieradata for all of these networks.

Switching from yaql to jinga templating to match the logic that
generates the hieradata.

Change-Id: Ic6c25aceb07ea3824a8fb23549bc5d1205e5cefc
Closes-bug: 1748023
Closes-Bug: 1748053
2018-02-08 12:50:04 +00:00
Thomas Herve e10d529b46 Introduce tests for yaql structures
We use yaql in many hot templates, but they are usually faily obscure
and not documented, which makes it hard to change them over time (to
make them perform better for example).

This introduces some basic infrastructure for testing them, as well as a
first modification to show how things would be easier to change.

Change-Id: I1a8d3a514d8fa0de4b024d3ae320723d791f3e7d
2018-01-17 14:36:06 +01:00
waleed mousa 615128f1c4 Add support for outputting role-specific parameters
It is possible to pass parameters only to a specific role by using an
environment with a <role name>Parameters dict.  This change adds support
for sample environments with such nested parameters.

Change-Id: I7eb91d4262013341fab733342ab313caf8cec118
2018-01-14 07:48:14 +00:00
Moshe Levi 609784ded1 fix generate_environments to run in python3
This patch fix the following error when running python3
TypeError: can only concatenate list (not "dict_keys") to list

Change-Id: Ic487bf4c4f6cb2bc35011416056bef1417a23076
2017-12-27 03:25:04 +02:00
waleed mousa 6035443151 Fixing bug in ""
The method "generate_environments" in
takes two arguments but 1 given in ""

Change-Id: I39abcf2340ce04f3d193d80c8af177027c512556
2017-12-27 03:25:04 +02:00
Ben Nemec 1c9553c37a Add pep8 check that generated environments are up to date
This check ensures that if a parameter is changed that would affect
a generated environment then the environment must be updated before
pep8 will pass.  It will also catch any mistaken hand edits to the
generated files.

bp generated-environments

Change-Id: I2d12992ed55f963285422e1282a4cee06e989b6d
2017-09-07 15:30:31 -05:00
Ben Nemec 7c06db3d1c Convert enable-internal-tls.yaml to be generated
All of the other SSL environments were converted, but this one was
missed.  That's an inconsistent user experience and should be
cleaned up.

This environment also exposed a bug in the tool where it did not
include the parameter_defaults section key if all the parameters
were marked static.

Change-Id: I19bc422c22b9f60f781e696ce703b026dc317786
Closes-Bug: 1713761
2017-08-29 10:50:00 -05:00
Ben Nemec 316dfbd5c6 Add ServiceData to hidden params
It looks like this was added recently and it doesn't appear to be a
parameter we want in the sample environments.

Change-Id: I0ac433553e7ad9b0a54c011b66c54b4692b44be0
2017-08-11 16:35:28 -05:00
Ben Nemec 204a582099 Add nested sample environments for inject-trust-anchor
Fix a bug that prevented these working.  A unit test and
documentation for the nested environment functionality is also

Change-Id: I2d4aeb584eb624178d601cfd6bc0a6473cb5289f
2017-06-12 15:02:50 -05:00
Ben Nemec 8d086b1710 Add storage sample environments
Starts converting storage-related sample environments to the tool,
and adds a few new ones for demonstration purposes.

This has required the addition of a new category of parameter
overrides in the tool.  There are some parameters that are part of
the public API of roles that should not normally be included in a
sample environment for that role.  Examples are EndpointMap and
ServiceNetMap.  Those are both passed into most (all?) roles, but
their template defaults are not useful (both default to {}).
Unless we are explicitly creating a sample environment that
overrides those defaults we don't want them included.

Parameters such as RoleName and RoleParameters are similar.  We
can't change them because they are part of the composable roles
interface and that would break any existing custom roles, but we
don't really want them included normally either.  It's possible
these could be made completely private, but there have been some
very preliminary discussions about generating role samples that
might actually want to set them.

In order to avoid issues with editing the unit test file in editors
that strip trailing whitespace, the minor formatting bug where
params like EndpointMap had a trailing space after the name has
also been fixed.

Change-Id: If11f30c734bfbc17d463a9890c736d7477186fb9
2017-06-12 15:02:50 -05:00
Ben Nemec f503d1b0e7 Support config dir for env generator input files
We're not going to want to list every single sample environment in
a single file, so let's also take a directory and just read every
yaml file in it.  This commit adds support for that as well as
some initial environments to demonstrate its use.

Change-Id: If2c608f2a61fc5e16784ab594d23f1fa335e1d3c
2017-06-12 15:02:50 -05:00
Ben Nemec 4e24c8cb6a Sample environment generator
This is a tool to automate the generation of our sample environment
files.  It takes a yaml file as input, and based on the environments
defined in that file generates a number of sample environment files
from the parameters in the Heat templates.  A tox genconfig target
is added that mirrors how the other OpenStack services generate
their sample config files.

A description of the available options for the input file is
provided in a README file in the sample-env-generator directory.

In this commit only a single sample config is provided as a basic
example of how the tool works, but subsequent commits will add
more generated sample configs.

Change-Id: I855f33a61bba5337d844555a7c41b633b3327f7a
bp: environment-generator
2017-06-12 15:02:50 -05:00