This changes to update the stack without using
the plan and also enables server side env merging
as we don't use the plan-environment.
Also makes changes to call derive params playbooks
without plan.
Depends-On: https://review.opendev.org/c/openstack/tripleo-ansible/+/772197
Change-Id: I8caad3e9185f1c6d23b0941b966192957ca8320b
This patch remove tripleo validator from tripleoclient to
tripleo-validations repository.
The change is related to the effort to make the validation
act as a plugin/component on top of tripleo and not being
part of the core of tripleo.
Change-Id: I1e08eb07ce1f0d138f9c5101b9fe770c5da9b3f9
According to: https://review.rdoproject.org/r/#/c/29059/
We decide to converge ansible roles/libraries/plugin and playbooks to
the /usr/share/ansible path instead of having a dedicated path for
validations.
This will avoid duplication since the validation-framework is now
an external framework from tripleo, the common validations between
validations-common and tripleo-validations can be stored at the same
place.
Add a fallback path for Updates and Upgrades
Change-Id: Ibe97d70f4d321d04900ca9fe88144bc16511611c
Don't use return code from ansible_runner and manage with a flag.
This also removes the fail_on_rc flag from run_ansible_playbook()
and makes it consistent to raise RuntimeError if rc !=0
everywhere including _standalone_deploy().
Change-Id: Ia5971af601d1d9500f33045768b38ac7937117f5
Closes-Bug: #1889394
The new --transport argument allows for specifying a transport
configuration to have set in the generated ansible configuration file.
The default value of "local" matches the previous hardcoded value.
To deploy multiple nodes with tripleo deploy, "--transport ssh" can be
used.
blueprint tripleo-deploy-multinode
Change-Id: I75cbde83dda1a79ad65bf522e57c8ef326bbc54d
Signed-off-by: James Slagle <jslagle@redhat.com>
The only effect the argument had previously was to prevent the command
from throwing an exception when it was not passed.
However, the argument did not actually prevent non-standalone
deployments. It was still possible to deploy multinode or single remote
node deployments using tripleo deploy even when passing --standalone.
This patch deprecates the --standalone argument and adds help text to
indicate that it's no longer needed. Continuing to pass the argument
will have no change in behavior.
blueprint tripleo-deploy-multinode
Change-Id: I6d390f3b201ad681d977a12fd627e610ad7b65f0
Signed-off-by: James Slagle <jslagle@redhat.com>
With rc being set to 1 initially, it was never reset to 0 when using
--output-only. This meant when running tripleo deploy with
--output-only, it always exited 1, causing an exception.
This patch instead initially sets the value to 0. The only other place
where rc is used is to capture the return from running ansible, so if
that returns non-zero, rc will be updated accordingly so that the
deployment will fail.
blueprint tripleo-deploy-multinode
Change-Id: Ie116b4c600317d618a9e55b91a945d03e71fb780
Signed-off-by: James Slagle <jslagle@redhat.com>
We need to prompt the user if we should upgrade earlier in the code so
we can handle additional things like package upgrades in the undercloud
upgrade after the prompt. This change pulls the confirmation logic out
of tripleo_deploy and reuses the prompt_user_for_confirmation code in
the tripleo upgrade and undercloud upgrade actions.
Change-Id: I8dbcae39e6f6d966c1337bad5fb5ba673f7be874
Closes-Bug: #1877825
This moves the node import into ansible.
Story: 2007126
Task: 38183
Change-Id: I7889228333dfc503529770d5cf06a35f39432f18
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This patch makes the VALIDATION_GROUPS list constants dynamic by getting
them from the 'groups.yaml' file. This file comes from
tripleo-validations and is present on the Undercloud at the location [1].
[1] => /usr/share/openstack-tripleo-validations
Change-Id: Ide8cba32a0ea14086714094291f1a8866b669491
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>
This patch adds logging support for validations. It introduces two new
Tripleo Validator commands to allow the user to get the executions
history and get the details of them.
Change-Id: Ie80318a7fa684adb7c3bf7c99d526b0de64b0904
Depends-On: I502c38d3f27db3c6f62a47190136dd03627956bc
Depends-On: I0cb2743d1d4d118320a799c6820d48f9b917498f
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
This change removes the use of mistral from the update static methods. These
methods will now call the required functions directly using
`deployment.config_download`, which will save time and improve reliability.
* The run_update_ansible_action method has been updated to ensure we're supporting
multi-playbook execution properly.
Story: 2007212
Task: 38435
Task: 38438
Change-Id: Ic324847341142829e986128d502fdcab2cbddcd8
Signed-off-by: Kevin Carter <kecarter@redhat.com>
Changes have been made to our ansible-runner usage to ensure we're not running into
file not found and directory creation collisions. This revert should allow us to
define callback plugins as needed and we shouldn't run into the failures we were
seeing before.
This reverts commit dbfe988265.
Change-Id: I0cb2743d1d4d118320a799c6820d48f9b917498f
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This change fixes our py27 tests. The issue was a cascading failure caused
by a bad mock for sys.stdout.flush which results in general failure when
calling system and other builtin functions.
Change-Id: Ia1a108e77dd50404b78d952c9c9a9c0d534a5d28
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This reverts commit 49d3f725f2.
Closes-Bug: #1858115
Change-Id: I49d560b2634847ea7f8bafefa8e7ee7f9726a193
Signed-off-by: Kevin Carter <kecarter@redhat.com>
The stdout callback plugin is being set within ansible runner making it
not respect the user defined configuration. This change adds the required
environment variable after the prepare function to ensure our defined
output callback plugin is always used.
Because we're allowing the assumed output callback plugin to be user defined
we need to cater for parallel playbook execution by defining
`directory_isolation_base_path` in config. Now, whenever parallel playbook
execution is required, `directory_isolation_base_path` will be used ensuring
there are no collisions at runtime.
Change-Id: I7c7e7472d238bd2ab3316cac3dbee01f6d0b10e9
Signed-off-by: Kevin Carter <kecarter@redhat.com>
copy_clouds_yaml is a new function that will be used to copy
/etc/openstack/clouds.yaml which is now generated by Ansible in TripleO
Heat Templates:
I585abc3e6a3b9b8ae9183e0b5170df2e39301e17
It also sets the right permissions to only allow the deployment user to
read the file which container sensible credentials (cloud admin).
Note that I286c71edf120e11d8e51c792e6078dc60147f026 will be rebased so
the revert can properly happen with the right dependencies with
tripleo-common.
Depends-On: https://review.opendev.org/#/c/696390/
Change-Id: I49b95bf7a03e39db3c2cbb91b2167c7645b09136
This change replaces all of the ansible shell commands with the
python library, ansible-runner. This library is supported by
upstream ansible, is approved by the openstack foundation, is
supported in global requirements, and provides a better, more
programatic interface into running ansible playbooks.
All tests that interacted with the old shell commands have been
updated to now test using the library.
Change-Id: I8db50da826e2fbc074f4e7986d6fd00f6d488648
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This patch hardly refactors the TripleO Validator CLI. The 'List'
subcommand has been split in different new subcommands which does "one
thing and only one thing". It also adds the support for the TripleO
history file and integrates better the CLI with the openstack client
library, which simplifies it a lot.
The subcommands are:
- To display the full list of validations or by group(s):
```
$ openstack tripleo validator list
```
- To display full details about one validation:
```
$ openstack tripleo validator show
```
- To display the available parameters for one or multiple validations,
export them in a JSON or YAML file:
```
$ openstack tripleo validator show parameter
```
- To display full details about the groups used by the validations:
```
$ openstack tripleo validator group info
```
- To run one or several validations by names or by groups:
```
$ openstack tripleo validator run
```
Depends-On: I56b9d39c113cfb30695fe8cf2740ae147b0dd3e4
Change-Id: I09e2cba484d3a91915fb9294bc351b5d7f3aca0f
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
This patch removes the possibility to run the validations through the
CLI with Mistral. Ansible is now the only way to run the validations.
Change-Id: I8bd9f4c4512279a291963d0b6df62316e7410f2b
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
Until now, we were using Mistral to get all the validations available on
the Undercloud. This patch drops the Mistral support for the validations
listing through the CLI and will parse them directly on the filesystem
instead of parsing from the *tripleo-validations* Swift container.
This change makes the CLI faster:
- ~6s with Mistral
- ~1.3s w/o Mistral
Change-Id: I5730fb2e23775f1d666276b6b4d46bcb550fc928
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
Like we do for the overcloud, add the --inflight-validations option,
disabled by default. Disable by default, we'll skip the
"opendev-validations" Ansible tags when running the playbooks.
Related-Bug: #1844446
Change-Id: Ia37b3d4cc657d994b6a63412d5792930d54a14dd
Instead of hardcoding 'undercloud' for the artifacts and password files,
let's name them:
- Directory: <stack_name>-install
- Archive: <stack_name>-install-%s.tar.bzip2
- tripleo-<stack_name>-passwords.yaml
Depends-On: Iab62ca6bb209524e923ad44a3d8d6079fc976cef
Change-Id: I1e65235017b0b223c53e1c8b48ad2ac573e4ccca
Reset is an application provided via ncurses which is useful in
resetting a terminal when things go wrong. Since podman is eating some
of the terminal output, it can leave the cmdline in a broken state at
the end of the undercloud install. We tried to address it outselves but
let's reuse an existing application instead. -I being passed to reset
skips the initialization strings which can erase the output.
Change-Id: Id9aa7910032588605977f4725385503cf0a235bc
Closes-Bug: #1833302
This patch adds a new parameter called 'gathering_policy' (Defaults to
None) to the 'run_ansible_playbook' function. This parameter will
control the default policy of the Ansible fact gathering. Sets to None
by default, it will use the default policy for Ansible (ie. 'implicit').
Change-Id: I0668241a1675dd4e344cc24b6ff2cbb8f93b7a45
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
The validations execution will use now a static inventory file to speed
up things a bit. Note that this file will be removed after execution.
Change-Id: Ibdc3fc7ea9d3dec9531a2ce29ec8a1477c33c780
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
When using local connection, all the validations targeting the overcloud
nodes are failing. Let's use the connection mode from the Ansible
inventory.
Change-Id: I188604f7767b51a0f485bd79ff3497af56586b13
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
This patch adds a new parameter called 'plan' (Defaults to "overcloud")
to the 'run_ansible_playbook' function. It will allow to execute
Validations with Ansible in a different plan or stack that the default
one through the TripleO CLI. Note That it was already possible only
while using Mistral but not for Ansible.
This patch also brings:
- Change the default values of the 'tags' and 'skip_tags' arguments
to 'None' and fixes their non Pythonic tests.
- Add '--stack' alias to the '--plan' argument for 'validator run'
command.
Change-Id: I6f8f55963f3f5261ec1497b650e0ca509d31dd32
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
The 'openstack tripleo validator list' subcommand can now get only the
available parameters for the validations using the new --parameters
argument.
```
$ openstack tripleo validator list \
--parameters \
[--validation-name <validation_id>[,<validation_id>,...] |
--group <group>[,<group>,...]]
```
Here is an output example:
```
Waiting for messages on queue 'tripleo' with no timeout.
{
"undercloud-cpu": {
"parameters": {
"min_undercloud_cpu_count": 8
}
},
"undercloud-ram": {
"parameters": {
"min_undercloud_ram_gb": 24
}
}
}
```
The --create-vars-file allow the operator to generate either a JSON or a
YAML file containing only the parameters of one or multiple validations.
This file will be available to pass as extra vars to the validations
execution.
```
$ openstack tripleo validator list \
--parameters \
--create-vars-file [json|yaml] /home/stack/myvars \
[--validation-name <validation_id>[,<validation_id>,...] |
--group <group>[,<group>,...]]
```
Change-Id: I6e2255c0d490ee8105f0757d02f5d8fba1d4fa20
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
We're seeing some escape sequences being dropped when we run the
deployment via the undercloud. This causes a user to have to run a reset
after the undercloud installation in order to be able to use their shell
command line again. We can avoid this if we send an terminal erase line
sequence to trim the rest of the line at the end of the deploy.
Change-Id: Ibb74bfebfaab2cd122abe1bc3cd5bbd2562ccfc5
Closes-Bug: #1833302
Add the ability to install an undercloud minion which is connected to an
original undercloud. This minion can have either heat-engine or
ironic-conductor deployed on it.
Adds two new openstack commands for the minion install and a new
minion.conf can be used to configure them.
openstack undercloud minion install
openstack undercloud minion upgrade
Depends-On: https://review.opendev.org/#/c/656984
Change-Id: I61832f5088be172eaf31b36a9cca8dc289580bb2
Related-Blueprint: undercloud-minion
The tripleo deploy command was duplicating a function call to the heat
stack that we already had a shared function for in utils. Let's use that
instead as it now has additional retry logic improvements.
Change-Id: I1f22b5b4aa79e5a1f7d340054e08423cd1b1c090
in running unittests on box with no sudo and
ansible 3 symlink missing ran into this issue.
There were 2 missing mock guards on unittests
that triggered this edge case
Change-Id: Ic4869cf8544e96bc38b26923fadc13be7c0970f9
Closes-Bug: #1819217
The symlink needs to be created with sudo because if validations are
enabled, the call to run ansible-playbook-3 is before the "sudo tripleo deploy
command".
This patch moves the symlink into a function, that will be called before
the preflight if enabled and during the tripleo deploy/upgrade command,
which should cover standalone deployment and upgrade.
Change-Id: I1ce9b3df6c937cd796728d4fc5921fcf1023e5db
This patch tries to reuse the same function _launch_ansible
for the multiple ansible executions in tripleo_deploy.py by
passing a list with the arguments to be used by the
ansible-playbook command.
Change-Id: Iad2c9b82478ae05d68fcfd5880270105c68abefb
Run a hostname preflight check when running the standalone as this can
cause issues with rabbit/mysql/pacemaker/puppet. The standalone check is
not run when an undercloud is being deployed or if output only is
specified. The undercloud has a different set of preflight checks run
prior to deployment so we do not need to rerun the hostname check when
the undercloud is being deployed.
Depends-On: https://review.openstack.org/#/c/634836/
Change-Id: I4de51d792f004575e0e802a1d340478d6ae9f848
Closes-Bug: #1814564
Standalone does not use any of the composable networks by
default. Deploy Standaloen using /dev/null as network data
so that these resources are not included when creating the
plan.
Undercloud uses only the External network for the external
VIP. Deploy the undercloud using the Undercloud specific
network_data_undercloud.yaml, ensures external_from_pool.yaml
is in the plan.
Related-Bug: #1809313
Depends-On: Ib11a134df93e59947168b40bc71fb1da9172d4ac
Change-Id: I102912851a3b9952daaf7c4d5a34a919f527f805
When rendering templates, check for services mapped to OS::Heat::None
and rewrite roles data to remove them. It reduces the numbers of
resources created, and improve in particular container image building.
Change-Id: I37aa673937681d5142feb74e89ca85fdb471db57
In order to be able to properly configure the python interpreter we want
to use for the ansible deployment process, we need to expose it to the
deployment process. If a user is deploying from a CentOS7 host to a
fedora28 host, we need to be able to specify that we want to use
python3. The opposite might be true for Fedora28 host to CentOS7. For
the undercloud or standalone deployment methods we use the version of
python that the openstack client is running under as our default. For
the overcloud deployment, we currently do not specify anything by
default which will fall back to the ansible default. This may need to
change as we begine to run deployments against python3 hosts in CI. But
for now this change lays the framework to be able to specify it in the
deployment commands as an option.
Depends-On: https://review.openstack.org/#/c/617793/
Depends-On: https://review.openstack.org/#/c/616203/
Change-Id: I425905110d701bb7e3079f1017a07678eca42456
Related-Blueprint: python3-support
Co-Authored-By: Alex Schultz <aschultz@redhat.com>
This changes the upgrade workflow for the standalone and undercloud
where we now run the post_upgrade_tasks after the deployment.
So the order is upgrade_tasks, deployment steps (docker/puppet),
then post_upgrade_tasks which is the same order as the overcloud.
It will allow us to execute some specific post upgrade tasks on standalone
and undercloud, like removing Docker containers when upgrading to Podman.
Change-Id: I311bfa17a3bbe52850092d501d563d53f7095a24
ramishra