This patch fixes 2 issues on MariaDB staged upgrade:
* After we landed service stopping on the data transfer step, the
actual MariaDB staged upgrade moved from step 1 to step 2 of
external_upgrade_tasks, but we only ran steps 0 and 1 during the
upgrade. Increase the number of steps to run step 2 as well.
* The mysql_upgrade container was being re-run on each upgrade-scaleup
even on nodes where it already ran before and MariaDB was already
running there, conflicting with the upgrade container. We fix this
by only running the MariaDB upgrade if there's no MariaDB running
yet on the particular node.
Change-Id: I2feba6c22cdf07ea6af406c5edd27f62ff285d87
Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>
Closes-Bug: #1841415
After bumping the Ansible version in Tripleo to 2.6, it was needed
to perform a change in the include module into include_tasks or
import_tasks, as include was getting deprecated [0]. The external
update/upgrades tasks got impacted by that change, but as they use
a loop to execute the tasks we couldn't use import_tasks.
The way include_tasks handles the tasks execution depending on the
tags differs from import_tasks (dynamic vs static) and as a consequence
when running the external upgrade run passing --tags container_image_prepare
we didn't see any tasks running. This behavior got fix in [1], which
seems to be the right way to preserve the import_tasks tags handling as
explained in [2]. But the external update tasks were missed to patch.
This patch includes the tags: always statement inside the external
update tasks and also syncs the variables content we pass into external_update_tasks
as many of those variables, which we do pass for the external_upgrade_playbook,
were missing in the external_update_playbook.
[0] - https://review.opendev.org/#/c/579844/
[1] - https://review.opendev.org/#/c/639642/1
[2] - https://github.com/ansible/ansible/issues/30882#issuecomment-380596557
Closes-Bug: #1839520
Change-Id: If7e7b4bbb3cead0887384cc543ce37e9ee5396ab
This helps to consolidate the Undercloud and Standalone deployments. It
also avoids an issue where the Ansible Python interperter cannot be
found.
Change-Id: I01a95be975011d2419a523da572503e0ebcfa49b
Resolves: rhbz#1733608
Signed-off-by: Luke Short <ekultails@gmail.com>
Commit a7661065743086961b8ef93056b810e7d2a49eda removed the
all-nodes-config.j2.yaml Heat template. This template had the support
for specifying hieradata to merge into the all_nodes hieradata via the
AllNodesExtraMapData. This commit restores the parameter which is now
set as an Ansible variable which will be used by the tripleo-hieradata
role.
Change-Id: Ib3838d404b45ae6684ef7e0aa211cb8015271866
Instead of writing a set of vars to hieradata_vars.yaml and then passing
that into tasks with vars_files, just set the data as group vars.
This makes for a simpler interface without having to remember to pass in
the data with vars_files on any task/play that might require them.
Change-Id: I851d9404861d55791bde1346c58bc1c94f591569
This var is no longer required to pass into the tripleo-ssh-known-hosts
role as using the template will be the only mode going forward.
Change-Id: I0bde2ac03edf26f5594381caa9542aa1eefd8f20
Depends-On: I4a2002fa42337e115f9679fcf2f2db71308389d6
Migrate to using the template mode of the tripleo-ssh-known-hosts role.
Detailed Changes:
common/deploy-steps.j2:
Remove ssh_known_hosts_hostnames parameter
Remove ssh_known_hosts variable in config-download output
Set tripleo_ssh_known_hosts_use_template var to true when including
the tripleo-ssh-known-hosts role to activate template mode.
extraconfig/tasks/ssh/host_public_key.yaml:
Template removed as it was deprecated and is no longer used.
extraconfig/tasks/ssh/known_hosts_config.yaml:
Template removed as it was deprecated and is no longer used.
overcloud-resource-registry-puppet.j2.yaml:
Mappings for Ssh known hosts resources removed
overcloud.j2.yaml:
Resources removed: SshKnownHostsConfig, SshKnownHostsHostnames,
{{role.name}}SshKnownHostsDeployment
Removed passing in ssh_known_hosts_hostnames parameter to
common/deploy-steps.j2
puppet/role.role.j2.yaml:
Removed SshHostPubKey and SshKnownHostsHostnames resources
Removed known_hosts_entry and known_hosts_hostnames stack outputs
Add role_networks to set Ansible group vars which is a list of enabled
networks for the role
Change-Id: I10d0de42acc0b88b7947b464e976b5d0b9067ca2
I9a37d1faec73a81a28d8f89d86375fb15ee765c7 moved to use ansible
for all_nodes hieradata. This also removed to create the
SERVICE_cell_node_names in in net_ip_list_map.j2.yaml.
This change addes nova_additional_cell to the global_vars that
the all_nodes hieradata via ansible can write cell service
hostnames.
Change-Id: I58c635b0fd92fa5a437cd6abd7c954d9cad720c9
The lookup function that was used previously still failed when these
vars were not defined. Using hostvars instead allows these to be
defaulted to [] when the vars are not actually defined.
Change-Id: I51f3c21f07a893848483a14e399a6d402a6205b8
Use the first_found Ansible lookup plugin to look for the NetworkConfig
script in both the role directory and server directory. This allows for
using a role generic NetworkConfig script in the role dir, while still
being backwards compatible.
The NetworkConfig script is also rendered from a template on the remote
node instead of just copied. Treating it as a template allows it to be
made generic per role instead of node specific.
Depends-On: I6bbca8eb256f387c4f74378baa5253890cfe1abf
Change-Id: Ibbb5dd2ccbdaec083ec636ac44dfdc75b65f89a7
Migrates these hieradata files to use the cloud_domain and fqdn
templates from tripleo-hieradata. The templates have more meaningful
names to actually represent the data that these hieradata files contain.
Detailed changes:
common/deploy-steps.j2:
Add parameter for CloudDomain and add it to the Ansible global vars
Remove the Per-host hieradata tasks and parameters as they are no longer used
Add cloud_domain and fqdn to list of templates for tripleo-hieradata
overcloud.j2.yaml:
Add role_networks to HieradataVars for each role
Replace {{role}} and host_extraconfig from hieradata hierarchy with
cloud_domain and fqdn.
puppet/role.role.j2.yaml:
Remove {{role}} hieradata from hieradata Datafiles as the template is
now used instead.
Remove per host hieradata resources and output as they are no longer
used.
Depends-On: I9c038399fccc4730b73e1a52281d7256ea689ee9
Change-Id: I136fb7aa864a2a3668f5a3845f3ded28b13bfe43
Migrates the net_ip_map hieradata from Heat to the tripleo-hieradata
role.
Detailed changes:
network/ports/port.j2:
cidr output added for the network subnet cidr of the port
port for a role
network/ports/noop.yaml:
cidr output for compatiblity with port template interface
puppet/role.role.j2.yaml:
{{network.name}}_cidr output added for the each networks subnet cidr
Remove the net_ip_map hieradata from the per host datafiles generated
by Heat as tripleo-hieradata template will be used instead.
overcloud.j2.yaml:
consume cidr outputs from the roles, as well as the ctlplane subnet
cidr and set them as hieradata vars that get passed into the
templating process for tripleo-hieradata
common/deploy-steps.j2:
Add a networks data structure to the global vars setting some needed
vars for tripleo-heiradata.
Use the net_ip_map template from tripleo-hieradata
Change-Id: Ib30e8347e1a50efec2b6a97482dc98c383cb7cf7
Depends-On: Iaec3654900f250eacf39c7748df166b716937358
Script is first located/generated on the undercloud node (aka ansible
host), meaning we have to use local_action.
We also deactivate the "become" in order to avoid useless privilege
escalation.
Change-Id: I8c1ed334dc5b578a87307a47656ee2d87f1e3688
Depends-On: https://review.opendev.org/668560
Related-Bug: #1834094
Update the condition to test if NetworkConfig should be run to also
include logic to run if it was never run previously. Matches prior
behavior when using a SoftwareDeployment resource
Change-Id: Ibd2c7d96a6ad8d207cbb484b7e211909f897e43f
Migrates the vip_data hieradata to the tripleo-hieradata role which will
use a jinja2 template to create this file on the deployed nodes. Allows
for removing the AllNodesConfig resource/template completely.
Change-Id: I5412f26423a1739088f12705fa47bac2377b9817
Depends-On: Iaf91f36a189d8e00627c21f14ca992a4d60fa10f
Migrate the generation of the all_nodes hieradata from Heat to using the
tripleo-hieradata role instead.
Change-Id: I9a37d1faec73a81a28d8f89d86375fb15ee765c7
Depends-On: I6e1e1c28dc09c9e04119db910068d62409a5afc8
Instead of generating and saving the hieradata in the bootstrap_node
hieradata file in Heat, use the tripleo-hieradata role instead.
Change-Id: Id6a7b10acc65ca6cd2135796a80fad0723078871
Depends-On: I9ccccbfee44fc7c1375dec7357ccad6cba909c16
Replace the Heat resources which used to manage the hiera config file
and now use Ansible with the new tripleo-hieradata role.
Remove {{server_resource_name}}Deployment and
{{server_resource_name}}Config resources not needed anymore.
Change-Id: I88fe3460af93b83cc086afeb1c2b959bbb720470
implements: blueprint reduce-deployment-resources
Depends-On: Ia5f11ea1a2f16b736ed7053c5182c7f5a7eb1f4b
Use the new tripleo-hieradata Ansible role to configure
the remaining Hieradata files that were created by Heat before.
{{server_resource_name}} isn't yet fully converted to Ansible, and
another patch will remove the resource when Ansible controls the
hiera.yaml file in /etc/puppet/hieradata.
Change-Id: Ib946b566d66300d23e7abc12498cc11d3f5fa5f8
implements: blueprint reduce-deployment-resources
The new AnsibleHostVarsMap output is a map of roles to Ansible host
vars, which later will be in config-download and populate the Ansible
host vars for each host in each role.
Change-Id: I3880f0f72beb24ee0b0868dca48afce6328144d3
Replace the HostsConfig SoftwareDeployment with an ansible task.
implements: blueprint reduce-deployment-resources
Change-Id: Ifd4bc4ce5618587c341ecbf37f82777ae6fc2f4a
Replace the Heat SoftwareDeployment with an ansible task for running the
deploy-artifacts.sh script.
Change-Id: Ib5f114440fcfcb7a7652111f0651ef01761d2601
implements: blueprint reduce-deployment-resources
Use ansible to apply the NetworkConfig resource instead of the
NetworkDeployment resource. The script module is used to run the same
rendered script as NetworkDeployment. In the future the script could be
refactored into a more proper ansible role, but the first step is to
remove the SoftwareDeployment for NetworkConfig and just use an ansible
task.
Change-Id: Ie60ddb90562f9630c24711337473f9e3acdcf4c1
Depends-On: Ie543782569de14d56bc41740611f7512e8357a22
implements: blueprint reduce-deployment-resources
Replaces the Heat SoftwareDeployment resoure for
AllNodesValidationConfig with an Ansible stask instead.
Depends-On: I0b3188ef45dc379f83c36561616b422418845b66
Change-Id: I6e8f8f29cf9ef906b9be55f27aa536d8cc200190
implements: blueprint reduce-deployment-resources
Write both paths with and without the .yaml extension until
https://review.opendev.org/#/c/663736 merges. This patch will allow the
tripleo-common patch to merge, then we can come back and clean this up.
Needed-By: I0b3188ef45dc379f83c36561616b422418845b66
Change-Id: I1434da248bb9bc06e38381a47f274e1480fd450c
Use a gather_facts variable to control fact gathering at the play level
instead of just the hardcoded gather_facts:no.
This allows for forcing facts to be gathered during a play if the
earlier fact gathering tasks had been skipped (such as when using
--start-at-task).
Change-Id: I2b7625baac4a09413cc50b84d2df1c00ec14b00a
During the run of external upgrade playbook, we connect to systems
with different OS releases via delegate_to, which seems to confuse the
autodetection of Python interpreter. Set it explicitly to
platform-python which should work both on EL7 and EL8.
Co-Authored-By: Sergii Golovatiuk <sgolovat@redhat.com>
Change-Id: Ieb486f0c3e4de7229cdd2be49d4d0f038668f06d
Closes-Bug: #1830713
By introducing update_serial variable we parallelize update
execution on non-pacemaker enabled nodes. Custom role data users
need to update their role files. By default we do serial 1 making
sure nothing changes for users who didn't update their role data.
Resolves: rhbz#1652057
Closes-Bug: #1831617
Change-Id: I4ee0110a6c2b9466d81e37e5df27f5f81a6eceb5
Replaces the SoftwareDeployment resource for AllNodesDeployment with the
usage of the new tripleo-hieradata role.
Change-Id: Ic174699ef038ca3028c21a49373658a5d43844f6
Depends-On: I28b5c2bedbba1c27da628c6bda4c7a57ca0f3ad7
implements: blueprint reduce-deployment-resources
There seems to be a difference between how include_tasks and
import_tasks work.
`include_tasks` applies properties on the inclusion itself, and if we
want to apply something on the included tasks, we need to use `apply`
dict. We previously had to add `always` tag onto `include_tasks` to
make the inclusion happen when we ran upgrade with `--tags`.
`import_tasks`, on the other hand, is processed more like a block than
an individual task, and all its properties get applied on the tasks
inside. This meant that the `always` tag got applied on all tags in
the upgrade playbook, instead of on the import itself, which broke use
of tags in `upgrade_tasks`. This is now fixed by removing the tags
from `import_tasks`. The import should happen always regardless if
there are any `tags` on the import.
Change-Id: I66a4ed99f9e0cc199899494813073b4a085d99e7
Closes-Bug: #1830892
Using EndpointMap to ensure we get the hostname/fqdn if possible
otherwise it fallbacks to the IP for Keystone public endpoint.
This is useful when the operator uses a certificate based on
hostname/fqdn and not an IP address.
Closes-Bug #1763776
Change-Id: Ifa9d55cca90caf5be0c83507cb47447e25311fce
The problem we want to selve is that the change
https://review.opendev.org/#/c/631486/ (moving iptables creation to the
host) never really worked.
The reason it never worked and we never noticed is two-fold:
A) It ran: -e include ::tripleo::profile::base::haproxy
the problem is that without quoting puppet basically does a noop
B) Once the quoting is fixed it breaks because 'export FACTER_step'
exports a custom fact but does not export a hiera key per-se (so calls
to hiera('step') would fail
So we add proper quoting only on the variables that are arguments to a
parameter so that there is no risk of ansible doing the wrong thing and
puppet gets the correct arguments.
We also explicitely set the step for hiera in the deploy_steps_tasks.
The reason we need it is because in non-HA the iptables rules would
be created at step 1. But since the deploy_steps_tasks run before the
actual tasks that set the step hieradata.we would get the following
error:
Error: Function lookup() did not find a value for the name 'step'
We can just write out the step hiera key during the deploy_steps_tasks,
it will be enforced again shortly afterwards once the
common/deploy-steps-tasks.yaml gets invoked.
We also switch back to puppet_execute: ::tripleo::profile::base::haproxy
even for the pacemaker profile. This was broken by the flattening of the
haproxy service (Id55ae44a7b1b5f08b40170f7406e14973fa93639)
Co-Authored-By: Luca Miccini <lmiccini@redhat.com>
Change-Id: Iab310207ca17a6c596470dda30a39e029c4fe09c
Closes-Bug: #1828250
Allow to override serial parameter for parallel execution
in Ansible
Needed-By: Iafd7c58aaf2f1b30cf46cd16122b5821c96c497c
Needed-By: I15511b4f36260292e0ea4100b15b8e65a701b38b
Change-Id: I707282bf3cfcc7e370af41f987700e39c1b0b268
This adds the ServiceNetMap value to global_vars for config-download.
This will make the value consumable from ansible tasks when running
config-download.
Additional values can be added in a similar fashion in the future to
allow for less hardcoded data coming out of Heat, and instead using
jinja expressions to consume data from Ansible variables when
config-download runs.
Change-Id: I8c442caac140f1c96123c1be47e858949419fd8f