This changes to update the stack without using
the plan and also enables server side env merging
as we don't use the plan-environment.
Also makes changes to call derive params playbooks
without plan.
Depends-On: https://review.opendev.org/c/openstack/tripleo-ansible/+/772197
Change-Id: I8caad3e9185f1c6d23b0941b966192957ca8320b
The updates and upgrades operations should not be executed as "root"
user, and the operator should use the regular deployment user (e.g.
"stack").
This enforces this expected behavior.
Change-Id: If23f12d6ab571bc3a3b41aef3dcbd58f98d80977
Update/Upgrade commands have now a prompt by default that ask for
confirmation before proceeding. It'll prevent an user to run the
command that may cause the problems to infrastructure.
This prompt can be skipped with --yes/-y argument.
Note: putting "UPDATE" and "UPGRADE" in uppercase to make sure this is
visible and clear. We have seen many users running the wrong command and
ending up doing an upgrade instead of an update.
Note2: this prompt will be ported to the upgrade and FFWD workflows to
prevent unexpected execution to prevent potential harm to
infrastructures.
Depends-On: https://review.opendev.org/741480
Change-Id: I838e6748879c668dd004ca2243b7b00b857c2a7b
As we aren't required to re-deploy any more during the converge step
the actions performed by the Upgrade Converge and Prepare are exactly
the same, the only difference is the lifecycle environment file we
use to set and unset parameters.
This patch refactors both classes, the UpgradePrepare and UpgradeConverge
clasess:
- UpgradePrepare: Stop forcing update-plan-only and then try to do most of the
actions from deployment into UpgradePrepare. This change will force config_download
to false (which will avoid running the full deployment and do only an stack update),
then download the config by invoking get_config from utils and lastly enable ssh
into overcloud nodes, as that part is perforemd in deployment only if config_download
is set to true. Add new attributes to UpgradePrepare so we can override them in
UpgradeConverge via inheritence.
- UpgradeConverge: Inherit from UpgradePrepare instead of DeployOvercloud. Set the
right value for the class attributes and let UpgradePrepare class do all the magic.
Change-Id: I6148511eb6ad1e3798a7bf40c721824830c7073d
This change will provide the operator the ability to better control
a given deployment or operational task while leveraging the
tripleoclient.
A utility has been added to sanitize user input. This will ensure
the parsed string is in valid ansible limit format.
Change-Id: I190f6efe8d728f124c18ce80be715ae7c5c0da01
Depends-On: I0056fdbe3d9807e6baf4a1645a632ab9eb1b2668
Signed-off-by: Luke Short <ekultails@gmail.com>
Co-Authored-By: Kevin Carter <kecarter@redhat.com>
This change will set the verbosity consistently whenever a playbook is
executed via the client.
All tests have been updated to ensure that the verbosity setting is always
defined when a playbook is executed.
Change-Id: I35b10d48344c8b7f71186bc529a300f75d7b8d63
Signed-off-by: Kevin Carter <kecarter@redhat.com>
The use of `yaml.load` is deprecated and considered unsafe.
The warning notice:
> /usr/lib/python3.6/site-packages/tripleoclient/utils.py:2349:
YAMLLoadWarning: calling yaml.load() without Loader=...
is deprecated, as the default Loader is unsafe. Please read
https://msg.pyyaml.org/load for full details.
This change removes the use of the deprecated function, which
corrects the issue.
Change-Id: I7b747fd124880f55a30f75e5efb27c22c57ac10c
Signed-off-by: Kevin Carter <kecarter@redhat.com>
The provision command was defaulting to id_rsa.pub, however the deploy
command uses id_rsa_tripleo for initial setup.
When using the deploy command for provision as well, use the public
key, not the private id_rsa_tripleo.
This option was being processed in several different ways, this change
normalize it by creating a single function in the Command class, which
all inheriting methods will consume. Tests have been updated to
accomodate this change.
Related-Bug: #1863920
Change-Id: I221480f3cfc77545a8fcbef777829239c3bad0a0
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This change removes several layers of abstration from our overcloud commands.
By cleaning up these layers we'll improve the understandability of the client
code while also improving the speed of these interactions.
Story: 2007212
Change-Id: I71617d4f47caaee9ce3025191ff0c0ebfe2adf79
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This change removes all of mistral from the config_download_deploy function
by calling the required functions directly.
- All of the inputs within this function have been cleaned up and documented.
- New log entires will be created when using this method giving the deployer
a better overall user experience.
- To ensure we're able to support the ability to reproduce commands, the ssh
args extravar has been moved to an environment variable.
- The methods get_config and get_key have been moved to the utils module.
This was done to help avoid circular imports.
Update methods have been changed to execute without running within a mistral
workflow. This was changed because there's no need to support the multiple
code paths anymore given config_download is now running directly.
Test classes have been updated to reduce duplication.
With the new streamlined execution process we should see improved deployment
times and better visability into the deployment process.
Task: 38422
Story: 2007212
Depends-On: I006291a2465aa4c950abce76f9e5f9459b76e330
Change-Id: Ide1a4503dd2bdd2d5e494cd1eac483b842a21acf
Co-authored-by: Luke Short <ekultails@gmail.com>
Co-authored-by: Dougal Matthews <dougal@dougalmatthews.com>
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This change uses the the generated mistral context to call the underlying workflow
function without needing mistral.
Story: 2007212
Task: 38436
Depends-On: https://review.opendev.org/#/c/706196/
Change-Id: I79ded325b904bf0f60408250611935d229eda426
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This change converts the deployment process to use the new enable ssh admin
playbook instead of the shell commands and a mistral workflow.
Tests have been updated to reflect the use of the new methods.
Story: 2007212
Task: 38427
Depends-On: https://review.opendev.org/#/c/706196/
Depends-On: I5ab7f4d9dc39a74714c9bcfa39fa50cc586e8e08
Change-Id: I5c744e116f00e81f92c014fc1f599ad3db11ada2
Signed-off-by: Kevin Carter <kecarter@redhat.com>
The OpenStack CLI defaults to 1 level of verbosity
which was being directly passed to Ansible methods.
When the CLI is given "-v", the verbosity level is set to 2.
This patch ensures that Ansible will not be verbose by default.
This lowers the amount of RAM a default
deployment/update/upgrade will use along with lowering
log file sizes.
Change-Id: Ic59cc6b2b9b979255c10d71e437afa733517e3e1
Signed-off-by: Luke Short <ekultails@gmail.com>
Added `--overcloud-ssh-enable-timeout` and
`--overcloud-ssh-port-timeout` to the openstack overcloud deploy command
to allow users the ability to tune these values. The ssh port timeout
might need to be tuned if the hardware being deployed is slow to post
and takes longer than 5 minutes to become ready.
These options have been added to:
* openstack overcloud deploy
* openstack overcloud admin
* openstack overcloud ffu upgrade
* openstack overcloud upgrade
Change-Id: I56f6001b5efeed7ccb3918a5c2622c3eca613c07
Closes-Bug: #1858222
The container name needs to be passed with the stack name value to the
update_nodes workflow so that the right config container is used during
the update.
Previously, this was not an issue since all update actions just
defaulted to overcloud-config even if the stack name wasn't overcloud.
However, with the multiple overclouds feature and the multi-stack
feature, we need to make sure the right config container is used that
matches the stack name.
Closes-Bug: #1854991
Depends-On: Id6b4a5913141fc558c40f2cfd29f5e0741544ab4
Change-Id: Ib25d8f8fbfde8fb13bd6518fd2383b7e3ae7abee
Without properly mocking the deployment action, these tests are trying
to execute deployment code including temp file creation which can lead
to package build failures. This change mocks out the deployment take
action for the UpgradeRun so we're only testing the upgrade code
itself.
Change-Id: I3ac295d2e2f86fbb2120f9491d4103434b6ac3da
Closes-Bug: #1838506
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
Fixes an issue wich cause warnings such as the one below on the
console when using python3.
ResourceWarning : unclosed file <_io.TextIOWrapper name='FILENAME'
Closes-Bug: #1836379
Change-Id: I4a80a339b2cc0c3fca992cfc456581a91dbf2eef
Add checking of mechanism driver during prep upgrade.
Prevent upgrading a stack to a version of tripleo templates or
environment that specifies neutron mechanism drivers that are
incompatible with the existing stack.
Change-Id: Ia6085ca63de0d8c48ab9c5b5811f6ac3e5141c85
Closes-Bug: #1823042
This will allow more specialized usage of upgrade run command, similar
to patterns used with 'external-upgrade run', e.g. during
reprovisioning of nodes.
Also, validation on '--skip-tags' is removed because we should no
longer assume what tags are defined by the upgrade tasks, and we
should allow more free-form usage.
Change-Id: Icf672cc0db50ea6daa57d3c5abe131c82553fc5a
Implements: blueprint upgrades-with-os
Currently we have nodes and roles options for upgrade run and update run
but it's a bit confusing. This both options it's same. Depecrated nodes
and roles and a new option limit. The options nodes and roles will be
remove in the future version.
Closes-Bug: #1813810
Change-Id: I4d33e7e5bd4b892219cfc2067e81938e0f6a8668
The tripleo-common allow us to provide a verbosity level
but its not possible to pass it with the cli.
This review catch the --debug option passed to the cli
and provide it to package_update workflow
Change-Id: Ib1fc8f6aab9a0b1a26945d441185b5c8ccb6e7dd
In commit 4cb7a22529 we changed
to set the DeployIndentifier to ''. This looks like an incorrect
change. It would set the DeployIdentifier to '' (from the existing
unique value), if --skip_deploy_identifier is used and this would
result in deployments applied on nodes, whereas we want exactly
the opposite. This checks the existing value in the stack for an
update.
Change-Id: Idd4178aa06aba5668dec6d8ba0427f57dfe5b5b6
Related-Bug: #1759326
Use a single queue for update and upgrade ansible run to avoid
race while consuming both messages from the web wocket
Change-Id: I8b04c2a39d35f530467ddb5f2c4305c08cbc44f1
Closes-Bug: #1804812
This change has two main parts:
* Ensure that when we ssh *via Mistral*, we always use
tripleo-admin. `--ssh-user` argument to any update/upgrade run
commands is now deprecated. The reason i didn't remove it completely
is that how current upgrade commands use ssh needed some mental
untangling, and i want us to keep an easy way back in case my
understanding is wrong here. Deprecation seems safer than outright
removal.
* On upgrade prepare and FFWD prepare, make sure that the
tripleo-admin user is created and authorized.
Note that for tripleo-admin creation and authorization, the
`--overcloud-ssh-user` parameter is used, and that one still
defaults to `heat-admin`, which is correct. So initially we connect
via whatever credentials user provided (which works on Nova+Ironic
envs unless user customized it), and from that point on we always
use tripleo-admin, which is common for all environments regardless
of any user customizations. This is how deployment already works
now, and after this change upgrades will work that way too.
Change-Id: Ib1a75a0a3f3b2a3bec00d8820c3a097620fa6256
Closes-Bug: #1801066
The convention is to use the same queue name ("tripleo") for all
workflows. This can lead to messages showing from other tripleoclient
triggered workflows showing up during message polling if multiple
workflows are running at the same time.
This patch adds a check that will filter out any messages that do not
belong to the execution that is being waited on by comparing the
execution id with the root_execution_id returned in the execution
payload.
Depends-On: Icbe80c338d69efc6ce8fceb0f73f833bec588536
Change-Id: Ie6473d6a1044cdf76552d62645b4d63da2df9398
Related-Bug: #1794277
Currently the container image registry references are passed with
--container-registry-file in prepare but as -e in converge for
the upgrade/update/ffwd-upgrade/ceph-upgrade clis.
There is no special handling of the file, and it is ultimately
included as any other environment file [1].
This removes the --container-registry-file parameter from
openstack overcloud [upgrade|update|ffwd-upgrade|ceph-upgrade]
prepare.
The related tripleo-common review is for removal of the parameter
from the mistral workflow and action
The related tripleo-docs patch is for updating the operator
instructions to include the container image registry references
using -e as any other environment file.
[1] 2c83dc964b/tripleo_common/actions/package_update.py (L60)
Related: https://review.openstack.org/570903 tripleo-docs
Partial-Bug: #1785825
Co-Authored-By: Jiri Stransky <jistr@jistr.com>
Depends-On: https://review.openstack.org/571186
Change-Id: Id2811dbef59d1be2a35cea062eb7116648f52145
The parameter was necessary before we inherited the update/upgrade
prepare commands from the deploy command. Nowadays this is not
necessary and can be achieved by 2 different means: either via the
*-prepare.yaml files setting CephAnsiblePlaybook parameter, or via
external_update_tasks and external_upgrade_tasks.
Partial-Bug: #1783949
Change-Id: If54e12dbac1a982e89d29b96d86f9f5299e3d810
Depends-On: If80f9cc2f4cb32610dc5eb87159872e28363a21a
New `openstack overcloud external-update run` and `openstack overcloud
external-upgrade run` commands are defined. These are meant to perform
updates and upgrades for services deployed via
external_deploy_tasks. A separate command is used because external
installers don't fit well within the --nodes and --roles selection
pattern we've established for the normal `update run` and `upgrade
run` commands.
Partial-Bug: #1783949
Depends-On: Ib2474e8f69711cd6610a78884d5032ffd19ad249
Depends-On: I982032a0eadfbfb7f1eadee9cae26c8cd5fcdbba
Change-Id: Ib2f32ae8ac234b0c25b0e1ff1f8f8d8e041185e0
This patch updates the code and associated tests to change from
directly calling the tripleo.deployment.overcloudrc action to
calling the tripleo.deployment.v1.create_overcloudrc workflow.
Change-Id: Ib100163a5c9a5d2e6cca6cbc5a186e8e5cbc19ce
Depends-On: I6074b4085b5b77f010f5f2972e7b120aa344f6c5
Partial-Bug: 1640436
This submission regenerate the overcloudrc file
when executing the upgrade prepare or the ffwd-upgrade prepare.
Closes-Bug: 1768036
Change-Id: If64e5f5fb329411ed8d7acd2235295ca924a90c5
The stack name being mocked was not matching
with the one used in the args. This didn't fail
because at the time of asserting mock_upgrade
we were passing still the wrong stack name, not
the one from the arguments.
This patch also adds the ceph_ansible_playbook
arg to verifylist.
Change-Id: I854ce652995c3381c389d5f4d7e6b68d881f78bb
This explicitly adds the upgrade-prepare.yaml environment file
or the upgrade-converge.yaml environment file, into the stored
swift plan, before continuing with the previous workflow (here
heat stack update in both cases). See dependency below in tht
and this tie-in is the biggest downside I can see.
The idea is to stop doing it in the tripleo-common like [1][2]
and move all the upgrade|update|ffwd-upgrade cli to use the
same and include the expected environment file.
Co-Authored-By: Lukas Bezdicka <social@v3.sk>
Co-Authored-By: Jiri Stransky <jistr@redhat.com>
[1] 6090d32b51/tripleo_common/actions/package_update.py (L62-L76)
[2] 6090d32b51/tripleo_common/actions/plan.py (L492-L502)
Depends-On: Icfe494e3219d6d6cd3251f75bb4329fc4d793c3c
Change-Id: I1288fe68ae8af02a5d77390d237ec467d88e43d2
When config-download becomes our only way of deploying, we'll be able
to assume that tripleo-admin user always exists. Until then it's not a
given, so we need to allow using custom user names for running updates
and upgrades playbooks. We default to heat-admin for now, as that's
expected to work on majority of current production environments.
Change-Id: I0df57002b2305c1e2504c9f7a7d0c326d8ffcaf7
Closes-Bug: #1759845
This exposes the --skip-tags which is useful for skipping the
service validations (--skip-tags validation). The expected
format is a comma-separated string for multiple values,
though 'validation' and 'pre-upgrade' are the only currently
supported tags for --skip-tags in the upgrade playbooks. The
full list of tags currently used in tripleo-heat-templates
upgrade_tasks is at [0]
openstack overcloud upgrade run --nodes foo
--skip-tags validation
[0]: 3eb0c62e47/tools/yaml-validate.py (L167)
Change-Id: Ie7fb8d9a388c6d53a31800406b03ddb8ed426401
Depends-On: I8544de64d3307e3dc925c1cecf2d9156e31d25a8
Currently --nodes and --roles have nargs="+" which is one or
more. Furthermore these are made into a list by argparse [1].
Passed as a list into the ansible playbook invocation will
not work.
Instead use defaults, remove nargs so one argument, which is a
string (the default) and which can be comma delimited if you want
to specify more than one --nodes "one, two" or --roles "foo, bar"
This will be passed to ansible-playbook verbatim in [2] and this
is the supported format "comma, delimited, string" for multiple
values.
[1] https://docs.python.org/3/library/argparse.html#nargs
[2] 3ac0ae3ec2/tripleo_common/actions/ansible.py (L387)
Change-Id: Ic8a9ed228e59845b904a7f839ce2da98ece72b66
This adds a required, mutually exclusive argument group for
--roles and --nodes. --roles replaces the existing --role.
This shifts the semantics of upgrading 'one at a time' (compute)
vs 'all together' (controller) to the operator. You can either
specify roles or nodes but not both. Using nodes for contollers
is not supported but we can only warn.
openstack overcloud upgrade run --roles Controller
openstack overcloud upgrade run --nodes overcloud-compute-0 overcloud-compute-1
#for the really ambitious
openstack overcloud upgrade run --roles Controller Networker
Change-Id: I0418883557113b5a67cfeaf181614dc5fe32ca94
This further refactors the update/upgrades cli and separates
the update and upgrade code. This adds the overcloud_upgrade.py
which defines the UpgradePrepare and UpgradeRun. The entry
points are now:
openstack overcloud upgrade prepare --container-registry-file ...
For the no-op heat stack update to refresh stack outputs and
openstack overcloud upgrade run --nodes foo --playbooks all
For running all the upgrade ansible playbooks. A corresponding
converge sub-command will be introduced in a subsequent patch.
Change-Id: I1880e8f546df8d509871ba3b4f02877e95c611c8
This new library provides the common logic used by openstack
clients. Replacing the imports from openstackclient and cliff.
Depends-On-External: https://review.rdoproject.org/r/#/c/2088/
Depends-On: I095209c241876c6ab48595962e07144fc5cc34a6
Closes-Bug: #1620964
Change-Id: I103a13ff6c0af94b82f5e48652380adfbf331d34
Use the OSC-compatible orchestration client instead of the home-grown
one. This reduces duplicate code, and relies on more thoroughly tested
code elsewhere.
Change-Id: Ib77d20086f86a260197c6f289fd9c270b766544c
Depends-On: I7aa0fd7c152ee91b6e2b956f7e594f8c8aacafa7
Rabi Mishra