This commit is to remove --deploy and --state from
'software deploy show' as they are not being used.
Test Plan:
PASS: build and deploy the iso on SX.
PaSS: verify both are removed.
Task: 51437
Story: 2010676
Change-Id: I490b5cbfdbf424c1e5f9e71488280a38c269ed08
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This commit introduces a progress indicator for the `do_upload` and
`do_upload_dir` commands. A `WaitThread` is used to display a progress
bar during long-running operations, such as uploading a software file
or directory.
Example output:
[sysadmin@controller-0 ~(keystone_admin)]$ software upload-dir patches
This operation will take a while. Please wait.
.............................................
+------------------------------------------------+-------------------+
| Uploaded File | Release |
+------------------------------------------------+-------------------+
| starlingx_upgrade_test_2024-11-03_12-00-06.iso | starlingx-25.03.0 |
+------------------------------------------------+-------------------+
Test Plan:
- PASS: Upload a patch using the `software upload` command. Verify the
progress bar is displayed and the execution finishes successfully.
- PASS: Execute the previous test with an .iso and .sig file.
- PASS: Run the `software upload-dir` command. Verify the progress bar
is displayed and the execution finishes successfully.
Closes-bug: 2089594
Change-Id: I3734c6389f945f8532c91ecf132004ff1d74ede8
Signed-off-by: Enzo Candotti <enzo.candotti@windriver.com>
The software upload command fails due to a timeout while waiting for
the HTTP response.
This issue arises specifically when uploading larger patches,
such as a 1GB file, for example:
software upload test.patch
To address this, increasing the timeout to the existing
UPLOAD_REQUEST_TIMEOUT (30 minutes) resolved the problem.
In a VBOX AIO-SX environment, the system required almost 10 minutes
to complete the upload and 7 minutes 30 seconds to delete the patch.
As a potential improvement, implementing the upload as an
asynchronous command could further enhance performance,
especially for large file uploads.
Test Done:
Tested software upload on AIO-SX with a 1GB patch file
Story: 2010676
Task: 51253
Change-Id: I46018bb6b139a0d067a74df2c39e33e0fc4dc347
Signed-off-by: Fabiano Correa Mercer <fabiano.correamercer@windriver.com>
Software client ignores return status from software api request,
when using 'sudo' or prebootstrap. This results, 'sudo software
deploy start' to return, rc=0 even on error.
This review fix http client to use status code from software api
to return correct code for commands on failure for 'sudo'.
Test Plan:
PASS: Verify prebootstrap rc for error and success,
'sudo software deploy start <release-id>'
PASS: Verify prebootstrap rc & error message for target release
that is already deployed (Eg. starlingx-24.09.0)
PASS: Verify prebootstrap software patch deploy and removal.
Story: 2010676
Task: 51191
Change-Id: I6099445adeaa9de2c859205ff2fbba03524aed84
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
This commit changes --local flag behavior in 'software upload'.
After the change, --local flag can be set on the remote terminal.
When the flag is set, the 'software upload' execution will look for
the specified files on the active controller. The file extension
validation is done on client side, while the file existence validation
is run on the server side.
Test Plans:
PASS: build and deploy iso on SX
PASS: upload files that are existing on server with --local set
PASS: upload files that are not existing on the server with --local set
PASS: upload files without --local set
Story: 2010676
Task: 51183
Change-Id: Ibaf4c0c9dc8fdab025067bc9fa97e1839323ac01
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This commit fix the current behavior of software bash completion which
was not completing for the software deploy subcommand.
PASS: Software deploy <tab><tab> list all options.
PASS: Software deploy st<tab> completes with 'start'.
PASS: Software deploy start <tab><tab> list flags.
Story: 2010676
Task: 51162
Change-Id: Icfcddbe861a1f0eeb309e3d0e105bdd3989cb768
Signed-off-by: Luis Eduardo Bonatti <luizeduardo.bonatti@windriver.com>
Updated tox configs on software and software-client
and update zuul jobs to call python tests on them using
their individual configs.
Fixed a few flake8 issues on both directories.
software-client pylint test set to non-voting for now.
Test Plan: In a venv:
pass: tox -c software/tox.ini -eflake8
pass: tox -c software/tox.ini -epylint
pass: tox -c software-client/tox.ini -eflake8
Story: 2010676
Task: 51082
Depends-On: https://review.opendev.org/c/starlingx/update/+/930489
Change-Id: I3b8e2612c4bc579e2284383f14b2879a07d0d7bd
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
This commit add a check for the commands software deploy start,
deploy-precheck to ensure it to be executed on controller-0.
Test Plan:
Test executed for major release:
PASS: Software upload blocked on controller-1
PASS: Software deploy start blocked on controller-1
PASS: Software deploy precheck|start on controller-0
PASS: Software upload on controller-0
Story: 2010676
Task: 51081
Change-Id: Ibb0d3445e2f1b3b71df24aaacb35d3fd06afe307
Signed-off-by: Luis Eduardo Bonatti <luizeduardo.bonatti@windriver.com>
The 'software help' command is exposing the commit-patch subcommand
that isn't fully implemented yet.
This commit removes the exposed subcommand 'commit-patch' from the
CLI subcommand options.
Closes-bug: 2081828
Test Plan:
PASS: Run the "software help" command and verify that 'commit-patch'
does not appear in the subcommand options.
Change-Id: I4457e9c12ce8557527ce5e38a6e5a6fe5cdf96f6
Signed-off-by: Humberto Wanderley <humberto.wanderley@windriver.com>
This commit improve the completion for software command and software
deploy subcommand, changes the bash_completion function to distinguish
between software command and deploy subcommand.
The code and behavior is based in the completion for the system command.
Test Plan:
PASS: software <tab><tab> options is listed.
PASS: software deploy <tab><tab> options is listed.
PASS: Checks if software completion works.
PASS: Checks if software deploy completion works.
PASS: Checks if flags is listed for software and deploy.
Story: 2010676
Task: 51038
Change-Id: I503bfc011ea29a4843d132078b45461c32818c49
Signed-off-by: Luis Eduardo Bonatti <luizeduardo.bonatti@windriver.com>
The return message in case of the iso or sig file doesn't exist was
also returning "cannot unpack non-iterable int object". This commit
fix this behavior.
Test Plan:
PASS: Software upload with non-existent files.
PASS: Software upload successfully.
Story: 2010676
Task: 51023
Change-Id: I8d487eee10d1471344ea3644d438c93bd880aa6c
Signed-off-by: Luis Eduardo Bonatti <luizeduardo.bonatti@windriver.com>
The 'software help deploy' command output does not seem to be formatted
correctly compared to other software commands.
This commit adjust the messages displayed according to standards.
Closes-bug: 2080517
Test Plan:
PASS: run "software help deploy" command and verify that formatting
is correct.
Change-Id: I4c53593252d9b70c275f3ea042a4a94302ad64d8
Signed-off-by: Humberto Wanderley <humberto.wanderley@windriver.com>
Deploy subcommands are overwriting release commands with same name
like "software help show" and "software deploy help show"
This commit adds a "deploy" prefix on dict used by help, avoiding
overwriting
Closes-bug: 2080375
Test Plan:
PASS: run "sotware help show" and verify that the output is correct.
PASS: run "software deploy help show" and verify that output is
correct.
Change-Id: Ic644cd648deb5348388636054868c0e0c70497e4
Signed-off-by: Humberto Wanderley <Humberto.Wanderley@windriver.com>
An extra "/" was appended to the SystemController usm internal endpoint
which resulted in an incorrect request URL.
Test plan:
PASS: Successfully run "software --os-region-name SystemController show"
PASS: Successfully run "software --os-region-name RegionOne show"
PASS: Successfully run "software show"
PASS: Successfully run "software --os-region-name SystemController
is-deployed"
PASS: Successfully run "software is-deployed"
PASS: Successfully run "software --os-region-name SystemController
delete"
PASS: Successfully run "software delete"
Closes-Bug: 2080067
Change-Id: I4df3b17ab6ebff83a7b8849af3692752ff104dad
Signed-off-by: lzhu1 <li.zhu@windriver.com>
Current configuration of software-client chooses public usm
endpoints by default. This is causing ssl errors when user
has HTTPS enabled w/ self-signed certificates, which
differs from other clients (e.g. cgtsclient).
Removed the code that set this, so that usm client uses the
one from env var 'os_endpoint_type'. This is set to 'internal'
when admin credentials are sourced.
Test plan:
PASS: Deploy DC + SX subcloud.
Change ssl cert to self-signed one.
Source admin credentials.
Execute 'software' commands w/ debug option in the
SystemController and subcloud and observe that they are
reaching the internal usm endpoints:
- software --debug list
- software --debug show <release>
- software --debug deploy show
- software --debug deploy host-list
- software deploy precheck <release> (tested by Heitor)
- software deploy start <major_release> (tested by Heitor)
PASS: Upgrade AIO-SX: stx 9 -> stx 10.
(code applied on both sides)
Story: 2009811
Task: 50722
Change-Id: I72e70725c0e82ec4cb43a87e006b12b1e4c55c1f
Signed-off-by: Marcelo de Castro Loebens <Marcelo.DeCastroLoebens@windriver.com>
This commit adds the option --all to 'software delete' command.
Usage:
software delete --all component-MM.mm
When using this option, all releases (patch or major) from
component-MM.mm will be deleted if all of them are in available
or unavailable state.
TestCase:
PASS: 'software delete --all'
> usage: software delete [--all] release [release ...]
PASS: 'software delete --all starlingx-24.09.1'
> The release ID must have the pattern: component-MM.mm
PASS: 'software delete --all starlingx-24.09 wrcp-24.09'
> --all require only one release ID
PASS: 'software delete --all starlingx-25.03'
with no starlingx-25.03 release.
> There are no starlingx-25.03 releases to delete.
PASS: 'software delete --all wrcp-24.09'
with at least one release not available or unavailable,
return a list of releases that are not ready to be deleted.
> Releases wrcp-24.09.0,wrcp-24.09.1 not ready to delete
PASS: 'software delete --all starlingx-24.09'
with all releases available or unavailable
> starlingx-24.09.0 has been deleted
> starlingx-24.09.1 has been deleted
> starlingx-24.09.2 has been deleted
Story: 2010676
Task: 50732
Change-Id: I37e9df297cdaa22653e8ad431a88a3c2dc18a643
Signed-off-by: Lindley Vieira <lindley.vieira@windriver.com>
Add support for installing a patch release to local host that is a
simplex or duplex system and fully installed. Installation should
support both locked or unlocked controller. This is similar to the
pre-bootstrap case, except that controller will have hostname,
networking etc. enabled.
This change will add a 'install-local' mode, that is enabled by
software command 'software install-local'. In this mode normal
software commands upload|start|deploy host|deploy activate|deploy
complete|deploy delete, can be used to install patch releases to
local controller.
The 'install-local' mode will be exited on successful completion of
'software deploy delete' or using 'software install-local --delete'
Test Plan:
PASS: Verify local patch deployment to SX in locked & unlocked mode
PASS: Verify local patch deployment to DX in locked & unlocked mode
PASS: Verify normal patch deployment in SX & DX
Story: 2010676
Task: 50612
Change-Id: I56674656dc9d5efbee7639459a8cea71fdaecf0f
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
Currently the exception returns http status
code 500, which doesn't help differentiate
between usm internal error or api call
rejected error.
Updating client code so that error message is
displayed properly.
Test Plan:
[PASS] Check proper status code is returned for
software deploy start
[PASS] Check proper status code is returned for
software deploy precheck
[PASS] Check proper status code is returned for
software deploy activate
[PASS] Check proper status code is returned for
software deploy complete
[PASS] Check proper status code is returned for
software deploy abort
[PASS] Check proper status code is returned for
software deploy activate_rollback
Story: 2010676
Task: 50481
Change-Id: Iaa90cf4706cd2ce930089fb5f69c6d674e369bfb
Signed-off-by: sshathee <shunmugam.shatheesh@windriver.com>
This commit implements the software deploy active-rollback command.
When deploy abort is executed after deploy activate, this command
is need to follow to deploy host-rollback. It will execute the upgrade
scripts with action activate-rollback. Deploy state is changed to
"activate-rollback-pending" after deploy abort, updated to
"activate-rollback" when activate-rollback is in progress and is
updated to "activate-rollback-done" when it completes with success
and "activate-rollback-failed" otherwise.
This commit also make some changes to the scripts log at software.log
file.
Test Plan:
PASS: activate-rollback with major release completed successfully
PASS: activate-rollback with major release failed
PASS: activate-rollback with major release after command failed
PASS: log related to activate_rollback is registered on software.log
Story: 2010676
Task: 50297
Change-Id: I5ede23229e0217f6eb98e27570be4ff40fdafdd8
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
This commit applies custom formatting enabled by [1] to
deploy host-list output, adding prefix "deploy-host-" to
the host states.
[1] https://review.opendev.org/c/starlingx/update/+/922245
Test Plan
PASS: run "deploy host-list" and verify the output
Story: 2010676
Task: 50392
Change-Id: Ie100acd99470af3e9e69bcbb6bcc6080445055df
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
This commit adds the capability to apply a custom formatting
function to the values returned by the API before presenting to
the user, and applies it initially on the deploy states.
Below is an example of the before/after:
Before:
+--------------+------------+------+-------+
| From Release | To Release | RR | State |
+--------------+------------+------+-------+
| 24.09.0 | 25.03.0 | True | host |
+--------------+------------+------+-------+
After
+--------------+------------+------+-------------+
| From Release | To Release | RR | State |
+--------------+------------+------+-------------+
| 24.09.0 | 25.03.0 | True | deploy-host |
+--------------+------------+------+-------------+
Test Plan
PASS: run deploy show and verify the output is correct
Story: 2010676
Task: 50368
Change-Id: Iaa06e796bd8661c05e3086e7c16cf39b09fde274
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
Currently the rejected deploy start returns
http status code 200, which is not ideal for
integration apps like VIM. Changing the status
code according to error.
Updating client code so that error message is
displayed properly.
Test Plan:
[PASS] Check proper status code is returned for
error
[PASS] Check error message is displayed properly
in terminal
Story: 2010676
Task: 50336
Change-Id: Ifdd42f06fec487b59e78c07beab8d541391d8ad0
Signed-off-by: sshathee <shunmugam.shatheesh@windriver.com>
This commit adds the commands to the software client
that will be used to rollback hosts to the previous
deployment ostree commit-id. The backend logic for
this command is not covered by this commit and will
be implemented in a follow-up commit.
The commit fixes minor tox issues as well.
Test Plan
PASS: run host-rollback command and verify the output
returned is correct
Story: 2010676
Task: 50201
Change-Id: Ib1267f493e274c7884d4e5ec3222050a96f42cc5
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
This commit implements the software deploy abort, it checks if the
deployment is in one of this states: activate, activate-done,
activate-failed, completed, host, host-done and host-failed.
Changes the order of from_release and to_release in order to
start the host-rollback so the target release will be the
from_release. It also changes the commit_id and feed_repo based on
from_release.
Set the correct state for the deploy, in case of a host-failed
a check against the sw_version will be needed to set the state as
rollback-deployed in case of the sw_version is the same as
from_release, or set rollback-pending otherwise.
Raises an exception case deployment is not major release because
the abort is not supported to patch releases.
Test Plan:
PASS: Deploy abort sets the correct data and states at
software.json after unlock of host deploy.
PASS: Deploy abort sets the correct data and states at
software.json before unlock of host deploy.
PASS: Right to_release and state with deploy failed state.
Task: 50223
Story: 2010676
Depends-On: https://review.opendev.org/c/starlingx/update/+/920763
Change-Id: Idce97e049fdadac2d3041a5f29488a9d77774493
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
Software client is updated to use correct 'Content-Type' when using
sudo software upload command with --local option.
Test Plan:
PASS: Verify software upload command with & without --local option
Closes-Bug: 2068555
Story: 2010676
Task: 50282
Change-Id: Iaca6db5e3f502c08c4009e6833599780dc698216
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
This commit adds the function to delete a deployment.
There are some validations and actions to be made regarding
each state of the deployment.
Deploy state completed:
Delete the from_release data, the deploy host entity and
deploy entity.
Deploy state aborted:
Delete the to_release data generated, the deploy host entity
and the deploy entity.
Deploy state start_failed or start_done:
Check if deploy host state is one of: [pending, failed] and
if all nodes are running from release software.
Delete the to_release data generated, the deploy host entity
and the deploy entity. If the release is major it will call the
deploy-cleanup scripts to delete the
/sysroot/upgrade/ostree_repo and /sysroot/upgrade/sysroot.
The data generated that was mentioned are the folders
[armada, config, deploy, fluxcd, helm, .keyring, puppet,
sysinv, nfv/vim] under /opt/platform/<folders>/<sw_version>
which will be deleted in case of a major release for all the
mentioned states [aborted, completed, start_done, start_failed].
Also will be deleted /var/lib/postgres/<sw_version>,
/var/lib/postgres/upgrade, /opt/etcd/<sw_version>,
/var/lib/rabbitmq/<sw_version> in case of major release deployment.
Test Plan:
PASS: Software deploy delete for patch release deployment.
PASS: Cleanup of staging data.
PASS: Failed to attempt deletion with a deployed host N+1 release
PASS: CLI command works with success.
PASS: Software deploy delete for a GA release deployment.
Story: 2010676
Task: 49979
Change-Id: I1789172edc730e6c94fa6bec7f5881c0bdfd7eab
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
Software cli uses HTTPClient when endpoint and token are provided
in the command line. HTTPClient fails when doing upload file/s
as it is not using passed token and insecure option.
This change updates HTTPClient to use token and insecure option
when using upload/upload-dir commands and remove some unused
functions from the client.
Test Plan:
PASS: Verify upload & upload-dir commands using software cli
(Eg. software --os-auth-token=$TOKEN \
--software-url=$URL upload-dir <dir>)
PASS: Verify upload & upload-dir commands using insecure cli
(Eg. software --insecure --os-auth-token=$TOKEN \
--software-url=$URL upload-dir <dir>)
PASS: Verify ca-file, cert-file, key-file options for upload
(Eg. software --ca-file=<ca file> \
--key-file=<key file> --cert-file=<cert file> \
--os-auth-token=$TOKEN --software-url=$URL \
upload-dir <upload dir>)
Closes-Bug: 2067293
Story: 2010676
Task: 50197
Change-Id: I32264cfd2178d8b75b02d4d1703c5e36c94828d3
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
Software cli should not display 'packages' attribute unless
'--packages' option is specified for command, 'software
show <release>'.
Test Plan:
PASS: Verify packages are absent for 'software show <release>'
PASS: Verify packages present, 'software show <release> --packages'
Closes-Bug: 2066037
Story: 2010676
Task: 50135
Change-Id: I20d4b26af868206c0d86c71d8b969c4c56f30293
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
When the deploy precheck condition is not met, ensure CLI exit code 1.
Closes-bug: 2065193
TCs:
Passed: verify CLI exit code 1 when precheck condition not met for
both software deploy precheck and software deploy start.
Passed: verify CLI exit code 0 when deploy precheck pass
Passed: verify CLI exit code 0 when deploy start pass
Change-Id: I0f73497140fcc82e3c8f583e9b421d2ff3e4db30
Signed-off-by: Bin Qian <bin.qian@windriver.com>
In the current http middle layer, authed and unauthed adapters are not
returning the same type of result. This causes the CLI having a hard
time to unified the display.
This change added data layer to provide identical response for
both authed and unauthed http adapters.
Also removed the code in unauth adapter that tries raise http exception,
align with authed adapter.
Fixed incorrect logic in check_rc function.
TCs:
authed and unauthed GET, POST and DELETE requests return identical
response.
Closes-Bug: 2064178
Change-Id: Ie8ba810bddeaa91c0b271fbf238ce59853f1169e
Signed-off-by: Bin Qian <bin.qian@windriver.com>
Updated the USM REST Api as agreed
Also applied the standardized CLI output w/ tabulate for CLI commands.
Fixed a few things:
1. software upload column header change to "Release"
2. use region_name from usm.conf to replace default "RegionOne" as local
region.
3. temporarily skip the ParsableErrorMiddleware.
4. do not recreate http exceptions based on the http status_code on
the client side, use common display function to display the http
error.
5. expose webob.exc.HTTPClientError to the client side.
6. updated --debug output to include CLI output at the end.
Test Cases:
passed all CLI commands, verify the endpoints and request body.
passed verify CLI requests compiled with defined REST Api.
Story: 2010676
Task: 49905
Change-Id: I5ab971b455aed527b7b1a21396b97334ba1e05ab
This change introduced state machines for release state, deploy state
and deploy host state.
This change removed the direct reference to the software metadata from
software-controller and other modules. Replaced with encapuslated
release_data module.
Also include changes:
1. removed required parameter for software deploy activate and software
deploy complete RestAPI.
2. ensure reload metadata for each request
3. added feed_repo and commit-id to the deploy entity, to be
subsequently passed to deploy operations.
4. fix issues
TCs:
passed: software upload major and patching releases
passed: software deploy start major and patching releases
passed: software deploy host (mock) major and patching release
passed: software activate major and patching release
passed: software complete major release and patching release
passed: redeploy host after host deploy failed both major and
patching release
Story: 2010676
Task: 49849
Change-Id: I4b1894560eccb8ef4f613633a73bf3887b2b93fb
Signed-off-by: Bin Qian <bin.qian@windriver.com>
Fix software deploy show and deploy host-list not displaying information
issue.
Also default the deploy host state to pending when a deploy host entity
is created.
Story: 2010676
Task: 49645
TCs:
passed: software deploy show and software deploy host-list show
deploy data after deploy start command accepted.
passed: display "No deploy in progress" for software deploy show
when there is no deploy.
Change-Id: I9dc50804c66d5cb07df7717fd6623c23d0fca522
Signed-off-by: Bin Qian <bin.qian@windriver.com>
When using Keystone auth for software cli, only user with 'admin' role
is allowed to run any commands. When using software cli without 'sudo',
all software commands require user with 'admin' role.
This review also update the exception handling and error reporting.
Test Plan:
PASS: A Keystone user in the 'admin' project with 'admin' role should
be able to run ALL 'software' commands WITHOUT SUDO
PASS: A Keystone user in the 'admin' project with only 'member' and/or
'reader' role should NOT be able to run ANY 'software' commands
WITHOUT SUDO
Story: 2010676
Task: 49754
Change-Id: I46653021b1a82bccded5eb870dc0907cd5c2351b
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
This commit is to fix 'software upload-dir' not having
respond that contains uploaded release info.
Test Plan:
PASS: upload files using 'software upload-dir'
Task: 49634
Story: 2010676
Change-Id: I635554fdbdb80fe31a38d1170202405fe6f32d3a
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This change adds checks before deleting software releases:
1. software release is available or unavailable
2. When it is on a system controller, the release is not being used by a
subcloud
This change also update the following:
1. removed the exception handling in controller level, moved to
exception hook
2. CLI code to display HTTP error, only handles 500 status code, by
displaying message from API, all other 4xx, 5xx status code display
HTTP error directly.
3. ensure CLI return 1 for unsuccessful requets (status code 500)
4. fixed some minor issues
Story: 2010676
Task: 49657
TCs:
passed: observe delection rejected because of release not found,
release is not in available or unavailable state.
passed: delete an available release
passed: on system controller, successfully delete scenarios
passed: (simulated) on system controller with subcloud, delete
release used by subcloud is rejected
Change-Id: I306b1d8604113b92d907384844e8e8107835a463
Signed-off-by: Bin Qian <bin.qian@windriver.com>
