Current configuration of software-client chooses public usm
endpoints by default. This is causing ssl errors when user
has HTTPS enabled w/ self-signed certificates, which
differs from other clients (e.g. cgtsclient).
Removed the code that set this, so that usm client uses the
one from env var 'os_endpoint_type'. This is set to 'internal'
when admin credentials are sourced.
Test plan:
PASS: Deploy DC + SX subcloud.
Change ssl cert to self-signed one.
Source admin credentials.
Execute 'software' commands w/ debug option in the
SystemController and subcloud and observe that they are
reaching the internal usm endpoints:
- software --debug list
- software --debug show <release>
- software --debug deploy show
- software --debug deploy host-list
- software deploy precheck <release> (tested by Heitor)
- software deploy start <major_release> (tested by Heitor)
PASS: Upgrade AIO-SX: stx 9 -> stx 10.
(code applied on both sides)
Story: 2009811
Task: 50722
Change-Id: I72e70725c0e82ec4cb43a87e006b12b1e4c55c1f
Signed-off-by: Marcelo de Castro Loebens <Marcelo.DeCastroLoebens@windriver.com>
Updated the USM REST Api as agreed
Also applied the standardized CLI output w/ tabulate for CLI commands.
Fixed a few things:
1. software upload column header change to "Release"
2. use region_name from usm.conf to replace default "RegionOne" as local
region.
3. temporarily skip the ParsableErrorMiddleware.
4. do not recreate http exceptions based on the http status_code on
the client side, use common display function to display the http
error.
5. expose webob.exc.HTTPClientError to the client side.
6. updated --debug output to include CLI output at the end.
Test Cases:
passed all CLI commands, verify the endpoints and request body.
passed verify CLI requests compiled with defined REST Api.
Story: 2010676
Task: 49905
Change-Id: I5ab971b455aed527b7b1a21396b97334ba1e05ab
When using Keystone auth for software cli, only user with 'admin' role
is allowed to run any commands. When using software cli without 'sudo',
all software commands require user with 'admin' role.
This review also update the exception handling and error reporting.
Test Plan:
PASS: A Keystone user in the 'admin' project with 'admin' role should
be able to run ALL 'software' commands WITHOUT SUDO
PASS: A Keystone user in the 'admin' project with only 'member' and/or
'reader' role should NOT be able to run ANY 'software' commands
WITHOUT SUDO
Story: 2010676
Task: 49754
Change-Id: I46653021b1a82bccded5eb870dc0907cd5c2351b
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
This change adds checks before deleting software releases:
1. software release is available or unavailable
2. When it is on a system controller, the release is not being used by a
subcloud
This change also update the following:
1. removed the exception handling in controller level, moved to
exception hook
2. CLI code to display HTTP error, only handles 500 status code, by
displaying message from API, all other 4xx, 5xx status code display
HTTP error directly.
3. ensure CLI return 1 for unsuccessful requets (status code 500)
4. fixed some minor issues
Story: 2010676
Task: 49657
TCs:
passed: observe delection rejected because of release not found,
release is not in available or unavailable state.
passed: delete an available release
passed: on system controller, successfully delete scenarios
passed: (simulated) on system controller with subcloud, delete
release used by subcloud is rejected
Change-Id: I306b1d8604113b92d907384844e8e8107835a463
Signed-off-by: Bin Qian <bin.qian@windriver.com>
This change add support for https with SSL protocol and certificate.
The USM client can work with either insecure (disable SSL/TLS
certificate verification) or with SSL certificate. The client is
also modified to support sessions and versions. These changes are
adapted from cgtsclient.
This adds three authorization modes, [token, keystone & local-root].
In token mode, a keystone token and software-url is used for auth.
Eg: $ software \
--software-url "http://192.168.204.1:5497" \
--os-auth-token "${TOKEN}" list
In keystone mode, sourced keystone configs in env is used for auth.
Eg: $ source /etc/platform/openrc; software list
In local-root mode, authorization is by privileged user (root/sudo)
of the controller where software application is running.
Eg: $ sudo software list
Optional arguments specific to https:
-k, --insecure
--cert-file CERT_FILE
--key-file KEY_FILE
--ca-file CA_FILE
Example usage for insecure connection:
software -k list
Story: 2010676
Task: 49666
Test Plan:
PASS: Verify software cli output for http endpoints
PASS: Verify software cli output for https endpoints
Change-Id: I2e2ff115b8d03cddb02e026da84f389918238dab
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
This commit add some changes on deploy host-list.
Adds a function to query the hostnames from sysinv
to deploy host-list entities during deploy start.
Changes endpoint to GET verb, the endpoint return in case
of no deployment in progress it will an empty list
and at CLI will print "No deploy in progress." In case
there is a deployment in progress the CLI will behave
the same but the endpoint will return the data below:
[{'hostname': '<hostname>',
'software_release': '<sw_version>',
'target_release': '<sw_version>',
'reboot_required': 'str<true/false>',
'host_state': '<host_deploy_state>'}]
This commit also changes the wait_for_install_complete function
to follow the new state logic.
Note: Software deploy host is affected by this change related
to states and will need a future commit regarding state changes
during deploy start and deploy host itself.
Test Plan:
PASS: Software deploy host-list with/without deployment in progress.
PASS: Deploy_host creation/update/get/delete.
PASS: Collect hostnames to deploy host entities during deploy start.
Story: 2010676
Task: 49586
Change-Id: I7b03df30fd8e326637a3ffc031e0fdf543cb6356
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
This commit add some changes to deploy show endpoint, the name was
changed to just deploy with GET verb and also changes the deploy to
be saved as a list of dict to attend the API requirements. Now the
api accepts from_release and to_release as optional params, in case
it is provided the endpoint will return a dict otherwise will return
a list of dict.
Test Plan:
PASS: Create deploy
PASS: Update deploy
PASS: Software deploy start
PASS: Software deploy show
Story: 2010676
Task: 49645
Change-Id: I68d243c05da88c7eecf2d866c7202c3c7be51a2b
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
This commit changes the wording on software upload command
to improve the feedback to the user.
Test Plan
PASS: run "software upload -h" and verify the output
Story: 2010676
Task: 49589
Change-Id: I8df2e67ef20627e96399d797c998e00c38eab849
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
When using 'software upload-dir' to upload files ,
this commit will only allow one iso file or one signature
to be uploaded in the directory. Multiple iso/signature
files in the directory to be uploaded won't be permitted
in order to prevent not enough disk space in /scratch
directory in the active controller and wrong signature
files being used.
Test Plan:
PASS: upload multiple iso files in the directory.
PASS: upload one iso file in the directory.
PASS: upload multiple sig files in the directory.
PASS: upload one sig file in the directory.
Task: 49576
Story: 2010676
Change-Id: Iba5af49ca50b4c836157027e140095a68ee5341b
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
As a follow-up commit for [1], this commit adds an optional
argument to the "software show" command to allow the user to
choose if the deb packages contained within a release should
be displayed along with the release information.
[1] https://review.opendev.org/c/starlingx/update/+/909046
Test Plan
PASS: use "software show" with "--packages" option and verify
deb packages information is displayed
PASS: use "software show" without "--packages" option and verify
deb packages information is not displayed
Story: 2010676
Task: 49579
Change-Id: I79e63cbb24dbe085d4f5dad2f2b1718a21044631
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
This commit is to enhance the 'software upload-dir' to handle when
the given parameter is invalid, and to improve the error messages to
indicate the error that occurred.
[sysadmin@controller-0 ~(keystone_admin)]$ software upload-dir abc
Invalid release directory: abc
No files to upload
[sysadmin@controller-0 ~(keystone_admin)]$
Test Plan:
PASS: run 'software upload-dir' with invalid directory
PASS: run 'software upload-dir' with valid directory
Task: 49527
Story: 2010676
Change-Id: I73a82eac713ec90e3df1ea70b033c3ddf1368910
This commit:
1. Changes metadata parsing to store package + version information
2. Changes output print function from software show to print the
packages contained in the release metadata
Test Plan
PASS: run "software show" for releases, verify the output shows
the name and version for deb packages included in the release
Story: 2010676
Task: 49573
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
Change-Id: I782c2d04ec665ea2330150add91f68182394a1b9
The software-api is versioned in order to allow for future upgrades.
This implementation add 'v1' version to all software REST APIs and
modify software client to use the updated 'v1' APIs.
Example:
- POST /v1/software/upload
- GET /v1/software/query
- GET /v1/software/show/<release>
- GET /v1/software/commit_patch/<release>
...
Story: 2010676
Task: 49478
Test Plan:
PASS: Verify REST interfaces of software APIs
PASS: Verify software client with new APIs
PASS: Verify client with sudo, without keystone auth prior bootstrap
PASS: Verify client without sudo, with keystone auth prior bootstrap
Change-Id: I10250676fbbcf7501913f21dedea769b581128af
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
This commit is to hide the traceback error messages when
there is no release to show in the 'software list'
[sysadmin@controller-0 ~(keystone_admin)]$ software list
No data
[sysadmin@controller-0 ~(keystone_admin)]$
Test Plan:
PASS: Remove all releases and run 'software list'
PASS: Upload one release and run 'software list'
Task: 49525
Story: 2010676
Change-Id: I7a00bfa6902c86acba46a9af2c593d86cac28813
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This commit is to hide the error traceback messages when
respond only contains error messages.
Test Plan:
PASS: uploaded only sig file and verified no traceback message
PASS: uploaded sig and iso files and verified correct result is displayed.
Change-Id: I0f997c861f45d7e59642833b13d23e6518ca7a69
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
After [1] the "software list" command is printing the
result in a random order, like:
Release RR State
================= == =========
starlingx-24.09.0 Y available
starlingx-24.03.2 Y available
starlingx-24.03.1 Y available
starlingx-24.03.0 Y deployed
starlingx-24.03.3 Y available
This commit enables sorting the output given a sort key,
which by default is the first element of each data row.
[1] https://review.opendev.org/c/starlingx/update/+/905320
Test Plan
PASS: run "software list" and verify the output is sorted
PASS: run "software upload" with multiple files and verify
the output is sorted
Story: 2010676
Task: 49491
Change-Id: Idbc4c59885ef3d8b8775da0271fb1ed301974969
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
This commit includes the cleanup below:
1. remove the sig file from the upload list
Uploaded File Id
============================= =================
starlingx-intel-x86-64-cd.iso starlingx-24.03.0
2. remove the software version column
from software list
Release RR State
================= == =========
starlingx-23.09.0 Y deployed
starlingx-24.03.0 Y available
Test Plan:
PASS: ran upload iso and sig file to check the list
PASS: ran software list to check the list
Task: 49396
Story: 2010676
Change-Id: I3f49a491ef7d1de0a108afa90da354392dbc44e6
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This commit supports keystone authentication for software client.
Test Plan:
[PASS] software CLI commands via remote CLI
[PASS] software CLI commands with sudo
[PASS] software CLI restricted commands without sudo give an error
[PASS] software CLI non-restricted commands without sudo work as
expected
Story: 2010676
Task: 49400
Change-Id: I608e32f905418698e784e89aa070a82bddeb3f49
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
When the valid files are being uploaded, the error messages were
mistakenly displayed due to missing absolute file path. The fix is
to ensure the absolute file path is added when filtering invalid files.
Test Plan:
PASS: uploaded valid files and no error message displayed.
PASS: uploaded invalid files and error message displayed.
Task: 49380
Story: 2010676
Change-Id: I6e5a68056892cbfdc3377e511974d7bfdb50b240
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This new --local option in software upload can take the
pre-uploaded files in active controller and import them
to the system.
It takes the absolute path to the file and upload the path string
to the software upload endpoint. The path is used to find the file
on the active controller and then proceed to load import.
Test Plan:
PASS: used --local to upload iso, sig and patch files
PASS: not use --local to upload iso, sig and patch files
PASS: uploaded a dir and unsupported file
Story: 2010676
Task: 49224
Change-Id: Ibdc7e603de79f43d3e6b57a6845942a0e7b9a6f1
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This commit is to add uploaded file info into 'software upload'
respond, in addition to existing respond messages.
This newly added uploaded file info is a list of dictionaries. Each
dictionary is taking the file name as the key and its value is a
dictionary in which id and sw_version are the keys.
Uploaded File Id SW Version
===================== =============== ==========
23.09_ALL_NODES.patch 23.09_ALL_NODES 23.09
Test Plan:
PASS: uploaded iso, sig and patch file and got response
Story: 2010676
Task: 49241
Change-Id: I5332268d238aa9e787dea68f8a23c6be42feb4cd
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This commit changes the software query-hosts command to software
host-list for usm. For the dict it keeps the previous one from
query_hosts, just added new values. The output of the command
has been changed, the from release and to release info is from deploy
entity.
Test Plan:
Command returning information according to host state.
Story: 2010676
Task: 49054
Change-Id: I5a9380c8c08b48d6f486caadb21d4f6203386687
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
This commit is to implement 'software deploy show'
Only one upgrade is allowed in the deployment every time.
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy show
From Release To Release Reboot Required State
============ ========== =============== =========
23.09 24.03 Yes deploying
Test Plan:
PASS: built and installed the iso and ran the command
with deploy in progress
PASS: built and installed the iso and ran the command
without deploy in progress
Task: 49134
Story: 2010676
Change-Id: I292837f1b9b39afbc589a9cae08b4c4b7f363b5e
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This commit adds an optional parameter '--force' to command
"software deploy precheck" to ignore alarms with lower severity
and non-mgmt affecting.
Test Plan:
PASS: force creation of a non-mgmt affecting alarm, such as [1],
run deploy precheck without '--force' option and verify the
command returns failure by the presence of alarms
PASS: same operation as the previous test case, but using '--force'
option, and verify the command returns success and absence of
alarms
PASS: force creation of a mgmt affecting alarm, such as [2], run
deploy precheck with and without '--force' option and verify
the command returns failure by the presence of mgmt affecting
alarms in both cases
[1] Non Mgmt Affecting Alarm Example:
Alarm ID: 100.104
Text: File System threshold exceeded ; threshold 80.00%, actual 84.12%
[2] Mgmt Affecting Alarm Example:
Alarm ID: 100.101
Text: Platform CPU threshold exceeded ; threshold 95.00%, actual 100.00%
Depends-on: https://review.opendev.org/c/starlingx/config/+/899750
Story: 2010676
Task: 49032
Change-Id: Ic4d327c3b072ce0f60e20ba7467ae6d9f4eb5bd4
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
Details: This commit allows 'software upload-dir' to upload
release iso and sig files from the given folder.
Test Plan:
PASS: upload iso/sig files from a folder
PASS: upload patches from a different folder
Task: 48983
Story: 2010676
Change-Id: Ie6179b6f0f9193983556ab6eae4353154aba6ca8
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This commit:
- Creates the "software deploy precheck" endpoint
- Implements a couple TODOs on the precheck code
- Replaces some hard-coded variables for constants
- Replaces "system" commands to retrieve system health
for sysinv client calls
- Creates separate health check classes for general and
upgrade-specific health checks
- Renames standalone precheck module to "deploy-precheck"
- Allows precheck module to receive keystone credentials
as parameters so that the precheck can be run against a
remote host (Use Case: system controller running precheck
against subclouds, but mind that DC support will be added
on a future commit)
- Creates an utility module that can be imported by standalone
USM upgrade scripts (copied to the ISO by [1])
Test Plan
PASS: run "software deploy precheck" and verify it returning
correctly, and verify that health checks output are valid
given the current system status
Story: 2010676
Task: 48902
Depends-on: https://review.opendev.org/c/starlingx/update/+/898474
Relates-to: [1] https://review.opendev.org/c/starlingx/tools/+/897484
Change-Id: I4d04804c1b1e147f486434df23a291c6d315842b
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
USM needs to build a python wheels package containing ONLY the
‘software’ cli client for inclusion in the Remote CLI Container.
Test Plan:
PASS Build software-client package
PASS Build ISO
PASS Install ISO
PASS Run "software list" and "software upload"
Story: 2010676
Task: 48851
Change-Id: Idb6d69a66d625291094501cacdb0bb84f511f81d
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
