Commit Graph

428 Commits (master)

Author SHA1 Message Date
Zuul 782a9b12c7 Merge "Add release upload for upgrade" 2023-09-20 14:36:36 +00:00
junfeng-li 90a03e65e6 Add release upload for upgrade
Details: Add uploading and listing release iso for upgrade.
This commit implements 'software upload' and 'software list'
for release iso load.

Test Plan:

PASS: Upload new release iso and sig file
PASS: Upload patch
PASS: Run software upload and software list
PASS: Reboot and run software list
PASS: Upload release iso, sig file and patch file
      at the same time

Task: 48383
Story: 2010676
Change-Id: I9c3ffca33b71a3e4f6d10dfacede0f24fef8fa39
Signed-off-by: junfeng-li <>
2023-09-19 16:52:21 -04:00
Jessica Castelino 7e7654ef54 software commit-patch implementation
This commit enables the command "software
commit-patch <Patch-ID>" in the Debian Env.

Test Plan:
[PASS] software commit-patch <patch-id>

Story: 2010676
Task: 48789
Change-Id: I5f8392ac71dc964f54fa02ac8f4086feab4858ee
Signed-off-by: Jessica Castelino <>
2023-09-18 22:08:37 +00:00
Jessica Castelino bd32299149 Fix: openstack-tox-docs job is broken with Sphinx >= 5.0.0
This commit aims to fix the broken zuul job for this repo.

Change-Id: Ic3a4703506ff53d12acf20ef7589809363ec465d
Signed-off-by: Jessica Castelino <>
Closes-Bug: 2033671
2023-08-31 17:01:12 +00:00
Jessica Castelino 0eb68e8416 "software install-local" implementation for USM
This commit enables "software install-local" in a Debian
env for a patch.

Test Plan:
[PASS] Verify software install-local installs the patch
[PASS] Verify state machine for install-local workflow

Story: 2010676
Task: 48607
Change-Id: I6fd44653210cecdc0e3e1131bfd4619a756556c8
Signed-off-by: Jessica Castelino <>
2023-08-25 16:11:20 +00:00
Zuul a4bf3a1dc8 Merge "update client to get the correct endpoint" 2023-08-18 12:16:42 +00:00
Christopher Souza fd9ad3f5c2 update client to get the correct endpoint
update client to get the correct endpoint when region name is provided.

Story: 2010676
Task: 48618

Test Case:
PASS: run software list command with region name SystemController

Change-Id: If8c7b5779868d2007819b98ef93ca8457b009aee
Signed-off-by: Christopher Souza <>
2023-08-14 16:45:01 -03:00
Luis Sampaio 2d289c0296 Update test-patches restart script
Sometimes the current script can trigger an issue while
restarting horizon so this commit updates the test patches
script to use a simpler script.

Test Plan:
PASS: build test patch and check the restart script inside
the patch file was updated.
PASS: Applied test patch on latest build

Closes-bug: 2031330
Signed-off-by: Luis Sampaio <>
Change-Id: Ia907d8b95526c35ba5992d7106178bf480c97986
2023-08-14 09:17:32 -07:00
Jessica Castelino f29561900f Bash completion for USM
This commit enables bash completion for Unified Software Management
commands using argcomplete module.

Test Plan:
[PASS] Build and install the iso
[PASS] Test bash completion for software commands

Story: 2010676
Task: 48540
Signed-off-by: Jessica Castelino <>
Change-Id: Id1d404ff48029c61040f2f137b0169d5ac6919d6
2023-08-03 16:21:00 +00:00
Heitor Matsui e8c8b24922 Create platform upgrade health plugin
This commit changes the previous platform upgrade health
check, executed with "system health-query-upgrade" to a
standalone executable to be called by the upcoming changes
on the upgrade framework.

More specifically, this commit remove direct sysinv code
importing from health check, converting the required calls
into CLI commands, and also removes deprecated code needed
only for CentOS to Debian upgrades and health checks that
won't matter anymore with the new upgrade approach.

There will be follow-up commits to address the TODO items
on the code, since they depend on work in progress related
to the new unified software management API and database.

Test Plan:
PASS: run the standalone upgrade precheck successfully

PASS: run "system health-query" successfully
PASS: run "system health-query-kube-upgrade" successfully
PASS: run "system health-query-upgrade" successfully

Story: 2010651
Task: 48058

Change-Id: Ifb76f7de09b2bffa559c90409f954aa43f172f32
Signed-off-by: Heitor Matsui <>
2023-07-28 09:23:45 -03:00
Zuul b836c809a6 Merge "Package the pecan templates with the code" 2023-07-19 15:42:41 +00:00
Zuul 29ee2e38a8 Merge "Rename CLI commands" 2023-07-17 15:56:56 +00:00
Al Bailey c38e16bd8b Package the pecan templates with the code
The pecan xml template files are used by the API
to format the results of certain operations in certain
response modes.

Those xml files are now included in the setuptools
packaging and are included in the debian files.

This change also moves all the package configuration into
the setup.cfg file.

PBR conflicts with the package_data definitions in the
setup files so pbr is no longer used for versioning.

Test Plan:
  PASS Bootstrap AIO-SX and verify files are included

Story: 2010676
Task: 48400
Signed-off-by: Al Bailey <>
Change-Id: Ibcea8d6f4da8b089b4b59c9c2ca96a988be0795f
2023-07-17 15:33:45 +00:00
Jessica Castelino 7f8c0f970b Rename CLI commands
This change renames the CLI commands as give below:
1) "software release list" is now "software list"
2) "software release upload" is now "software upload"
3) "software release delete" is now "software delete"
In addition, this change updates the upload and delete
functionalities to use the major version instead of what is
provided in patch metadata while creating directories.

Test Plan:
[PASS] Verify CLI commands have been renamed

Story: 2010676
Task: 48382
Signed-off-by: Jessica Castelino <>
Change-Id: I969ca9d4409a8f51e75a44ab6a1dc5284b396908
2023-07-14 20:09:05 +00:00
Zuul 661fabdbb3 Merge "Improvements to sneaky patch utility" 2023-07-11 17:15:58 +00:00
Al Bailey 6feb85e1f9 Improvements to sneaky patch utility
Adds validation to whether the .deb files exist.
Validates if the restart script is executable.
This adds in yaml support as the input.
This adds multiple patch support (through the yaml).
Adds support for a sem-ver sw-version (major.minor.micro)
Validates if a sequence of patches use the same
restart script (prohibited).

Test Plan:
 Make a sneaky patch using command line args
 Make a sneaky patch sequence of 2 patches using yaml
 and verify they can both be applied and installed.

Story: 2010547
Task: 48323
Signed-off-by: Al Bailey <>
Change-Id: If8e7745b2054b21a1476b9685629559a376a3841
2023-06-30 22:15:48 +00:00
Zuul d11fde3d5b Merge "Add timeout option in test patches script" 2023-06-30 19:28:39 +00:00
Zuul 33b379914e Merge "USM State Machine" 2023-06-30 14:44:58 +00:00
Dostoievski Batista b6e4ab3a27 Add timeout option in test patches script
When the build server is not strong enough we encounter timeout on the
aptly's operation. This change add the option to increase the timeout
directly into the script.

Test Plan:
Pass: ./ --software-version \
    "${SW_VERSION}" --time-out 15
Pass: ./ --software-version \
Pass: Build patches
Pass: Validate metadata content

Story: 2009969
Task: 48314

Change-Id: I545510f250e1c51d0a2db9386c73624e3bebeb00
Signed-off-by: Dostoievski Batista <>
2023-06-30 11:00:34 -03:00
Jessica Castelino 3d2e806453 USM State Machine
This commit implements the state machine such that the acceptable
values are as given below:
state - available, unavailable, deploying-start, deploying-host,
        deploying-activate, deploying-complete, removing, deployed,
        aborted, committed

Test Plan:
1) Verify the exit states after each software operation
2) Verify the state progression in 3 scenarios:
    (a) upload, deploy start, deploy host,
        deploy activate, deploy complete,
        deploy start, deploy host,
        deploy activate, deploy complete, delete
    (b) upload, deploy start <newer release>,
        deploy start <older release>, delete
    (c) upload, delete

Story: 2010676
Task: 48190
Signed-off-by: Jessica Castelino <>
Change-Id: I3d8cfe21af76d19586fd1b42a347167c9f808392
2023-06-29 17:53:39 +00:00
Al Bailey 451374161d Update kernel.env after creating ostree deployment
When an ostree deployment is created, it uses a kernel.env
from the ostree environment which does not contain local
changes such as selecting a real time kernel.

This change add a post-deployment step to update the kernel.env
using a utility method from puppet to ensure that the proper
kernel is selected, and that other values are preserved from
previous deployments.

Note:  Not all patches encounter this issue.
 This problem would only be seen if the patch introduced a
new directory under /boot/ostree

Test Plan:
 - Unlock a AIO-SX RT system.
 - lock / host-install a reboot-required patch / unlock.
 -- observe the system is running RT mode after reboot.
 -- observe that the /boot/ostree was different

Closes-Bug: #2023318
Signed-off-by: Al Bailey <>
Change-Id: I571b05ddf92500f4823132f94a07297082bd7a5d
2023-06-08 20:56:42 +00:00
Zuul 0f76dd885f Merge "software deploy start implementation" 2023-05-29 20:43:31 +00:00
Jessica Castelino 8ecbecbef1 software deploy start implementation
This commit enables "software deploy start" in a Debian env for
all minor releases.

Test Plan:
[PASS] software deploy start

Story: 2010676
Task: 48051
Signed-off-by: Jessica Castelino <>
Change-Id: Icec0535e210c279ae79364ece9633155519e797e
2023-05-29 14:32:18 +00:00
Zuul b7e06cc6ee Merge "Utility to create a patch directly on controller" 2023-05-19 18:09:24 +00:00
Al Bailey 25f0ecb888 Utility to create a patch directly on controller
This provides a sneaky patching mechanism to interact
with ostree and build an ostree commit through the
installation of debian packages in a chroot env.

Test Plan:
  PASS: successfully created a reboot required patch
    for a single .deb file
  PASS: successfully created a reboot required patch
    for multiple .deb files
  PASS: successfully created a no-reboot required patch
    for multiple .deb files

Story: 2010547
Task: 48029
Signed-off-by: Al Bailey <>
Change-Id: I31b2828869c3eaeb42c2fb546bd02ad0007265b2
2023-05-19 17:22:06 +00:00
Zuul f1c83a78e4 Merge "Remove ssh host-key from new rootfs before patch" 2023-05-19 15:48:05 +00:00
Jessica Castelino 7f395e5870 Cosmetic changes for software CLI output
This commit updates the CLI output for all software
commands and also updates variable names to eliminate
the word patch wherever possible.

Test Plan:
[PASS] software deploy query-hosts
[PASS] software release list
[PASS] software release show
[PASS] uploaded and deleted a patch

Story: 2010676
Task: 48053
Signed-off-by: Jessica Castelino <>
Change-Id: I803bcd0dcd621fa70d8e45c3feb1892aafd04ee7
2023-05-18 17:49:59 +00:00
Al Bailey 379bc6a89a Rename exceptions for unified software management
The exceptions were copied from the patching system, and
now their names are updated to properly indicate their

Test Plan:
  PASS: upload/apply a software release (patch)

Story: 2010676
Task: 48039
Signed-off-by: Al Bailey <>
Change-Id: I80625d9955d417b99ce80d41499ee095cae07d67
2023-05-17 16:06:37 +00:00
Al Bailey a58c23ca01 Fix invalid patch alarm on unlocked AIO-SX
When an AIO-SX is unlocked, the patching and unified
software management systems were sharing a flag file
to indicate if the system had encountered a patch failure.

The flag was being incorrectly set by the unified software
management system.

This fixes the code that was setting of the flag, and also
renames the flag in the unified software management system.

Test Plan:
  PASS: Tox
  PASS: build component
  PASS: Boostrap/Unlock AIO-SX (no alarms)
Story: 2010676
Task: 48024
Signed-off-by: Al Bailey <>
Change-Id: I681a641f563e851d3f4dc4d5a63b74526dd77892
2023-05-16 15:59:45 +00:00
Jessica Castelino e14546dbcc "software deploy delete" implementation
This commit enables "software deploy delete" in a Debian env for
all minor releases.

Test Plan:
[PASS] software deploy delete

Story: 2010676
Task: 47987
Signed-off-by: Jessica Castelino <>
Change-Id: I5fe248109282788290279fae34283299e20f1462
2023-05-15 21:24:37 +00:00
Jessica Castelino 63f1a44e43 "software deploy query-hosts" implementation
This commit enables software deploy query-hosts in a Debian env for
all minor releases.

Test Plan:
software deploy query-hosts

Story: 2010676
Task: 47982
Signed-off-by: Jessica Castelino <>
Change-Id: If55e23a4c6a1f5bcdeeb57453a953efa771a4c7a
2023-05-12 18:11:58 +00:00
Jessica Castelino 4138c37835 software deploy create implementation
This commit enables software deploy create in a Debian env for
all patches.

Test Plan:
software deploy create

Story: 2010676
Task: 47961
Signed-off-by: Jessica Castelino <>
Change-Id: I7c4bfb9811be41ae416154e985d1a418f35a9a01
2023-05-11 15:08:46 +00:00
Al Bailey 796df7ddd3 Implement software release upload / delete / list / show
The unified software management 'release' commands for
 - upload
 - upload-dir
 - delete
 - list
 - show
are similar to the previous sw-patch commands

This component is not currently being used however
these functions appear to be operational.

Test Plan:
  PASS: tox
  PASS: software release upload --release TEST.patch
  PASS: software release upload-dir --release TEST.patch
  PASS: software release list
  PASS: software release show --release TEST
  PASS: software release delete --release TEST

Story: 2010676
Task: 47959
Signed-off-by: Al Bailey <>
Change-Id: I723648b8a522c8ef7e1d6e23e806553294780653
2023-05-10 21:12:02 +00:00
Jessica Castelino c863d8e47d Software deploy host implementation
This commit enables software deploy host in a Debian env for
all patches. It also includes the ostree_utils required to
checkout a sysroot ostree commit, install/uninstall packages
from a checked-out commit, and write commit to feed ostree -
all of which are not being invoked by software deploy host
as of today but are intended for future use.

Test Plan:
[PASS] software deploy host

Story: 2010676
Task: 47948
Signed-off-by: Jessica Castelino <>
Change-Id: I2178010f3af5cd8e69b28a06866e567248088cba
2023-05-08 17:11:53 +00:00
Al Bailey 5015668dd7 Convert software CLI commands to argparse
- converted CLI to use argparse
- added stubbed CLI commands for:
 - software
   - deploy
     - activate
     - complete
     - host
     - list
     - query
     - start
   - release
     - delete
     - list
     - show
     - upload
     - upload-dir
- added in the gir dependency used by ostree commands to
unblock enabling the CLI command functions.

Test Plan:
  PASS: tox
  PASS: software commands passing (-h)

Story: 2010676
Task: 47945
Signed-off-by: Al Bailey <>
Change-Id: I28023b690dea312e78eb4e4b4ef1c9be9c2af69e
2023-05-05 14:23:03 +00:00
Zuul 3c395fa898 Merge "Update patch-alarm tox for python3" 2023-05-02 14:36:06 +00:00
Al Bailey 31366985ab Update unit tests for new software component
- cmd files are replaced by software_client
 - is renamed to and includes
the previous config setup.
 - unit tests from sw-patch are being migrated here

Story: 2010676
Task: 47917
Signed-off-by: Al Bailey <>
Change-Id: I886b4abd63a9b7057efd2b6440211a9c1f06f6f3
2023-05-01 21:09:15 +00:00
Al Bailey 4624457333 Fix runtime patching error during upgrade
During an upgrade, multiple patch directories may
be detected, and the patch controller may attempt to
iterate over those folders to determine the packages.

The code that iterates over those folder can raise
 RuntimeError: dictionary changed size during iteration
if the directory does not exist, but a package does.

The fix is to iterate over the copy of the list.

This issue only occurs in a duplex env, when the neighbor
sync is being done.  It will be encountered when a patch
import is invoked after the older directory is removed.

Test Plan:
   AIO-DX create multiple folders under /var/www/pages/feed
  and trigger a patch-import

Closes-Bug: #2017916
Signed-off-by: Al Bailey <>
Change-Id: Ic78d30019bdaa93b9493bf57c7122f6de7cc6dab
2023-04-27 16:58:18 +00:00
Dostoievski Batista f3aa4b35ab Remove ssh host-key from new rootfs before patch
The ssh host-keys are changing during the patch build.
This change remove the ssh host-key from the new
rootfs dir before creating the patch.

Test plan:
- Run ./ create --patch-recipe patch_recipe_rr_sample.xml
    \ --clone-repo /localdisk/deploy/ostree_test/
- Check if folder {new_rootfs}/usr/etc/ssh/ssh_host has no key file
- Check if patch is built correctly

Story: 2009969
Task: 47898

Change-Id: I902b88987209f80ebf4a48405e9ef51265d14617
Signed-off-by: Dostoievski Batista <>
2023-04-26 23:14:24 +00:00
Jessica Castelino 965a6d3639 Setup controller and agent processes for "software"
Current Status:
 - CLI commands
 - API endpoints
 - controller process
 - agent process
 - Logging

Test Plan:
  PASS: Build and Install ISO
  PASS: software-agent and software-controller
        services are active post unlock
  PASS: tox

Story: 2010676
Task: 47817
Signed-off-by: Jessica Castelino <>
Change-Id: I394780ce40fee398c4eacb3aacb575a03ff93332
2023-04-20 19:22:30 +00:00
Al Bailey 70f9755c1f Update patch-alarm tox for python3
- Removed the old python2 entries
- Replaced mock with unittest.mock
- Unsuppressed most flake8 and pylint checks
- Added the startup script to flake8 checking and fixed
a python3 syntax issue in the error handling.
- Regenerated an updated /  modern pylint.rc
- Updated the upper-constraints to be Debian and python3

Test Plan:
  Pass: Build / Install / Unlock
  Pass: trigger a patch alarm

Story: 2010642
Task: 47815
Signed-off-by: Al Bailey <>
Change-Id: Id8c612a677253dfc53931719af7823572d1536ef
2023-04-12 18:28:07 +00:00
Al Bailey ba472d6ede Adding the software-wheel debian build target
The entry added to by [1]
means that the wheel generation will fail because the
control file does not have an entry for that wheel.

The version needs to be 1.0-1 rather than 1.0.1

Adding the appropriate directives.


Closes-Bug: 2015404
Signed-off-by: Al Bailey <>
Change-Id: I5752d72531aeb521e24d9e8661f55748ef4d0624
2023-04-10 17:14:26 +00:00
Zuul 3de2a431f6 Merge "Revert "Revert "Fix github mirroring for this repo""" 2023-04-05 13:50:41 +00:00
Al Bailey 314c01be47 Revert "Revert "Fix github mirroring for this repo""
This reverts commit 3e920a1641.

Reason for revert: It looks like after a post zuul job merges, the change is not used by its first invocation.  The original code should work.

Change-Id: Ieab56ef652b69ab47f9de6055c5e71520ea67375
2023-04-05 13:32:29 +00:00
Zuul 646fae2523 Merge "Revert "Fix github mirroring for this repo"" 2023-04-04 19:26:19 +00:00
Al Bailey 3e920a1641 Revert "Fix github mirroring for this repo"
This reverts commit 3770c7bc99.

Reason for revert: Did not work

Change-Id: Ib84486374c5c874b8a8f9e1dfdc3b63dacc75383
2023-04-04 19:08:25 +00:00
Zuul dab0d59301 Merge "Fix github mirroring for this repo" 2023-04-04 18:51:02 +00:00
Al Bailey 3770c7bc99 Fix github mirroring for this repo
Updating the rsa ssh host key based on:

Note: In the future, StarlingX should have a zuul job and
secret setup for all repos so we do not need to do this
for every repo.

Needed to rename the secret

Partial-Bug: #2015246
Signed-off-by: Al Bailey <>
Change-Id: I2b98d3500eac816e72fd5e513763d7633ddc51f3
2023-04-04 14:31:12 +00:00
Al Bailey 7f76bea9d3 Upload skeleton for unified software management
Current Status:
 - minimal debian packaging
 - python skeleton
 - tox
 - zuul integration
 - pecan API skeleton
 - unit tests

Next Steps:
 - CLI commands
 - API endpoints
 - controller
 - agent
 - expanded packaging

Test Plan:
  PASS: Build and Install ISO
  PASS: tox
  PASS: 'software' is in the PATH and can be invoked.
  PASS:  wget returns 204 when
    software-api is running

Story: 2010676
Task: 47754
Signed-off-by: Al Bailey <>
Change-Id: I8e2d993ac85dcff7bb970ddf05fd63db9f2abe11
2023-04-03 14:03:04 +00:00
Zuul 1b9f78d1bb Merge "Ignore ownership during ostree patch build" 2023-03-29 17:47:39 +00:00