ostree pull with --mirror creates a new ref for the sysroot
ostree which isn't kept up to date by the subsequent
pull operations. This commit removes the --mirror from
ostree pull to avoid this inconsistent behavior.
Test plan:
[PASS] - Verified the --mirror messes the sysroot ostree refs
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I798ff7119442a09007692b3bf416e4e96125a8d4
Closes-Bug: 1997491
When reusing initramfs during the patch creation
the patch_work/patch_repo commit that is included
in the patch has a different sha from the ostree_repo.
This happens because in order to reuse the initramfs
we need to checkout the commit change the files and
check it back which generates a new commit ID, as
this work is done under the patch_repo we need to
sync the deploy/ostree_repo by doing an ostree
pull-local when the initramfs is reused. Otherwise if
we keep the repos out of sync we can't create a
second patch due to the missmatch in the base sha.
Test:
Pass: build patch reusing initramfs
Pass: build patch without reuse
Story: 2009969
Task: 46877
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I69f7e789881a0b51ec4c580348acb753961cf791
This commit updates the make_patch prepare
stage to use hardlinks instead of copying
the ostree_repo directory. This will save
time and space while building patches.
It also updates the 'sed' command that get
the list of commits in an ostree_repo, an
error can happen if the ostree history has
a message that contains the word 'commit'
but it is not the commit sha line.
* setting chmod back to +x and updating the
shebang to be more generic
Story: 2009969
Task: 45573
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I9454c45fffec384baf98d96783d2b5c390f2c0ad
The delta-dir was not being found during patch generation.
This was because the code to detect initramfs here:
https://review.opendev.org/c/starlingx/update/+/861184
was changing to a different directory, so delta-dir was
being created in a patch_work directory, and was not
able to be included in the patch.
Test Plan:
PASS: python make_patch.py --create
Story: 2008862
Task: 46731
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Ide73f4a2dea1945f987cb36fae818bd1973706f9
This wheel is required by stx-keystone-api-proxy
Test Plan:
PASS: Build cgts-patch package. Verify that the wheel is generated,
and it's included in the wheels tarball.
PASS: Build stx-keystone-api-proxy image that requires cgts-patch
wheel. Verify that the image is created and the container is working
as expected.
Story: 2009831
Task: 46787
Signed-off-by: Hugo Brito <hugo.brito@windriver.com>
Change-Id: I5d097ae68a943895844cd667b9523c832dc002b3
This commit adds some additional steps within sw-patch host-install
to support the current implementation of kubernetes-unversioned.
Test Plan:
[PASS] Run kubectl after installing in-service patch
Closes-Bug: 1995973
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: Ic89da6cd2e1cbd771cf1f3620a83ffa37ea832ca
The base-initramfs file is rather large and will appear to change
frequently. This will expand the size of a patch in Debian for no good
reason as the original initramfs can generally be reused. To solve
this bloat problem, a new empty package called initramfs-trigger was
introduced. When making debian patch, if the patch version of this new
'initramfs-trigger' package does not change, use the original initramfs
file to shrink the patch.
TEST PLAN:
PASS - make patch between two images with same initramfs-trigger,
base-initframs can be reused
PASS - make patch between two images with different initramfs-trigger,
base-initframs can not be reused
Story: 2008862
Task: 46731
Co-developed-by: Luis Sampaio <luis.sampaio@windriver.com>
Signed-off-by: Zhang Xiao <xiao.zhang@windriver.com>
Change-Id: Id270a8a386cc21a0a8a99c5e04554d19d28e2c84
This commits updates the build-image call to
use --keep as the build-image was updated
to use --clean by default. As we need to
preserve the ostree_repo during this
build-image call the --keep was added.
Story: 2009969
Task: 45635
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I102504f3fb933c8b5adbbc68df9dde79e1a34308
This commits fixes a bug with the sw_version
in the metadata xml for the INSVC patch
and also the requires ID in the patch C.
Test Plan:
Pass: build patches
Pass: validate metadata content
Story: 2009969
Task: 45635
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I18da57c36dea67444471927fc52fd9096dae4445
This commit enables re-run of host-install if the first
attempt failed.
Test:
Pass: Error during ostree pull
Pass: Error during ostree deployment
Pass: Error during mount
Story: 2009969
Task: 46579
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I9e0a565145ecbf34236cfe50cb5c5ee17afea8a8
The access control for patching API was changed to accept GET requests
from users with reader role and presence in either admin or services
project. For other requests, it is required from the user that it has
admin role and presence in either project admin or services.
As all default system users have admin role and are present in either
admin or services project, this change should not cause regressions.
Test Plan:
PASS: Successfully deploy an AIO-SX using a Debian image with this
change present and create user "readeruser" with reader role. Log in the
Horizon interface using "readeruser" user, access page "Admin" ->
"Software Management" with no errors (a GET patches list request is
executed successfully here), try to upload a patch and check that it
fails. Repeat the steps for user "admin" and check that the patch upload
succeeds.
PASS: Successfully deploy a DC with 1 subcloud using a Debian image with
this change present and create user "readeruser" with reader role. Log
in the Horizon interface using "readeruser" user, access page
"Distributed Cloud Admin" -> "Software Management" with no errors (a GET
patches list request is executed successfully here), try to upload a
patch and check that it fails. Repeat the steps for user "admin" and
check that the patch upload succeeds.
Story: 2010149
Task: 46561
Depends-on: https://review.opendev.org/c/starlingx/gui/+/860701
Signed-off-by: Joao Victor Portal <Joao.VictorPortal@windriver.com>
Change-Id: I1b0b06ebeaadc82cd14174a46bf148c564dc7c08
We are currently using rsync to sync the ostree repos
between our controllers. However, it is not the best
choice because we don't have control over the updates
that are being applied while the other hosts are being
installed. Thus, we could get a partial view of the
updated objects and metadata via rsync if someone decides
to apply a patch while a host is installing. On the
contrary, ostree pull is based on a commit reference (SHA)
so the objects synced will be based on that reference.
If a new commit is added or removed during the pull, the
history of that commit will be consistent with the
objects and metadata.
Thus, this commit fixes the problem and uses ostree pull
for feed repo sync between controllers.
Story: 2009969
Task: 46515
Depends-On: https://review.opendev.org/c/860581
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I42c274079631a3c197015e636e03de1bc96de28b
Remove the installation of per-package preset installs
since they are centrally managed now by the ISO install
for the following packages:
- cgcs-patch
Story: 2009968
Task: 46406
Test Plan
PASS Build package
PASS Build ISO
PASS Check for non-existant preset file in /etc/systemd/system-preset
Depends-On: https://review.opendev.org/c/starlingx/integ/+/853653
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: Iaec2b4dad28f590e9fb6ee7cf8a2c0512b0060f0
Removed conf files from /etc/pmon.d/
as they are being moved to another location.
This is part of an effort to allow pmon conf files
to be selected at runtime by kickstarts.
The change is debian-only, since centos support
will be dropped soon.
Centos' pmon conf files remain in /etc/pmon.d/
Test Plan:
PASS - deb doesn't install anything to /etc/pmon.d/
PASS - AIOSX unlocked-enabled-available
PASS - Standard 2+2 unlocked-enabled-available
Story: 2010211
Task: 46308
Depends-On: https://review.opendev.org/c/starlingx/metal/+/855095
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: Ic406157798161d67520913579edd1065b6626128
Details: For feature parity, the upgrade-start-pkg-extract has
to be updated to support upgrading to Debian 22.12 and
CentOS 22.12.
Preview change in this CR
https://review.opendev.org/c/starlingx/update/+/850906
only runs on Debian to Debian upgrade.
This change will allow the package extraction running
on CentOS to Debian upgrade.
Test Plan:
PASS: manually tested the script on Debian
PASS: manually tested the script on CentOS
Task: 46269
Story: 2009303
Signed-off-by: Junfeng (Shawn) Li <junfeng.li@windriver.com>
Change-Id: I05f9e631b0bbddcad3e9ea21000ff7f283b2f097
These changes will not affect the behaviour of sw-patch.
- Remove fm-api from tox dependencies for sw-patch.
- fm-api is not used by sw-patch
- Cleanup flake8 error:
H401 docstring should not start with a space
- Cleanup pylint error:
R1710 inconsistent-return-statements
- Cleanup pylint error:
W1505 deprecated-method
The following are the deprecated methods
- LOG.warn -> LOG.warning
- ConfigParser.readfp -> ConfigParser.read_file
- Added missing description for a suppressed pylint error
- W3101 missing-timeout
- Added two additional tox utilities:
Neither tool is wired into the default tox, or zuul jobs.
- tox -e prospector
prospector calls multiple linters and validators
- tox -e vulture
vulture is a tool that detects unused code.
Test Plan:
- Run tox
- Build / Install / Unlock AIO-DX
- Apply a reboot required patch
Story: 2009969
Task: 46265
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I7fcd386d11ec3836059b1036fb334dcae9bedeb1
In Debian Patching, every time we run a host-install,
new deployments are created. For in-service patching,
we need this deployment history as some unmodified
files in pending deployments have symlinks to older
deployments. However, when the machine is rebooted
(say, due to power failure), the pending deployment
becomes active and we no longer need the entire
deployment history. Thus, we can delete all deployments
except the active and fallback deployment after a
reboot to save some space.
Test Plan:
1) Deployments are deleted after reboot
2) Deployments are not deleted after patch-agent restart
3) Test with in-service and RR patches
Story: 2009969
Task: 46139
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I763f945f7c3c5902034cfb4f0273c9eed2e0a50d
This commit adds the inservice patch test to the script
and now we have the following 3 test patches that can be
built:
PATCH A) Reboot required - all nodes
Update package - logmgmt
rebuild the pkg
build-image to generate a new ostree_repo commit
build a patch
PATCH B) In Service patch
Update the metadata
Uses the example-restart script
Uses the same ostree commit as PATCH A so they
can't be applied together
build a patch
PATCH C) Patch with dependency
(reboot required, depends on PATCH A)
build PATCH A
update package - logmgmt
build-image to generate a new ostree commit
build Patch C (requires A)
Story: 2009969
Task: 45635
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I3c91a761b4d165c7a6179cdf9f69a8bf078dfc9e
Pylint 2.15 came out on Aug 26th which added a new check for
missing-timeout.
https://pylint.pycqa.org/en/latest/whatsnew/2/2.15/index.html
This causes zuul jobs to fails and the submissions in the update
repo are blocked as a result. This commit disables the check
to fix the issue.
Closes-Bug: 1988226
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: Ic0f8ac3c08eba3f70e117a9066dcda9920753530
Created a duplicate install of /etc/pmon.d/*.conf files
to /usr/share/starlingx/pmon.d/
This is part of an effort to allow pmon conf files
to be selected at runtime by kickstarts.
Test Plan:
PASS: duplicate conf on deb
Story: 2010211
Task: 46115
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I1379b3c7f2fbdec1a265632ef60f0c86285ff64c
Details: For feature parity, the upgrade-start-pkg-extract has
to be updated to support upgrading to Debian 22.12 and
CentOS 22.12
Test Plan:
PASS: manually tested the script on Debian
PASS: manually tested the script on CentOS
Task: 45864
Story: 2009303
Signed-off-by: Junfeng (Shawn) Li <junfeng.li@windriver.com>
Change-Id: I0582197096f0cb4aca02aea19124fb86133701ae
sysinv files were being imported to provide authentication
features like policy enforcement and request contexts.
Those are now replaced with oslo imports.
Test Plan: (Debian)
PASS: AIO-DX bootstrap/unlock
PASS: CLI upload/apply/host-install RR patch
PASS: Horizon patching operations work
PASS: NFV patching operations work
PASS: no (new) errors in patching logs
Story: 2009969
Task: 45998
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I15d80441201755673f827529469a7f4feaa7f0ee
The patch storage in Debian holds software.tar files
associated with each patch as well as it's restart scripts.
Once a patch is committed, these files are expected to be
deleted to free up disk space in patch storage. This action
is irreversible (similar to the behavior in CentOS).
Test:
1) Verify software.tar and restart scripts (if any) are
deleted
2) Verify metadata is moved to committed directory
3) Verify that patch can only be committed if it has a
REL status
4) Verify that once a patch is committed, it cannot be
removed
Story: 2009969
Task: 45943
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: Iea89c4404d1a55a2ae12fc7dffaeb96486fb27e5
The sysinv method is no longer valid.
The code needs to be updated to accomodate the new
signature.
Test Plan:
PASS: upload/apply a patch
PASS: Apply a patch using NFV
Closes-Bug: #1983504
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I12a8defabc64155dd085f028b12c2d67bb52d99d
This commit changes the make_patch and
make_patching_workspace file permission to
+x (100755)
Story: 2009969
Task: 45542
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: Ib2761fffcfd0b1642b4b3c2a521a3a5a93f72138
Host-install for DX fails for RR patch because controller-1
tries to do a ostree pull from it's own feed ostree repo
which hasn't been synced. We need to have some mechanism in
place that would sync the feed ostree among the controllers
after each sw-patch command is issued.
This commit enables host-install in a duplex debian env.
Story: 2009969
Task: 45821
Depends-On: https://review.opendev.org/q/850779
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I654848481f8bd85d4e40af2d115398e1e9583915
If /etc/resolv.conf is empty the patch controller
will raise an uncaught exception due to osprofiler
attempting to validate dns.
osprofiler is unused by patching, so the module is
disabled by the patch controller.
All sw-patch CLI commands will fail prior to
setting up resolv.conf (bootstrapping) without this
fix, since there is no working patch controller
sysinv.common.utils also pulls in dns, so the two
utility methods are cloned into patching repo.
Test Plan:
PASS Build/Install AIO-SX Debian
PASS upload a patch
Story: 2009969
Task: 45838
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I0975f5b54a17a0989a78f6ac39160af0b3e26013
- Adds the --modify to the make_patch script which
updates the STATUS tag to one of the supported options:
DEV, REL or OBS
Example:
./make_patch.py modify --status development \
--patch-file PATCH_0001.patch
* --formal can also be used if the patch needs to be signed
- Updates the deploy dir path to:
localdisk/deploy as now we have a single iso for std and rt.
- Remove deprecated make_test_patch script
Test Plan:
Pass: create and modify patch status
Pass: modify patch and formal sign it
Story: 2009969
Task: 45842
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I508c2a723a07f48f8fc84ed043968eadc5804dcf
Patch scripts will now be staged in /opt/patching/patch-scripts
instead of /etc/patching/patch-scripts. For in-service patching,
we re-mount /etc and /usr directories that makes these directories
non-writable. Thus, in the future if you have to delete the
scripts, it's not possible.
This commit fixes the issue by staging the patch scripts in /opt
during upload.
Test Plan:
Apply, Install, Remove and Delete an in-service patch
Story: 2009969
Task: 45820
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: Ib90e5892d5d3e365d6e2da6abb41cfbc6be2c263
- Adding missing requirements for sw-patch
- Cleaning up redundant dependencies
- Invoke clean as part of debian build
- Rename Starlingx to StarlingX
- Display error details for failed sw-patch commands
- Fix a python3 string comparison affecting duplex
- Fix report-app-dependencies
- Remove 'six' from sw-patch since the code is py3 only
- Fix a KeyError if restart_script not found in metadata.
- Allow bandit to ignore 'input' checks on python3 since
it is considered safe
Test Plan:
PASS: Build/Bootstrap/Unlock AIO-SX Debian
PASS: sw-patch report-app-dependencies --app foo
PASS: Upload/Apply/Remove/Delete a patch on AIO-DX system
host-install was not tested on AIO-DX as that
capability is still under active developement.
Closes-Bug: #1976535
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I94ec47e2f2087989da26ebab0e05c3b0d8f303e6
This commit enables installation of in-service patches in a
Debian env by doing a hot update to make the pending
deployment live.
Test:
1) Verify that "sw-patch upload" stages the restart scripts
(if any) at /etc/patching/patch-scripts
2) Verify that "sw-patch delete" deletes the restart scripts
(if any) from /etc/patching/patch-scripts
3) Verify restart scripts are executed after a patch is applied
4) Apply and install in-service patch
Story: 2009969
Task: 45585
Depends-On: https://review.opendev.org/c/849134
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I93a39c0b4de73c5043592acaad1a62e17144d186
make_patching_workspace executes the following steps:
01 - Setup env
02 - Download binaries and sources
03 - Build all pkgs and image or rsync data from source build (build-avoidance)
04 - replace build ostree_repo by ostree_repo_source
Prereqs:
- Build pre reqs for Minikube or Kubernetes
- The script uses env variables to init the repo and build the
environment.
- Env variables to export prior to running the script:
export PROJECT="stx-patch"
export STX_BUILD_HOME="/localdisk/designer/${USER}/${PROJECT}"
export MY_REPO="${STX_BUILD_HOME}/repo/cgcs-root"
# Kubernetes:
export STX_PLATFORM="kubernetes"
export STX_K8S_NAMESPACE="${USER}-${PROJECT}"
export KUBECONFIG="/build/.kube/config"
# Minikube
export STX_PLATFORM="minikube"
# Manifest and branch
export MANIFEST_URL="https://opendev.org/starlingx/manifest.git"
export MANIFEST_BRANCH="master"
export MANIFEST="default.xml"
Examples:
- Create workspace and build all packages
./make_patching_workspace.py \
--ostree-source /path_to_source_build/ostree_repo --build-all
- Create workspace and rsync source build data (aptly/mirrors)
source dir can be a source build home or a directory that contains
a copy of aptly and mirrors, these directories will be copied into
the patch build environment
./make_patching_workspace.py \
--ostree-source=/path_to_ostree/ostree_repo/ \
--build-avoidance --build-avoidance-dir=/path_to_source_dir
Test Plan:
Pass: create workspace using build-all
Pass: create workspace using build-avoidance
Pass: build-image on new workspace
* This commit also fixes the chmod permission for the
build_test_patches script.
Story: 2009969
Task: 45751
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I46d9531fad88abbaef8f4b05bb6a0e16b5e54e14
Adds the patch_id as prefix for the restart script name
Story: 2009969
Task: 45573
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I1ab1a67c078a24089819f803d2d23eb99f6a94f8
Jessica Castelino