This allows creation of debian patches (RR and INSVC) based on ostree.
Steps to create a debian patch:
1) Export env variables, e.g:
export PROJECT="stx-debian-desktop"
export STX_BUILD_HOME="/localdisk/designer/${USER}/${PROJECT}"
2) Run prepare to create a clone of the STX_BUILD_HOME ostree_repo
./make_patch.py prepare --clone-repo ostree_test
3) Make changes to the environment
- update a package
- build-pkgs -c -p <package name>
- build-image
This will create a new commit in the build ostree_repo
4) Create your patch.xml (samples provided)
- patch_recipe_rr_sample.xml
- patch_recipe_insvc_sample.xml
5) Run create to build the patch
./make_patch.py create --patch-recipe patch_recipe_rr_sample.xml \
--clone-repo ostree_test
Once the script is done the .patch file can be located at:
$STX_BUILD_HOME/localdisk/lat/std/deploy/
Story: 2009969
Task: 45542
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I902c42bc22601810fb91e45c630671af14906dcd
lxml library tostring() accepts only one argument
instead of 2 like it was before. This commit removes
the second argument.
Closes-Bug: 1977869
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I9bb43a758bb660c8ab6edbf54984ff993b7f8598
This commit ensures that sw-patch show command gives details about
the contents of a Debian patch i.e. number of ostree commits,
the base commit on which the patch can be applied, and the commit
IDs that are associated with the patch.
Test:
sw-patch show <PATCH-ID>
Story: 2009969
Task: 45536
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I1c7e48f299e566fea9c2ee4069a44580052df8f5
Adding unit tests for:
- patch_client (sw-patch) CLI
- help
- query
-Deprecation and Syntax warnings are now ignored by tox.
These are caused by netaddr and other 3rdparty components.
Those components are outside of the control of this repo.
- Pylint error codes are now suppressed individually.
Previously all (C)onvention and (R)efactor error checks
were being suppressed, including those that were passing.
- All the python3 enable checks are removed, since sw-patch
pylint runs in python3, and that was only needed for running
in python2.
- SafeConfigParser is renamed ConfigParser since python 3.2
and this is now fixed to satisfy the latest pylint.
Story: 2009969
Task: 45542
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I60a4340ea6f2c2303d0baa17e303938d55e8d278
- Add a fix for install-local so the patch agent attempts
to query the feed commit if no socket is established.
Test Plan:
Debian: Build / Bootstrap / Unlock / Reboot AIO-SX
Debian: Verify install-local works
Story: 2009969
Task: 45494
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I8f05728b6fcdafd5d54aa3eba4fe17954085bdee
- Assign 'system_mode' to 'simplex' when it is unknown.
The system_mode is not set until after bootstrap. Without
this change, it defaults to duplex.
- Remove the Clean RPMS step from sw-patch init
Debian does not use rpm, so this method can be removed.
- Remove rpm-audit utility.
Debian does not use rpm, so this utility can be removed.
- Remove 'ID' as a 'required' field for make_test_patch
since the utility has a default, and will not use an ID
for some of its sub-commands.
- Remove the SafeConfigParser workaround which is no
longer needed in Debian env.
- Add a fix for install-local so that the feed commit
is not sent if the host has not been provisioned.
Test Plan:
Debian: Build / Bootstrap / Unlock / Reboot AIO-SX
Verify logs clean
Verify no patch alarms
Verify make_test_patch prepare does not prompt for ID
Story: 2009969
Task: 45409
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I75ada6e262533d9c6477721836b6ecdf213c25dc
During the host-install step, the sysroot repo pulls from
the remote defined inside /sysroot/ostree/repo/config file.
This is a better approach than doing pull-local and it allows
the sysroot repo to be a mirror of the feed repo.
Test:
Running host-install after sw-patch remove <PATCH-ID>
should get rid of the commits associated with the patch
in the sysroot repo.
Story: 2009969
Task: 45435
Depends-On: https://review.opendev.org/c/starlingx/metal/+/842568
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: If3e7db2a47f085bbc4abd59eb34565261e30c061
Once a patch is removed from the system, we need to update the
summary file to make sure that it updates the head of the feed
repo. This ensures that when a pull is done from the feed repo,
the ostree repo knows what the latest head of the feed ostree is.
Test:
Run sw-patch remove <PATCH-ID> and verify that the summary file
is updated by running "ostree remote summary debian"
Story: 2009969
Task: 45430
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: Ib83522302b84f9793c7046c4c332f9bf87497631
When patch service starts up, it invokes "install"
to see if there are patches that need to be installed.
- The logic was incorrectly reporting a failure if updates
were not installed. This is invalid in the scenario where
there are no updates (patches) that need to be installed.
- The logic will now only return failure if updates were
expected, but were not installed.
Test Plan:
Debian AIO-SX: Verify Bootstrap / Unlock / Reboot
Verify no patching alarms
Story: 2009969
Task: 45391
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Ieb2b0c056151f9e989871a300b54e31a07a4c5b0
This commit fixes the patch current implementation
in Debian env. It checks the sysroot commit, feed
commit and active deployment commit to report if a node
is patch current or not.
Test:
- Build/ Bootstrap/ Unlock / Reboot and verify that
patching does not trigger a reboot loop.
Story: 2009969
Task: 45394
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I43c02af3c7f5b3b0430502b1c5992d6b7bcd1915
The purpose for these minor changes is to help
when making more than one patch for testing.
- Adding ID argument to passing in the patch ID
- Adding clean-mode argument to automatically delete
the delta directory
Test Plan:
python make_test_patch.py --create --repo ostree_repo \
--clone-repo ostree-clone --id=PATCH0002 --clean-mode
Note: restart changes which are still hard coded to Patch1
are not being changed until support for the scripts is
implemented.
Story: 2009969
Task: 45342
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Ifdcf3526bb1c0653f0a638fb40efa346870b12ee
This commit enables patch remove in a Debian env. Patch remove
command operates on the feed directory. It resets the feed ostree
HEAD to the base commit of the patch being removed and then deletes
all commits that belong to the patch.
Test:
sw-patch remove works as expected in Debian
Story: 2009969
Task: 45327
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: If01ab1a1ab4e4e58ee1713cc622feb57481219b9
This is a check of ostree log output (commit and checksum)
for determining if a patch is currently installed or not.
The ostree commands are being moved to their own file.
Additional ostree commands will be refactored in later
commits.
Test Plan:
Verify controller-0 is Patch current when no patches
installed.
Verify controller-0 is Patch current when patches
uploaded, but not applied.
Verify controller-0 is NOT Patch current when a patch is
applied.
Story: 2009969
Task: 45323
Co-Authored-By: Jessica Castelino <jessica.castelino@windriver.com>
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I84b238868badf1159c6312575704441bd996f741
Debian uses ostree for software patching.
Centos uses rpm and dnf for software patching.
The dnf API to track rpms is being removed from the debian
code. Calls to ostree interaction may be added later.
Note: One dnf call remains in upgrade-start-pkg-extract
This has been left, as upgrade for Debian is
still being investigated.
This change eliminates sw-patch-agent service failures
caused by calling dnf and therefore also eliminates the
200.006 major alarm raised due to service failures.
Test Plan:
PASS: Build / Boot / Bootstrap / Unlock AIO-SX
PASS: Upload / Apply a patch on AIO-SX
PASS: reboot after applying a patch and verify
no alarms or excessive error logs for patching.
PASS: Delete a patch on AIO-SX
Depends-On: https://review.opendev.org/c/starlingx/update/+/840721
Story: 2009969
Task: 45242
Co-Authored-By: Jessica Castelino <jessica.castelino@windriver.com>
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I823b45efb90b4106fc8b684ee49453e2354e9315
This commit enables sw-patch delete and cleans up the
ostree tarball from the /opt/patching/packages directory.
Test:
sw-patch delete <patchID>
Story: 2009969
Task: 45289
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I8afff9511391aa7cdf540e09bd53007032b98aa6
This commit enables sw-patch apply functionality in a Debian env.
Patch apply extracts software.tar and store its contents in a
temporary location. The extracted contents include objects, refs,
extensions, summary, tmp, state, etc. folders. All these folders
are then recursively copied to the feed ostree stored at
/var/www/pages/feed/rel-<SW-VERSION>/ostree_repo/
Test case:
sw-patch apply
Story: 2009969
Task: 45244
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I9134f90350265e9388d8b317adda7d671e72e2d9
This allows creation of a developer-signed patch
based on ostree.
There are still some issues to work out but this
allows for a starting point in developer patch testing.
Story: 2009969
Task: 45215
Co-Authored-By: Luis Sampaio <luis.sampaio@windriver.com>
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: If86d28daff2505a1f6fdc5194e1d0b92dc742ef9
In CentOS, the contents of software.tar are extracted and
saved under /opt/patching/packages directory. In Debian,
we rename software.tar to <PatchID>-software.tar before moving
it to /opt/patching/packages as a part of patch upload and
clean up the tarball as a part of patch delete.
Test:
sw-patch upload
sw-patch delete
Verify that <PatchID>-software.tar is created and cleaned up
Story: 2009969
Task: 45223
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: Idea3ff424153039367310c483ce33d058439b9c0
- Removed the py27 target for sw-patch since it is
python3 only.
- Set the base python in tox.ini to python3.
- Removed the site-packages directive for pylint since
site level rpm component no longer needs to be installed.
- Added the pep8 target (it just calls flake8).
- Removed redundant settings already set at testenv level.
- Cleaned up bandit suppressions that were not needed.
- Cleaned up the flake8 suppressions that were not needed.
- Cleaned up the pylint suppressions that were not needed.
- Minor code cleanup to reduce number of flake8 suppressions
- Minor code cleanup to reduce number of pylint suppressions
- Updated the copyright dates for updated source files
Test Plan:
Tox
Story: 2009969
Task: 45209
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Ifccf2a274530b14bacb6ce2dc32f8cca01e26217
The PatchData object is updated to reflect the fields that would be
used in the Debian env. Similarly, the code to parse metadata has
been updated to incorporate the newer ostree commits and checksums.
Tests:
sw-patch upload
sw-patch delete
Story: 2009969
Task: 45203
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Change-Id: I9efca439f212f1ab91655b8024bf4f8937ff882f
The previous patching scripts were specific to the
centos rpm patching directories, so these scripts
are cloned for the debian env and updated to reflect
the new python and binary locations.
These scripts affect build commands and not runtime.
These scripts are not operational as they point to
utilities and imports that are not converted to ostree
yet.
Test Plan:
Verify build and ISO creation for debian succeed.
Story: 2009969
Task: 45198
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Ia903c6b4f07e3d1409c04c2cc17ff40a2e5a4c7f
The sw-patch bash-completion file needs to be located at
usr/share/bash-completion/completions/sw-patch
Test Plan:
Build / Boot / Bootstrap / Install on Debian
Verify sw-patch <tab> shows the sw-patch commands
Story: 2009969
Task: 45201
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Icdb3915fbc36f753d058a643bc04a6fdb0923867
Remove the rpm imports from debian patching code.
The dnf imports will be removed in a later commit.
The patch code had methods, variables and subprocesses
that reference 'rpm'. Most of these have been removed
or renamed. The remaining 'rpm' references will be
removed as functionality related to those calls is
implemented for debian ostree.
The code is being converted to ostree, so these changes
are not currently runnable, nor were the rpm calls on
debian.
The createrepo calls are also removed, ostree equivalent
calls may (or may not) be added in a followup commit.
The subprocess exceptions are made more generic, as
any uncaight exception in API handling could make the patch
controller non-responsive. Robustness improvements may be
investigated in a followup commit.
Test Plan:
Verify build/install/bootstrap/unlock on Debian.
Verify sw-patch upload /delete do not report failures
using a signed patch. (Note: used an rpm patch for centos)
Story: 2009969
Task: 45192
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I0590950868805b89dd1e302397d83f1a6f5e244a
systemd reports the following:
systemd[1]: /lib/systemd/system/sw-patch.service:11:
Standard output type syslog+console is obsolete,
automatically updating to journal+console.
Please update your unit file, and consider removing
the setting altogether.
This change is similar to controllerconfig service file
771e6ca734
Test Plan:
Build / Bootstrap / Unlock on Debian
Verify that /var/log/daemon.log does not show the 'obsolete' errors
Verify that logs during sw-patch service start are logged.
Story: 2009969
Task: 45145
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I1e900f12884f8626e76e72e79de2453de32691cf
On Debian running python3, patch dev signature verification
fails because the expected string becomes malformed using
the 'update' method.
This fixes the issue, by not calling 'update' and instead
directly passing the signature string to the constructor.
Test-Plan:
Verify on Debian that a sample designer patch can be
imported (when the dev certificate is installed).
Verify that altering the DEV_CERT_CONTENTS causes the
dev certificate to be rejected and the patch to not import.
Co-Authored-By: Jessica Castelino <jessica.castelino@windriver.com>
Story: 2009969
Task: 44950
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I9c2d2ce3cbcf75f41d7886057959e2dbebcff084
The original cgcs-patch is rpm based which requires a
complete re-write to work on ostree/dpkg systems like Debian.
The code has been forked, since the older Centos env and
python2.7 are end-of-life.
Forking the code allows all new development to not
require re-testing on Centos.
The debian folder under cgcs-patch has been moved
under sw-patch
Renaming and refactoring will be done in later commits.
pylint is un-clamped in order to work on python3.9
Some minor pylint suppressions have been added.
Test Plan:
Verify that this builds on Debian
Verify that the ISO installs the new content on Debian without
breaking packages that import cgcs_patch.
Verify patching service runs on Debian
Co-Authored-By: Jessica Castelino <jessica.castelino@windriver.com>
Story: 2009101
Task: 43076
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I3f1bca749404053bae63d4bcc9fb2477cf909fcd
Luis Sampaio