229 Commits

Author SHA1 Message Date
Scott Little
a997e11c7f Make sbuild nice to it's host
StarlingX build tools consume excessive cpu and io which can starve
non-build services, including kubernetes and calico. This results in
timeouts, restarts, and general instability of calico and kubernetes
which in turn can the build to fail. This affect is observed on the
main StarlingX build server.

The major build steps should employ 'nice' and 'ionice' to ensure
that critical services on the build host are not starved for cpu or io.

Closes-bug: 2095512
Change-Id: Id228c43188812df373a17602336ae7a61c5569a6
Signed-off-by: Scott Little <scott.little@windriver.com>
(cherry picked from commit f360849703b54612f497c3516a96f6cef32799ed)
2025-01-23 08:35:35 -05:00
Scott Little
6e16428c26 Make builder nice to it's host
StarlingX build tools consume excessive cpu and io which can starve
non-build services, including kubernetes and calico. This results in
timeouts, restarts, and general instability of calico and kubernetes
which in turn can the build to fail. This affect is observed on the
main StarlingX build server.

The major build steps should employ 'nice' and 'ionice' to ensure
that critical services on the build host are not starved for cpu or io.

Closes-bug: 2095512
Change-Id: I9f9fd3c6cb3daef40c90299184ca09bc5a0d2415
Signed-off-by: Scott Little <scott.little@windriver.com>
2025-01-22 09:09:01 -05:00
Scott Little
b9e695f8f7 fix: stx-pkgbuilder failed to build due to snapshot.debian.org 503
Upversion from debootstrap_1.0.126+nmu1_all.deb
to debootstrap_1.0.128+nmu2+deb12u1_all.deb.

The new version is part of snapshot.debian.org/archive/debian/20211128T160803Z
And mirror.starlingx.windriver.com has a mirror of that snapshot.
This in turn allows the stx-pkgbuilder container to pull debootstrap
from either of two sources.

Closes-Bug: 1996940
Change-Id: Ifc5b5d1a4197df8cc9bcae0e1e55c36dcf51173c
Signed-off-by: Scott Little <scott.little@windriver.com>
2024-11-13 13:40:44 -05:00
Dostoievski Batista
3412f5818b Fixing error when trying to index schroot
The decompose functions uses .split('-') to separate the components of
the schroots, but if the username has a dash it will fail. This changes
adds the username as argument and remove it from the schroots folder's
name before splitting it.

Test plan:
    PASS - With username without '-' run build-pkgs -a -c --parallel 3
    PASS - With an '-' in the username, run build-pkgs -a -c
        --parallel 3

Closes-bug: 2084170

Change-Id: Ibc94a23d4d38dd3391d0e3891d67749d8094f68d
Signed-off-by: Dostoievski Batista <dostoievski.albinobatista@windriver.com>
2024-10-10 17:17:41 -03:00
Zuul
661876b665 Merge "build-pkgs may fail if --parallel <n> is not specified" 2024-10-09 19:41:00 +00:00
Scott Little
da6e8acd61 build-pkgs may fail if --parallel <n> is not specified
partial-bug: 2081843
Change-Id: Idcfe636fa14171ee252685ad74ccddb0d7ac6f9d
Signed-off-by: Scott Little <scott.little@windriver.com>
2024-10-09 14:52:51 -04:00
Davlet Panech
db26697886 minikube: better error handling
* stx control stop: don't auto-start minikube
* return non-zero to shell when minikube profile is stopped

Closes-Bug: 2083862
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: Idd8dcc382c189fa36d40eb9b544e83a84a7ce1d5
2024-10-07 15:36:37 -04:00
Zuul
71604e5dfb Merge "fix early exit of clone_chroot" 2024-10-07 18:06:51 +00:00
Scott Little
948070c723 fix early exit of clone_chroot
Change-Id: I4c26e50cf1372f4ae0ab3c5bb3b28de7caa65104
Signed-off-by: Scott Little <scott.little@windriver.com>
2024-10-07 13:55:24 -04:00
Zuul
8377504c72 Merge "stx control start: increase minikube start timeout" 2024-10-07 17:41:46 +00:00
Davlet Panech
720332e74d stx control start: increase minikube start timeout
Increase minikube startup timeout to 120 seconds.

Closes-Bug: 2083849

Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I7db6a5e6bd8e56fe61c08c114064eedfffd54e6f
2024-10-07 12:15:27 -04:00
Scott Little
eb2f57c3a4 relocate set_unique_id to avoid errors
partial-bug: 2081843
Change-Id: I55f465ec8ec66b86c3088c52311744370c62e07f
Signed-off-by: Scott Little <scott.little@windriver.com>
2024-10-07 11:52:50 -04:00
Scott Little
5f6223bf08 Add post build cleanup of tmpfs chroots
The memory tied up in tmpfs filesystems needs to be released
after building packages.

This update also tries to simplify some of the more tortured
logic, overly complex functions, code duplications and
confusing or inconsistent namings.

partial-bug: 2081843
Change-Id: I59c12602929a8da1c5076c27243d6e6dca951c61
Signed-off-by: Scott Little <scott.little@windriver.com>
2024-10-03 14:41:22 -04:00
Scott Little
bd9a4b6959 ihandle case where tmpfs_percentage is ommitted from clone request
Closes-bug: 2083023
Change-Id: I754179c03ccb9c8f213bfd22301dd0e14e015f63
Signed-off-by: Scott Little <scott.little@windriver.com>
2024-09-27 10:55:47 -04:00
Zuul
8059deb2d5 Merge "Add option to use tmpfs during build-pkgs" 2024-09-26 15:54:28 +00:00
Zuul
cb525b9eab Merge "Set flag ignore errors to false to rm old chroot" 2024-09-26 13:00:51 +00:00
Scott Little
36c035f7ef Add option to use tmpfs during build-pkgs
Building in a tmpfs speeds the build process and greatly reduces
the disk io required during a build, leaving the host far more
responsive to other tasks.

Added option --tmpfs_percentage=N to the build-pkgs command
to control how much memory can be used for tmpfs build environments.
The valid range is 0-50%. The default is 0% resulting in a traditional
disk only build.

The available memory on the host is multiplied by the percentage
to determine how much is availabe for tmpfs in total.  This total
is then assigned to up to N-1 of the N parallel build environments,
with the minimum tmpfs environment size being 10GB.
One environment is reserved for disk based builds.  First time
package builds are assigned to tmpfs based build environments in
preference over the disk base environments. However, if a package
has failed a prior build attempt, subsequents attemps will only
occure on a disk based build environmnet.  This may help if the
build failed due to a too small tmpfs build environment.  It also
may leave the environment intact for mor indepth debugging.

Testing has revealed that 20gb is required to build the largest
packages (linux, ceph, kubernetes).  In order to avoid costly
rebuilds of these large packages, the choice of tmpfs percentage
and number of parallel builds is important.

e.g. on a single user host with 128GB of memory, an appropriate
choice might be...

    --parallel=4 --tmpfs_percentage=50

...yielding 3 tmpfs build environments of aprox 20GB each.
A higher parallelism, or a lower tempfs percentage will result
in build environments that drop below 20GB, and you might start
seeing rebuilds of large packages.

Further development is suggested.  If we can add an advisory to the
dsc metadata suggesting a minimum space requireemnt for the build of
a package, we can proactively assign large package to a build environment
that is large enough to support it, avoiding rebuilds.

Testing:
   - build-pkgs without --tmpfs_percentage
     Only disk base build envoronments are used.
   - build-pkgs --parallel=4 --tmpfs_percentage=20
     On 128GB machine, Only 2 10-12 GB tmpfs are used,
     the other two remanin disk based.
     Large packages fail in 10GB tmpfs, and a pass when retried on disk.
   - build-pkgs --parallel=4 --tmpfs_percentage=50
     On 128GB machine, Creates 3 18-20 GB tmpfs.
     Large packages build in tmpfs without need for rebuild.

partial-bug: 2081843
Change-Id: I09dd2f60afc3e866ec8f86b6898d41f19a419d87
Signed-off-by: Scott Little <scott.little@windriver.com>
2024-09-25 10:35:28 -04:00
Fabricio Henrique Ramos
339837067d Set flag ignore errors to false to rm old chroot
When removing old disused chroot, the instruction shutil.rmtree is
invoked using flag ignore_errors=True, this commits changes that flag to
ignore_errors=False, so that if anything wrong happens, an exception is
thrown and the error is logged. Current way no errors are logged.

Test Plan:
PASS: build package
      rebuild container pkgbuilder forcing new chroot creation
      rebuild package
      force old chroot to fail to be removed (use chattr -i on a file
from old chroot)
      error is logged to /localdisk/pkgbuilder.log
PASS: build package
      rebuild container pkgbuilder
      build package

Change-Id: I6c9488e9b224cba3791b14c1f964c158b003cbd5
Signed-off-by: Fabricio Henrique Ramos <fabriciohenrique.ramos@windriver.com>
2024-09-24 17:19:05 -03:00
Igor Frazzon
543c9d1c26 stx: Fix Build env failing after rebooting.
Added checks to ensure the Minikube profile exists and is running
before executing stx commands. Provided user instructions for creating
the profile and starting pods when necessary.

Solution: fix the log message's format string
* Added a check to verify if the Minikube profile exists before
  executing stx commands.
* Logged an error message and provided instructions to create
  the Minikube profile if it is missing.
* Included guidance to run `./stx-init-env` from the `stx-tools`
  directory to create the missing profile.
* Added a message to prompt the user to run `stx control start
  --wait` when attempting to run `stx shell` if the builder
  container is not running.
* Ensured that execution halts if the profile is missing, avoiding
  potential errors.

TESTS
========================
* Stop the Minikube profile and run `stx control status`, ensure
  the message `stx control start --wait` is shown.
* Ensure that running `stx control start --wait` starts the Minikube
  profile and allows commands to run without errors.
* Stop the Minikube profile and run `stx control stop`, ensure the
  profile starts and the command runs without errors.
* Remove the containers with `stx control stop` and run `stx shell`,
  ensure the message `stx control start --wait` is displayed.
* Remove the Minikube profile and ensure the `stx control start/stop/
  status` commands show the message `./stx-init-env` to create the
  profile.
* stx control start/stop/status` commands function correctly after
a machine restart
Closes-Bug: 2080681

Change-Id: I474ad59947fb98af434e7616859f5256a226f903
2024-09-19 15:09:30 -03:00
Leonardo Fagundes Luz Serrano
d5eec5ba18 Add 7z cmd support to LAT container
This utility can extract files from ISOs
and is used to create the pre-patched ISO
in the patch-iso-debian script.

Test Plan:
pass - build LAT container
pass - confirm 7z is available in LAT

Story: 2011098
Task: 50849

Change-Id: Iefc0aeca3c70f06def3e5d6e943dae74fe20ce25
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
2024-08-15 18:02:44 -03:00
Dostoievski Batista
1ee6982c89 Simplifying patch-builder use in builder container
To make easier for the user to utilize the patch-builder tool we
made 3 different changes:
- stx-builder.Dockerfile: Add python modules to be installed when
building the image.
- buildrc: Add /stx/patch to $PATCH env variable, making "patch-builder"
script more easy execute.
- userenv: Update information in the "welcome" message for builder
container.

Test plan:
    PASS: Rebuild all containers, using:
        "./stx-init-env --rebuild"
    PASS: Test usual build process, using:
        "downloader"
        "build-pkgs"
        "build-image"
    PASS: Test patch builder, using:
	"patch-builder --recipe <path>"

Story: 2010676
Task: 50438

Change-Id: Ic458f58d376a64d3654c6f19e5d11d0e99f8d6f9
Signed-off-by: Dostoievski Batista <dostoievski.albinobatista@windriver.com>
2024-07-23 14:59:50 -03:00
Zuul
aa67db0b02 Merge "Add support for pre-patched ISO on LAT container" 2024-07-05 14:46:38 +00:00
Dostoievski Batista
3c9fa8296e Add support for pre-patched ISO on LAT container
This commit adds the necessary packages and env variables necessary in
order to create pre-patched iso using apt-ostree inside inside the
lat container.

Test plan:
    PASS: Ran "./stx-init-env --rebuild"
        Check if packages are installed
        Check if env variable are set and available
    Other tests are covered by:
        https://review.opendev.org/c/starlingx/root/+/922827

Story: 2010676
Task: 50437

Change-Id: I9d5b6de52ad41525b581dc6956529ae94e7f3f53
Signed-off-by: Dostoievski Batista <dostoievski.albinobatista@windriver.com>
2024-07-04 15:11:10 -03:00
Leonardo Fagundes Luz Serrano
e5972742ce Zuul: Fix test-requirements and satisfy pep8 requirements
Fixes:
- Missing comma in test-requirements.txt
- pep8 requires base classes to inherit from 'object'

Test Plan:
pass - run tox

Closes-Bug: 2070937

Change-Id: Ibe659bd0a02a7ca047d4406b9555a2419fdb915f
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
2024-06-26 17:27:29 -03:00
Dostoievski Batista
b918126a47 Adding ISO tools in builder container
We need to be able to work with ISO images inside builder container.
This change makes the stx-builder privileged so we can mount a
filesystem inside it and we add mkisofs, isomd5sum and syslinux-utils
packages to be installed when building the image.

Test plan:
    PASS: Build stx-builder image.
    PASS: Run ./stx-init-env --rebuild with success.

Story: 2010676
Task: 50189

Change-Id: I44b0d99965befd9e78934ff3dbcbe4bdf02514af
Signed-off-by: Dostoievski Batista <dostoievski.albinobatista@windriver.com>
2024-05-24 17:13:49 -03:00
Davlet Panech
44ccc58f55 stx-init-env: get LAT installer from $STX_MIRROR
LAT docker file downloads the installer from a hard-coded URL that
points to https://mirror.starlingx.windriver.com . Allow users to
override this location by defining STX_MIRROR_URL in the host
environment. By default, guess mirror location from stx.conf.

TESTS
==========================
* Rebuild LAT container and make sure it uses the mirror URL from
  stx.conf
* Set STX_MIRROR_URL prior to calling stx-init-env and make sure it
  gets picked up by the docker file

Story: 2010055
Task: 49883

Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: Id8ea88407f74003db934337efd574451658633d8
2024-04-17 16:23:05 +00:00
Davlet Panech
14d5030d09 aptly: update expired GPG key
Aptly repos are signed with a GPG key embedded in environment
containers. That key expired today (2024-02-23).

Replace key with a new one that does not expire at all.

Partial-Bug: 2054862
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I41a5c7a785a23eb8c9546e99865ecf62faaf506a
2024-02-23 22:47:45 -05:00
Zuul
fa10efb041 Merge "stx control stop: faster shutdown" 2024-02-20 21:31:23 +00:00
Davlet Panech
f5ddd163a1 stx control stop --wait: helm compatibility
Don't use --wait with helm uninstall because it requires helm >= 3.7,
and even in those versions doesn't work correctly.

Story: 2011038
Task: 49549

Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I4f3be32bf4ce84e1670e7884fc09c3ddac00b85a
2024-02-16 08:23:07 -05:00
Davlet Panech
86d219ada7 stx control stop: faster shutdown
Make sure aptly & builder containers catch and handle SIGTERM. Otherwise
"stx stop" sends the signal, 2 out of 6 containers ignore it, then
docker waits for ~15 seconds and SIGKILL's them.

* stx-builder.Dockerfile: change default image command from plain "bash"
  to "tini" that starts "sleep infinity". Tini catches and broadcasts
  signals to its own children (sleep), enabling graceful shutdown to
  work

* aptly: replace call to "supervisord" to "exec supervisord", to make
  sure it runs as PID 1 and actually receives signals from docker.

* stx_control.py: slightly reduce loop sleep time in "stx control stop"

TESTS
==================
* Retest "stx control start --wait"
* Make sure builder's entry point executes "finisSetup.sh" script, as
  before this change
* Make sure "stx control stop --wait" exits quickly (~4 seconds on my
  machine, down from ~15 seconds)

Story: 2011038
Task: 49577

Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I984846fc45349be045c069b84186f12179fe36ad
2024-02-15 14:28:54 -05:00
Davlet Panech
4187e73f86 Commands to reset the build environment
* stx script:
- New command "stx control is-started" to complement start/stop
- New option "stx control {start,stop} --wait"

* stx-init-env:
- new option --reset: delete chroots + restart pods
- new option --reset-hard: stop pods, delete local workspaces,
  chroots, aptly, docker & minikube profile
- rename option "--nuke" to "--delete-minikube-profile"; old spelling
  is still accepted with a warning
- renamed & refactored some functions

* import-stx:
- new env var STX_RM_METHOD: may be optionally set to "docker" for
  deleting root-owned files via "docker run", rather than "sudo"

TESTS
=========================
* Misc sanity checks using minikube & k8s
* Manually tested blacklist checks in safe_rm()
* rm via "sudo" vs "docker run"
* Using minikube:
- stx-init-env
- stx-init-env --rebuild
- stx start, build all packages, --reset, build all packages
- stx start, build all packages, --reset-hard, stx-init-env,
  build all packages

Story: 2011038
Task: 49549

Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: Ife4172ae9fa7b58332ac7ad65beb99525bc2a1a3
2024-02-14 13:19:31 -05:00
Zhang Xiao
1275dc128f LAT: update to lat-sdk-20231206
Update lat-sdk.sh for the meta-lat commit:
https://github.com/Wind-River/meta-lat/commit/3ddaf92342

Test plan:
  PASS: Run "stx-init-env --rebuild".
        Run "stx shell" to enter new builder container.
        Run "build-image -c".
  PASS: Check the boot checksum of new build ostree repos, the order
        is always standard kernel, rt kernel, vmlinuz and initrd.
  PASS: Qemu boot OK;

Closes-Bug: 2045914

Change-Id: I256c83d213bde76b82879711d8a0e038bcbf394d
Signed-off-by: Zhang Xiao <xiao.zhang@windriver.com>
2023-12-07 23:36:35 +08:00
Scott Little
fee99ba459 cengn reference removal - debian
mirror.starlingx.cengn.ca no longer exists. CENGN is kindly forwarding
requests to the new location mirror.starlingx.windriver.com for now, but
that will only last a few months. We need to replace all the references
with the new URL.

I will also remove as many 'cengn' references as possible, replacing
them
with 'stx_mirror'

Partial-Bug: 2033555
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: Icea255bcf628d0535991156a729ef21421646bfc
2023-11-07 21:31:00 -05:00
Davlet Panech
1f7a760068 Remove obsolete stx.conf upgrade
Remove code that automagically changes debian archive URLs in stx.conf
to public WR mirror URLs. This upgrade was added last year and was only
relevant to un-upgraded development environments that existed at that
time (eg on developer desktops). It is no longer relevant. Removing it
because it breaks the ability of the user to explicitly set those URLs
to the real Debian archive repository under some circumstances.

How to reproduce:
* Remove existing stx.conf
* Copy stx.conf.example => stx.conf
* Replace the key:
  stx config --add stx config --add project.debian_snapshot_base http://snapshot.debian.org/archive/debian
* Upgreade configuration:
  stx config --upgrade
* ===> URL gets reset back to default (mirror) in stx.conf

TESTS
==================================
* Re-run above steps and make sure this doesn't happen

Closes-Bug: 2042854

Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I3ff011d747f2804ade0152ad0296b00ba2785ebf
2023-11-07 11:14:20 -05:00
Zuul
3752f6adf1 Merge "cengn reference removal - debian" 2023-11-07 15:36:53 +00:00
Scott Little
2684e9e3fd cengn reference removal - debian
mirror.starlingx.cengn.ca no longer exists. CENGN is kindly forwarding
requests to the new location mirror.starlingx.windriver.com for now, but
that will only last a few months. We need to replace all the references
with the new URL.

I will also remove as many 'cengn' references as possible, replacing
them with 'stx_mirror'

TESTS
=====
Run "stx config --upgrade" with various combinations of the old,
  new & missing values.
Run "./stx-init-env --rebuild"  and verify stx-localrc is updated
  as well.

Partial-Bug: 2033555
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: I18491fb5aa0087029d51bb35c236613c1dc092bc
2023-11-06 19:51:14 +00:00
Luan Nunes Utimura
9694971db1 debbuilder: Support layer-specific repo in sbuild
In [1], a mechanism was proposed to create layer-specific aptly binary
repositories in addition to the existing `deb-local-binary`, to better
separate the binary dependencies of each layer.

For this new mechanism to work, new repositories must be dynamically
taken into account by `sbuild`, by adding a new `--extra-repository`
argument referring to the layer of the package being built, which is
precisely what this change does.

[1] https://review.opendev.org/c/starlingx/root/+/893095

Test Plan:
PASS - Build packages of different layers successfully

Story: 2010797
Task: 48698

Change-Id: Ib09ddf042584bc418c6a07773e920901961938ee
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
(cherry picked from commit 25c1d0807b6142955f828469c06425ace43e317d)
2023-10-20 15:43:48 -03:00
Li Zhou
48f28a811d Remove kernel pkgs with abiname in yaml config files
Remove the kernel image packages with abiname in LAT yaml config files.
After commit <Add pkgs without abiname for image/headers>, packages
for linux unsigned images without abiname are created.
So replace
linux-image-5.10.0-6-amd64-unsigned
linux-rt-image-5.10.0-6-rt-amd64-unsigned
with
linux-image-stx-amd64
linux-rt-image-stx-amd64

This is a preparation for kernel upgrading with major version.

Test plan:
 PASS: Build all the packages and iso successfully.
 PASS: The rt/std installations are fine for both qemu and lib.
 PASS: No warning appears for insmod/modprobe.

Depends-On: https://review.opendev.org/c/starlingx/kernel/+/896187

Story: 2010643
Task: 48816

Signed-off-by: Li Zhou <li.zhou@windriver.com>
Change-Id: I22c1481527b5b0f73d63f7811f8c4f07401839eb
2023-09-22 02:30:41 -04:00
Davlet Panech
5188f106a0 stx cleanup: new sub-command
Add new subcommand to stx tool:

  stx cleanup [--force][--dry-run] [docker|minikube]

It runs "docker system prune --volumes" in the builder docker instance,
or in the minikube's docker instance, or both (default).

Partial-Bug: 2020807
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I35a7a439e6a52455111e5bc3fff1e9377a407085
2023-09-18 15:16:06 -04:00
Scott Little
30822201fa Update ca-certificates within chroot
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: I2e8c4fb6dba75a700d69c0fa15d165e0f90aa631
2023-07-26 18:31:23 -04:00
Scott Little
bfa65efc74 Update the root certificates of the Debian build containers
mirror.starlingx.cengn.ca will only authenticate against the
latest certificates.

Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: I9ea940edb529cb8bf44ab1b1433f9c3d62d7b225
2023-07-24 15:33:36 -04:00
Li Zhou
72ca1de615 LAT: update to lat-sdk-20230510
Update lat-sdk.sh for the meta-lat commit:
d3210e17b9

The lat-sdk.sh is built by common script from now on.

Test plan:
 PASS: Run "stx-init-env --rebuild".
       Run "stx shell" to enter new builder container.
       Run "build-image -c".
 PASS: The bzImage/bzImage.sig are links to vmlinuz-5.10.0-6-amd64/
       vmlinuz-5.10.0-6-amd64.sig.
       The test team verified that RR patch apply successfully
       after initial INSVC patch apply and removal with this meta-lat
       commit.
 PASS: Qemu boot OK; AIO-DX lab installation is OK.

Closes-Bug: 2020736

Signed-off-by: Li Zhou <li.zhou@windriver.com>
Change-Id: Ie56562ee9e5b6b8b0272bfadfda8b39fb403b03c
2023-05-24 23:36:08 -04:00
Zuul
5616b0575c Merge "get_pod_name: Only return running pods" 2023-05-18 15:35:36 +00:00
Davlet Panech
ae1ba8e3d7 stx: docker pod: persist /var/lib/docker
Make sure docker pod's container FS layers are persisted on the host.
Previously, we persisted /var/run/docker in $BUILD_HOME/docker/run, but
that directory contains only sockets and similar files internal to the
docker daemon. This patch changes the helm chart to persist
/var/lib/docker in $BUILD_HOME/docker/var ; that directory contains
docker FS layers & docker images generated during the build.

TESTS
================================
* Start stx-tools containers
* Pull an image inside "stx shell"
* Stop stx-tools containers
* Make $BUILD_HOME/docker/var is not empty
* Re-start stx-tools containers & make sure the previously-pulled image
  still exists and didn't disappear

Story: 2010055
Task: 47861

Change-Id: Ibbd60f48989ecd9663b98da3167e491a9c2493c7
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
2023-04-20 10:35:50 -04:00
Scott Little
ae18863a34 get_pod_name: Only return running pods
get_pod_name was observed to return a list of all
pods satisfying the query without consideration of
the state of the pod. In one observed scenario the name of
a runing pod and an terminating pod were returned.

All subsequent commands used the podname with out checking
for multiple entries.  They clearly expect only a single
name, not multiple.

Modify get_pod_name to only return a single rinning pod name.

Closes-bug: 2015885
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: I0b7bf22c4481fc4a81012fc14ac9d388afd6b065
2023-04-12 14:46:06 -04:00
Zhang Xiao
0f1d695379 Debian: pkgbuilder: adjust apt key and repositories of sbuild
Without related apt key, `apt update` on related repository will lead
to warning messages as below, add apt key to avoid it:
W: GPG error: http://stx-stx-repomgr:80/deb-local-binary bullseye
InRelease: The following signatures couldn't be verified because the
public key is not available: NO_PUBKEY FAAD0555200D6582

There are duplicate repositories in apt sources list, remove one of
them to avoid warning messages as below:
W: Target Packages (main/binary-amd64/Packages) is configured multiple
times in /etc/apt/sources.list:1 and
/etc/apt/sources.list.d/sbuild-extra-repositories.list:1
...

Test Plan:
PASS: ./stx-init-env --rebuild
PASS: build-pkgs -a; build-image
PASS: No such warning message any more

Story: 2010643
Task: 47639

Signed-off-by: Zhang Xiao <xiao.zhang@windriver.com>
Change-Id: Ia4907fa92a92afeeca73abc35dfb712ea6ae8cb4
2023-03-15 09:24:53 +08:00
Li Zhou
066f34ea9e LAT: upgrade to 20221226
We need to improve secure boot on below issues:
(1)The shim/grub are compiled in debian stx instead of lat now,
   but they need to take affect by rebuilding lat sdk;
(2)The kernel/initramfs are signed by keys embedded in meta-secure-core
   (upstream source code related with lat sdk);
(3)There is no automatic process for signing all of them
   (on signing server) and building them into image by lat, although
   the signing interface is designed on signing server;
(4)We need to divide signed build and unsigned build because most
   developers don't have access to signing server.

So these commits are brought out in this topic:
https://review.opendev.org/q/topic:secure_boot_enhancement

To corporate with these commits, changes are done for lat-sdk.sh:
(1) When we move shim/grub images out of lat sdk and use the packages
built by starlingx debian, issues occur. Below changes are done
to solve the errors reported:
  When grub.cfg's path in grub-efi-amd64*.deb is same with the path
it is to be copied as "BOOT_GRUB_CFG", this copying should be avoided.
  There are grub mods installed in grub-efi-amd64*.deb in path
/boot/efi/EFI/BOOT/x86_64-efi, so flag "-r" is needed when rootfs_efi
is copied.
  When EFI_SECURE_BOOT is disabled, bootx64-nosig.efi is copied to
bootx64.efi and there is no need to check if bootx64.efi exits or
not.
(2) Remove all the gpg signings from genimage and add script interface
initramfs-sign-script for signing initramfs and mini_initramfs.
The files about grub.cfg aren't verified by starlingx debian
any more, so the signings aren't needed.
The LockDown.efi and kernel images will be signed in script
rootfs-post-scripts.

Test plan:
 The tests are done with all the changes which involve tools/integ/root
 repo and lat-sdk.sh and signing tools on signing server.
 - PASS: Follow below build process to build unsigned image
         and build successfully:
         build-pkgs
         build-image
 - PASS: Follow below build process to build signed image
         and build successfully:
         Prepare keys on dockers and signing server to setup access
         to signing server without password.
         build-pkgs
         export SIGNING_SERVER="user@signing_server_ip"
         <PATH>/sign-secure-boot_debian
         build-image
         The <PATH> is:
         /localdisk/designer/${USER}/stx/cgcs-root/build-tool
 - PASS: Do AIO-DX installation successfully for both
         unsigned image on secure boot disabled lab and
         signed image on secure boot enabled lab.

Depends-On: https://review.opendev.org/c/starlingx/tools/+/868918

Story: 2009221
Task: 47097

Signed-off-by: Li Zhou <li.zhou@windriver.com>
Change-Id: Ia144b8e04c110af46ba5732e270ed239adf37aa1
2023-02-28 20:23:24 -05:00
Li Zhou
c87e465a01 tools: add command 'stx control keys-add'
From https://review.opendev.org/c/starlingx/root/+/868920,
build process for signed image is as below:
(1) Prepare keys on dockers and signing server to setup access
to signing server without password:
Use "ssh-keygen -t rsa" to create a ssh key pair, e.g.
    id_rsa.pub and id_rsa.
Prepare on lat docker (default user is root):
    mkdir ~/.ssh
    copy id_rsa to ~/.ssh directory
Prepare on builder docker:
    mkdir ~/.ssh
    copy id_rsa to ~/.ssh directory
    sudo mkdir /root/.ssh
    sudo copy id_rsa to /root/.ssh directory
Prepare on signing server:
    append id_rsa.pub to the file on signing server:
    /home/${signing_user}/.ssh/authorized_keys

(2) Run on builder docker:
export SIGNING_SERVER="signing_user@signing_server_ip"
<PATH>/sign-secure-boot_debian
build-image

The <PATH> is:
/localdisk/designer/${USER}/stx/cgcs-root/build-tools

An extension to the 'stx' tool to inject keys into the
relevant containers is added as
'stx control keys-add --key-type=signing-server --key=[key file]'
which can replace all the work on dockers in (1).

Test plan:
 - PASS: run 'stx control keys-add --key-type=signing-server --key=[key
         file]' after containers start up,
         and check the keys mentioned above are right in containers.
 - PASS: run the complete build process as below and get a bootable iso
         with secure boot enabled:
         stx control keys-add --key-type=signing-server --key=[key file]
         stx shell
         export SIGNING_SERVER="signing_user@signing_server_ip"
         <PATH>/sign-secure-boot_debian
         build-image

Story: 2009221
Task: 47097

Signed-off-by: Li Zhou <li.zhou@windriver.com>
Change-Id: Id84c5d96b8565a7c7d8da018730f2591d86e2d24
2023-02-28 20:21:18 -05:00
Li Zhou
c181eda91b debian: secure boot enhancement
We need to improve secure boot on below issues:
(1)The shim/grub are compiled in debian stx instead of lat now,
   but they need to take affect by rebuilding lat sdk;
(2)The kernel/initramfs are signed by keys embedded in meta-secure-core
   (upstream source code related with lat sdk);
(3)There is no automatic process for signing all of them
   (on signing server) and building them into image by lat, although
   the signing interface is designed on signing server;
(4)We need to divide signed build and unsigned build because most
   developers don't have access to signing server.

So below changes (cowork with other changes in integ/root repo and
lat-sdk.sh and signing tools on signing server) are done to fix
above issues.
(1) Use shim/grub images from debian stx instead of LAT sdk sysroots:
Change the LAT config file to build iso using the shim/grub
images built by StarlingX debian instead of those from
LAT sdk sysroots. And update the paths accordingly for:
grubx64.efi in rootfs-post-scripts and bootx64-nosig.efi in
iso-post-script.

(2) Make secure boot build disabled by default:
Set EFI_SECURE_BOOT as disable.

(3) Add hooks for gpg signing of initramfs:
Define initramfs-sign-script.

(4) When lat docker is running to build image, gpg signings are
done by the signing scripts which are hooks of lat build process,
so the tool for accessing remote signing server is needed.

Test plan:
 The tests are done with all the changes which involve tools/integ/root
 repo and lat-sdk.sh and signing tools on signing server.
 - PASS: Follow below build process to build unsigned image
         and build successfully:
         build-pkgs
         build-image
 - PASS: Follow below build process to build signed image
         and build successfully:
         Prepare keys on dockers and signing server to setup access
         to signing server without password
         build-pkgs
         export SIGNING_SERVER="user@signing_server_ip"
         <PATH>/sign-secure-boot_debian
         build-image
         The <PATH> is:
         /localdisk/designer/${USER}/stx/cgcs-root/build-tool
 - PASS: Do AIO-DX installation successfully for both
         unsigned image on secure boot disabled lab and
         signed image on secure boot enabled lab.
 - PASS: Failure path tests include:
         [NG]unsigned image on secure boot enabled
         [NG]image signed with wrong key on secure boot enabled
         [OK]signed image on secure boot disabled
         NG: fail to boot; OK: succeed to boot.

This commit works together with:
https://review.opendev.org/c/starlingx/root/+/868920
https://review.opendev.org/c/starlingx/integ/+/868919

Story: 2009221
Task: 47097

Signed-off-by: Li Zhou <li.zhou@windriver.com>
Change-Id: I3c79dad2f915707cede8dff67fbe245bde1f4ad0
2023-02-23 02:23:58 -05:00
Luis Sampaio
591c6e1564 Update dockername for lat container
the k8s selector matchs the exact name and as the lat
container name is 'lat-tool' it can't find the pod when
we run stx shell --container lat. This commits adds the
suffix -tool when trying to get the lat pod name.

Test Plan:
Pass: stx shell
Pass: stx shell --container repomgr
Pass: stx shell --container lat
Pass: stx shell --container docker
Pass: stx shell --container pkgbuilder
Pass: stx shell --container builder-files-http

Story: 2010294
Task: 47307
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I6ef2af93c26215f3de4e4f32907f819c32d8c413
2023-02-13 13:11:20 -08:00