root/build-tools/stx/image-layers.conf
Luan Nunes Utimura 01f8933531 Create layer-specific aptly binary repositories
In the current state of the build environment, all layers -- common,
flock, distro, compiler, etc -- share the same aptly binary repository:
`deb-local-binary`.

While this works for the vast majority of scenarios, there is one
specific scenario that the build environment does not currently support,
but which would be of great values to us.

Consider the following example:

```
Imagine that our goal is to build `python-barbicanclient`.

This particular package is required for both the platform, StarlingX,
and the containerized application, StarlingX OpenStack. However, for the
platform, this package needs to be built in the `Victoria` version,
while for the application, in the `Antelope` version.

If we compare the dependencies of each version by looking at their
respective `control` files, we can see, for example, that `Victoria`
lists `python3-keystoneauth1` as one of the dependencies *without*
specifying version constraints [1]:

* python3-keystoneauth1,

Which means that, if we attempt to build `python-barbicanclient` for the
platform, it will use whatever version is available in
`deb-local-binary`. In this case, since `base-bullseye.lst` -- from
`common` layer -- specifies version `4.2.1-2`, that would be the version
used.

On the other hand, `Antelope` lists this same dependency with a specific
version [2]:

* python3-keystoneauth1 (>= 5.1.1),

Which means that, if we attempt to build `python-barbicanclient` for the
application, it will *fail*, because there is no `python3-keystoneauth1`
available in `deb-local-binary` that matches the criteria.

However, even if we had *both* binaries in `deb-local-binary` there
would be another problem:

If we were to build `python-barbicanclient` for the platform, because
`Victoria` has no version constraint for `python3-keystoneauth1`
dependency, the most recent version available would be used, which means
that the package would be built with the source code of the `Victoria`
version, with dependencies of the `Antelope` version. This would
certainly cause problems at runtimes.

This is only a problem because *different layers use the same aptly
binary repository*.
```

Therefore, to avoid having to create a patch for each package built on
top of wrong dependencies -- when multiple version of the same package
are available -- this change proposes the use of *layer-specific aptly
binary repositories* in addition to the existing `deb-local-binary`.

The `deb-local-binary` will still exist, for the `common` layer, but
every other layer will have its own aptly binary repository, e.g.:

  * deb-local-binary-flock;
  * deb-local-binary-distro;
  * deb-local-binary-compiler;
  * deb-local-binary-containers;
  * deb-local-binary-openstack;
  * deb-local-binary-<layer>.

Regardless of the package and/or layer built, `deb-local-binary` will
continue to be present in the `sources.list`. However, if the package is
from the `openstack` layer, for example, the corresponding repository --
`deb-local-binary-openstack` -- will be added to the `sources.list` so
that the build can access other dependencies that are uniquely and
exclusively relevant to that layer.

This control, in particular, is implemented in the `Depends-On` change.

[1] https://salsa.debian.org/openstack-team/clients/python-barbicanclient/-/blob/debian/5.0.1-2/debian/control#L20
[2] https://salsa.debian.org/openstack-team/clients/python-barbicanclient/-/blob/debian/5.5.0-1/debian/control#L20

Test Plan:
PASS - Run `downloader` -- and its layer variants -- successfully:
       * downloader -l compiler
       * downloader -l containers
       * downloader -l distro
       * downloader -l flock
       * downloader -l openstack
PASS - Verify that multiple binary repositories were created:
       * repo_manage.py list
PASS - Run `build-pkgs -c -a --refresh_chroots` -- and its layer
       variants -- successfully:
       * build-pkgs -c -l compiler --refresh_chroots
       * build-pkgs -c -l containers --refresh_chroots
       * build-pkgs -c -l distro --refresh_chroots
       * build-pkgs -c -l flock --refresh_chroots
       * build-pkgs -c -l openstack --refresh_chroots
PASS - Run `build-image` successfully

Story: 2010797
Task: 48697

Depends-On: https://review.opendev.org/c/starlingx/tools/+/896770

Change-Id: I496cceeab084112b7b8e27024ead96e8da0c6a11
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
(cherry picked from commit f953c4a671)
2023-10-05 00:39:09 +00:00

19 lines
632 B
Plaintext

# This configuration file contains the list of layers that should be taken
# into account by the `build-image` script.
#
# The rationale behind this list is that not every layer -- and their
# respective binaries -- are needed to create an ISO.
#
# Layers like `containers` and `openstack`, for example, list binaries that
# are only relevant for building container images (via the
# `build-stx-base.sh` and `build-stx-images.sh` scripts).
#
# Therefore, only layers that matter for the ISO creation process are listed
# here.
#
# Note: The `common` layer -- despite not being listed -- is always considered.
compiler
distro
flock