Add initial doc for compose pinning dependency jobs
This patch adds documentation for CentOS compose pinning dependency work. It explains how users can create their own jobs to use compose-repos dependency to test against a CentOS compose. It also removes two deprecated dependency pipelines. Change-Id: I39091a656bef91978a2fd08e4a4a6cfcf32a4c9f Signed-off-by: Douglas Viroel <dviroel@redhat.com>
This commit is contained in:
parent
39ddc0ce71
commit
2bc0dbe511
|
@ -14,10 +14,9 @@ Currently we have `promotion and component pipeline`_ set up to detect
|
||||||
OpenStack projects related issues early.
|
OpenStack projects related issues early.
|
||||||
|
|
||||||
In order to detect the breakages from non-openstack projects, TripleO
|
In order to detect the breakages from non-openstack projects, TripleO
|
||||||
dependency pipeline comes into existance. we have added two types of
|
dependency pipeline has come into existence. Currently we have a single
|
||||||
pipeline:
|
type of pipeline enabled:
|
||||||
|
|
||||||
* packages coming from specific module
|
|
||||||
* packages coming from specific repo
|
* packages coming from specific repo
|
||||||
|
|
||||||
The configurations for each pipeline can be found under
|
The configurations for each pipeline can be found under
|
||||||
|
@ -25,56 +24,15 @@ tripleo-quickstart/src/branch/master/config/release/dependency_ci/<module/repo n
|
||||||
|
|
||||||
Current OpenStack Dependency Pipeline jobs
|
Current OpenStack Dependency Pipeline jobs
|
||||||
------------------------------------------
|
------------------------------------------
|
||||||
* openstack-dependencies-containertools - for testing container tools dependencies
|
|
||||||
* openstack-dependencies-centos8stream - for testing base operating system dependencies coming from CentOS-8 stream repo.
|
|
||||||
* openstack-dependencies-openvswitch - for testing OVS and OVN dependencies coming from NFV sig repo.
|
* openstack-dependencies-openvswitch - for testing OVS and OVN dependencies coming from NFV sig repo.
|
||||||
|
* openstack-dependencies-centos-compose - for testing jobs pinned to a specific CentOS compose build.
|
||||||
|
|
||||||
UnderStanding Module dependency pipeline
|
.. note::
|
||||||
----------------------------------------
|
The following pipelines were deprecated in favor of CentOS Stream 9 adoption:
|
||||||
|
- openstack-dependencies-containertools - for testing container tools dependencies
|
||||||
|
- openstack-dependencies-centos8stream - for testing base operating system dependencies coming from CentOS-8 stream repo.
|
||||||
|
|
||||||
Under openstack-dependencies-containertools pipeline,
|
Understanding Package Dependency Pipeline
|
||||||
We test the latest podman/buildah version comes from container tools RHEL-8 module.
|
|
||||||
|
|
||||||
The release specific configuration for `containertools`_ pipeline can be found here:
|
|
||||||
|
|
||||||
.. code-block:: yaml
|
|
||||||
|
|
||||||
dependency_modules:
|
|
||||||
- module_name: container-tools
|
|
||||||
control_version: 2.0
|
|
||||||
test_version: rhel8
|
|
||||||
dep_repo_cmd_after: |
|
|
||||||
sudo dnf repolist;
|
|
||||||
sudo dnf module list;
|
|
||||||
{% for item in dependency_modules %}
|
|
||||||
sudo dnf module disable {{ item.module_name }}:{{ item.control_version }} -y;
|
|
||||||
sudo dnf module enable {{ item.module_name }}:{{ item.test_version }} -y;
|
|
||||||
{% endfor %}
|
|
||||||
sudo dnf clean metadata
|
|
||||||
sudo dnf clean all;
|
|
||||||
sudo dnf update -y;
|
|
||||||
|
|
||||||
|
|
||||||
What do the above terms mean:
|
|
||||||
* `module_name`: Name of the DNF module to pull packages from
|
|
||||||
* `control_version`: Name of the version to disable for the particular module
|
|
||||||
* `test_version`: Name of the version to enable for the particular module
|
|
||||||
|
|
||||||
In the above cases, it is going to test container-tools module in which
|
|
||||||
it will disable container-tools:2.0 and enable container-tools:rhel8 module.
|
|
||||||
All the container-tools packages will be coming here.
|
|
||||||
|
|
||||||
The above config is used by `get-dependency-module-content.yaml`_ to enable or
|
|
||||||
disable the above module during the deployment.
|
|
||||||
|
|
||||||
- jobs running in `openstack-dependencies-containertools`_ pipeline on review.rdoproject.org.
|
|
||||||
* periodic-tripleo-ci-centos-8-standalone-container-tools-master
|
|
||||||
* periodic-tripleo-ci-centos-8-standalone-container-tools-container-build-master
|
|
||||||
|
|
||||||
The job definitions are defined under `rdo-jobs/zuul.d/dependencies-jobs.yaml`_ and
|
|
||||||
their triggers are defined here: `rdo-jobs/zuul.d/project-templates-dependencies.yaml`_.
|
|
||||||
|
|
||||||
UnderStanding package dependency pipeline
|
|
||||||
-----------------------------------------
|
-----------------------------------------
|
||||||
|
|
||||||
openstack-dependencies-openvswitch is a package dependency pipeline where we
|
openstack-dependencies-openvswitch is a package dependency pipeline where we
|
||||||
|
@ -135,8 +93,70 @@ Below are the jobs running in `openstack-dependencies-openvswitch`_ pipeline on
|
||||||
vars:
|
vars:
|
||||||
force_periodic: false
|
force_periodic: false
|
||||||
|
|
||||||
|
Understanding CentOS Compose Pinning Dependency
|
||||||
|
-----------------------------------------------
|
||||||
|
The dependency `compose-repos`_ works in the same line as package dependency jobs, with the difference that instead of
|
||||||
|
setting up a single repository at a time, it consumes metadata from the provided compose URL and generates a set of
|
||||||
|
repos as specified in the configuration snippet below:
|
||||||
|
|
||||||
Ensuring correct module or repo is used
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
...
|
||||||
|
add_repos:
|
||||||
|
- type: compose_repos
|
||||||
|
compose_url: "https://odcs.stream.centos.org/production/latest-CentOS-Stream/compose/"
|
||||||
|
release: centos-stream-9
|
||||||
|
disable_conflicting: true
|
||||||
|
variants:
|
||||||
|
- AppStream
|
||||||
|
- BaseOS
|
||||||
|
- HighAvailability
|
||||||
|
- CRB
|
||||||
|
disable_repos:
|
||||||
|
- quickstart-centos-appstreams.repo
|
||||||
|
- quickstart-centos-base.repo
|
||||||
|
- quickstart-centos-highavailability.repo
|
||||||
|
- quickstart-centos-crb.repo
|
||||||
|
|
||||||
|
The `compose_repos` repo's type was created to generate a set of repos based on a compose repos URL and information about
|
||||||
|
variants and conflicting repos. The `variants` will define which repos should be created from compose metadata, while
|
||||||
|
`disable_conflicting` and `disable_repos` should guarantee that conflicting repos will be disabled in favor of the new
|
||||||
|
ones.
|
||||||
|
For more details on how repos are generated, please check `yum-config-compose`_ in setup-role and `yum-config`_ tool.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
The process of setting up compose-repos starts earlier in the job, before any call to `repo-setup`, in one of the pre
|
||||||
|
playbooks defined in base jobs. You shall see the `centos-compose-repos.yml`_ playbook running in jobs that have
|
||||||
|
`dependency` set to *centos-compose*, which sets up those repos using the same tools mentioned above. The purpose of
|
||||||
|
the dependency config here is to keep those repos enabled when any other playbook or role calls `repo-setup`.
|
||||||
|
|
||||||
|
Testing Jobs Using Compose Pinning Dependency
|
||||||
|
---------------------------------------------
|
||||||
|
|
||||||
|
In order to test any job against a CentOS compose build, which can be a compose newer or older than the available on
|
||||||
|
CentOS mirrors, you will need to setup a new job definition and provide the following job variables:
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
- job:
|
||||||
|
name: tripleo-ci-centos-9-standalone-compose-pinning
|
||||||
|
parent: tripleo-ci-centos-9-standalone
|
||||||
|
vars:
|
||||||
|
dependency: centos-compose
|
||||||
|
centos_compose_url: https://odcs.stream.centos.org/production/latest-CentOS-Stream/compose/
|
||||||
|
build_container_images: true
|
||||||
|
containers_base_image: quay.io/centos/centos:stream9
|
||||||
|
|
||||||
|
* `dependency`: need to be set to *centos-compose*.
|
||||||
|
* `centos_compose_url`: CentOS compose URL to be tested. Note that the full URL ends with `compose`, because it is where
|
||||||
|
compose `metadata` lives, required by `yum-config` tool to generate the repos. The default value is set to latest
|
||||||
|
compose, which might be ahead of mirror's compose.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
In the example above, there is an enabled flag for `build_container_images`. It means that process of building
|
||||||
|
containers will also use compose repositories.
|
||||||
|
|
||||||
|
Ensuring Correct Module or Repo is Used
|
||||||
---------------------------------------
|
---------------------------------------
|
||||||
|
|
||||||
Once a jobs runs and finishes in the dependency pipeline, we need to navigate
|
Once a jobs runs and finishes in the dependency pipeline, we need to navigate
|
||||||
|
@ -149,9 +169,11 @@ two log files:
|
||||||
All the above operation is done `rdo-jobs/playbooks/dependency/diff-control-test.yaml`_ playbook which uses
|
All the above operation is done `rdo-jobs/playbooks/dependency/diff-control-test.yaml`_ playbook which uses
|
||||||
`compare_rpms`_ project from ci-config/ci-scripts/infra-setup/roles/rrcockpit/files.
|
`compare_rpms`_ project from ci-config/ci-scripts/infra-setup/roles/rrcockpit/files.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
The dependency `compose-repos` doesn't support rpm diff control test yet.
|
||||||
|
|
||||||
.. _`promotion and component pipeline`: https://docs.openstack.org/tripleo-docs/latest/ci/stages-overview.html
|
.. _`promotion and component pipeline`: https://docs.openstack.org/tripleo-docs/latest/ci/stages-overview.html
|
||||||
.. _`openvswitch dependency pipeline`: https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/release/dependency_ci/openvswitch/repo_config.yaml
|
.. _`openvswitch dependency pipeline`: https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/release/dependency_ci/openvswitch/repo_config.yaml
|
||||||
.. _`containertools`: https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/release/dependency_ci/container-tools/repo_config.yaml
|
|
||||||
.. _`openstack-dependencies-containertools`: https://review.rdoproject.org/zuul/builds?pipeline=openstack-dependencies-containertools
|
.. _`openstack-dependencies-containertools`: https://review.rdoproject.org/zuul/builds?pipeline=openstack-dependencies-containertools
|
||||||
.. _`openstack-dependencies-openvswitch`: https://review.rdoproject.org/zuul/builds?pipeline=openstack-dependencies-openvswitch
|
.. _`openstack-dependencies-openvswitch`: https://review.rdoproject.org/zuul/builds?pipeline=openstack-dependencies-openvswitch
|
||||||
.. _`rdo-jobs/zuul.d/dependencies-jobs.yaml`: https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/dependencies-jobs.yaml
|
.. _`rdo-jobs/zuul.d/dependencies-jobs.yaml`: https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/dependencies-jobs.yaml
|
||||||
|
@ -160,3 +182,7 @@ All the above operation is done `rdo-jobs/playbooks/dependency/diff-control-test
|
||||||
.. _`get-dependency-module-content.yaml`: https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/dependency/get-dependency-module-content.yaml
|
.. _`get-dependency-module-content.yaml`: https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/dependency/get-dependency-module-content.yaml
|
||||||
.. _`rdo-jobs/dependency/get-dependency-repo-content.yaml`: https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/dependency/get-dependency-repo-content.yaml
|
.. _`rdo-jobs/dependency/get-dependency-repo-content.yaml`: https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/dependency/get-dependency-repo-content.yaml
|
||||||
.. _`compare_rpms`: https://github.com/rdo-infra/ci-config/tree/master/ci-scripts/infra-setup/roles/rrcockpit/files/compare_rpms
|
.. _`compare_rpms`: https://github.com/rdo-infra/ci-config/tree/master/ci-scripts/infra-setup/roles/rrcockpit/files/compare_rpms
|
||||||
|
.. _`compose-repos`: https://github.com/openstack/tripleo-quickstart/blob/3f3f93da95c8531a4542c9a9aaa6424f2f6364c6/config/release/dependency_ci/centos-compose/repo_config.yaml
|
||||||
|
.. _`yum-config-compose`: https://github.com/openstack/tripleo-quickstart/blob/3f3f93da95c8531a4542c9a9aaa6424f2f6364c6/roles/repo-setup/tasks/yum-config-compose.yml
|
||||||
|
.. _`yum-config`: https://github.com/openstack/tripleo-repos/blob/cbbdde6cb6c73692b3ce9d0f6931f1b6e6fe6c91/plugins/modules/yum_config.py
|
||||||
|
.. _`centos-compose-repos.yml`: https://github.com/openstack/tripleo-ci/blob/5246cc282819f0248d997db79bc16c0f00a9e2f8/playbooks/tripleo-ci/centos-compose-repos.yml
|
Loading…
Reference in New Issue