Merge "Add initial doc for compose pinning dependency jobs"
This commit is contained in:
commit
942bb6648e
|
@ -14,10 +14,9 @@ Currently we have `promotion and component pipeline`_ set up to detect
|
|||
OpenStack projects related issues early.
|
||||
|
||||
In order to detect the breakages from non-openstack projects, TripleO
|
||||
dependency pipeline comes into existance. we have added two types of
|
||||
pipeline:
|
||||
dependency pipeline has come into existence. Currently we have a single
|
||||
type of pipeline enabled:
|
||||
|
||||
* packages coming from specific module
|
||||
* packages coming from specific repo
|
||||
|
||||
The configurations for each pipeline can be found under
|
||||
|
@ -25,56 +24,15 @@ tripleo-quickstart/src/branch/master/config/release/dependency_ci/<module/repo n
|
|||
|
||||
Current OpenStack Dependency Pipeline jobs
|
||||
------------------------------------------
|
||||
* openstack-dependencies-containertools - for testing container tools dependencies
|
||||
* openstack-dependencies-centos8stream - for testing base operating system dependencies coming from CentOS-8 stream repo.
|
||||
* openstack-dependencies-openvswitch - for testing OVS and OVN dependencies coming from NFV sig repo.
|
||||
* openstack-dependencies-centos-compose - for testing jobs pinned to a specific CentOS compose build.
|
||||
|
||||
UnderStanding Module dependency pipeline
|
||||
----------------------------------------
|
||||
.. note::
|
||||
The following pipelines were deprecated in favor of CentOS Stream 9 adoption:
|
||||
- openstack-dependencies-containertools - for testing container tools dependencies
|
||||
- openstack-dependencies-centos8stream - for testing base operating system dependencies coming from CentOS-8 stream repo.
|
||||
|
||||
Under openstack-dependencies-containertools pipeline,
|
||||
We test the latest podman/buildah version comes from container tools RHEL-8 module.
|
||||
|
||||
The release specific configuration for `containertools`_ pipeline can be found here:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
dependency_modules:
|
||||
- module_name: container-tools
|
||||
control_version: 2.0
|
||||
test_version: rhel8
|
||||
dep_repo_cmd_after: |
|
||||
sudo dnf repolist;
|
||||
sudo dnf module list;
|
||||
{% for item in dependency_modules %}
|
||||
sudo dnf module disable {{ item.module_name }}:{{ item.control_version }} -y;
|
||||
sudo dnf module enable {{ item.module_name }}:{{ item.test_version }} -y;
|
||||
{% endfor %}
|
||||
sudo dnf clean metadata
|
||||
sudo dnf clean all;
|
||||
sudo dnf update -y;
|
||||
|
||||
|
||||
What do the above terms mean:
|
||||
* `module_name`: Name of the DNF module to pull packages from
|
||||
* `control_version`: Name of the version to disable for the particular module
|
||||
* `test_version`: Name of the version to enable for the particular module
|
||||
|
||||
In the above cases, it is going to test container-tools module in which
|
||||
it will disable container-tools:2.0 and enable container-tools:rhel8 module.
|
||||
All the container-tools packages will be coming here.
|
||||
|
||||
The above config is used by `get-dependency-module-content.yaml`_ to enable or
|
||||
disable the above module during the deployment.
|
||||
|
||||
- jobs running in `openstack-dependencies-containertools`_ pipeline on review.rdoproject.org.
|
||||
* periodic-tripleo-ci-centos-8-standalone-container-tools-master
|
||||
* periodic-tripleo-ci-centos-8-standalone-container-tools-container-build-master
|
||||
|
||||
The job definitions are defined under `rdo-jobs/zuul.d/dependencies-jobs.yaml`_ and
|
||||
their triggers are defined here: `rdo-jobs/zuul.d/project-templates-dependencies.yaml`_.
|
||||
|
||||
UnderStanding package dependency pipeline
|
||||
Understanding Package Dependency Pipeline
|
||||
-----------------------------------------
|
||||
|
||||
openstack-dependencies-openvswitch is a package dependency pipeline where we
|
||||
|
@ -135,8 +93,70 @@ Below are the jobs running in `openstack-dependencies-openvswitch`_ pipeline on
|
|||
vars:
|
||||
force_periodic: false
|
||||
|
||||
Understanding CentOS Compose Pinning Dependency
|
||||
-----------------------------------------------
|
||||
The dependency `compose-repos`_ works in the same line as package dependency jobs, with the difference that instead of
|
||||
setting up a single repository at a time, it consumes metadata from the provided compose URL and generates a set of
|
||||
repos as specified in the configuration snippet below:
|
||||
|
||||
Ensuring correct module or repo is used
|
||||
.. code-block:: yaml
|
||||
|
||||
...
|
||||
add_repos:
|
||||
- type: compose_repos
|
||||
compose_url: "https://odcs.stream.centos.org/production/latest-CentOS-Stream/compose/"
|
||||
release: centos-stream-9
|
||||
disable_conflicting: true
|
||||
variants:
|
||||
- AppStream
|
||||
- BaseOS
|
||||
- HighAvailability
|
||||
- CRB
|
||||
disable_repos:
|
||||
- quickstart-centos-appstreams.repo
|
||||
- quickstart-centos-base.repo
|
||||
- quickstart-centos-highavailability.repo
|
||||
- quickstart-centos-crb.repo
|
||||
|
||||
The `compose_repos` repo's type was created to generate a set of repos based on a compose repos URL and information about
|
||||
variants and conflicting repos. The `variants` will define which repos should be created from compose metadata, while
|
||||
`disable_conflicting` and `disable_repos` should guarantee that conflicting repos will be disabled in favor of the new
|
||||
ones.
|
||||
For more details on how repos are generated, please check `yum-config-compose`_ in setup-role and `yum-config`_ tool.
|
||||
|
||||
.. note::
|
||||
The process of setting up compose-repos starts earlier in the job, before any call to `repo-setup`, in one of the pre
|
||||
playbooks defined in base jobs. You shall see the `centos-compose-repos.yml`_ playbook running in jobs that have
|
||||
`dependency` set to *centos-compose*, which sets up those repos using the same tools mentioned above. The purpose of
|
||||
the dependency config here is to keep those repos enabled when any other playbook or role calls `repo-setup`.
|
||||
|
||||
Testing Jobs Using Compose Pinning Dependency
|
||||
---------------------------------------------
|
||||
|
||||
In order to test any job against a CentOS compose build, which can be a compose newer or older than the available on
|
||||
CentOS mirrors, you will need to setup a new job definition and provide the following job variables:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
- job:
|
||||
name: tripleo-ci-centos-9-standalone-compose-pinning
|
||||
parent: tripleo-ci-centos-9-standalone
|
||||
vars:
|
||||
dependency: centos-compose
|
||||
centos_compose_url: https://odcs.stream.centos.org/production/latest-CentOS-Stream/compose/
|
||||
build_container_images: true
|
||||
containers_base_image: quay.io/centos/centos:stream9
|
||||
|
||||
* `dependency`: need to be set to *centos-compose*.
|
||||
* `centos_compose_url`: CentOS compose URL to be tested. Note that the full URL ends with `compose`, because it is where
|
||||
compose `metadata` lives, required by `yum-config` tool to generate the repos. The default value is set to latest
|
||||
compose, which might be ahead of mirror's compose.
|
||||
|
||||
.. note::
|
||||
In the example above, there is an enabled flag for `build_container_images`. It means that process of building
|
||||
containers will also use compose repositories.
|
||||
|
||||
Ensuring Correct Module or Repo is Used
|
||||
---------------------------------------
|
||||
|
||||
Once a jobs runs and finishes in the dependency pipeline, we need to navigate
|
||||
|
@ -149,9 +169,11 @@ two log files:
|
|||
All the above operation is done `rdo-jobs/playbooks/dependency/diff-control-test.yaml`_ playbook which uses
|
||||
`compare_rpms`_ project from ci-config/ci-scripts/infra-setup/roles/rrcockpit/files.
|
||||
|
||||
.. note::
|
||||
The dependency `compose-repos` doesn't support rpm diff control test yet.
|
||||
|
||||
.. _`promotion and component pipeline`: https://docs.openstack.org/tripleo-docs/latest/ci/stages-overview.html
|
||||
.. _`openvswitch dependency pipeline`: https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/release/dependency_ci/openvswitch/repo_config.yaml
|
||||
.. _`containertools`: https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/release/dependency_ci/container-tools/repo_config.yaml
|
||||
.. _`openstack-dependencies-containertools`: https://review.rdoproject.org/zuul/builds?pipeline=openstack-dependencies-containertools
|
||||
.. _`openstack-dependencies-openvswitch`: https://review.rdoproject.org/zuul/builds?pipeline=openstack-dependencies-openvswitch
|
||||
.. _`rdo-jobs/zuul.d/dependencies-jobs.yaml`: https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/dependencies-jobs.yaml
|
||||
|
@ -160,3 +182,7 @@ All the above operation is done `rdo-jobs/playbooks/dependency/diff-control-test
|
|||
.. _`get-dependency-module-content.yaml`: https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/dependency/get-dependency-module-content.yaml
|
||||
.. _`rdo-jobs/dependency/get-dependency-repo-content.yaml`: https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/dependency/get-dependency-repo-content.yaml
|
||||
.. _`compare_rpms`: https://github.com/rdo-infra/ci-config/tree/master/ci-scripts/infra-setup/roles/rrcockpit/files/compare_rpms
|
||||
.. _`compose-repos`: https://github.com/openstack/tripleo-quickstart/blob/3f3f93da95c8531a4542c9a9aaa6424f2f6364c6/config/release/dependency_ci/centos-compose/repo_config.yaml
|
||||
.. _`yum-config-compose`: https://github.com/openstack/tripleo-quickstart/blob/3f3f93da95c8531a4542c9a9aaa6424f2f6364c6/roles/repo-setup/tasks/yum-config-compose.yml
|
||||
.. _`yum-config`: https://github.com/openstack/tripleo-repos/blob/cbbdde6cb6c73692b3ce9d0f6931f1b6e6fe6c91/plugins/modules/yum_config.py
|
||||
.. _`centos-compose-repos.yml`: https://github.com/openstack/tripleo-ci/blob/5246cc282819f0248d997db79bc16c0f00a9e2f8/playbooks/tripleo-ci/centos-compose-repos.yml
|
Loading…
Reference in New Issue