481 Commits

Author SHA1 Message Date
Vladimir Kozhukalov
f9c47d05e0 Update glance image build variables
* Add the python-cinderclient and oslo-rootwrap packages
  to the Glance image. When Glance uses Cinder to store
  image data it needs:
  A) The python-cinderclient python package to communicate with
     Cinder's API and
  B) The oslo-rootwrap python package to execute actions that
     require privilege escalation like attaching an iSCSI target via
     iscsadm.

* Add qemu-utils to the Glance image.
  When Glance is configured to do image conversion after an image is
  uploaded but before it's stored, it will rely on the 'qemu-img'
  tool to perform image conversion from the source to the
  destination image format.

Co-Authored-By: Kaloyan Kotlarski <kaloyan.kotlarski@storpool.com>
Change-Id: Iccc6441e38ad0bca081d3028dfcf56cd15198e01
2025-02-24 17:41:33 -06:00
Vladimir Kozhukalov
6f325450ad Build master with ubuntu:noble
Change-Id: If67d6b3d02a3d24b89a419e1a3c906a53fa43558
2025-02-13 07:32:59 -06:00
Vladimir Kozhukalov
799e35d609 Revert "Temporarily run two promote jobs for docker.io and quay.io"
This reverts commit 455c06b78fca3312f048aad75cc4c9a5e7a61968.

Reason for revert: https://review.opendev.org/c/zuul/zuul-jobs/+/881524 is merged which solves the initial issue

Change-Id: Ie50e4473b9f733774200dd3d1fc3e98036bd2b90
2025-02-11 13:08:43 +00:00
Vladimir Kozhukalov
455c06b78f Temporarily run two promote jobs for docker.io and quay.io
After this https://review.opendev.org/c/zuul/zuul-jobs/+/881524
is merged we will be able to run a single promotion job that
loops over two registries docker.io and quay.io

Change-Id: I2b43835541faf9c193cd5daff45bf7fca11304f2
2025-01-14 21:58:41 +00:00
Vladimir Kozhukalov
ddc70da7d5 Add quay.io api token
Promote-container-image role includes remove-registry-tag
role which requires api token to be defined for quay.io instead of username
and password.

Change-Id: Id672e28c39ab5bb016978bccf21e9a1f3053b8f0
2025-01-14 15:58:05 -06:00
Vladimir Kozhukalov
90092adfa1 Use periodic pipeline for nightly builds
Depends-On: I020b0bc990a8ac8466c1696a8787221792e2ef28
Change-Id: I70bb36c27b7147bc3e2f10ea7426827ca159b2a4
2025-01-10 15:57:39 -06:00
Vladimir Kozhukalov
1ec7a09e9e Rename .zuul.d -> zuul.d
This is to align it with other Openstack-Helm projects.
We use same test jobs with multiple git repos and some
jobs have file filters including smth like ^zuul\.d.*$
which prevents those jobs to start if the directory
named the other way.

Change-Id: Icc4c84a8ec6d13e67a573c5837e595f787b4cf81
2025-01-10 15:53:07 -06:00
Vasyl Saienko
639122fc11 Drop dances around virtualenv
* Drop dances around virtualenv installtion, install it via get-pip.py
* Allow to specify pip,wheel and setuptools versions as older openstack
  releases may not work with latest versions

Change-Id: I59a905dd08e1797135d805bee83a85c300746294
2024-12-30 15:39:16 +00:00
Zuul
4f6f16ae26 Merge "Use *-container-image roles" 2024-12-06 21:45:19 +00:00
Vladimir Kozhukalov
a7e69684b4 Use *-container-image roles
Instead of using *-docker-image roles for
building and uploading images let's use
*-container-image roles which allow upload
images to OCI compatible registries (not only docker hub).

Change-Id: I706045d465de6217c82632148bdaf3b0eb4f6d95
2024-12-05 17:39:19 -06:00
dbiletskiy
6224a5a40a Ignore already installed packages for pip
Co-Authored-By: vsaienko <vsaienko@mirantis.com>

Change-Id: I39b46d3af33edabf2d0da9cc6ac1c7cf5373ad34
2024-12-04 21:55:41 +00:00
Vasyl Saienko
614016bdb7 Allow to build libraries from sources
It may be required to install library from source code that is present
in upper-constraints.txt. Just clone your libraries before building
images into loci/data directory and trigger build with KEEP_ALL_WHEELS=True

Change-Id: Ia7d17285d11c3a10a42eb3d572a19e45f8511bbf
2024-12-03 14:35:51 +00:00
Vasyl Saienko
933d92a431 Align Dockerfile defaults with best practises
Do not use bash style assignment for docker ARGs

Change-Id: I19f9f1474cc608ff1328bf0cb576c2c1606213ae
2024-11-28 14:57:43 +00:00
Vasyl Saienko
d0a8108744 Allow to specify python extras during project install
Add PROJECT_PIP_EXTRAS that allows to specify which python extras
to use during project install.

Change-Id: Ic574ff530d23cc04301949a0bcdb637737055da4
2024-11-28 14:57:37 +00:00
Vasyl Saienko
185487e8c8 Build horizon static during image build
We do not need to build static files each time horizon restarts
instead build static files once during image build.

Add  argument to specify list of pannels
to enable during horizon build.

Change-Id: Ib1f8e5f01c9c1d1498b1ae8d986dd0869a7d1699
2024-11-28 14:56:50 +00:00
Vasyl Saienko
4c8bfa468e Use rbash for nova user instead of bash
We do not need to allow execute all commands, rbash is more than enough

Change-Id: I266e2723aee4dcf608f2a29bb5adc970de4d3b56
2024-11-26 07:51:23 +00:00
dbiletskiy
97263c21d1 Add tss user/group for nova image
When using emulated TPM, needs to set the user/group
that swtpm binary runs as.

Related-Prod: PRODX-37352
Change-Id: Ife7b0000a66936f3581f968d8825294b71ac3c49
2024-11-26 07:51:03 +00:00
Zuul
1b578c3d51 Merge "Honor global-requirements when buiding wheels" 2024-11-18 18:19:22 +00:00
Vladimir Kozhukalov
b1bd2bf27b Test images before publishing them
Also build job for 2023.2 Ubuntu Jammy

Change-Id: If391d17f14f054afca061036487574805c393634
2024-11-14 21:05:58 -06:00
Vasyl Saienko
8fa7edab42 Honor global-requirements when buiding wheels
Not all dependencies are present in upper-constraints
one of examples in setuptools. Since global-requirements
may limit allowed versions add it here explicitly
Add global-requirements when building wheels explicitly.

Change-Id: Ie50485859c0ebea90e26e4c98ea71d66b6323f2f
2024-11-14 07:07:01 +00:00
Vasyl Saienko
3669c19c08 Drop 2023.1 release is unmaintained
The release is unmaintained and don't have stable branch

Change-Id: I182a944c6634a381767b64e40e5aaad3be7aa748
2024-11-14 07:06:54 +00:00
Michael Polenchuk
8488ce67a1 Set default locale
Neutron tempest plugin (common/ssh.py) gets locale for
execute scripts on remote machines, so set it to default
value to avoid an empty one.

Closes-Bug: PRODX-5583
Change-Id: If1b558c1fe85137716c0836b7049274affeef2c4
2024-10-23 06:23:55 +00:00
Vladimir Kozhukalov
4251c1c4e8 Temporary customization for 2024.2
There have been some bugs fixed in oslo.messaging
and we would like to install it from stable/2024.2
branch.

See for example https://bugs.launchpad.net/oslo.messaging/+bug/2084168

This customization should be removed after these
changes are released.

Change-Id: I9ca545b0b140b339c55615c321f7336b68a3f9a0
2024-10-15 12:14:28 -05:00
Vasyl Saienko
977cd4ba10 Allow to install extra projects from source
It may be required to install additional python packages from source.
To do this download them into loci/data directory and specify names of
projects via EXTRA_PROJECTS argument during build.

Change-Id: Iadf9a14aa0aa9082abb4ba015b08d2e37a20f521
2024-10-09 18:43:39 +00:00
Vasyl Saienko
447845c0f7 Allow to pass project from external directory
This patch allows to pass directory with source code of
projects to conteiner during build. This is useful when
it is not possible to clone project in anonymous mode.

Change-Id: Id157ad614b790a87d8c8910a438a29887f9fc46e
2024-10-09 18:43:33 +00:00
Vladimir Kozhukalov
54fc5a3838 Use zuul-jobs roles for building images
Later will add jobs to test images with Openstack-Helm
before publishing them.

Change-Id: I1c173350ddd92d31b5a691a6e3820d655a8c1aee
2024-10-01 14:43:19 -05:00
Vasyl Saienko
047353213d Introduce per project customization scripts interface
Do project specific installations by calling project specific
scripts located in scripts/project_specific/<project_name> directory.
The scripts are executed in alphabetical order.
Also move keystone oidc and nova console into own files

Change-Id: I1f381e85f095eee2bede3f4925f51c9103d83467
2024-09-10 13:19:18 +00:00
Vladimir Kozhukalov
4226e755ee Add networking-baremetal to pydep.txt
Networking-baremetal package provides ironic-neutron-agent
and is not among the requirements of neither Neutron itself
nor global upper-constraints. So when we try to install it
using PIP_PACKAGES build env variable pip can not find it
among pre-downloaded wheels. So let's add it to pydep.txt
and when necessary to install it together with neutron
we can pass the baremetal profile.

Change-Id: I0f3eed5e1babd7fafcfbe2bae18a8b6b3b93397b
2024-06-20 10:35:59 +00:00
Vladimir Kozhukalov
73d2ed2657 Fix requirements build for master
The requirements.sh script fails on master due to this
https://review.opendev.org/c/openstack/requirements/+/921656

Also Ansible docker_container module is not compatible
with the recent docker daemon.

Change-Id: Ib9440cbf2a5196c31a7669edd73472d685a754a5
2024-06-20 05:33:01 -05:00
Zuul
0842d310b3 Merge "Fix building python-nss==1.0.1 for Focal images" 2024-04-26 19:49:52 +00:00
Vladimir Kozhukalov
6aa92e029d Stop building Ubuntu Focal images for master
Master branch is not compatible with Ubuntu Focal.

Some projects like Octavia recently bumped
the minimal required python version to 3.9 while
the default python3 version in Focal is 3.8.

Also Nova requires Libvirt/Qemu newer than the
default versions on Ubuntu Focal.

See for example:
https://review.opendev.org/c/openstack/octavia/+/915970
https://review.opendev.org/c/openstack/nova/+/887255

Change-Id: Ida62539a250554e776eaae3fdf283716b98f8298
2024-04-25 17:01:22 -05:00
Vladimir Kozhukalov
3fa460580d Fix building python-nss==1.0.1 for Focal images
Change-Id: I7c6e676087b4ed1f8acc9dd39558dfa79e1b0dff
2024-04-24 19:25:57 -05:00
ricolin
eb2ce826ef Revert "Remove uwsgi by default"
This reverts commit 39aaeb3f329993046703233d2f52c9f7470a8de1.

Change-Id: Ib607f3ecd5ada46ffe724fbc0d245d08f2a0fe01
2024-01-18 21:01:12 +08:00
Johannes Kulik
f4cc70548a Make fetch-wheels PEP8 compliant
This is just a little cleanup.

Change-Id: I148369869e9aac1b3f4ba25d3780d6bca380d4c9
2023-12-05 23:22:49 +00:00
Vladimir Kozhukalov
6537fe480e Add nodesets.yaml and stop using tmpfs for docker
Change-Id: I6fa07a45f491e7101f85cb90731611409a279719
2023-12-05 17:06:42 -06:00
Zuul
235b8e9d65 Merge "Remove pyldap from pydep" 2023-11-02 18:25:06 +00:00
liyingjun
75b269d8d1 fix(nova): enable login ability for nova resizing or migrating
Login should be enabled for nova resizing or migrating between
hypervisors[1].

[1]: https://docs.openstack.org/nova/latest/admin/ssh-configuration.html

Change-Id: I1db692c3fbb371873ebe5432495130f4ecfdc930
2023-08-21 07:21:10 +00:00
okozachenko
82082bf82d Copy default configuration files to /etc/
This will allow us to shorten the values.yaml
in many cases.

This is disabled by default. Set COPY_DEFAULT_CONFIG_FILES=yes
to enable it.

Change-Id: I244c29844d3ceb29b73e9a60b284d900bbeeb4ec
2023-08-16 00:39:53 +03:00
Vladimir Kozhukalov
2f88cf4644 Fix build of those wheels which depend on python-nss
In the https://review.opendev.org/c/openstack/loci/+/888351
we build python-nss separately because we need to
patch it before building. Then we moved the result
.whl file to the root directory.

Now let's add --find-links / to all pip wheel commands.

Change-Id: I5faca92eb229af989b4bf63e6e044e49c7bacb5a
2023-07-13 03:25:03 +03:00
Vladimir Kozhukalov
ce9fea8fe6 Fix building python-nss on Ubuntu Jammy
The libnss3 headers in Ubuntu Jammy are compatible
with python-nss===1.0.1. Ubuntu Jammy itself
provides the binary package python-nss and
they apply the patch that renames RSAPublicKey/DSAPublicKey
types into PyRSAPublicKey/PyDSAPublicKey.

This change applies the same patch before
building python-nss wheel.

Change-Id: I5211aac7b6a3bbc1f1b74e8662170dc8932525f9
2023-07-12 22:45:24 +03:00
Vladimir Kozhukalov
ec70407d68 Build jammy images
Change-Id: I639588a618816f6ff70a7736963589977bb36eb1
2023-07-11 19:34:19 +03:00
Vladimir Kozhukalov
7c3c53c375 Add more info to the /etc/image_info
Change-Id: I956985410f55293c998e75c9fa8960e2b9106d45
2023-06-16 20:30:51 +03:00
Vladimir Kozhukalov
e09a032257 Fix setup-gate.yaml playbook
- Use roles from zuul-jobs
- Unpin docker python package version. It used
  to be pinned to 2.7 and it is incompatible
  with the latest docker api.
- Also do not build Ubuntu Bionic images which
  by default use Ussuri release (not supported).

Change-Id: I958677e3df88028d00ac0df812c0030419d517a7
2023-06-15 19:58:47 +03:00
Vladimir Kozhukalov
c152ea081b Install dist packages before pip packages
Sometimes we need dist packages before
installing pip packages. For example
we might want gcc to be able to install
some pip packages from source.

Change-Id: I820a35f61999acbbed40a83b33ea523a0c15fcc6
2023-04-03 06:05:42 +03:00
Gage Hugo
c4281d2291 Remove pyldap from pydep
The pyldap library was needed only through the queens release, which
we no longer support. This change removes the entry as well as
removes the checks for python-ldap for these older releases.

Change-Id: Id387d4d0bd901682d0aec87ef5e5ee784b9af10c
2023-02-16 06:03:04 +00:00
Vlad Gusev
f725a1256a Install libldap-2.4-2 bindep library for the py3 ldap keystone profile
Change-Id: I86051fc48af907be05cfdd4fbb86d35aae704c4d
2022-12-06 18:18:46 +02:00
Zuul
5806885d68 Merge "Drop UWSGI from always being installed" 2022-11-22 22:56:15 +00:00
Sam Yaple
39aaeb3f32 Remove uwsgi by default
Not all deploys will use uwsgi by default but there is no way to disable
it currently. Since uwsgi itself isnt truly a python app (just a C
program with a python building shim).

The user will still be able to install binary package based uwsgi or
include uwsgi in the extra pip packages to install if they wish to.

Related-Id: I76612794c1ba8dbc45b62dff00cee43c6ba10a34
Change-Id: I15008d41633168fda31e061003ccf4681cade68e
2022-11-18 16:44:58 +00:00
Gage Hugo
7dd55124c3 Add cryptography back for placement
The compute-kit jobs are currently failing in osh-images due to
placement not having cryptography available. At this time the
placement project does not specify cryptography in their
requirements, so in the meantime this change adds it back but
specifically for the placement project.

Change-Id: I4e7d8c81d3793d4d3b85b450ded673ad122dc266
2022-11-08 10:49:18 -06:00
Zuul
5688c6ffa3 Merge "Update fetch_wheels to add Accept header" 2022-10-06 04:10:38 +00:00