4448 Commits

Author SHA1 Message Date
Clark Boylan
bea96e613e Setup C.UTF-8 as a RPM install lang in yum-minimal
The yum-minimal element was configuring C, en_US, and en_US.UTF-8 as
languages to be installed by packages notably leaving out C.UTF-8. This
appears to be due to CentOS 7 not supporting C.UTF-8 (its glibc was too
old). Since then CentOS 8, CentOS 8 Stream, and CentOS 9 Stream all
reportedly support C.UTF-8. Since we don't support CentOS 7 any longer
it should be safe to update yum-minimal to include C.UTF-8.

This ensures we have modern C locales available for use which are
starting to be expected (for example Ansible apparently wants a UTF8
locale and some hardcode it to use C.UTF-8).

Change-Id: I0446def63ea0fbf6bb53a0cc56c2d62f47e0ff21
3.35.0
2024-09-30 09:05:01 -07:00
Zuul
527f049a31 Merge "Update functests" 3.34.0 2024-09-18 01:02:14 +00:00
Zuul
5bf57caed7 Merge "Drop logic for old python versions" 2024-09-17 23:59:22 +00:00
Zuul
1aee4a8a6d Merge "dib-functests: run on bookworm" 2024-09-17 23:13:54 +00:00
Zuul
772ec6a5a0 Merge "Adapt to upstream CentOS Stream mirror changes" 2024-09-17 23:02:11 +00:00
Takashi Kajinami
db2955b8b5
Drop logic for old python versions
Now minimum python version supported is 3.6, so any logic for older
python versions can be removed.

Change-Id: Ie26cc300c823c3e0615f54453dc9ca9366d7a2e0
2024-09-17 09:27:27 +10:00
Dr. Jens Harbott
1d3a4de4a8 Update functests
The arm64-deb builds are failing for jammy since debootstrap is too old,
let's run them on bookworm instead of bullseye. For consistency do the
same for the x86 builds. Also add functests building noble.

Depends-On: https://review.opendev.org/c/zuul/nodepool/+/929573
Change-Id: I5b25239a7cacda62d3a46f7b15306446b3cc08e6
2024-09-16 20:47:48 +00:00
Steve Baker
3e4062e35d Fix hashbang of non-executed bash libs
These files have #!/bin/bash but do not have the execute bit set, and
are never run directly, only sourced. This has triggered RDO packaging
linting. Also outfilter.py can be run directly but does not have an
execute bit set.

This change fixes these issues.

Change-Id: Idd5cac19ec6f10103495bbe217cc5fe41c48e91b
2024-09-06 14:55:15 +12:00
Zuul
6632aaa80e Merge "New elements replace-partition(-redhat)" 2024-09-05 18:26:57 +00:00
Zuul
4b90051ce9 Merge "Add block device rollback for lvm volumes and groups" 2024-09-05 18:04:15 +00:00
Zuul
3d9a2bcc29 Merge "Add tox-py312 job for testing" 2024-09-05 13:58:20 +00:00
Zuul
b62a6e01cb Merge "Add DIB_SKIP_BASE_PACKAGE_INSTALL to base element" 2024-09-04 02:11:42 +00:00
Zuul
0e72643425 Merge "Add DIB_SKIP_GRUB_PACKAGE_INSTALL for bootloader" 2024-09-04 02:11:40 +00:00
Steve Baker
33fb961127 Add block device rollback for lvm volumes and groups
If mkfs fails then the rollback won't delete logical volumes or the
volume group. This prevents unmounting the loop back, and subsequent
image build runs will fail because the volume group still exists.

This change adds lvremove and vgremove rollback actions.

Change-Id: Ib93dfc43f5b3ef90fcec38538e828f135e514f8b
2024-09-04 11:30:17 +12:00
Zuul
aa92b26105 Merge "4k Block device support" 2024-09-03 22:15:17 +00:00
Steve Baker
835a2468dc New elements replace-partition(-redhat)
replace-partition
-----------------
A standalone element which consumes a base image which was created with
``diskimage-builder`` and rebuilds it without making any packaging changes. This
allows the image contents to be copied to a new block device layout. Use cases
for this element include:

* Rebuilding a whole-disk image with a different partition layout by setting
  ``DIB_BLOCK_DEVICE_CONFIG``
* Rebuilding a whole-disk image with the same partitions but with the sector
  size increased to 4096 bytes

replace-partition-redhat
------------------------

A redhat family specific version of the ``replace-partition`` element.

Change-Id: I7399c4bf6a4d6acfef43f871df0a40e2961ed44e
2024-09-04 09:30:30 +12:00
Steve Baker
14c651ca44 Add DIB_SKIP_BASE_PACKAGE_INSTALL to base element
Along with DIB_AVOID_PACKAGES_UPDATE it is now possible to invoke the
base element without invoking any package management. This is useful
when using a dib created image as the base image.

Change-Id: Ic6da340627bd55d4d34b1669ebf3a494131f72a6
2024-09-04 09:23:00 +12:00
Steve Baker
52e2965c21 Add DIB_SKIP_GRUB_PACKAGE_INSTALL for bootloader
Some base images will be known to have the required grub packages
installed already so package installation can be skipped.

This is specifically for the case where the base image was built by dib,
and installing any packages is undesirable because repos are not set up.

Change-Id: I0beab9e93838845a0ee533c7e6c7a4980cadd9b3
2024-09-04 09:23:00 +12:00
Zuul
110dde388c Merge "Refactor selinux-fixfiles-restore into its own element" 2024-09-03 21:04:59 +00:00
Curt Moore
cb55b089c8 Adapt to upstream CentOS Stream mirror changes
When using DIB_DISTRIBUTION_MIRROR for an internal mirror with
CentOS Stream >= 9, ensure that the yum repository configuration
files reference the new upstream directory structure layout.

Change-Id: I8360bb77e5ef9370ec69f9c0c327726e3554b7a4
2024-09-03 11:52:50 -05:00
Zuul
fb0c1e8702 Merge "Allow to override Fedora subrelease regexp" 2024-09-03 06:04:45 +00:00
Charalampos Kominos
7e1bb74831 feat: Add new autoupdates element
This patch proposes a new element which creates a mechanism for the
image to have automatic updates enabled from the first boot. A custom
config file, is expected during build time, which is injected into
the image.

Change-Id: Ib4c7513db4e00d592447fda1b1d0ed2bc649e1cf
Signed-off-by: Charalampos Kominos <hkominos@gmail.com>
2024-09-02 21:54:54 +03:00
Julia Kreger
2df6d9b914 4k Block device support
Adds support for the block device size to be asserted for the purpose of
writing out new images, which may be critical for operators with hardware
which requires logical blocks which are 4096 bytes long.

Change-Id: I5c16a042eacfbb94a905b93a0eb9fbc73de0a890
2024-08-30 13:30:27 -07:00
Steve Baker
aae1f5f020 Refactor selinux-fixfiles-restore into its own element
This is for a use-case where running the rest of rpm-distro is not
desirable (rebuilding an image previously build by dib)

Change-Id: I85f4d7fd2ab8a7278f4c76b133104f638a055e74
2024-08-30 12:28:53 +12:00
Zuul
e89f59393a Merge "Not duplicate grub defaults" 2024-08-19 06:01:25 +00:00
Zuul
ffbdd94fcf Merge "Change centos DIB_RELEASE default to 9-stream" 2024-08-05 16:34:33 +00:00
Zuul
9923387f0c Merge "Remove EOLed CentOS 7 and RHEL 7" 2024-08-05 13:13:04 +00:00
Julia Kreger
b8ed6035d5 Change centos DIB_RELEASE default to 9-stream
Since the public mirrors for centos8 are really not available
anymore since it has gone End of Life, changes the default
invocation for a quick disk image to 9-stream.

Users who wish specific behavior should explicitly express
the version they desire.

Change-Id: I12cb10258ab80bae1c754e86aadcf926cdfae805
2024-08-02 12:52:13 -07:00
afazekas
be4ff60d9c Not duplicate grub defaults
The GRUB_TIMEOUT was duplicated in
the /etc/grub/defaults.
Changing the DIB_GRUB_TIMEOUT default to the base
image typical default.
Note: Hold shift if you want to interact
with the grub menu.

Change-Id: If4d5ae467706b98d89f9dd6714ef06a5933ccfb3
2024-08-01 16:24:33 +02:00
Zuul
14de4278cd Merge "Run functest jobs on bookworm" 2024-07-19 22:04:41 +00:00
Zuul
637a0a7960 Merge "Add podman to bindep" 2024-07-18 19:31:48 +00:00
Clark Boylan
67d7ff3dab Run functest jobs on bookworm
We were running these test jobs on bullseye which is getting a bit old.
Additionally, it seems that the debootstrap on bullseye may not be able
to build jammy on arm64? Attempt to address that via executing on a
newer platform.

Change-Id: I54195c3f4500dcbcb92c5abb2a14f3078b2461dc
2024-07-11 15:53:49 -07:00
Gregory Thiemonge
c0850ae0ca Add podman to bindep
podman is a requirement when building rockylinux images, it must be
included in bindep.txt

Note: zuul ensures that podman is installed in CI jobs, but the
dependency is missing when deploying devstack outside CI.

Change-Id: I17aaac32fdb435b78a2da23b179f2db8d9b0d531
2024-07-11 17:27:30 +00:00
Dmitriy Rabotyagov
01a4ab7724 Allow to override Fedora subrelease regexp
This gives more flexability into defenition and won't result in users to
be blocked until new DIB release will happen with fixed regexp.

Change-Id: I5b6d15dc7cbc6f41b600c5b33f5030df5bdfa793
2024-07-11 12:31:49 +00:00
Dmitriy Rabotyagov
7cd5268d15 Fix regexp for Fedora element SUBRELEASE parsing
Current regexp  is catching a wrong extra record for HTML-based mirrors [1]

Modifying regexp allows to workaround fetching extra string.

[1] https://paste.openstack.org/show/bVBUEW88mDF58OKKp3kw/

Change-Id: Ibd8806cad10ad7d993b7953eee2d456e92a8f7cf
2024-07-11 12:31:45 +00:00
Dr. Jens Harbott
0f9ba78dac Add tox-py312 job for testing
Change-Id: I4e93962305b1d5614e4f4368cfd9e9428dcc1637
2024-07-10 21:35:51 +02:00
Steve Baker
eb395ec558 Remove EOLed CentOS 7 and RHEL 7
CentOS 7 reached EOL on 30th June 2024[1] and RHEL 7 ended its
maintenance support 2 phase[2] the same date.

This change removes the ablity to build images derived from these base
images.

The centos and centos-minimal elements now default to a DIB_RELEASE
value of 9-stream.

[1] https://www.redhat.com/en/topics/linux/centos-linux-eol
[2] https://www.redhat.com/en/technologies/linux-platforms/enterprise-linux/rhel-7-end-of-maintenance

Change-Id: Ic50e08d9f84bbd319129be236d799eade5f40be8
2024-07-05 09:53:29 +12:00
Zuul
55c8644ef3 Merge "bootloader: Strip prior console settings" 2024-07-04 03:18:37 +00:00
Zuul
175685ec0c Merge "CentOS-7 EOL: remove jobs" 2024-07-04 00:35:30 +00:00
Steve Baker
e9be113e8f CentOS-7 EOL: remove jobs
CentOS-7 went EOL on June 30th. This change removes the job as
mirrorlist.centos.org no longer resolves. Follow-up changes will remove
CentOS-7 from existing elements.

Change-Id: I23a00d2aa2a056dec33516eccb917126f8b766b2
2024-07-04 10:12:17 +12:00
Julia Kreger
abe2a4e239 bootloader: Strip prior console settings
Prior to this change, source images could introduce variation in the
the result of the bootloader element results, if the configuration
is changed later which would could re-introduce arguments not desired
which come from the source image.

Since we assert, bsaed upon user supplied input, new console configuration
we now strip the console configuration from /etc/default/grub configuration
before starting to work on putting in place the user's desired end state.

This also ends up helping reduce/remove the possibility of multiple
console elements as some were sourcing from the original source image.

Change-Id: Ia1f9987d989f1d0e1cbda619f7abdf7005b47fbc
2024-06-28 08:16:33 -07:00
Zuul
1d48e5afde Merge "growvols, enforce pvcreate and overwrite old VG signature" 2024-06-26 22:00:36 +00:00
Zuul
a1425cc19b Merge "Fix uninstall gentoo packages" 2024-06-26 20:25:31 +00:00
Zuul
7d0fa0b737 Merge "Fix setting apt mirror for noble" 2024-06-26 20:20:45 +00:00
Zuul
89ce30d7b3 Merge "minor ci: quick cleanup and remove centos8" 2024-06-26 20:18:48 +00:00
Julia Kreger
2897c5be79 minor ci: quick cleanup and remove centos8
Just a follow-up from one of the recent changes, it was noted we could
just remove the job entirely instead of commenting it out. As such,
this change removes the entry from the zuul configuration.

Change-Id: I1661498493e48f34a3a1a9242b484360cf39e829
2024-06-25 10:28:51 -07:00
Julia Kreger
81355c4124 remove console entries when console is disabled
A huge problem with latency sensitive baremetal worklaods
is interrupts being triggered spuriously. Unfortunately
when we start with cloud images, often they default to
logging to a serial console which means every write is
an interrupt, which is far from ideal and can result
in packet loss and service degredation quite quickly.

So instead, if the console logging has been disabled,
and no virtual terminal has been defined, we now strip
the console entries from the resulting image.

In testing with Centos9 Stream, with booting a VM, the console
in this scenario jumps to the framebuffer once started, and
after thousands of lines being written to the console, even with
delays, locally I'm at 219 interrupts, with a bulk of the entries
coming from before I interacted with the console (~135 from just
boot).

Change-Id: Id9b19c4d9804b88e6db05a20e26c8264bb357734
2024-06-25 17:27:43 +00:00
Julia Kreger
69187bae3f Provide an ability to disable serial console injection
By default, we attempt to inject a serial console, which may, or may not
be needed, for example, Centos Stream 9 cloud images already configure
a console setting, and repeating it just might be undesirable and cause
workload performance degredation if the kernel or an application has to
report anything to a console.

This change generally results in original console entries being preserved,
which might actually be a bug and get fixed in a latter patch.

Generally, users of dib *should* likely be specific what they want to do
with their console setting, and without setting the new
DIB_BOOTLOADER_USE_SERIAL_CONSOLE paramter to false, the default will
be adhered to, and any pre-existing serial console entries will *not* be
de-duplicated from the base image. That too is *likely* a bug, but a
harder one to fix.

Change-Id: Icdfb5ed021b1a91e2de3c9a22bb2ff7fe9882bcd
2024-06-25 17:27:30 +00:00
Ian Wienand
5256991346
dib-functests: run on bookworm
Run dib-functests on bookworm, which should mean that we don't need to
pull in the unstable debootstrap
(c.f. 7b6247554eaecb22d95e626f44452350cc69645b).

This should be good for a while :)
(c.f. 1a5c5639205a1ce84719aa999dc7a99a3ead5d37)

Change-Id: Ib9240c086eff84e02740d98f02e1846a8a39298d
2024-06-25 11:06:48 +10:00
Riccardo Pittau
fc4fa1e81f Disable Stream-8, non-vote FC 37, and assume yes on debian functest
CentOS Stream 8 and Fedora 37 are EOL and their jobs are just
failing.

Disabled the Centos Stream 8 job, and marked the Fedora Core 37 job
as non-voting fo the time being so we are encouraged to fix by adding
a FC 38 or FC 39 job.

Also removing CS8 and Fedora jobs from dib-functests, and
setting the debian functional build to assume yes to try and
minimize prompting on package upgrades.

Change-Id: Iba7a52a7882de4293f87b98a61bda04faf07cbe8
2024-06-24 22:00:02 +00:00