1378 Commits

Author SHA1 Message Date
Clark Boylan
bea96e613e Setup C.UTF-8 as a RPM install lang in yum-minimal
The yum-minimal element was configuring C, en_US, and en_US.UTF-8 as
languages to be installed by packages notably leaving out C.UTF-8. This
appears to be due to CentOS 7 not supporting C.UTF-8 (its glibc was too
old). Since then CentOS 8, CentOS 8 Stream, and CentOS 9 Stream all
reportedly support C.UTF-8. Since we don't support CentOS 7 any longer
it should be safe to update yum-minimal to include C.UTF-8.

This ensures we have modern C locales available for use which are
starting to be expected (for example Ansible apparently wants a UTF8
locale and some hardcode it to use C.UTF-8).

Change-Id: I0446def63ea0fbf6bb53a0cc56c2d62f47e0ff21
2024-09-30 09:05:01 -07:00
Zuul
527f049a31 Merge "Update functests" 2024-09-18 01:02:14 +00:00
Zuul
5bf57caed7 Merge "Drop logic for old python versions" 2024-09-17 23:59:22 +00:00
Zuul
772ec6a5a0 Merge "Adapt to upstream CentOS Stream mirror changes" 2024-09-17 23:02:11 +00:00
Takashi Kajinami
db2955b8b5
Drop logic for old python versions
Now minimum python version supported is 3.6, so any logic for older
python versions can be removed.

Change-Id: Ie26cc300c823c3e0615f54453dc9ca9366d7a2e0
2024-09-17 09:27:27 +10:00
Dr. Jens Harbott
1d3a4de4a8 Update functests
The arm64-deb builds are failing for jammy since debootstrap is too old,
let's run them on bookworm instead of bullseye. For consistency do the
same for the x86 builds. Also add functests building noble.

Depends-On: https://review.opendev.org/c/zuul/nodepool/+/929573
Change-Id: I5b25239a7cacda62d3a46f7b15306446b3cc08e6
2024-09-16 20:47:48 +00:00
Steve Baker
3e4062e35d Fix hashbang of non-executed bash libs
These files have #!/bin/bash but do not have the execute bit set, and
are never run directly, only sourced. This has triggered RDO packaging
linting. Also outfilter.py can be run directly but does not have an
execute bit set.

This change fixes these issues.

Change-Id: Idd5cac19ec6f10103495bbe217cc5fe41c48e91b
2024-09-06 14:55:15 +12:00
Zuul
6632aaa80e Merge "New elements replace-partition(-redhat)" 2024-09-05 18:26:57 +00:00
Zuul
4b90051ce9 Merge "Add block device rollback for lvm volumes and groups" 2024-09-05 18:04:15 +00:00
Zuul
b62a6e01cb Merge "Add DIB_SKIP_BASE_PACKAGE_INSTALL to base element" 2024-09-04 02:11:42 +00:00
Zuul
0e72643425 Merge "Add DIB_SKIP_GRUB_PACKAGE_INSTALL for bootloader" 2024-09-04 02:11:40 +00:00
Steve Baker
33fb961127 Add block device rollback for lvm volumes and groups
If mkfs fails then the rollback won't delete logical volumes or the
volume group. This prevents unmounting the loop back, and subsequent
image build runs will fail because the volume group still exists.

This change adds lvremove and vgremove rollback actions.

Change-Id: Ib93dfc43f5b3ef90fcec38538e828f135e514f8b
2024-09-04 11:30:17 +12:00
Zuul
aa92b26105 Merge "4k Block device support" 2024-09-03 22:15:17 +00:00
Steve Baker
835a2468dc New elements replace-partition(-redhat)
replace-partition
-----------------
A standalone element which consumes a base image which was created with
``diskimage-builder`` and rebuilds it without making any packaging changes. This
allows the image contents to be copied to a new block device layout. Use cases
for this element include:

* Rebuilding a whole-disk image with a different partition layout by setting
  ``DIB_BLOCK_DEVICE_CONFIG``
* Rebuilding a whole-disk image with the same partitions but with the sector
  size increased to 4096 bytes

replace-partition-redhat
------------------------

A redhat family specific version of the ``replace-partition`` element.

Change-Id: I7399c4bf6a4d6acfef43f871df0a40e2961ed44e
2024-09-04 09:30:30 +12:00
Steve Baker
14c651ca44 Add DIB_SKIP_BASE_PACKAGE_INSTALL to base element
Along with DIB_AVOID_PACKAGES_UPDATE it is now possible to invoke the
base element without invoking any package management. This is useful
when using a dib created image as the base image.

Change-Id: Ic6da340627bd55d4d34b1669ebf3a494131f72a6
2024-09-04 09:23:00 +12:00
Steve Baker
52e2965c21 Add DIB_SKIP_GRUB_PACKAGE_INSTALL for bootloader
Some base images will be known to have the required grub packages
installed already so package installation can be skipped.

This is specifically for the case where the base image was built by dib,
and installing any packages is undesirable because repos are not set up.

Change-Id: I0beab9e93838845a0ee533c7e6c7a4980cadd9b3
2024-09-04 09:23:00 +12:00
Zuul
110dde388c Merge "Refactor selinux-fixfiles-restore into its own element" 2024-09-03 21:04:59 +00:00
Curt Moore
cb55b089c8 Adapt to upstream CentOS Stream mirror changes
When using DIB_DISTRIBUTION_MIRROR for an internal mirror with
CentOS Stream >= 9, ensure that the yum repository configuration
files reference the new upstream directory structure layout.

Change-Id: I8360bb77e5ef9370ec69f9c0c327726e3554b7a4
2024-09-03 11:52:50 -05:00
Zuul
fb0c1e8702 Merge "Allow to override Fedora subrelease regexp" 2024-09-03 06:04:45 +00:00
Charalampos Kominos
7e1bb74831 feat: Add new autoupdates element
This patch proposes a new element which creates a mechanism for the
image to have automatic updates enabled from the first boot. A custom
config file, is expected during build time, which is injected into
the image.

Change-Id: Ib4c7513db4e00d592447fda1b1d0ed2bc649e1cf
Signed-off-by: Charalampos Kominos <hkominos@gmail.com>
2024-09-02 21:54:54 +03:00
Julia Kreger
2df6d9b914 4k Block device support
Adds support for the block device size to be asserted for the purpose of
writing out new images, which may be critical for operators with hardware
which requires logical blocks which are 4096 bytes long.

Change-Id: I5c16a042eacfbb94a905b93a0eb9fbc73de0a890
2024-08-30 13:30:27 -07:00
Steve Baker
aae1f5f020 Refactor selinux-fixfiles-restore into its own element
This is for a use-case where running the rest of rpm-distro is not
desirable (rebuilding an image previously build by dib)

Change-Id: I85f4d7fd2ab8a7278f4c76b133104f638a055e74
2024-08-30 12:28:53 +12:00
Zuul
e89f59393a Merge "Not duplicate grub defaults" 2024-08-19 06:01:25 +00:00
Zuul
ffbdd94fcf Merge "Change centos DIB_RELEASE default to 9-stream" 2024-08-05 16:34:33 +00:00
Zuul
9923387f0c Merge "Remove EOLed CentOS 7 and RHEL 7" 2024-08-05 13:13:04 +00:00
Julia Kreger
b8ed6035d5 Change centos DIB_RELEASE default to 9-stream
Since the public mirrors for centos8 are really not available
anymore since it has gone End of Life, changes the default
invocation for a quick disk image to 9-stream.

Users who wish specific behavior should explicitly express
the version they desire.

Change-Id: I12cb10258ab80bae1c754e86aadcf926cdfae805
2024-08-02 12:52:13 -07:00
afazekas
be4ff60d9c Not duplicate grub defaults
The GRUB_TIMEOUT was duplicated in
the /etc/grub/defaults.
Changing the DIB_GRUB_TIMEOUT default to the base
image typical default.
Note: Hold shift if you want to interact
with the grub menu.

Change-Id: If4d5ae467706b98d89f9dd6714ef06a5933ccfb3
2024-08-01 16:24:33 +02:00
Dmitriy Rabotyagov
01a4ab7724 Allow to override Fedora subrelease regexp
This gives more flexability into defenition and won't result in users to
be blocked until new DIB release will happen with fixed regexp.

Change-Id: I5b6d15dc7cbc6f41b600c5b33f5030df5bdfa793
2024-07-11 12:31:49 +00:00
Dmitriy Rabotyagov
7cd5268d15 Fix regexp for Fedora element SUBRELEASE parsing
Current regexp  is catching a wrong extra record for HTML-based mirrors [1]

Modifying regexp allows to workaround fetching extra string.

[1] https://paste.openstack.org/show/bVBUEW88mDF58OKKp3kw/

Change-Id: Ibd8806cad10ad7d993b7953eee2d456e92a8f7cf
2024-07-11 12:31:45 +00:00
Steve Baker
eb395ec558 Remove EOLed CentOS 7 and RHEL 7
CentOS 7 reached EOL on 30th June 2024[1] and RHEL 7 ended its
maintenance support 2 phase[2] the same date.

This change removes the ablity to build images derived from these base
images.

The centos and centos-minimal elements now default to a DIB_RELEASE
value of 9-stream.

[1] https://www.redhat.com/en/topics/linux/centos-linux-eol
[2] https://www.redhat.com/en/technologies/linux-platforms/enterprise-linux/rhel-7-end-of-maintenance

Change-Id: Ic50e08d9f84bbd319129be236d799eade5f40be8
2024-07-05 09:53:29 +12:00
Julia Kreger
abe2a4e239 bootloader: Strip prior console settings
Prior to this change, source images could introduce variation in the
the result of the bootloader element results, if the configuration
is changed later which would could re-introduce arguments not desired
which come from the source image.

Since we assert, bsaed upon user supplied input, new console configuration
we now strip the console configuration from /etc/default/grub configuration
before starting to work on putting in place the user's desired end state.

This also ends up helping reduce/remove the possibility of multiple
console elements as some were sourcing from the original source image.

Change-Id: Ia1f9987d989f1d0e1cbda619f7abdf7005b47fbc
2024-06-28 08:16:33 -07:00
Zuul
1d48e5afde Merge "growvols, enforce pvcreate and overwrite old VG signature" 2024-06-26 22:00:36 +00:00
Zuul
a1425cc19b Merge "Fix uninstall gentoo packages" 2024-06-26 20:25:31 +00:00
Zuul
7d0fa0b737 Merge "Fix setting apt mirror for noble" 2024-06-26 20:20:45 +00:00
Julia Kreger
81355c4124 remove console entries when console is disabled
A huge problem with latency sensitive baremetal worklaods
is interrupts being triggered spuriously. Unfortunately
when we start with cloud images, often they default to
logging to a serial console which means every write is
an interrupt, which is far from ideal and can result
in packet loss and service degredation quite quickly.

So instead, if the console logging has been disabled,
and no virtual terminal has been defined, we now strip
the console entries from the resulting image.

In testing with Centos9 Stream, with booting a VM, the console
in this scenario jumps to the framebuffer once started, and
after thousands of lines being written to the console, even with
delays, locally I'm at 219 interrupts, with a bulk of the entries
coming from before I interacted with the console (~135 from just
boot).

Change-Id: Id9b19c4d9804b88e6db05a20e26c8264bb357734
2024-06-25 17:27:43 +00:00
Julia Kreger
69187bae3f Provide an ability to disable serial console injection
By default, we attempt to inject a serial console, which may, or may not
be needed, for example, Centos Stream 9 cloud images already configure
a console setting, and repeating it just might be undesirable and cause
workload performance degredation if the kernel or an application has to
report anything to a console.

This change generally results in original console entries being preserved,
which might actually be a bug and get fixed in a latter patch.

Generally, users of dib *should* likely be specific what they want to do
with their console setting, and without setting the new
DIB_BOOTLOADER_USE_SERIAL_CONSOLE paramter to false, the default will
be adhered to, and any pre-existing serial console entries will *not* be
de-duplicated from the base image. That too is *likely* a bug, but a
harder one to fix.

Change-Id: Icdfb5ed021b1a91e2de3c9a22bb2ff7fe9882bcd
2024-06-25 17:27:30 +00:00
Radoslaw Smigielski
34cc4c5fb9
growvols, enforce pvcreate and overwrite old VG signature
In some situations, when growvols run on disk which was already used or
growvols runs on the same driver, pvcreate finds old VG signature and
refuse to re-create PV on a block device.

  [ERROR] Running command failed: cmd "pvcreate /dev/sda6", stdout "",
          stderr " Can't initialize physical volume "/dev/sda6" of volume group
          "vg" without "-ff" /dev/sda6: physical volume not initialized.

Closes-Bug: 2069487
Change-Id: Ie8e0641d5637e8c0c2edb23e820bc7961b7d319b
2024-06-18 22:06:26 +02:00
Riccardo Pittau
5b2aef2018 Fix uninstall gentoo packages
The square brackets cause the check to fail because of the piped
grep.
Removing them works just fine.

Change-Id: I85dfaa436d09edf958315579c7050165b80bf721
2024-06-11 16:41:56 +02:00
Lukas Kranz
7ab9adfb5f Fix setting apt mirror for noble
With noble the structure of the apt sources file has changed.
(see https://repolib.readthedocs.io/en/latest/deb822-format.html)
This brakes the current implementation for setting mirrors when building images.

Change-Id: I2ba0344b49c78e499ead681338e1a2c275e8e6f1
2024-05-31 08:17:53 +02:00
Julia Kreger
ac180b9ac6 simple-init: Swap continue for true
Continue is not valid, and in the current syntax usage for 80-simple-init,
the error is logged. Swapped to true to act as a no-op.

/tmp/in_target.d/post-install.d/80-simple-init: line 30: continue: \
    only meaningful in a `for', `while', or `until' loop

Change-Id: I9ddb9aadc1114c0f4dea14d31b9751d83a887a6b
2024-05-22 14:37:33 -07:00
Julia Kreger
e544379010 ConfigDrive: Disable cloud-init auto dhcp by default
If your using configuration drive, cloud init tries to DHCP by
default anyway, which delays the boot sequence as it seeks out a
network attached metadata source.

So instead, disable the automatic activation so the configuration
drive data source is acutally leverage. Additionally add further
notes to provide guidance to users of DIB.

Change-Id: Ie7c522f23deb3f08ee4ec002e0e2020f382359aa
2024-04-29 18:39:41 +00:00
Zuul
4d1e1712b1 Merge "Overwrite EFI grub.cfg when it exists" 2024-04-17 19:18:19 +00:00
Zuul
d95c475c3a Merge "Write fedora download redirect info to stderr" 2024-04-17 19:17:18 +00:00
Steve Baker
c03e46d9e1 Overwrite EFI grub.cfg when it exists
Currently a partition image with existing EFI assets may produce an
invalid image because the EFI grub.cfg may reference a partition UUIDs
that don't exist in the partition image case.

This change replaces the EFI grub.cfg if the EFI directory exists at
all, ensuring the EFI grub.cfg is valid in more cases.

Also, add missing EFI_BOOT_DIR for fedora-container element.

Change-Id: Id4f8cd2cb16bbb1a392f531cf5b8455f81162c42
2024-04-17 14:21:03 +12:00
Zuul
ee0aeef1a6 Merge "fix(rocky): don't uninstall linux-firmware" 2024-04-17 01:21:10 +00:00
Zuul
30a8948271 Merge "allow passing arbitrary build options in containerfile element" 2024-04-12 16:17:20 +00:00
Zuul
833a2e17a2 Merge "Get rid of 3rd party mock" 2024-04-08 13:57:25 +00:00
Zuul
c81f77b9f3 Merge "Remove pre-bullseye release workaround" 2024-03-25 12:03:53 +00:00
Birger J. Nordølum
7a3b9396ab
fix(rocky): don't uninstall linux-firmware
This is included in the container itself, and should not be removed.

Ref. change: 899372

Change-Id: Ib00e11c7af8202935e808a297c34a1a6982bd2c7
Signed-off-by: Birger J. Nordølum <contact@mindtooth.no>
2024-03-13 23:50:15 +01:00
Zuul
2ba78f79a1 Merge "Call grub2-mkconfig with --update-bls-cmdline" 2024-03-11 16:24:38 +00:00