Re-read the partition table with 'partx -a', rather than 'partx -u'.
This should fix an timing issue where the bootloader installation
fails to mount the EFI partition from a whole disk image since it
is not yet aware of the new partitions (observed with both, the
iscsi and the direct deploy interface).
Change-Id: If5da3075e813ae01df3decf8f0647aba111b0515
(cherry picked from commit dc8c1f16f9)
(cherry picked from commit ab3e8a75ba)
I accidently put colons on the test data and remembered taking the
colon character out of the regex I was working on, but apparently
left it in, and accounted for the active entry indicator flag
which appears to have inconsistent support across vendors.
The regex has been fixed, and a test added from a Lenovo SR650
which has some additional string entry data in the UEFI output
which may separate entries.
Change-Id: I1f67b0fb1f645fa82e98bd7c7bba3ffc7755cc74
(cherry picked from commit e10f052c06)
(cherry picked from commit 87a30ff4fa)
(cherry picked from commit 25f89e4d88)
Some firmware seems to take an objection with EFI nvram
entries being deleted after one is added, resulting in the
entire entry table being reset to the last known good state.
This is problematic, as ultimately deployments can time out
if we previously booted with Networking, and the machine, while
commanded to do other wise, reboots back to networking regardless.
We will now delete entries first, before proceeding.
Additionally, for general use, this pattern may serve the
community better by avoiding cases where we would have
previously just relied upon efibootmgr[0] to warn us of duplicate
entries.
[0]: 103aa22ece/src/efibootmgr.c (L228)
Change-Id: Ib61a7100a059e79a8b0901fd8f46b9bc41d657dc
Story: 2009649
Task: 43808
(cherry picked from commit 67eddfa7e3)
(cherry picked from commit 33b39705a5)
(cherry picked from commit 8fca145739)
(cherry picked from commit 47ac40a7f9)
When debugging boot manager problems it can be advantageous to
see all the full entries rather then just their labels.
Depends-On: https://review.opendev.org/c/openstack/ironic/+/818394
Change-Id: I6a1bb78acaf5a4284727bdf533d4be6db2099f50
(cherry picked from commit caf695f70a)
(cherry picked from commit 9753b61a51)
(cherry picked from commit 750934a207)
Adds support to identify and utilize a CSV file to signal which
bootloader to utilize, and set it when the OS is running as opposed
to when EFI is running. This works around EFI loader potentially
crashing some vendors hardware types when entry stored in the
image does not match the EFI loader record which was utilzied to
boot.
Grub2+shim specifically specifically needs the CSV file name
and entry label to match what the system was booted with in order
to prevent the machine from potentially crashing.
See https://storyboard.openstack.org/#!/story/2008962
and https://bugzilla.redhat.com/show_bug.cgi?id=1966129#c37
for more information.
Change-Id: Ibf1ef4fe0764c0a6f1a39cb7eebc23ecc0ee177d
Story: 2008962
Task: 42598
Co-Authored-By: Bob Fournier <bfournie@redhat.com>
(cherry picked from commit 2fab70c36b)
(cherry picked from commit bfa97cbbc2)
(cherry picked from commit 474d2a91b0)
To make this function useful for purposes other than efibootmgr
entries, this change moves the path manipulation to _run_efibootmgr.
This change also adds boot*.efi entries to BOOTLOADERS_EFI so that it
includes every entry in the UEFI Spec 2.9[1] Table 3-2 UEFI Image
Types.
[1] https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf
Story: 2008923
Task: 42521
Change-Id: Ibe02786609aa0de65115897d8f4a9b4f36c8aed2
(cherry picked from commit 10d18c4113)
(cherry picked from commit d61b7bd843)
(cherry picked from commit 350a67e203)
The _manage_uefi code has a check where it attempts to just
identify the precise partition number of the device, in order
for configuration to be parsed and passed. However, the same code
did not handle the existence of a `p1` partition instead of just a
partition #1. This is because the device naming format is different
with NVMe and Software RAID.
Likely, this wasn't an issue with software raid due to how complex the
code interaction is, but the docs also indicate to use only whole disk
images in that case.
This patch was pulled down my one RH's professional services folks
who has confirmed it does indeed fix the issue at hand. This is noted
as a public comment on the Red Hat bugzilla.
https://bugzilla.redhat.com/show_bug.cgi?id=1954096
Story: 2008881
Task: 42426
Related: rhbz#1954096
Change-Id: Ie3bd49add9a57fabbcdcbae4b73309066b620d02
(cherry picked from commit fe825fa97e)
Adds support for the EFI partition to be appended to fstab so the
filesystem can be automounted and EFI loader updated should the
deployed operating system need to do so.
This should enable bootloaders to be upgraded by linux based
operating systems after the instance has been deployed when
a partition image was utilized for the initial deployment.
Change-Id: Iec28a8841cc01ec8b01a3f5cca070c934c7a2531
Story: 2008070
Task: 40754
(cherry picked from commit a12a5744b6)
Partition images can sometimes contain a /boot folder structure
event he assets for EFI booting on that filesystem. Which is a
good thing. The conundrum is that Ironic does not handle this
properly and potentially replaces the bootloader in this sequence
such that grub2-install is used instead of signed bootloader assets.
As such, we should be preserving the assets and using them from
a partition image much like we do when we have a wholedisk
image and can identify the assets.
Now we will preserve the EFI boot assets, copy them to the new EFI
boot partition, and call the EFI setup methods to manage the EFI
nvram.
Note, this change also splits the logic path out that performs the
end call of the EFI boot manager into a reusable method but does
not retool all of the testing as it is intertwined in the
install_grub2 testing.
Also adds some additional debug logging, as much of the bootloader
installation code has multiple fallback/cleanup points which makes
it difficult to debug from logs.
Story: 2008070
Task: 40753
Change-Id: If17d4b4c06df5504987e61a1fde6662e9acd6989
(cherry picked from commit f9870d5812)
Some hardware is very well intentioned. However this intention
can result in the UEFI NVRAM table being full which prevents us
from adding new records to the table. We can't be sure what to
delete, so in this case some operators just need the ability to
tell ironic "it is okay if this fails, it will still work."
The added ``ignore_bootloader_failure`` option adds
this capability which can be set per-node either in the agent
configuation via the ramdisk image, or in the pxe_append_params
configuration parameter for the node itself with a
``ipa-ignore-bootloader-failure`` option in order to prevent
the failure from being raised.
Change-Id: If3c83fb2ea2025fce092d495a64f32077c70d2d6
Story: 2008386
Task: 41309
(cherry picked from commit 7a83773fbc)
(cherry picked from commit 07cf2c1b79)
In order to ensure grub2 finds all files it needs, mount all
vfat partitions specified in the deployed image.
Story: #2007618
Task: #39629
Change-Id: Ie5b6e0abc3f266409562f9ecb26538126b667056
(cherry picked from commit c5022790b3)
When we added software raid support, we started calling bootloader
installation. As time went on, we ehnanced that code path for non
RAID cases in order to ensure that UEFI nvram was setup
for the instance to boot properly.
Somewhere in this process, we missed a possible failure case where
the iscsi client tgtadm may return failures. Obviously, the correct
path is to not call iscsi teardown if we don't need to.
Since it was always semi-opportunistic teardown, we can't blindly
catch any error, and if we started iSCSI and failed to tear the
connection down, we might want to still fail, so this change
moves the logic over to use a flag on the agent object which
one extension to set the flag and the other to read it and take
action based upon that.
Change-Id: Id3b1ae5e59282f4109f6246d5614d44c93aefa7c
Story: 2007937
Task: 40395
(cherry picked from commit 2a56ee03b6)
This has been a popular guidance, and diskimage-builder has recently
started following it.
Change-Id: I794c846fb191c15b0a30546bf64d624dfbde0fd4
(cherry picked from commit ba3caa6c64)
It does not return anything, so it makes no point for it to be
synchronous. Ironic always calls it with wait=True, so there is
no problem with backward compatibility either.
Change-Id: I44fec2e0cb54486328ce71263613d8592e384870
(cherry picked from commit 7e5fe1121e)
The proposed changes concern two steps:
First, when creating the RAID configuration, have a GPT partition
table type (this is not necessary, but more natural with UEFI).
Also, leave some space, either for the EFI partitions or the BIOS
boot partitions, outside the Software RAID.
Secondly, when installing the bootloader, make sure the correct
boot partitions are created or relocated.
Change-Id: Icf0a76b0de89e7a8494363ec91b2f1afda4faa3b
Story: #2006379
Task: #37635
Somewhat common are dual boot images that have both a MBR loader
and the contents required for a UEFI boot, as largely the pointer
to where to begin reading the rest of the boot loader occurs in
the first few hundred bytes on disk which redirects the disk to
begin reading from a known address.
This goes sideways on UEFI machines where this method of booting
is not recognized nor supported. Thus we need to return false when
we encounter this state.
Change-Id: I8c0b42bb71b9e26ed7fec8894e21ce7fc06b94a1
Story: 2007455
Task: 39133
With the fix to the uefi code path so secure boot works properly
and is not accidently stomped on, we forgot to rescan the device
and force the partition table to update, which is vital for iscsi
based deploys.
Depends-On: https://review.opendev.org/706960
Change-Id: Ic2f338be075e93a2ce8c76c706d37db9bf8792ea
Story: 2007276
Task: 38713
This patch changes the workflow for whole disk images when using uefi.
If we can identify the bootloader and it's valid we can update using
efibootmgr since grub2-install have problems specially on secure boot
mode.
We also updated the regex to search for the uefi partition on the disk,
since in some cases the parted command output can be without the FS
for the partition with esp Flag.
Change-Id: I7167e71e5d2352a045565289b200e5530d0ba11d
Story: #2006847
Task: #37435
Lets not do silly things and if the disk looks bootable,
and we're not trying to do UEFI, then let us assume the
proper thing will occur upon power-up.
Looks at the boot sector data and if an executable is
found in the first 218 bytes, then it bypasses loading
a boot loader.
Also adds a dependency on the "file" linux distribution
package.
Change-Id: I11bc26670a08ee13174a43d7cd0f1ab9c1bd35cf
Story: 2006474
Task: 36410
When deploying an image to a software raid array, it is currently
required that the deployed image assembles the md arrays automatically
so that the rootfs can be mounted. In order to remove this
requirement/limitation on the deployed image we can add rd.md.uuid to
the kernel command line with the raid array's uuid.
Story: 2006648
Task: 36884
Change-Id: I42cb198753ecd84b7eaef6b5aa7c2064535bfe0e
Falls back to attempt to use findfs to locate
a UUID or PARTUUID match as opposed to trying to
list and enumerate through lsblk output.
Can confirm that tinycore 8.x's findfs binary works
expected.
Story: 2006724
Task: 37141
Change-Id: I4d488e4a1ab680eb1353b158c3339cb30b056ada
This patch proposes to extend the IPA to be able to deploy on
software RAID devices. The IPA needs to be able to detect an
md device, find the underlying holder disks and configure grub
on them.
Change-Id: Ieb2c95ff130b5cc1e643fcde500066d9458ddbec
Story: #2004581
Task: #29102
grub-mkconfig runs a lvs command that attempts to access /run/lvm
once for each block device, currently it times out after 10 seconds
for each device and moves on. Multiple 10 second delays become
a problem (causing IPA API timeouts) when multiple block devices
are present. Bind mounting in /run avoids the delay and the
timeouts.
Task: 30616
Story: 2005507
Change-Id: Iae8b7808a35bff121f64971aadd4bd36b5f5bb71
Installs the grub bootloader to the PreP Boot partition when the
prep_boot_partition_uuid is provided. This is required when
booting a partition image locally on ppc64* systems.
This change also passes the cpu_arch along to work_on_disk so
that the PReP partition is created when partitioning disks for
local boot on ppc64* systems,
Change-Id: I70667d43af962b357e6eeccba258f4fa5a91a09e
Depends-On: I2bc9f13ec605de7b7b96d96a1a4edebee0af76dc
Story: #1749057
Task: #22999
If mounting the root partition fails for some reason, we try to unmount
the EFI partition, which is not mounted at this point. This results in
a new exception hiding the real failure. This change fixes it.
Change-Id: I0ec636a361eda71b4149e4a7ba1538a9bbf6ec34
Closes-Bug: #1732932
Have our unit tests derive from the base unit test class in
ironic_python_agent/tests/unit/base: IronicAgentTest
This is so if we add additional global common features to our base
test class, all of our tests will get those common features.
Change-Id: I5188112f06dcfda4f5b0fd41fa9b9dd270cde8d7
Add missing 'autospec' keyword argument to mock.patch and
mock.patch.object calls. Use 'autospec=True' except for a few cases
where it fails because the mocked function is a @classmethod and it
doesn't work. In that case explicity set it to 'autospec=False'
Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
Use the flake8 plugin flake8-import-order to check import ordering. It
can do it automatically and don't need reviewers to check it.
Change-Id: I946457e9079ce0b54c7fe0ad554d024a1c61dce0
there is no guarantee that 'bash' is present in a user's image,
while 'sh' as system shell must be there.
As we do not use any bash-specific syntax (just starting commands in a
subshell) when installing the bootloader from chroot-ed user image,
let's be more supportive and use 'sh' instead of 'bash' for that.
Change-Id: I1fb82068b9c55da35166d8d2ecf9f0ba41356adb
Closes-Bug: #1657096
The following files added utf-8 encoding but never used. So we can
removes them at all.
ironic_python_agent/extensions/image.py
ironic_python_agent/extensions/iscsi.py
ironic_python_agent/tests/unit/extensions/test_image.py
ironic_python_agent/tests/unit/extensions/test_iscsi.py
Change-Id: Ifc98faf025ee81f885ea58d9203037c6357ea7f2
In systems where the nvram may have been reset, add
\EFI\BOOT\BOOTx64.EFI to the EFI system partition so that
the image continues to boot. This will be particularly useful
if the NVRAM wasn't written correctly or if the baremetal
server is be simulated on virt and the NVRAM was been reset
on start (as is the case in libvirt).
Change-Id: I5005ae02c283cdc01540ea46c341a7b0bf9d4c42
Closes-Bug: #1632637
This patch updates the _install_grub2 function in image.py
so that we use 'mount -t sysfs' instead of bind mounting
the hosts /sys into the chroot.
This resolves issues which can occur with unmounting
the chroot mounts where 'target is busy' errors
may occur when you are debugging things with a serial
console.
Change-Id: I5c9a6546d048cbf54695329d9744fda55127bd30
Closes-bug: #1503385
Bring ironic-python-agent in line with the other ironic projects.
Stop ignoring all E12* errors except E129
Stop ignoring E711
Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
Moved tests/extensions to tests/unit/extensions, otherwise they
wouldn't have run. Renamed the files in tests/unit/extensions to be
prefixed with 'test_', to match the rest of the test files. Did a
general clean up of tox.ini so we have standard whitespace.
Change-Id: I5615bfbf2d6ff4a4bc3e66e71c2472b0d1e3a2d7
This commit adds running 'udevadm settle' after
running partx so that the partition table is re-read
properly by lsblk which follows it.
Closes-bug: 1432918
Change-Id: I1c7dddd3ed8cab175e608e09ecc4ae7f289cfc31
This commit adds support for uefi systems in the image
extension so that grub can be installed onto efi system
partition for uefi machines.
Implements: blueprint local-boot-support-with-partition-images
Change-Id: I8fbb4b2ebdff991d41c7b618a4d654af26311a56
Initially this extension supports installing a bootloader so the user
image can boot from the local disk.
Change-Id: Ia588aafc240b55119c02f1254addc0cf796f88c5
Arne Wiebalck
Julia Kreger
Derek Higgins
Steve Baker
Dmitry Tantsur
Riccardo Pittau
Raphael Glon
mvpnitesh
Iury Gregory Melo Ferreira
Andrei Nistor
Matthew Thode
Michael Turek
Dmitry Tantsur
John L. Villalovos
Vasyl Saienko
Pavlo Shchelokovskyy
Cao Xuan Hoang
Dmitry Tantsur
Dan Prince
Josh Gachnang
Ramakrishnan G
Lucas Alvares Gomes