2299 Commits

Author SHA1 Message Date
Dmitry Tantsur
f1ee454a0e Add mount and parted -l to the collected commands
Change-Id: I1c759552220291890704d0002a62ea3f51701691
2022-02-14 13:01:32 +01:00
Dmitry Tantsur
b8b1991bea Clean up release notes
Change-Id: I568d7edfe81e928e6d7f09bd4a7933ca72b8813a
2022-02-03 14:49:36 +01:00
Dmitry Tantsur
b921b76167 Make the standalone job voting again
We forgot to revert it. This job covers software RAID and manual
cleaning, so it's very important to avoid regressions, even if it costs
us some rechecks from time to time.

Change-Id: I2446afeaca866ffc3131b5e9f266526f35fc5ed7
2022-02-01 10:14:48 +01:00
Dmitry Tantsur
6ebf041704 Use canonical device name for RAID device for ESP
It seems like tinyIPA silently replaces /dev/md/esp with /dev/md127.
Find the next free /dev/md device and use it instead.

Also rescan the resulting device before copying files.

Change-Id: Ie04f530be434c4b1561e75f387b9da679e4607e0
Depends-On: https://review.opendev.org/c/openstack/ironic/+/827129/
2022-02-01 10:06:31 +01:00
Arne Wiebalck
62c5674a60 SoftwareRAID: Use efibootmgr (and drop grub2-install)
Move the software RAID code path from grub2-install to
efibootmgr:

- remove the UEFI efibootmgr exception for software RAID
- create and populate the ESPs on the holder disks
- update the NVRAM with all ESPs (the component devices
  of the ESP mirror, use unique labels to avoid unintentional
  deduplication of entries in the NVRAM)

Story: #2009794

Change-Id: I7ed34e595215194a589c2f1cd0b39ff0336da8f1
2022-01-26 14:43:40 +01:00
Zuul
e06dd22e78 Merge "Burn-in: Dynamic network pairing" 2022-01-20 21:17:38 +00:00
Arne Wiebalck
7f15455d8d Burn-in: Dynamic network pairing
Pair nodes dynamically via a distributed coordination backend for
network burn-in. The algorithm uses a group to pair nodes: after
acquiring a lock, a first node joins the group, releases the lock,
waits for a second node, then they both leave, and release the lock
for the next pair.

Story: #2007523
Task: #42796

Change-Id: I572093b144bc90a49cd76929c7e8685ed45d9f6e
2022-01-10 11:31:33 +01:00
Arne Wiebalck
0b69890c11 [trivial] Fix typo in __init__.py
Change-Id: I67810abbfb975c0d0ad0faf9807318c462580528
2021-12-16 22:03:51 +01:00
Ghanshyam Mann
bdf735c662 Re-add python 3.6/3.7 in classifier
We have updated the yoga testing runtime to keep the
py36 testing.

- https://review.opendev.org/c/openstack/governance/+/820195

Unit tests job template is also updated to keep python
3.6 as a voting job. So with the py3.6 and py3.9 testing as voting
job template, we are keeping python 3.6, 3.7, 3.8, and 3.8 as
tested versions in the Yoga cycle.

- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286

This commit re-add the python 3.6/3.7 versions in setup.cfg classifier.

Change-Id: I0f03a7f5bb2aa07c2ec2aab1a8ebfddc0c70ca87
2021-12-13 19:42:33 -06:00
Zuul
fa5cccd137 Merge "Burn-in: Add options for named log files" 2021-12-09 11:54:17 +00:00
Zuul
29432512ab Merge "Use only Yoga tests" 2021-12-09 11:40:24 +00:00
Zuul
60df149c8f Merge "Instruct qemu-img to write image zeros to disk." 2021-12-09 11:00:50 +00:00
Zuul
8abc930d97 Merge "Burn-in: Add SMART self test to disk burn-in" 2021-12-09 09:38:39 +00:00
Zuul
3cd964fa84 Merge "Prepare for bugfix release" 2021-12-08 19:27:57 +00:00
Arne Wiebalck
e751218059 Burn-in: Add options for named log files
In order to ease logging of the various burn-in steps, this patch
proposes options to define the outpout files for all burn-in steps:
{'agent_burnin_cpu', 'agent_burnin_vm', 'agent_burnin_fio_network',
'agent_burnin_fio_disk'}_outputfile  via a node's driver-info.

Story: #2007523
Task: #44102

Change-Id: I327cae5949d38e738d3c535487b3795d00ad8f1e
2021-12-08 17:47:19 +01:00
Derek Higgins
12f5f30e63 Instruct qemu-img to write image zeros to disk.
Doing this will cause it not to zero out the entire
block device which can be very costly on a slow HDD.

Story: 2009227
Task: 43315

Change-Id: I62ba2afc037d9844387e6b0984fe5008779d95d2
2021-12-08 15:56:05 +00:00
Riccardo Pittau
9b827944b0 Prepare for bugfix release
Change-Id: Ic9437c6d55879db45aa1aa3dc548ea62ed5ca10d
2021-12-07 10:38:58 +01:00
Riccardo Pittau
a35c77e62e [trivial] Fix Xena release notes versions
Change-Id: Iaf511e6aeae59409ac6d1ba4543d297c5bb2ec01
2021-12-06 11:09:03 +01:00
Arne Wiebalck
c6b1cb1c32 Burn-in: Add SMART self test to disk burn-in
Add the option to run a SMART self test right after
the disk burn-in. The disk burn-in step will fail if
the SMART test on any of the disk fails.

Story: #2007523
Task: #43383

Change-Id: I1312d5b71bedd044581a136af0b4c43769d21877
2021-12-06 09:09:35 +01:00
Riccardo Pittau
c31835e248 Use only Yoga tests
TC has decided to keep support for Python 3.6 during the Yoga cycle.
For more info see [1]

[1] http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026164.html

Change-Id: Icfe518fafa2b012e034a2e8ff18c242843df0086
2021-12-03 08:56:39 +01:00
Riccardo Pittau
87a42cc887 Test python 3.6 for distributions compatibility
As discussed during the ironic meeting on 29/11/2021, the community has
decided to continue testing the code against Python 3.6 to keep
compatibility with distributions that have native support for it, such
as CentOS Stream 8, Ubuntu Bionic, Debian 10, and openSuse Leap, at
least until it's possible and reasonable considering the support granted
from other dependencies and libraries, and not beyond the Yoga cycle.

Change-Id: I97ad7c4acdb278e2849a598c5a02a71fc67cc81a
2021-11-30 10:52:59 +01:00
Zuul
45ff41704a Merge "Updating yoga tested python versions in classifier" 2021-11-26 12:37:18 +00:00
Iury Gregory Melo Ferreira
4042e7b08c Get rid of lambda in RealFilePartitioningTestCase
This commit changes the lambda usage in the RealFilePartitioningTestCase
to autospec to avoid problems with unexpected args.

Change-Id: I21356a7783f105dde9ff0d3777e2a06f3f28a786
2021-11-25 11:21:32 +01:00
Ghanshyam Mann
dda53b42cb Updating yoga tested python versions in classifier
Yoga testing runtime has been updated with py38 and py39
as voting and removed the py36 testing. Unit tests update are
handled by the job template change in openstack-zuul-job and you
can see the updated jobs running in gate.

- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/818609

this commit updates the tested py versions in setup.cfg classifier.

[1] https://governance.openstack.org/tc/reference/runtimes/yoga.html

Change-Id: I47cd32e5690b55b983827c650efba4fd99e7be55
2021-11-24 20:07:26 -06:00
Zuul
bcf2846553 Merge "Trivial: split away efibootmgr helpers" 2021-11-23 12:26:17 +00:00
Zuul
4954fe3702 Merge "Call execute from ironic-lib in hardware.py" 2021-11-22 20:04:40 +00:00
Dmitry Tantsur
5cf61e797a Trivial: split away efibootmgr helpers
These are very useful for downstream deploy steps, make them public.

Change-Id: I26106a07049f751d3e3cc646431e2176001f4645
2021-11-19 17:27:27 +01:00
Dmitry Tantsur
abe38a6a5f Fix compatibility with disk_utils.find_efi_partition
This function returns the complete block device record, not just number.
Fixes regression in 89bc73aa0105850c6ae44428642e31802bba3b20.

Also fix the incorrect job in the gate queue, which prevented us from
catching this issue on merging.

Change-Id: I4cbc359ceabfc193ce18fed14a1952359460e7d9
2021-11-19 14:51:27 +01:00
Zuul
de385cb291 Merge "Remove metalsmith centos7 job" 2021-11-19 11:08:16 +00:00
Dmitry Tantsur
89bc73aa01 Use two more functions from disk_utils
Change-Id: If01c9cd7f95b4495509369786360741b731161db
2021-11-18 13:49:51 +01:00
Riccardo Pittau
7b03fbbb36 Call execute from ironic-lib in hardware.py
Replace the execute wrapper from utils with execute from ironic-lib in
hardware.py

Adjust unit tests as needed.

Change-Id: I63a3b0407b2ca2246bd0e6624bfa0f748c0d73f7
2021-11-18 07:52:48 +01:00
Dmitry Tantsur
36d4a18fbc Move manage_uefi from the image extension to a public location
This call is very useful for custom deploy implementations, such as one
we maintain for OpenShift. Splitting it out also makes image.py slightly
more manageable.

The get_partition call is moved to partition_utils.

Change-Id: I60a6a2823d3eb27a4ae78e913e3655dae7b54ffe
2021-11-16 17:58:16 +01:00
Zuul
f5efbc3e7e Merge "Simplify error messages when running clean/deploy step" 2021-11-13 07:35:50 +00:00
Julia Kreger
ce296a43d6 Remove metalsmith centos7 job
Change-Id: I523382b5b06b479ba34dce5a1dc7262076122514
2021-11-10 11:30:03 -08:00
Riccardo Pittau
a799dcc422 Move rescan device function to general utils
We use basically the same function in two modules in the same way, let's
put that in a common place.

Change-Id: I4016e43f2cb102d4327bafcc8a2f90112a6f944a
2021-11-10 15:34:37 +01:00
Dmitry Tantsur
c5fb191393 Simplify error messages when running clean/deploy step
The caller knows what step it invokes, there is no point in repeating
it in the error message. There is also no need to wrap the exception
if it's a RESTError or an ironic-lib exception already since they
are normally detailed enough.

Only leave a detailed message when an unexpected exception happens.

Change-Id: I1d8ca1e7ed1462159e4ae5f0bcf58686f6a2681c
2021-11-09 13:58:44 +01:00
Riccardo Pittau
23e67b5fea Re-read the partition table with partx -a, part 2
Use add instead of update to re-read the partition table with partx.

See [1] for more details.

Co-authored-by: Arne Wiebalck <arne.wiebalck@cern.ch>

[1] https: //opendev.org/openstack/ironic-python-agent/commit/dc8c1f16f9a00e2bff21612d1a9cf0ea0f3addf0

Change-Id: I2336e22dadc790cfbde87904612fcaa3b8c501db
2021-11-09 13:03:14 +01:00
Arne Wiebalck
dc8c1f16f9 Re-read the partition table with partx -a
Re-read the partition table with 'partx -a', rather than 'partx -u'.

This should fix an timing issue where the bootloader installation
fails to mount the EFI partition from a whole disk image since it
is not yet aware of the new partitions (observed with both, the
iscsi and the direct deploy interface).

Change-Id: If5da3075e813ae01df3decf8f0647aba111b0515
2021-11-06 13:43:48 +01:00
Zuul
0b56cca7f0 Merge "Fix UEFI record regex" 2021-11-05 14:59:35 +00:00
Julia Kreger
c5268bbdbb Fix UEFI record regex
I accidently put colons on the test data and remembered taking the
colon character out of the regex I was working on, but apparently
left it in, and accounted for the active entry indicator flag
which appears to have inconsistent support across vendors.

The regex has been fixed, and a test added from a Lenovo SR650
which has some additional string entry data in the UEFI output
which may separate entries.

Change-Id: I1f67b0fb1f645fa82e98bd7c7bba3ffc7755cc74
2021-11-04 09:45:25 -07:00
Zuul
a4b73058ee Merge "Always include the oslo_log log file in ramdisk logs" 2021-11-04 15:14:33 +00:00
Zuul
65827b3015 Merge "Stop requiring mocking of utils.execute if ironic-lib execute is mocked" 2021-11-03 14:19:52 +00:00
Julia Kreger
67eddfa7e3 Delete EFI boot entry duplicate labels first
Some firmware seems to take an objection with EFI nvram
entries being deleted after one is added, resulting in the
entire entry table being reset to the last known good state.

This is problematic, as ultimately deployments can time out
if we previously booted with Networking, and the machine, while
commanded to do other wise, reboots back to networking regardless.

We will now delete entries first, before proceeding.

Additionally, for general use, this pattern may serve the
community better by avoiding cases where we would have
previously just relied upon efibootmgr[0] to warn us of duplicate
entries.

[0]: 103aa22ece/src/efibootmgr.c (L228)

Change-Id: Ib61a7100a059e79a8b0901fd8f46b9bc41d657dc
Story: 2009649
Task: 43808
2021-11-01 06:59:26 -07:00
Dmitry Tantsur
2cedaa53c2 Always include the oslo_log log file in ramdisk logs
Even if journald is present, there is no guarantee that IPA logs there
(this is the case in container-based ramdisks).

Change-Id: Iceeab0010827728711e19e5b031ccac55fe1efde
2021-10-28 18:32:40 +02:00
Dmitry Tantsur
b360214cf2 Stop requiring mocking of utils.execute if ironic-lib execute is mocked
Based on unit tests, this was done intentionally, but I don't see
reasons for that. It makes refactoring much harder, because sometimes
you need to mock both execute functions and test them separately.

In the end, utils.execute should be removed.

Change-Id: I5a9c694ebe626c54f219d4870eab0a592777518d
2021-10-27 14:24:00 +02:00
Zuul
c824dda7a5 Merge "Fix error messages in burnin code" 2021-10-26 19:50:15 +00:00
Dmitry Tantsur
9a8c6a831a Fix error messages in burnin code
Change-Id: Ib920b6f8da2c2c9d72dbb8bbd8915bde767bddd2
2021-10-26 11:33:01 +02:00
Dmitry Tantsur
8a66978666 Respect global parameters when downloading a configdrive
* Use the same TLS parameters as everything else
* Respect image_download_connection_timeout
* Do not ignore HTTP errors

Change-Id: I84f8021f731186d82e44ac3d4ef2d12df13f830a
2021-10-20 15:11:16 +02:00
Arne Wiebalck
333ed70c94 Assert EFI part UUID is not None before editing fstab
The EFI partition UUID may be None and this will break
the fstab editing. While this is not necessarily fatal when
instantiating a node, it creates an exception at the end of
bootloader installation, so only attempt to add a line to
fstab when the UUID is not None.

Change-Id: I68799980e67c05afe4ca68ca9733605dd166d54d
2021-10-08 08:35:29 +02:00
Arne Wiebalck
9d707e9f4b Software RAID: Call udev_settle before creation
This patch fixes a race during software RAID creation:
we create the partition with parted, the kernel then
notifies udev, but we need to wait for udevd to create
the device files before calling mdadm to create the
md device.

Credits to jcosmao for finding this.

Change-Id: I642f28acc351cf50263e37dfbc8468bf59de2cc5
2021-10-05 11:42:49 +02:00