210 Commits

Author SHA1 Message Date
Zuul
5dcff4d2b3 Merge "Add raid.apply_configuration deploy step" 2020-04-21 09:32:40 +00:00
Zuul
6a95e216f6 Merge "Simplify deduplicate_steps" 2020-04-21 00:37:15 +00:00
Dmitry Tantsur
c0502649ba Add raid.apply_configuration deploy step
For compatibility with out-of-band RAID deploy steps, we need to have
one apply_configuration step, not a create/delete pair.

Change-Id: I55bbed96673c9fa247cafdac9a3ade3a6ff3f38d
Story: #2006963
2020-04-20 12:50:14 +02:00
Riccardo Pittau
3966871f47 Move logic to calculate raid sectors to raid_utils
Some more raid related logic moved to raid_utils.

Change-Id: I08c73ad14e5b01ebac2490b83997c5452506d4a2
2020-04-09 15:03:37 +02:00
Zuul
83b5a8b202 Merge "Move logic for raid start sector to raid_utils" 2020-04-09 12:31:29 +00:00
Zuul
b9e320e76f Merge "Add an ability to run in-band deploy steps" 2020-04-09 09:31:49 +00:00
Riccardo Pittau
f32a4a2b29 Move logic for raid start sector to raid_utils
A starting tentative in reducing size of raid related functions.

Change-Id: I81f912d0dc0ad138d8cc776cdb4ee3b5251ec3ba
2020-04-08 17:32:38 +02:00
Arne Wiebalck
66c32784af Editing follow-up for UEFI Software RAID support
This is a follow-up to https://review.opendev.org/#/c/696156/

Change-Id: I0fd2c09045ff07a57374934c35d4a3a8467f5e99
Story: #2006379
Task: #37635
2020-04-06 18:03:25 +02:00
Dmitry Tantsur
079f61d09c Simplify deduplicate_steps
The same result can be achieved using a multi-component sorting key.

Change-Id: Ieacf9fcecb2a6de7b4ccd8889f789099af39aa37
2020-04-06 10:30:31 +02:00
Mark Goddard
1b4ce47921 Add an ability to run in-band deploy steps
Mostly adaptation of cleaning methods.

Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com>
Change-Id: Ife0502391bbece46d619a20a825dfdb191d5c2b4
Story: 2006963
Task: 37791
2020-04-06 10:24:08 +02:00
Raphael Glon
9343348106 Software RAID: Add UEFI support
The proposed changes concern two steps:

First, when creating the RAID configuration, have a GPT partition
table type (this is not necessary, but more natural with UEFI).
Also, leave some space, either for the EFI partitions or the BIOS
boot partitions, outside the Software RAID.

Secondly, when installing the bootloader, make sure the correct
boot partitions are created or relocated.

Change-Id: Icf0a76b0de89e7a8494363ec91b2f1afda4faa3b
Story: #2006379
Task: #37635
2020-04-02 18:02:19 +02:00
Zuul
d71a8375fa Merge "Only check for partitions on devices that are part of software RAID" 2020-04-02 14:56:45 +00:00
Zuul
ab8c7c05bc Merge "Allow specifying target devices for software RAID" 2020-04-01 17:36:50 +00:00
Dmitry Tantsur
34b58f6024 Only check for partitions on devices that are part of software RAID
Now that an operator can pick the devices that participate in RAID,
it no longer makes sense to verify all devices.

Change-Id: Id5d8d539183f0db4ba3c4132ce6bc9919f9cd1ea
Story: #2006369
2020-04-01 16:02:05 +02:00
Riccardo Pittau
a332a19a57 Bump hacking to 3.0.0
Change-Id: I1032ea6a2e9d79aeaecb1458c319cbeb15ac1fff
2020-03-30 12:55:46 +02:00
Julia Kreger
bf0bb7a87a Improve debug logging around Raid/Bootloader
Change-Id: I7d34b918a859972a2d5650494824d3333016dd11
2020-03-28 08:55:32 -07:00
Dmitry Tantsur
ddbba07021 Allow specifying target devices for software RAID
This change adds support for the physical_disks RAID parameter in
a form of device hints (same as for root device selection).

Change-Id: I9751ab0f86ada41e3b668670dc112d58093b8099
Story: #2006369
Task: #39080
2020-03-17 13:03:24 +01:00
Julia Kreger
c1da514645 Ignore pyudev errors about device number
Pyudev is used to return extra data about a device
using the udev interface.

Sometimes that lookup doesn't quite work, like on
md devices after restarting them. As such, we will
now tollerate the failure and continue the process
as before.

Change-Id: Ibbc1759fe2cd3d7d09019b4e80d3c61d54c844dd
Story: 2007281
Task: 38726
2020-02-11 20:44:18 +00:00
Julia Kreger
cd7b2693f8 Skip read-only devices with metadata erase
HPE "Virtual Install Devices" appear as read-only block
devices, and may... or may not be visible depending on the
bios configuration state.

These devices can no longer be disabled from the bios settings
so the simplest course of action seems to be that we should
handle the existence of a read-only device.

In the event of secure erase, this is treated as a hard failure
case and a driver_internal_info flag has been added to enable
a future bypass method for knowledgable operators.

Change-Id: Ief8b360d11e654d8fae3a04a2a9f8d474a06e167
Story: 2007229
Task: 38502
2020-01-29 13:41:25 -06:00
Dmitry Tantsur
31b73b4984 Expose collector and hardware manager names via introspection data
This change adds a new introspection data field 'configuration'
with two lists: managers and collectors.

Change-Id: Ice0d7e6ecff3f319bc3a4f41617059fd6914e31c
2020-01-22 11:15:38 +01:00
Zuul
98838597c6 Merge "Allow reading root_device from instance_info" 2020-01-14 08:32:35 +00:00
Zuul
12b62d6c3a Merge "Collect lsblk and /proc/mdstat with ramdisk logs" 2020-01-10 09:22:29 +00:00
Dmitry Tantsur
96a094646b Allow reading root_device from instance_info
For the future deployment API we need to be able to set root_device
per deployment in addition to per node. This change adds it.

Change-Id: I1dd046c2e5fca211a84290bac8daa7550b21614f
Story: #2006910
Task: #37955
2020-01-03 17:29:05 +01:00
Zuul
6032643a04 Merge "Stop using six library" 2019-12-03 13:09:30 +00:00
Dmitry Tantsur
4354bc04f9 Replace netaddr dependency with stdlib ipaddress
netaddr is quite a big library, and all we need is covered by
the built-in ipaddress module (available in python 3).

Also add a safeguard for invalid 'ip route' output.

Change-Id: I9d76a8d1c1b6b1585e301a9c63b37fab3b98746f
2019-12-02 12:13:04 +01:00
Riccardo Pittau
ca7a46b113 Stop using six library
Since we've dropped support for Python 2.7, it's time to look at
the bright future that Python 3.x will bring and stop forcing
compatibility with older versions.
This patch removes the six library from requirements, not
looking back.

Change-Id: I4795417aa649be75ba7162a8cf30eacbb88c7b5e
2019-11-29 10:18:14 +01:00
Julia Kreger
e4659c94cf RAID 5/6
Looks like adding RAID 5/6 support may be easier than most
could immagine. The code, as written appears to be safe and
logical creating a RAID 5 or RAID 6 volume.

Not that we can really test this in CI, but it seems only
validation code needs to be changed to loosen the constraint.

Change-Id: Ib891b3c97f0bfb02af3b59581a451c4b25e03b85
2019-11-27 20:30:28 +00:00
Andrei Nistor
1975478097 Set rd.md.uuid kernel parameter when deploying on software raid
When deploying an image to a software raid array, it is currently
required that the deployed image assembles the md arrays automatically
so that the rootfs can be mounted. In order to remove this
requirement/limitation on the deployed image we can add rd.md.uuid to
the kernel command line with the raid array's uuid.

Story: 2006648
Task: 36884
Change-Id: I42cb198753ecd84b7eaef6b5aa7c2064535bfe0e
2019-10-17 11:14:04 +00:00
Dmitry Tantsur
11976c9d2b Collect lsblk and /proc/mdstat with ramdisk logs
This should improve debugability of partitioning problems.

Change-Id: I3c7ae3f2831c9900a3f0d24daec6dd6b8bea6a60
2019-10-14 15:28:08 +02:00
Zuul
3968ec9d5a Merge "Delete_configuration, consider removed raid members as well" 2019-09-24 11:51:06 +00:00
raphael.glon
316dd3f952 Revert "Software raid: mbr/gpt partition table alternative"
This reverts commit 258d963e406c512bb90295c700ee22e4609abcd0.

Remove the mbr/gpt choice from softraid features for now, as it cannot be
directly used without additional commits, to be pushed soon. Furthermore, it
could even lead to instance spawn misconfiguration, if the disk label
specified in instance_info cannot fit with the boot mode + partitioning layout
(example: you build softraid over gpt, then you lose the mbr gap. Thus you need
an additional bios boot partition, in BIOS boot mode, or an esp, in UEFI boot
mode, outside the raid, for grub to install itself inthere, to be able to
assemble raid and find root device correctly).

Change-Id: I3a0a704ea99a40eb3fc8e879270dfbd356951488
2019-09-23 12:26:37 +02:00
Raphael Glon
37385dd9dd Delete_configuration, consider removed raid members as well
Change-Id: Ie4f62d8855d3f30a55f7032918dfed1f1d8b5b31
Story: #2006535
Task: #36591
2019-09-16 11:27:27 +02:00
Zuul
6d3d1367a9 Merge "Handle floppy disk controllers" 2019-08-27 07:53:00 +00:00
Julia Kreger
b98bc678b9 Handle floppy disk controllers
Cleaning presently fails on floopy disk controllers.

While they may be uncommon in server hardware, they
can exist in virtual machines, and even as virtual
devices on some hardware chassises.

Change-Id: I8ba07bfd5ca1e503f46c1bac4fffb5f509186939
Story: 2006419
Task: 36309
2019-08-22 08:58:24 -04:00
Zuul
0eff77ac8d Merge "Softraid: partitioning fixes" 2019-08-21 11:58:22 +00:00
Zuul
187d5da057 Merge "Software RAID: Ignore missing component devices or holder disks" 2019-08-19 13:27:41 +00:00
Arne Wiebalck
9dca5d76b3 Software RAID: Ignore missing component devices or holder disks
With the addition of devices with type 'MD' during
list_all_block_devices (for "empty" devices), partitions on top
of md devices are now returned as well. These don't need deletion,
so skip these during software RAID cleanup.

Story: #2006355
Task: #36121

Change-Id: Idff9b9b0cbe42b57aa315ae07c0b51c35c9ade1e
2019-08-16 11:35:07 +02:00
Raphael Glon
8ad9267aa0 Softraid: partitioning fixes
Fixes:
- Softraid creation: size_gb unit conversion to default parted unit.
- Softraid creation issue when we specify several logical disks with
  more than one having its size specified (not 'MAX').
- Softraid creation issue when a logical disk with size_gb 'MAX' is
  not last in the list.

Change-Id: I13e087611354a35acc2925037f1b10047477d972
Story: 2006352
Task: 36118
2019-08-15 23:48:15 +02:00
Zuul
680143a4da Merge "Fixes get_holder disks with nvme drives" 2019-08-14 09:14:55 +00:00
Zuul
ac6542cab8 Merge "Software RAID: Handle empty target_raid_config" 2019-08-13 10:01:19 +00:00
Raphael Glon
c546749423 Fixes get_holder disks with nvme drives
Change-Id: I195ffdeeb3c13bdec5fc1735b82efa53c8d9d3de
2019-08-13 10:37:18 +02:00
Zuul
6b778e171b Merge "Fixes physical memory fetching for aarch64" 2019-08-12 16:02:44 +00:00
Arne Wiebalck
5dd548dc7b Software RAID: Handle empty target_raid_config
If the target_raid_config is empty, we should not try
to create software RAID devices.

Change-Id: I5a29dc34a07903be0cc1b19594f160ac63c6ed2d
2019-08-12 11:38:28 +02:00
Dmitry Tantsur
3d5fc82ece Ignore /dev/ramX devices for any purposes
While most modern Linux distributions do not create RAM disks, newer
TinyCoreLinux actually does. Ignore them since we don't want to deploy
to RAM.

Change-Id: Ia810c7fb47f7098b8173d765ff9c33b48a223ff1
2019-08-12 09:22:52 +02:00
Dmitry Tantsur
c7c307c497 Software RAID: try to assemble RAID on start-up
TinyIPA does not do it, which prevents deployment on RAID devices.

Story: #2004581
Task: #36196
Change-Id: I6c1d90b4669102ab59588ab15f7166626c4d72be
2019-08-09 13:28:25 +02:00
Raphael Glon
258d963e40 Software raid: mbr/gpt partition table alternative
GPT/MBR partition table type alternative on softraid creation.

Change-Id: Ib3d00034fe687328a1ea3e038f6f86d2746e63a0
2019-08-08 11:44:25 +02:00
Dmitry Tantsur
88796a8936 Do not try to use zRAM devices for anything
These devices are mapped into memory, but nonetheless can appear
in lsblk output. Ignore them explicitly.

Change-Id: I2bb595ef29abdc98bd887bdf41c3c7f64cc3a7e6
2019-08-07 16:10:51 +02:00
Raphael Glon
570584f9ab Adds software raid creation support on nvme drives.
Change-Id: Ia725fcaee6018255af3dd7ff0eb2fb1cfbd2b322
Story: #2006341
Task: #36091
2019-08-06 13:23:08 +02:00
liyingjun
cd50fd1151 Fixes physical memory fetching for aarch64
The output for aarch64[1] `lshw -quiet -json` is a bit different from
x86_64, this will lead to 0 physical memory calculated currently. This
patch aims to make it compatible for aarch64 when getting physical memory

[1]: http://paste.openstack.org/show/754789/

Change-Id: Ib7b4ee857d8550e3d75ac8a53c61f0ecab45feff
2019-07-30 15:51:17 +08:00
Zuul
6af02b25ea Merge "Trivial: Add missing quote when detecting raid devices" 2019-07-16 10:07:46 +00:00