165 Commits

Author SHA1 Message Date
Dmitry Tantsur
c7c307c497 Software RAID: try to assemble RAID on start-up
TinyIPA does not do it, which prevents deployment on RAID devices.

Story: #2004581
Task: #36196
Change-Id: I6c1d90b4669102ab59588ab15f7166626c4d72be
2019-08-09 13:28:25 +02:00
Raphael Glon
258d963e40 Software raid: mbr/gpt partition table alternative
GPT/MBR partition table type alternative on softraid creation.

Change-Id: Ib3d00034fe687328a1ea3e038f6f86d2746e63a0
2019-08-08 11:44:25 +02:00
Dmitry Tantsur
88796a8936 Do not try to use zRAM devices for anything
These devices are mapped into memory, but nonetheless can appear
in lsblk output. Ignore them explicitly.

Change-Id: I2bb595ef29abdc98bd887bdf41c3c7f64cc3a7e6
2019-08-07 16:10:51 +02:00
Raphael Glon
570584f9ab Adds software raid creation support on nvme drives.
Change-Id: Ia725fcaee6018255af3dd7ff0eb2fb1cfbd2b322
Story: #2006341
Task: #36091
2019-08-06 13:23:08 +02:00
Zuul
6af02b25ea Merge "Trivial: Add missing quote when detecting raid devices" 2019-07-16 10:07:46 +00:00
Zuul
6bcd935092 Merge "Software RAID: Detect component & holder disks in broken RAIDs" 2019-07-16 10:07:42 +00:00
Arne Wiebalck
8e5430951a Trivial: Add missing quote when detecting raid devices
Change-Id: I26008295c62abaeb31a32682abd1de21c20d15a9
2019-07-16 08:59:43 +02:00
Zuul
21a742751a Merge "Include empty devices when listing block devices" 2019-07-15 17:18:20 +00:00
Arne Wiebalck
fbfdc49140 Include empty devices when listing block devices
When listing block devices, empty devices are not listed by
default. In order to clean up broken software RAID devices,
however, we need to list these as well. This patch adds the
corresponding option to 'lsblk', makes sure the device type
filtering handles the additional device type ('md' instead
of 'raid*' when an md device has only spares), and takes
care of lsblk reporting size '' for empty devices.

Story: #2006215
Task: #35807
Change-Id: I523e2ed90f3a477b5f5f27054b65fcfd26239ba2
2019-07-15 12:23:42 +02:00
Arne Wiebalck
6d2f9b986c Software RAID: Detect component & holder disks in broken RAIDs
This patch amends the component and holder disk detection code
to be able to also extract information from inactive/broken md
devices. This may be required in case of broken RAID-0 (which
will show up as an empty inactive device after a reboot which
none the less keeps its spare devices 'in-use' and prevents
RAID re-creation).

Story: #2006215
Task: #35808
Change-Id: I4ce41b335843e18c67668192d2fccff1c6cdfb92
2019-07-12 18:03:53 +02:00
Riccardo Pittau
8c9fd6dffc Using from_device_file from Devices
The from_device_file method in pyudev.Device is deprecated since
version 0.18
Using the same method from pyudev.Devices is now recommended.

Increasing minimum version of pyudev to 0.18 and correctly handling
exception raised by from_device_file.

Change-Id: I0637585241c81a36f69aef8017f1134f5c526eb1
2019-07-09 13:59:32 +02:00
Dmitry Tantsur
69064a8610 Correct formatting of a warning when lshw cannot be run
Change-Id: I7c55aee4b3b52cb414a595cf548e1f22e954f619
2019-06-26 11:37:07 +02:00
Dmitry Tantsur
94048fe97e Stop logging lshw output, collect it with other logs instead
The lshw output is huge even on virtual machines, and it pollutes
the debug logging. This change silences it. Instead, the lshw output
is collected as part of the ramdisk logs.

Depends-On: https://review.opendev.org/#/c/665635/
Change-Id: I6a3015b2d8d09f6f48b5cbd39dc84bd75b72f909
2019-06-17 14:00:26 +02:00
Zuul
b1e5c4b645 Merge "Software RAID: Protect RAID metadata upon cleaning" 2019-06-13 10:40:42 +00:00
Zuul
6e0749a1ec Merge "Get the hostname of the introspected host" 2019-06-12 15:57:22 +00:00
Arne Wiebalck
48f9914ee7 Software RAID: Protect RAID metadata upon cleaning
During automatic cleaning, the generic hardware manager erases the
data and the metadata on all block devices. If this is done on RAID
members, the RAID device is destroyed. This patch proposes to exclude
RAID members from (meta)data cleaning.

Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077
Story: #2004581
Task: #33640
2019-06-12 15:47:39 +02:00
Brad P. Crochet
7b3469ee0f Get the hostname of the introspected host
This will retrieve the hostname of the host being introspected, and
return it as part of the introspection data.

Change-Id: I54084251e1b0f3a40fe5ac760b2a7e45199c9fdb
Story: #2005867
Task: #33674
2019-06-12 13:00:21 +00:00
Arne Wiebalck
5140c3bda2 Software RAID: Don't preformat log strings
Remove the preformatting of log messages to avoid any formatting
is done when the corresponding log level not enabled.

Additional nits addressed:
- make the supported RAID levels a top-level frozenset
- remove unused return value from validate_configuration

Follow-up to Id20302537f7994982c7584af546a7e7520e9612b

Change-Id: If6484847a16f938c80523f0b7ec6ae52ea9cf598
2019-06-07 14:53:47 +02:00
Zuul
9bf743a768 Merge "Software RAID: Fix shell commands" 2019-06-07 02:31:46 +00:00
Arne Wiebalck
4535305c1d Software RAID: Fix shell commands
Pass an array to utils.execute() or enforce string usage
via 'shell=True', but don't mix.

Follow-up to Id20302537f7994982c7584af546a7e7520e9612b

Change-Id: I2e6c628360aecf81039089af78b19fe6a956e564
2019-06-06 13:59:10 +02:00
Zuul
3c91e55886 Merge "Software RAID: Add IPA deploy support" 2019-06-05 19:40:09 +00:00
Zuul
09d2db7c39 Merge "Software RAID: Create/delete configurations" 2019-06-05 07:34:01 +00:00
Arne Wiebalck
8fe390ecdd Software RAID: Add IPA deploy support
This patch proposes to extend the IPA to be able to deploy on
software RAID devices. The IPA needs to be able to detect an
md device, find the underlying holder disks and configure grub
on them.

Change-Id: Ieb2c95ff130b5cc1e643fcde500066d9458ddbec
Story: #2004581
Task: #29102
2019-06-04 12:43:09 +02:00
Arne Wiebalck
2db123d318 Software RAID: Create/delete configurations
This patch proposes to extend the IPA to be able to configure software
RAID devices. For this, the {create,delete}_configuration methods of
the GenericHardwareManager are implemented.

Change-Id: Id20302537f7994982c7584af546a7e7520e9612b
Story: #2004581
Task: #29101
2019-06-04 12:33:40 +02:00
Dongcan Ye
eb875cc9a2 Add more channel number for detecting BMC IPv6 address
This is a missing fix for patch.[1]

[1] https://review.opendev.org/#/c/656275/

Change-Id: I8443ae0c08487afea383e111b6c2b6d5941d4024
Task: 30654
Story: 2005528
2019-05-20 11:08:31 +00:00
Dongcan Ye
66d3c8e453 Add more channel number for detecting BMC IP address
IPMI SPEC[1] assignment channel number 1-11 for vendors
implementation(We an search "Channel Numbers" in section 6.3).
We met an baremetal product, like H3C, use number 8 as LAN channel.

Current code limit channel number to 1~7,
I think this can increase to 1~11.

[1] https://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/ipmi-second-gen-interface-spec-v2-rev1-1.pdf

Change-Id: I63a6ff91d702b990ef09da13e6e2e269e7274cce
Task: 30654
Story: 2005528
2019-05-05 10:29:58 +00:00
Zuul
e6eb4133b6 Merge "Ironic python agent does not extract correct available memory" 2019-04-16 15:16:05 +00:00
Uros Orozel
d8018b9f35 Ironic python agent does not extract correct available memory
This patch fixes parsing of memory class output of lshw output.
Task: 30201
Story: 2005308

Change-Id: I0e7ff9f86a266d2f798457136fed0100c78569e2
2019-04-15 10:59:42 +00:00
Zuul
55e3266c89 Merge "Discover IPv6 BMC address" 2019-04-10 03:49:03 +00:00
Kaifeng Wang
4cb2ac4ae4 Fix docs job failure due to malformated docstring
Change-Id: Ic3532e51481fd07e2f816aeacb07ded2d56791ee
2019-04-09 10:24:17 +08:00
Nikolay Fedotov
518c338ca3 Discover IPv6 BMC address
Change-Id: Ia7f24c1df782de4809a5195876bbf0309b481fd7
Story: #2005133
Task: #29817
2019-04-08 15:58:33 +03:00
Mark Goddard
b81b7c1c87 Add 'interface' to get_clean_steps docstring
Change-Id: Ice547f5eec849650828e1bd1aac48e7f041eeb34
2019-03-21 17:26:24 +00:00
Corey Wright
96961070ee Allow erasing metadata from disk partitions
Modify the metadata erasing call chain to retrieve a list of devices
that includes partitions in addition to disks so it can erase metadata
from all of them, otherwise incidentally recreating disk partitions
causes the Linux kernel to discover and automatically recreate some
types of storage entities (eg LVM PVs, VGs, & LVs, RAID members &
devices).

Change-Id: If8f47a083966051856439e3291a6872929b93e3b
Story: #2003673
Task: #26192
2018-11-13 00:36:18 -06:00
Zuul
3176ea483c Merge "Fix multi-device behavior" 2018-11-12 13:56:57 +00:00
Zuul
67a516bdcf Merge "Parallel erase disk devices" 2018-10-22 20:31:34 +00:00
Zhenguo Niu
1ee42cc3ff Parallel erase disk devices
Currently we erase the disks one by one, which takes a long
time to finish, this patch adds support to the IPA so that
it can erase disks in parallel if told so.

Story: 1546949
Task: 11548
Co-Authored-By: yuan liang <leetpy2@gmail.com>
Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com>

Change-Id: If5cfb6ec000a654d07103c4b378d4c135249e238
2018-10-22 13:56:51 +08:00
Pierre Riteau
8eecadb0a8 Fix BMC IP address detection on CoreOS images
On CoreOS images, IPA runs in a Debian Jessie chroot which uses mawk as
its default awk implementation. However, mawk doesn't support POSIX
character classes such as [:space:], which means get_bmc_address() fails
to parse the BMC IP address from the output of ipmitool.

This patch replaces the use of [[:space:]] by [ \t] which is equivalent
for the purpose of parsing the output of ipmitool. Note that matching on
tab characters is not strictly required as the packaged version of
ipmitool only uses space characters, but is left in case tabs are used
in other versions.

Change-Id: I0e3306a4d4584ca28e03608e9f7270b770960a39
Story: #2004121
Task: #27571
2018-10-19 14:22:05 +01:00
Julia Kreger
c540731aee Fix multi-device behavior
ATARAID is functionally a version of software
RAID where the setup is managed by the controller
and the Operating System takes over managing the
RAID after boot. Most commonly this is found for
mirrored boot devices.

Prior to this patch, we were looking for non-dependent
items (i.e. base block devices), with a type of disk.

Now we will permit the "disk" to be added to the list
if lsblk indicates that it is a type containing "raid".
The lsblk results should not change as we explicitly
look for disk objects.

Change-Id: Ia4a03b33cc06ce42e1bc33026683c28b31901cb7
Story: #2003445
Task: #24647
2018-09-24 19:10:38 -07:00
zhengyong
561be77303 Collect IPv6 address during introspection
This patch adds support to retrieve IPv6 address.

A new field ``ipv6_address`` is added to NetworkInterface
and store the assigned IPv6 address (if any).

Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com>
Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9
Closes-Bug: #1744064
Story: #1744064
Task: #11604
2018-07-24 09:22:40 +08:00
Zuul
105e580310 Merge "Provide knob to disable ata secure erase" 2018-06-29 15:12:37 +00:00
Julia Kreger
d9f59d94f2 Provide knob to disable ata secure erase
We need to allow the operator to able to
explicitly disable secure erase, in case
it is problematic in their environment
or hardware.

Change-Id: I4c68efa65cdd7f88f54f8dd9a8bcbeee9e8124a8
Story: #2002546
Task: #22108
2018-06-28 22:20:15 +00:00
Julia Kreger
aef703b879 Refuse secure erase if ATA command does not work
Adds dependency upon smartmontools's binary smartctl to
query the block devices via ATA mode which fails on pass-thru
buses such as ATA over SCSI and ATA over USB, in an effort
to prevent the initiation of ATA secure erase with one
of these interfaces in place which may render the disk
unreachable after security options are enabled for
ATA Secure Erase or upon the Secure Erase command being
sent to the Hard Disk.

Change-Id: I7635a197eb000650e919fac386b38ac15ef17041
Story: #2002546
Task: #22109
Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-28 18:38:10 +00:00
Julia Kreger
0f7b5a0896 Try to unlock failed device before proceeding
When a hard error has occured with secure erase,
we should attempt an unlock of the device becuase
the current mode can prevent disk IO. This may upset
some things like raid controllers even if they are
in a pass-through mode.

Change-Id: I32e1d962fbbb4a305d5dbebea92ac48ebd9b67ca
Story: #2002546
Task: #22107
2018-06-18 19:43:40 +00:00
Will Szumski
aaf76e2cfb rework ATA secure erase
hdparm versions prior to 9.51 interpret the value, NULL, as a
password with string value: "NULL".

Example output of hdparm with NULL password:

    [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda
    security_password="NULL"

    /dev/sda:
    Issuing SECURITY_UNLOCK command, password="NULL", user=user
    SECURITY_UNLOCK: Input/output error

Example output of hdparm with "" as password:

    [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda
    security_password=""

    /dev/sda:
     Issuing SECURITY_UNLOCK command, password="", user=user

Note the values of security_password in the output above. The output
was observed on a CentOS 7 system, which ships hdparm 9.43 in the
offical repositories.

This change attempts to unlock the drive with the empty string if an
unlock with NULL was unsucessful.

Issuing a security-unlock will cause a state transition from SEC4
(security enabled, locked, not frozen) to SEC5 (security enabled,
unlocked, not frozen). In order to check that a password unlock attempt
was successful it makes sense to check that the drive is in the unlocked
state (a necessary condition for SEC5). Only after all unlock attempts
fail, do we consider the drive out of our control.

The conditions to check the drive is in the right state have been
adjusted to ensure that the drive is in the SEC5 state prior to issuing
a secure erase. Previously, on the "recovery from previous fail" path,
the security state was asserted to be "not enabled" after an unlock -
this could never have been the case.

A good overview of the ATA security states can be found here:

  http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs

Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad
Story: 2001762
Task: 12161
Story: 2001763
Task: 12162
2018-05-16 13:18:15 +00:00
Hamdy Khader
4f9ee6ae5e GenericHardwareManager: get mac address using netifaces
Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:12:03 +02:00
Mike Turek
2877fc53d4 Use lshw in place of dmidecode for the default hardware manager
Currently the generic hardware manager uses dmidecode to get the
total physical memory and system details. This patch switches the
generic hardware manager to use lshw, as it is capable of reading
more than DMI [0]. This enables systems that do not support DMI
to use the generic hardware manager, such as IBM Power systems.

[0] https://github.com/lyonel/lshw/blob/master/README.md

Closes-Bug: #1715790
Change-Id: Ie370331df6bb5ef131c5cb60f458877e2a7ad71a
Depends-On: Idaf05b8efce28cd0cbf339cf693db4f55a693d9b
2018-01-30 15:54:10 +00:00
Zuul
e54bf7a8ae Merge "Report /dev/disk/by-path on inspection followup" 2017-12-14 19:09:52 +00:00
Ilya Etingof
8a05c0cee8 Report /dev/disk/by-path on inspection followup
This is the followup patch for
commit d0a53149f82a3587515a4371f0f4cad8570dc715) fixing
issues with the unit tests not addressed initially.

Change-Id: I7889bf908bcb64b79bf303c6ae356fd3f4e94a83
2017-12-14 01:12:16 +01:00
Ruby Loo
6da0268ebe Fix off-by-one error in warning
This fixes an off-by-one error in a warning message.

This is a follow-up to 3189c16a5e95ade468fa8bc37302eb9979f5a8c9.

Change-Id: I89b56974c1b919f4c03498873d3ce9860d5644c5
Related-Bug: #1670916
2017-11-09 10:42:22 -05:00
Zuul
ce32efc82b Merge "Fix waiting for target disk to appear" 2017-10-17 14:24:05 +00:00