595 Commits

Author SHA1 Message Date
Zuul
c5f31db691 Merge "Attempt to read the partition table after writing an image" 2019-01-04 08:01:22 +00:00
Sam Betts
fc2dfcee60 Attempt to read the partition table after writing an image
This patch adds code that tries to read the partition table after we've
successfully written an image to make sure the image that we wrote has a
valid partition table so we can more easily guarantee that what we've
written is bootable and not just junk. Without a valid partition table
writing a config drive will fail for whole disk images.

Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com>
Change-Id: I5cfd8c433a4db3e0d2d5086250e629d16234b7a4
Story: 2001760
Task: 12159
2018-11-19 18:57:23 +01:00
Corey Wright
96961070ee Allow erasing metadata from disk partitions
Modify the metadata erasing call chain to retrieve a list of devices
that includes partitions in addition to disks so it can erase metadata
from all of them, otherwise incidentally recreating disk partitions
causes the Linux kernel to discover and automatically recreate some
types of storage entities (eg LVM PVs, VGs, & LVs, RAID members &
devices).

Change-Id: If8f47a083966051856439e3291a6872929b93e3b
Story: #2003673
Task: #26192
2018-11-13 00:36:18 -06:00
Zuul
3176ea483c Merge "Fix multi-device behavior" 2018-11-12 13:56:57 +00:00
Zuul
f63099ebb6 Merge "Allow streaming raw partition images" 2018-10-26 14:14:55 +00:00
Kaifeng Wang
6f42768756 Follow up to parallel disk erasure
Improve test to verify apply_async is called twice as expected.

Story: 1546949
Task: 11548

Change-Id: I41736dfb2932dd0036bbc4cbc51929bf61a16569
2018-10-24 10:51:43 +08:00
Zuul
67a516bdcf Merge "Parallel erase disk devices" 2018-10-22 20:31:34 +00:00
Zhenguo Niu
1ee42cc3ff Parallel erase disk devices
Currently we erase the disks one by one, which takes a long
time to finish, this patch adds support to the IPA so that
it can erase disks in parallel if told so.

Story: 1546949
Task: 11548
Co-Authored-By: yuan liang <leetpy2@gmail.com>
Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com>

Change-Id: If5cfb6ec000a654d07103c4b378d4c135249e238
2018-10-22 13:56:51 +08:00
Zuul
5ad766ebcb Merge "Enhanced checksum support" 2018-10-20 11:10:55 +00:00
Pierre Riteau
8eecadb0a8 Fix BMC IP address detection on CoreOS images
On CoreOS images, IPA runs in a Debian Jessie chroot which uses mawk as
its default awk implementation. However, mawk doesn't support POSIX
character classes such as [:space:], which means get_bmc_address() fails
to parse the BMC IP address from the output of ipmitool.

This patch replaces the use of [[:space:]] by [ \t] which is equivalent
for the purpose of parsing the output of ipmitool. Note that matching on
tab characters is not strictly required as the packaged version of
ipmitool only uses space characters, but is left in case tabs are used
in other versions.

Change-Id: I0e3306a4d4584ca28e03608e9f7270b770960a39
Story: #2004121
Task: #27571
2018-10-19 14:22:05 +01:00
Dmitry Tantsur
29136bf68d Allow streaming raw partition images
Currently we support streaming raw whole disk images, but not
partition ones. This change enables it.

Change-Id: Ie95102aa3f2054a6b429f3d3e0926e90923c5faf
Story: #2003809
Task: #26558
2018-10-17 11:16:04 +02:00
Kaifeng Wang
ec2bf8667d Enhanced checksum support
Adds enhanced checksum support to IPA, when os_hash_algo and os_hash_value
are passed in via image_info, it will be used to calculate image checksum
and verification.

In other cases, the old md5 checksum is used.

Change-Id: I1d2f33e7059910326b4ac3f7786543b333a93a5a
Story: 2003938
Task: 26846
2018-10-15 17:15:38 +08:00
Julia Kreger
c540731aee Fix multi-device behavior
ATARAID is functionally a version of software
RAID where the setup is managed by the controller
and the Operating System takes over managing the
RAID after boot. Most commonly this is found for
mirrored boot devices.

Prior to this patch, we were looking for non-dependent
items (i.e. base block devices), with a type of disk.

Now we will permit the "disk" to be added to the list
if lsblk indicates that it is a type containing "raid".
The lsblk results should not change as we explicitly
look for disk objects.

Change-Id: Ia4a03b33cc06ce42e1bc33026683c28b31901cb7
Story: #2003445
Task: #24647
2018-09-24 19:10:38 -07:00
Tao Li
8a70f12435 Replace assertEqual(True/False, expr) with assertTrue/assertFalse in tests
In some cases, If the result of expr is a boolen value, we shoud use
assertTrue/assertFalse to instead. Beacause it is clear and simple.

Change-Id: Ie61369f6335a90b09bb24192282d33da5272c13f
Story: #2003785
Task: #26490
2018-09-18 16:14:20 +08:00
Giulio Fidente
bc21b5b140 Clear GPT and MBR headers with dd to avoid sgdisk CRC errors
This change adds a dd before the existing sgdisk -Z command to
workaround CRC verification errors.

Change-Id: Ia1ac4e1c0faf14ad4bb11c2a1c796c93ca8cb5e3
Closes-Bug: #1737556
Story: 1737556
Task: 11496
2018-08-08 16:40:22 +00:00
Zuul
667589bb00 Merge "fall back to PARTUUID if UUID not found." 2018-08-03 15:40:51 +00:00
Zuul
d009263349 Merge "Collect IPv6 address during introspection" 2018-08-02 14:05:14 +00:00
Matthew Thode
7e0bcbbeee fall back to PARTUUID if UUID not found.
Change-Id: Icbf1fd8179658bd8bcd15f99aaaee796288dbf6f
Story: 2002052
Task: 19714
2018-07-31 13:01:33 -07:00
zhengyong
561be77303 Collect IPv6 address during introspection
This patch adds support to retrieve IPv6 address.

A new field ``ipv6_address`` is added to NetworkInterface
and store the assigned IPv6 address (if any).

Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com>
Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9
Closes-Bug: #1744064
Story: #1744064
Task: #11604
2018-07-24 09:22:40 +08:00
Michael Turek
b32750f5c4 Install grub to PReP partition when prep_boot_part_uuid is provided
Installs the grub bootloader to the PreP Boot partition when the
prep_boot_partition_uuid is provided. This is required when
booting a partition image locally on ppc64* systems.

This change also passes the cpu_arch along to work_on_disk so
that the PReP partition is created when partitioning disks for
local boot on ppc64* systems,

Change-Id: I70667d43af962b357e6eeccba258f4fa5a91a09e
Depends-On: I2bc9f13ec605de7b7b96d96a1a4edebee0af76dc
Story: #1749057
Task: #22999
2018-07-20 16:07:16 +00:00
Zuul
105e580310 Merge "Provide knob to disable ata secure erase" 2018-06-29 15:12:37 +00:00
Julia Kreger
d9f59d94f2 Provide knob to disable ata secure erase
We need to allow the operator to able to
explicitly disable secure erase, in case
it is problematic in their environment
or hardware.

Change-Id: I4c68efa65cdd7f88f54f8dd9a8bcbeee9e8124a8
Story: #2002546
Task: #22108
2018-06-28 22:20:15 +00:00
Julia Kreger
aef703b879 Refuse secure erase if ATA command does not work
Adds dependency upon smartmontools's binary smartctl to
query the block devices via ATA mode which fails on pass-thru
buses such as ATA over SCSI and ATA over USB, in an effort
to prevent the initiation of ATA secure erase with one
of these interfaces in place which may render the disk
unreachable after security options are enabled for
ATA Secure Erase or upon the Secure Erase command being
sent to the Hard Disk.

Change-Id: I7635a197eb000650e919fac386b38ac15ef17041
Story: #2002546
Task: #22109
Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-28 18:38:10 +00:00
Julia Kreger
0f7b5a0896 Try to unlock failed device before proceeding
When a hard error has occured with secure erase,
we should attempt an unlock of the device becuase
the current mode can prevent disk IO. This may upset
some things like raid controllers even if they are
in a pass-through mode.

Change-Id: I32e1d962fbbb4a305d5dbebea92ac48ebd9b67ca
Story: #2002546
Task: #22107
2018-06-18 19:43:40 +00:00
Bill Dodd
3c30088c1e Add min/max values to integer config options
None of the existing ironic-python-agent integer config options included
min or max values. Added appropriate min/max values for the integer
config options.

Two of the integer options are for ports (listen_port and
advertise_port). These were changed to use the more appropriate
oslo_config cfg.PortOpt instead of cfg.IntOpt. PortOpt has the proper
min and max values built in.

Change-Id: I98709a45d099aea62c9973beb6817591cb445a9c
Story: 1731950
2018-05-23 12:08:42 -05:00
Zuul
cd96627760 Merge "write byte objects when using os.write" 2018-05-18 14:30:31 +00:00
Zuul
bcb4f3652d Merge "Fix for fatal error when GPT was used and only MBR was cleaned" 2018-05-18 14:30:29 +00:00
Zuul
696d1cf421 Merge "rework ATA secure erase" 2018-05-17 19:39:54 +00:00
Matthew Thode
a03661c4a8
write byte objects when using os.write
Change-Id: I184a9d0bf4a0ba0776d519b3a3b9ccd39151b4ae
Story: 2002052
Task: 19713
2018-05-17 11:11:55 -05:00
Olivier Bourdon
5fcf3f066c Fix for fatal error when GPT was used and only MBR was cleaned
You can generate this error if after having provisioned a node
using GPT partitioning, you clean its MBR using say
dd if=/dev/zero bs=1024 count=1 of=/dev/sda
and then cleanup all Ironic/Bifrost informations to get it
reprovisioned.
In this case sgdisk -Z returns an error and last_error field
in Ironic contains:
Error writing image to device: Writing image to device
        /dev/sda failed with exit code 2
Caution: invalid main GPT header, but valid backup;
        regenerating main header\nfrom backup!\n
        \nInvalid partition data!\

Change-Id: Ib617737fff5e40cb376edda0232e0726d9c71231
2018-05-17 12:51:55 +02:00
Will Szumski
aaf76e2cfb rework ATA secure erase
hdparm versions prior to 9.51 interpret the value, NULL, as a
password with string value: "NULL".

Example output of hdparm with NULL password:

    [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda
    security_password="NULL"

    /dev/sda:
    Issuing SECURITY_UNLOCK command, password="NULL", user=user
    SECURITY_UNLOCK: Input/output error

Example output of hdparm with "" as password:

    [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda
    security_password=""

    /dev/sda:
     Issuing SECURITY_UNLOCK command, password="", user=user

Note the values of security_password in the output above. The output
was observed on a CentOS 7 system, which ships hdparm 9.43 in the
offical repositories.

This change attempts to unlock the drive with the empty string if an
unlock with NULL was unsucessful.

Issuing a security-unlock will cause a state transition from SEC4
(security enabled, locked, not frozen) to SEC5 (security enabled,
unlocked, not frozen). In order to check that a password unlock attempt
was successful it makes sense to check that the drive is in the unlocked
state (a necessary condition for SEC5). Only after all unlock attempts
fail, do we consider the drive out of our control.

The conditions to check the drive is in the right state have been
adjusted to ensure that the drive is in the SEC5 state prior to issuing
a secure erase. Previously, on the "recovery from previous fail" path,
the security state was asserted to be "not enabled" after an unlock -
this could never have been the case.

A good overview of the ATA security states can be found here:

  http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs

Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad
Story: 2001762
Task: 12161
Story: 2001763
Task: 12162
2018-05-16 13:18:15 +00:00
Julia Kreger
3164053f08 Fix gate and bump CoreOS version to latest stable.
Increases the amount of ram for CoreOS IPA to 2GB
as the base CoreOS image is now 310MB.

Bumped CPU count for CoreOS runs to 2 CPUs as the
concurrency helps boot times for the CoreOS ramdisk.

Adds netbase, udev, and open-iscsi to debian jessie container
as they are no longer present in the default container.

Explicitly set path variable for execution in the debian
container as udevadm is in /sbin, and we may not have
/sbin on the path that is passed through to the
chroot.

Also fixed new pep8 test failures.

Story: #1600228
Task: #16287
Change-Id: I488445dfd261b7bca322a0be7b4d8ca6105750a3
2018-05-10 15:50:05 -07:00
Hamdy Khader
4f9ee6ae5e GenericHardwareManager: get mac address using netifaces
Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:12:03 +02:00
Moshe Levi
7631902ec7 mlnx hardware_manager: get mac address using netifaces
Change-Id: I62e4ca6afb0f8a256ddf9bbc48225fbb96f9e817
2018-03-11 12:50:08 +02:00
Zuul
de32d66ce3 Merge "When ouputting to /dev/null should open in write mode" 2018-02-07 21:45:49 +00:00
John L. Villalovos
944a77c9c6 When ouputting to /dev/null should open in write mode
Even though it was working opening the file in 'read' mode, it really
should be opened in 'write' mode, since we are redirecting the output
to the file.

Interestingly it does fail in 'read' mode if the command is:
    echo something

But passes in 'write' mode.

Change-Id: Ic67091881e0be377e527b78d270ab48962881ae0
2018-02-06 23:01:53 -08:00
Zuul
ca710ca692 Merge "Replace use of functools.wraps() with six.wraps()" 2018-02-06 01:27:41 +00:00
John L. Villalovos
8873f5eb91 Replace use of functools.wraps() with six.wraps()
In Python 2.7, functools.wraps() does not provide the '__wrapped__'
attribute. This attribute is used by
oslo_utils.reflection.get_signature() when getting the signature of a
function. If a function is decorated without the '__wrapped__'
attribute then the signature will be of the decorator rather than the
underlying function.

From the six documentation for six.wraps():
    This is exactly the functools.wraps() decorator, but it sets the
    __wrapped__ attribute on what it decorates as functools.wraps()
    does on Python versions after 3.2.

Change-Id: Ic0f7a6be9bc3e474a0229b264d1bfe6c8f7e6d85
2018-02-01 15:15:00 -08:00
Zuul
7dfd44dc13 Merge "Use lshw in place of dmidecode for the default hardware manager" 2018-02-01 00:00:31 +00:00
Shivanand Tendulker
f08636fe8b Follow-up patch for rescue extension for CoreOS
This patch addresses few minor comments in commit
a659306272542dd38420cb118cc7b04b1e8cf377

Change-Id: Id5b48e3cc96c8807c471c947da3e233cebdf687e
Related-Bug: #1526449
2018-01-30 19:00:13 +00:00
Mike Turek
2877fc53d4 Use lshw in place of dmidecode for the default hardware manager
Currently the generic hardware manager uses dmidecode to get the
total physical memory and system details. This patch switches the
generic hardware manager to use lshw, as it is capable of reading
more than DMI [0]. This enables systems that do not support DMI
to use the generic hardware manager, such as IBM Power systems.

[0] https://github.com/lyonel/lshw/blob/master/README.md

Closes-Bug: #1715790
Change-Id: Ie370331df6bb5ef131c5cb60f458877e2a7ad71a
Depends-On: Idaf05b8efce28cd0cbf339cf693db4f55a693d9b
2018-01-30 15:54:10 +00:00
yuan liang
f55b8a34c4 Execute error in _detect_cna_card
A list type parameter pass to utils.execute will raise OSError.

Change-Id: Ic5dd30f7e819e433d05bf9cc888902abe7a82def
2018-01-26 18:34:56 +00:00
Zuul
0607774be9 Merge "Remove /ironic-python-agent/api/app.wsgi" 2017-12-19 23:41:11 +00:00
Zuul
5ee16ee2e8 Merge "Do not try unmounting the EFI partition if it was not mounted" 2017-12-15 08:40:16 +00:00
Dmitry Tantsur
db4694de24 Do not try unmounting the EFI partition if it was not mounted
If mounting the root partition fails for some reason, we try to unmount
the EFI partition, which is not mounted at this point. This results in
a new exception hiding the real failure. This change fixes it.

Change-Id: I0ec636a361eda71b4149e4a7ba1538a9bbf6ec34
Closes-Bug: #1732932
2017-12-14 19:52:53 -05:00
Zuul
e54bf7a8ae Merge "Report /dev/disk/by-path on inspection followup" 2017-12-14 19:09:52 +00:00
Vu Cong Tuan
e6033e733e Remove /ironic-python-agent/api/app.wsgi
/ironic-python-agent/api/app.wsgi is an empty file.
As suggestion from John L. Villalovos, we probably should delete it.

Change-Id: I695aca42b76dfad1b74418c05a48c5cba3b7d71e
2017-12-14 14:20:14 +07:00
Ilya Etingof
8a05c0cee8 Report /dev/disk/by-path on inspection followup
This is the followup patch for
commit d0a53149f82a3587515a4371f0f4cad8570dc715) fixing
issues with the unit tests not addressed initially.

Change-Id: I7889bf908bcb64b79bf303c6ae356fd3f4e94a83
2017-12-14 01:12:16 +01:00
Zuul
e55d68f11e Merge "Include IPA Version during heartbeat" 2017-12-13 17:18:25 +00:00
Zuul
e789ec8898 Merge "Catch OSError thrown when hexdump is missing" 2017-12-12 21:08:13 +00:00