openstack/ironic-python-agent
Code Issues Proposed changes
77ca889d9e
ironic-python-agent/ironic_python_agent/tests/unit/test_hardware.py

3176 lines
136 KiB
Python
Raw Normal View History

Use # instead of """ for copyright blocks Reformats copyright messages to be comments rather than docstring-style blocks. Change-Id: I4d863f53b67bb49d03bda0952b9e6179b6d23c59
2014-04-09 14:31:31 -07:00
# Copyright 2013 Rackspace, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
add tests of hardware module
2014-01-07 16:36:59 -08:00
Add a new Hardware Manager for CNA network card This patch adds a new hardware manager, which will disable the embedded LLDP agent on Intel CNA network cards in order to allow the gathering of LLDP data during the inspection process. Change-Id: I572756ac6a7bf67a7f446738ba9d145e1c1bdc48 Closes-Bug: #1623659
2016-11-23 17:14:15 +01:00
import binascii
Parallel erase disk devices Currently we erase the disks one by one, which takes a long time to finish, this patch adds support to the IPA so that it can erase disks in parallel if told so. Story: 1546949 Task: 11548 Co-Authored-By: yuan liang <leetpy2@gmail.com> Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: If5cfb6ec000a654d07103c4b378d4c135249e238
2016-07-25 19:07:27 +08:00
import multiprocessing
Wait for at least one suitable disk to appear on start up Some kernel modules take substantial time to initialize. For example, with mpt2sas RAID driver inspection and deployment randomly fail due to IPA starting before the driver finishes initialization. This problem is probably impossible to solve in a generic case, as modern Linux environment do not have a notion of "hardware is fully initialized" moment. All hardware is essentially hotplug. To solve it at least for the simplest case, this patch adds a wait loop on start up waiting for at least one suitable disk to appear in inventory. Note that root device hints are not considered, as the node might not be known at that moment yet. Change-Id: Id163ca28f7c140c302ea04947ded3f3c58b284de Partial-Bug: #1582797
2016-05-24 10:04:12 +02:00
import os
import time
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
from ironic_lib import disk_utils
add tests of hardware module
2014-01-07 16:36:59 -08:00
import mock
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
import netifaces
Fix error in in-band disk erase using shred in-band disk erase using shred fails with error "'module' object has no attribute 'ProcessExecutionError'". This commit is to fix the issue. Change-Id: Ia0c426074b2f0e9d534ed96a3e213933160edc61 Closes-Bug:#144799
2015-04-24 15:08:41 +05:30
from oslo_concurrency import processutils
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
from oslo_config import cfg
Fix get_os_install_device() Instead of silently failing, raise DeviceNotFound when no root device hints were provided and all found block devices are smaller than 4GB. Change-Id: Idd2e2c5905adf847f00ad15a84a817c3715225dd Closes-Bug: #1490761
2015-09-02 15:21:04 +00:00
from oslo_utils import units
Add support for root device hints This patch add support for root device hints on IPA. Instead of picking the first disk >= 4G, if the hints are specified IPA will look at it to decide which device it should pick for the deployment. The initial patch supports the following hints: Size, model, WWN, serial, vendor. Implements: blueprint ipa-as-default-ramdisk Implements: blueprint root-device-hints Change-Id: I2b00b3fb3b61001033750dd8951f9353d6f2e361
2015-03-10 11:20:40 +00:00
import pyudev
Parallel erase disk devices Currently we erase the disks one by one, which takes a long time to finish, this patch adds support to the IPA so that it can erase disks in parallel if told so. Story: 1546949 Task: 11548 Co-Authored-By: yuan liang <leetpy2@gmail.com> Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: If5cfb6ec000a654d07103c4b378d4c135249e238
2016-07-25 19:07:27 +08:00
import six
Fix HardwareManager loading order bug The agent attempts to sort the HardwareManager implementations found by stevedore in order of suitability. This patch fixes a bug in the sorting order that caused the GenericHardwareManager to always be loaded. Fixes bug 1327923 Change-Id: I1cc20e5351a704325f0cf41dab205c9164799f6a
2014-06-08 18:45:40 -07:00
from stevedore import extension
add tests of hardware module
2014-01-07 16:36:59 -08:00
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
from ironic_python_agent import errors
Replacing teeth/overlord with ipa/ironic
2014-03-19 16:19:52 -07:00
from ironic_python_agent import hardware
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
from ironic_python_agent import netutils
Prevent tests' unmocked access to utils.execute() This change introduces a new base test class that mocks out utils.execute and forces an exception if it gets called. This has rooted out many tests that were doing this as a side effect of calling other functions, doing things like modprobe and running iscsi on the host's actual machine. The tests are all now appropriately patched in places where this was happening, and the new base class permanently prevents this from accidentally happening again. If you really want to call utils.execute() then you need to re-mock it in your unit test. Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-04-03 15:32:44 +10:00
from ironic_python_agent.tests.unit import base
Deprecate hardware._cmd in favor of utils.execute This removes the custom function that was in place before our move to Openstack in favor of the utils method integrated with oslo. Change-Id: If668e53ccc97743b00c4beae39d153ee63ed015a
2014-04-10 14:12:16 -07:00
from ironic_python_agent import utils
add tests of hardware module
2014-01-07 16:36:59 -08:00
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
CONF = cfg.CONF
CONF.import_opt('disk_wait_attempts', 'ironic_python_agent.config')
CONF.import_opt('disk_wait_delay', 'ironic_python_agent.config')
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
HDPARM_INFO_TEMPLATE = (
'/dev/sda:\n'
'\n'
'ATA device, with non-removable media\n'
'\tModel Number: 7 PIN SATA FDM\n'
'\tSerial Number: 20131210000000000023\n'
'\tFirmware Revision: SVN406\n'
'\tTransport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, '
'SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0\n'
'Standards: \n'
'\tSupported: 9 8 7 6 5\n'
'\tLikely used: 9\n'
'Configuration: \n'
'\tLogical\t\tmax\tcurrent\n'
'\tcylinders\t16383\t16383\n'
'\theads\t\t16\t16\n'
'\tsectors/track\t63\t63\n'
'\t--\n'
'\tCHS current addressable sectors: 16514064\n'
'\tLBA user addressable sectors: 60579792\n'
'\tLBA48 user addressable sectors: 60579792\n'
'\tLogical Sector size: 512 bytes\n'
'\tPhysical Sector size: 512 bytes\n'
'\tLogical Sector-0 offset: 0 bytes\n'
'\tdevice size with M = 1024*1024: 29579 MBytes\n'
'\tdevice size with M = 1000*1000: 31016 MBytes (31 GB)\n'
'\tcache/buffer size = unknown\n'
'\tForm Factor: 2.5 inch\n'
'\tNominal Media Rotation Rate: Solid State Device\n'
'Capabilities: \n'
'\tLBA, IORDY(can be disabled)\n'
'\tQueue depth: 32\n'
'\tStandby timer values: spec\'d by Standard, no device specific '
'minimum\n'
'\tR/W multiple sector transfer: Max = 1\tCurrent = 1\n'
'\tDMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5\n'
'\t Cycle time: min=120ns recommended=120ns\n'
'\tPIO: pio0 pio1 pio2 pio3 pio4\n'
'\t Cycle time: no flow control=120ns IORDY flow '
'control=120ns\n'
'Commands/features: \n'
'\tEnabled\tSupported:\n'
'\t *\tSMART feature set\n'
'\t \tSecurity Mode feature set\n'
'\t *\tPower Management feature set\n'
'\t *\tWrite cache\n'
'\t *\tLook-ahead\n'
'\t *\tHost Protected Area feature set\n'
'\t *\tWRITE_BUFFER command\n'
'\t *\tREAD_BUFFER command\n'
'\t *\tNOP cmd\n'
'\t \tSET_MAX security extension\n'
'\t *\t48-bit Address feature set\n'
'\t *\tDevice Configuration Overlay feature set\n'
'\t *\tMandatory FLUSH_CACHE\n'
'\t *\tFLUSH_CACHE_EXT\n'
'\t *\tWRITE_{DMA|MULTIPLE}_FUA_EXT\n'
'\t *\tWRITE_UNCORRECTABLE_EXT command\n'
'\t *\tGen1 signaling speed (1.5Gb/s)\n'
'\t *\tGen2 signaling speed (3.0Gb/s)\n'
'\t *\tGen3 signaling speed (6.0Gb/s)\n'
'\t *\tNative Command Queueing (NCQ)\n'
'\t *\tHost-initiated interface power management\n'
'\t *\tPhy event counters\n'
'\t *\tDMA Setup Auto-Activate optimization\n'
'\t \tDevice-initiated interface power management\n'
'\t *\tSoftware settings preservation\n'
'\t \tunknown 78[8]\n'
'\t *\tSMART Command Transport (SCT) feature set\n'
'\t *\tSCT Error Recovery Control (AC3)\n'
'\t *\tSCT Features Control (AC4)\n'
'\t *\tSCT Data Tables (AC5)\n'
'\t *\tData Set Management TRIM supported (limit 2 blocks)\n'
'Security: \n'
'\tMaster password revision code = 65534\n'
'\t%(supported)s\n'
'\t%(enabled)s\n'
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
'\t%(locked)s\n'
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
'\t%(frozen)s\n'
'\tnot\texpired: security count\n'
Added support for secure erase enhanced Added support for secure erase enhanced option when doing an ATA secure erase. Also made it prefer the enhanced version if hdparm claims the device supports it. Change-Id: I2ce04d0b967d6c176acd40ea10a05bcab5dbf2d0
2015-02-05 17:03:33 -06:00
'\t%(enhanced_erase)s\n'
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
'\t24min for SECURITY ERASE UNIT. 24min for ENHANCED SECURITY '
'ERASE UNIT.\n'
'Checksum: correct\n'
Update hacking and fix hacking violations This does a few things: * Update hacking to the version in global-requirements. Old hacking was installing a version of pbr that was breaking other packages. * Fix all the hacking/pep8 rules that updating hacking raised. * Do some general docstring cleanup, while already in there cleaning up a bunch of docstrings due to H405 violations. Change-Id: I1fc1e59d4c3d7b14631f8b576e3f3854bc452188 Closes-Bug: #1461717
2015-06-03 16:56:50 -07:00
) # noqa
# NOTE(jroll) noqa here is to dodge E131 (indent rules). Since this is a
# massive multi-line string (with specific whitespace formatting), it's easier
# for a human to parse it with indentations on line continuations. The other
# option would be to ignore the 79-character limit here. Ew.
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
BLK_DEVICE_TEMPLATE = (
'KNAME="sda" MODEL="TinyUSB Drive" SIZE="3116853504" '
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
'ROTA="0" TYPE="disk" SERIAL="123"\n'
Add support for root device hints This patch add support for root device hints on IPA. Instead of picking the first disk >= 4G, if the hints are specified IPA will look at it to decide which device it should pick for the deployment. The initial patch supports the following hints: Size, model, WWN, serial, vendor. Implements: blueprint ipa-as-default-ramdisk Implements: blueprint root-device-hints Change-Id: I2b00b3fb3b61001033750dd8951f9353d6f2e361
2015-03-10 11:20:40 +00:00
'KNAME="sdb" MODEL="Fastable SD131 7" SIZE="10737418240" '
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
'ROTA="0" TYPE="disk"\n'
'KNAME="sdc" MODEL="NWD-BLP4-1600 " SIZE="1765517033472" '
' ROTA="0" TYPE="disk"\n'
'KNAME="sdd" MODEL="NWD-BLP4-1600 " SIZE="1765517033472" '
' ROTA="0" TYPE="disk"\n'
'KNAME="loop0" MODEL="" SIZE="109109248" ROTA="1" TYPE="loop"'
)
Fix get_os_install_device() Instead of silently failing, raise DeviceNotFound when no root device hints were provided and all found block devices are smaller than 4GB. Change-Id: Idd2e2c5905adf847f00ad15a84a817c3715225dd Closes-Bug: #1490761
2015-09-02 15:21:04 +00:00
# NOTE(pas-ha) largest device is 1 byte smaller than 4GiB
BLK_DEVICE_TEMPLATE_SMALL = (
'KNAME="sda" MODEL="TinyUSB Drive" SIZE="3116853504" '
'ROTA="0" TYPE="disk"\n'
'KNAME="sdb" MODEL="AlmostBigEnough Drive" SIZE="4294967295" '
'ROTA="0" TYPE="disk"'
)
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
BLK_DEVICE_TEMPLATE_SMALL_DEVICES = [
hardware.BlockDevice(name='/dev/sda', model='TinyUSB Drive',
size=3116853504, rotational=False,
vendor="FooTastic"),
hardware.BlockDevice(name='/dev/sdb', model='AlmostBigEnough Drive',
size=4294967295, rotational=False,
vendor="FooTastic"),
]
Fix get_os_install_device() Instead of silently failing, raise DeviceNotFound when no root device hints were provided and all found block devices are smaller than 4GB. Change-Id: Idd2e2c5905adf847f00ad15a84a817c3715225dd Closes-Bug: #1490761
2015-09-02 15:21:04 +00:00
Fix multi-device behavior ATARAID is functionally a version of software RAID where the setup is managed by the controller and the Operating System takes over managing the RAID after boot. Most commonly this is found for mirrored boot devices. Prior to this patch, we were looking for non-dependent items (i.e. base block devices), with a type of disk. Now we will permit the "disk" to be added to the list if lsblk indicates that it is a type containing "raid". The lsblk results should not change as we explicitly look for disk objects. Change-Id: Ia4a03b33cc06ce42e1bc33026683c28b31901cb7 Story: #2003445 Task: #24647
2018-08-16 10:57:59 -07:00
# NOTE(TheJulia): This list intentionally contains duplicates
# as the code filters them out by kernel device name.
RAID_BLK_DEVICE_TEMPLATE = (
'KNAME="sda" MODEL="DRIVE 0" SIZE="1765517033472" '
'ROTA="1" TYPE="disk"\n'
'KNAME="sdb" MODEL="DRIVE 1" SIZE="1765517033472" '
'ROTA="1" TYPE="disk"\n'
'KNAME="sdb" MODEL="DRIVE 1" SIZE="1765517033472" '
'ROTA="1" TYPE="disk"\n'
'KNAME="md0" MODEL="RAID" SIZE="1765517033470" '
'ROTA="0" TYPE="raid1"\n'
'KNAME="md0" MODEL="RAID" SIZE="1765517033470" '
'ROTA="0" TYPE="raid1"'
)
RAID_BLK_DEVICE_TEMPLATE_DEVICES = [
hardware.BlockDevice(name='/dev/sda', model='DRIVE 0',
size=1765517033472, rotational=True,
vendor="FooTastic"),
hardware.BlockDevice(name='/dev/sdb', model='DRIVE 1',
size=1765517033472, rotational=True,
vendor="FooTastic"),
hardware.BlockDevice(name='/dev/md0', model='RAID',
size=1765517033470, rotational=False,
vendor="FooTastic"),
]
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
SHRED_OUTPUT_0_ITERATIONS_ZERO_FALSE = ()
SHRED_OUTPUT_1_ITERATION_ZERO_TRUE = (
Enforce all flake8 rules except E129 Bring ironic-python-agent in line with the other ironic projects. Stop ignoring all E12* errors except E129 Stop ignoring E711 Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
'shred: /dev/sda: pass 1/2 (random)...\n'
'shred: /dev/sda: pass 1/2 (random)...4.9GiB/29GiB 17%\n'
'shred: /dev/sda: pass 1/2 (random)...15GiB/29GiB 51%\n'
'shred: /dev/sda: pass 1/2 (random)...20GiB/29GiB 69%\n'
'shred: /dev/sda: pass 1/2 (random)...29GiB/29GiB 100%\n'
'shred: /dev/sda: pass 2/2 (000000)...\n'
'shred: /dev/sda: pass 2/2 (000000)...4.9GiB/29GiB 17%\n'
'shred: /dev/sda: pass 2/2 (000000)...15GiB/29GiB 51%\n'
'shred: /dev/sda: pass 2/2 (000000)...20GiB/29GiB 69%\n'
'shred: /dev/sda: pass 2/2 (000000)...29GiB/29GiB 100%\n'
Add fallback secure erase using shred This prevents devices from failing cleaning if they don't support ATA security erase. Instead, the device will be erased by overwriting using GNU shred. Change-Id: Ie4f88b97c8c34efe2397538662622abd0b963aea
2015-03-17 16:26:06 -07:00
)
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
SHRED_OUTPUT_2_ITERATIONS_ZERO_FALSE = (
'shred: /dev/sda: pass 1/2 (random)...\n'
'shred: /dev/sda: pass 1/2 (random)...4.9GiB/29GiB 17%\n'
'shred: /dev/sda: pass 1/2 (random)...15GiB/29GiB 51%\n'
'shred: /dev/sda: pass 1/2 (random)...20GiB/29GiB 69%\n'
'shred: /dev/sda: pass 1/2 (random)...29GiB/29GiB 100%\n'
'shred: /dev/sda: pass 2/2 (random)...\n'
'shred: /dev/sda: pass 2/2 (random)...4.9GiB/29GiB 17%\n'
'shred: /dev/sda: pass 2/2 (random)...15GiB/29GiB 51%\n'
'shred: /dev/sda: pass 2/2 (random)...20GiB/29GiB 69%\n'
'shred: /dev/sda: pass 2/2 (random)...29GiB/29GiB 100%\n'
)
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
LSCPU_OUTPUT = """
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 45
Model name: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz
Stepping: 7
CPU MHz: 1290.000
CPU max MHz: 2400.0000
CPU min MHz: 1200.0000
BogoMIPS: 4800.06
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 10240K
NUMA node0 CPU(s): 0-3
"""
LSCPU_OUTPUT_NO_MAX_MHZ = """
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 12
On-line CPU(s) list: 0-11
Thread(s) per core: 2
Core(s) per socket: 6
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Stepping: 2
CPU MHz: 1794.433
BogoMIPS: 6983.57
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 15360K
NUMA node0 CPU(s): 0-11
"""
Returns CPU flags in the CPU inventory These flags will be processed in a new ironic-inspector plugin to support setting capabilities like cpu_vt (virtualization enabled). Change-Id: I5fe9310c316841eabdd2d5e2ef2ae30afa03d29a Partial-Bug: #1571580
2016-04-20 13:58:14 +02:00
# NOTE(dtanstur): flags list stripped down for sanity reasons
CPUINFO_FLAGS_OUTPUT = """
flags : fpu vme de pse
"""
Ironic python agent does not extract correct available memory This patch fixes parsing of memory class output of lshw output. Task: 30201 Story: 2005308 Change-Id: I0e7ff9f86a266d2f798457136fed0100c78569e2
2019-03-27 22:38:59 +11:00
LSHW_JSON_OUTPUT_V1 = ("""
Use lshw in place of dmidecode for the default hardware manager Currently the generic hardware manager uses dmidecode to get the total physical memory and system details. This patch switches the generic hardware manager to use lshw, as it is capable of reading more than DMI [0]. This enables systems that do not support DMI to use the generic hardware manager, such as IBM Power systems. [0] https://github.com/lyonel/lshw/blob/master/README.md Closes-Bug: #1715790 Change-Id: Ie370331df6bb5ef131c5cb60f458877e2a7ad71a Depends-On: Idaf05b8efce28cd0cbf339cf693db4f55a693d9b
2017-11-30 16:30:42 +00:00
{
"id": "fuzzypickles",
"product": "ABC123 (GENERIC_SERVER)",
"vendor": "GENERIC",
"serial": "1234567",
"width": 64,
"capabilities": {
"smbios-2.7": "SMBIOS version 2.7",
"dmi-2.7": "DMI version 2.7",
"vsyscall32": "32-bit processes"
},
"children": [
{
"id": "core",
"description": "Motherboard",
"product": "ABC123",
"vendor": "GENERIC",
"serial": "ABCDEFGHIJK",
"children": [
{
"id": "memory",
"class": "memory",
"description": "System Memory",
"units": "bytes",
"size": 4294967296,
"children": [
{
"id": "bank:0",
"class": "memory",
"physid": "0",
"units": "bytes",
"size": 2147483648,
"width": 64,
"clock": 1600000000
},
{
"id": "bank:1",
"class": "memory",
"physid": "1"
},
{
"id": "bank:2",
"class": "memory",
"physid": "2",
"units": "bytes",
"size": 1073741824,
"width": 64,
"clock": 1600000000
},
{
"id": "bank:3",
"class": "memory",
"physid": "3",
"units": "bytes",
"size": 1073741824,
"width": 64,
"clock": 1600000000
}
]
},
{
"id": "cpu:0",
"class": "processor",
"claimed": true,
"product": "Intel Xeon E312xx (Sandy Bridge)",
"vendor": "Intel Corp.",
"physid": "1",
"businfo": "cpu@0",
"width": 64,
"capabilities": {
"fpu": "mathematical co-processor",
"fpu_exception": "FPU exceptions reporting",
"wp": true,
"mmx": "multimedia extensions (MMX)"
}
}
]
},
{
"id": "network:0",
"class": "network",
"claimed": true,
"description": "Ethernet interface",
"physid": "1",
"logicalname": "ovs-tap",
"serial": "1c:90:c0:f9:4e:a1",
"units": "bit/s",
"size": 10000000000,
"configuration": {
"autonegotiation": "off",
"broadcast": "yes",
"driver": "veth",
"driverversion": "1.0",
"duplex": "full",
"link": "yes",
"multicast": "yes",
"port": "twisted pair",
"speed": "10Gbit/s"
},
"capabilities": {
"ethernet": true,
"physical": "Physical interface"
}
}
]
}
Remove support for older psutil versions Global requirements was recently updated to force psutil=>3.0.1. This patch removes support for older versions of psutil as well as changing to opportunistically attempt to work if a version >5 is released but doesn't change the interface we use. Change-Id: I1f7fab33fd275fb8b5cd7704dc13375402756d06 Related-bug: #1659137
2017-01-25 08:52:46 -08:00
""", "")
Ironic python agent does not extract correct available memory This patch fixes parsing of memory class output of lshw output. Task: 30201 Story: 2005308 Change-Id: I0e7ff9f86a266d2f798457136fed0100c78569e2
2019-03-27 22:38:59 +11:00
LSHW_JSON_OUTPUT_V2 = ("""
{
"id" : "bumblebee",
"class" : "system",
"claimed" : true,
"handle" : "DMI:0001",
"description" : "Rack Mount Chassis",
"product" : "ABCD",
"vendor" : "ABCD",
"version" : "1234",
"serial" : "1234",
"width" : 64,
"configuration" : {
"boot" : "normal",
"chassis" : "rackmount",
"family" : "Intel Grantley EP",
"sku" : "NULL",
"uuid" : "00010002-0003-0004-0005-000600070008"
},
"capabilities" : {
"smbios-2.8" : "SMBIOS version 2.8",
"dmi-2.7" : "DMI version 2.7",
"vsyscall32" : "32-bit processes"
},
"children" : [
{
"id" : "core",
"class" : "bus",
"claimed" : true,
"handle" : "DMI:0002",
"description" : "Motherboard",
"product" : "ABCD",
"vendor" : "ABCD",
"physid" : "0",
"version" : "1234",
"serial" : "1234",
"slot" : "NULL",
"children" : [
{
"id" : "memory:0",
"class" : "memory",
"claimed" : true,
"handle" : "DMI:004A",
"description" : "System Memory",
"physid" : "4a",
"slot" : "System board or motherboard",
"children" : [
{
"id" : "bank:0",
"class" : "memory",
"claimed" : true,
"handle" : "DMI:004C",
"description" : "DIMM Synchronous 2133 MHz (0.5 ns)",
"product" : "36ASF2G72PZ-2G1A2",
"vendor" : "Micron",
"physid" : "0",
"serial" : "101B6543",
"slot" : "DIMM_A0",
"units" : "bytes",
"size" : 17179869184,
"width" : 64,
"clock" : 2133000000
},
{
"id" : "bank:1",
"class" : "memory",
"claimed" : true,
"handle" : "DMI:004E",
"description" : "DIMM Synchronous [empty]",
"product" : "NO DIMM",
"vendor" : "NO DIMM",
"physid" : "1",
"serial" : "NO DIMM",
"slot" : "DIMM_A1"
},
{
"id" : "bank:2",
"class" : "memory",
"claimed" : true,
"handle" : "DMI:004F",
"description" : "DIMM Synchronous 2133 MHz (0.5 ns)",
"product" : "36ASF2G72PZ-2G1A2",
"vendor" : "Micron",
"physid" : "2",
"serial" : "101B654E",
"slot" : "DIMM_A2",
"units" : "bytes",
"size" : 17179869184,
"width" : 64,
"clock" : 2133000000
},
{
"id" : "bank:3",
"class" : "memory",
"claimed" : true,
"handle" : "DMI:0051",
"description" : "DIMM Synchronous [empty]",
"product" : "NO DIMM",
"vendor" : "NO DIMM",
"physid" : "3",
"serial" : "NO DIMM",
"slot" : "DIMM_A3"
}
]
},
{
"id" : "memory:1",
"class" : "memory",
"claimed" : true,
"handle" : "DMI:0052",
"description" : "System Memory",
"physid" : "52",
"slot" : "System board or motherboard",
"children" : [
{
"id" : "bank:0",
"class" : "memory",
"claimed" : true,
"handle" : "DMI:0054",
"description" : "DIMM Synchronous 2133 MHz (0.5 ns)",
"product" : "36ASF2G72PZ-2G1A2",
"vendor" : "Micron",
"physid" : "0",
"serial" : "101B6545",
"slot" : "DIMM_A4",
"units" : "bytes",
"size" : 17179869184,
"width" : 64,
"clock" : 2133000000
},
{
"id" : "bank:1",
"class" : "memory",
"claimed" : true,
"handle" : "DMI:0056",
"description" : "DIMM Synchronous [empty]",
"product" : "NO DIMM",
"vendor" : "NO DIMM",
"physid" : "1",
"serial" : "NO DIMM",
"slot" : "DIMM_A5"
},
{
"id" : "bank:2",
"class" : "memory",
"claimed" : true,
"handle" : "DMI:0057",
"description" : "DIMM Synchronous 2133 MHz (0.5 ns)",
"product" : "36ASF2G72PZ-2G1A2",
"vendor" : "Micron",
"physid" : "2",
"serial" : "101B6540",
"slot" : "DIMM_A6",
"units" : "bytes",
"size" : 17179869184,
"width" : 64,
"clock" : 2133000000
},
{
"id" : "bank:3",
"class" : "memory",
"claimed" : true,
"handle" : "DMI:0059",
"description" : "DIMM Synchronous [empty]",
"product" : "NO DIMM",
"vendor" : "NO DIMM",
"physid" : "3",
"serial" : "NO DIMM",
"slot" : "DIMM_A7"
}
]
},
{
"id" : "memory:4",
"class" : "memory",
"physid" : "1"
},
{
"id" : "memory:5",
"class" : "memory",
"physid" : "2"
}
]
}
]
}
""", "")
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
SMARTCTL_NORMAL_OUTPUT = ("""
smartctl 6.2 2017-02-27 r4394 [x86_64-linux-3.10.0-693.21.1.el7.x86_64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
ATA Security is: Disabled, NOT FROZEN [SEC1]
""") # noqa
SMARTCTL_UNAVAILABLE_OUTPUT = ("""
smartctl 6.2 2017-02-27 r4394 [x86_64-linux-3.10.0-693.21.1.el7.x86_64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
ATA Security is: Unavailable
""") # noqa
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
Discover IPv6 BMC address Change-Id: Ia7f24c1df782de4809a5195876bbf0309b481fd7 Story: #2005133 Task: #29817
2019-03-04 15:31:14 +03:00
IPMITOOL_LAN6_PRINT_DYNAMIC_ADDR = """
IPv6 Dynamic Address 0:
Source/Type: DHCPv6
Address: 2001:1234:1234:1234:1234:1234:1234:1234/64
Status: active
IPv6 Dynamic Address 1:
Source/Type: DHCPv6
Address: ::/0
Status: active
IPv6 Dynamic Address 2:
Source/Type: DHCPv6
Address: ::/0
Status: active
"""
IPMITOOL_LAN6_PRINT_STATIC_ADDR = """
IPv6 Static Address 0:
Enabled: yes
Address: 2001:5678:5678:5678:5678:5678:5678:5678/64
Status: active
IPv6 Static Address 1:
Enabled: no
Address: ::/0
Status: disabled
IPv6 Static Address 2:
Enabled: no
Address: ::/0
Status: disabled
"""
Software RAID: Detect component & holder disks in broken RAIDs This patch amends the component and holder disk detection code to be able to also extract information from inactive/broken md devices. This may be required in case of broken RAID-0 (which will show up as an empty inactive device after a reboot which none the less keeps its spare devices 'in-use' and prevents RAID re-creation). Story: #2006215 Task: #35808 Change-Id: I4ce41b335843e18c67668192d2fccff1c6cdfb92
2019-07-12 14:50:14 +02:00
MDADM_DETAIL_OUTPUT = ("""/dev/md0:
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
Version : 1.0
Creation Time : Fri Feb 15 12:37:44 2019
Raid Level : raid1
Array Size : 1048512 (1023.94 MiB 1073.68 MB)
Used Dev Size : 1048512 (1023.94 MiB 1073.68 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Fri Feb 15 12:38:02 2019
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Consistency Policy : resync
Name : abc.xyz.com:0 (local to host abc.xyz.com)
UUID : 83143055:2781ddf5:2c8f44c7:9b45d92e
Events : 17
Number Major Minor RaidDevice State
0 253 64 0 active sync /dev/vde1
1 253 80 1 active sync /dev/vdf1
""")
Discover IPv6 BMC address Change-Id: Ia7f24c1df782de4809a5195876bbf0309b481fd7 Story: #2005133 Task: #29817
2019-03-04 15:31:14 +03:00
Software RAID: Detect component & holder disks in broken RAIDs This patch amends the component and holder disk detection code to be able to also extract information from inactive/broken md devices. This may be required in case of broken RAID-0 (which will show up as an empty inactive device after a reboot which none the less keeps its spare devices 'in-use' and prevents RAID re-creation). Story: #2006215 Task: #35808 Change-Id: I4ce41b335843e18c67668192d2fccff1c6cdfb92
2019-07-12 14:50:14 +02:00
MDADM_DETAIL_OUTPUT_BROKEN_RAID0 = ("""/dev/md126:
Version : 1.2
Raid Level : raid0
Total Devices : 1
Persistence : Superblock is persistent
State : inactive
Working Devices : 1
Name : prj6ogxgyzd:1
UUID : b5e136c0:a7e379b7:db25e45d:4b63928b
Events : 0
Number Major Minor RaidDevice
- 8 2 - /dev/sda2
""")
Fix HardwareManager loading order bug The agent attempts to sort the HardwareManager implementations found by stevedore in order of suitability. This patch fixes a bug in the sorting order that caused the GenericHardwareManager to always be loaded. Fixes bug 1327923 Change-Id: I1cc20e5351a704325f0cf41dab205c9164799f6a
2014-06-08 18:45:40 -07:00
class FakeHardwareManager(hardware.GenericHardwareManager):
def __init__(self, hardware_support):
self._hardware_support = hardware_support
def evaluate_hardware_support(self):
return self._hardware_support
Prevent tests' unmocked access to utils.execute() This change introduces a new base test class that mocks out utils.execute and forces an exception if it gets called. This has rooted out many tests that were doing this as a side effect of calling other functions, doing things like modprobe and running iscsi on the host's actual machine. The tests are all now appropriately patched in places where this was happening, and the new base class permanently prevents this from accidentally happening again. If you really want to call utils.execute() then you need to re-mock it in your unit test. Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-04-03 15:32:44 +10:00
class TestHardwareManagerLoading(base.IronicAgentTest):
Fix HardwareManager loading order bug The agent attempts to sort the HardwareManager implementations found by stevedore in order of suitability. This patch fixes a bug in the sorting order that caused the GenericHardwareManager to always be loaded. Fixes bug 1327923 Change-Id: I1cc20e5351a704325f0cf41dab205c9164799f6a
2014-06-08 18:45:40 -07:00
def setUp(self):
super(TestHardwareManagerLoading, self).setUp()
# In order to use ExtensionManager.make_test_instance() without
# creating a new only-for-test codepath, we instantiate the test
# instance outside of the test case in setUp, where we can access
# make_test_instance() before it gets mocked. Inside of the test case
# we set this as the return value of the mocked constructor, so we can
# verify that the constructor is called correctly while still using a
# more realistic ExtensionManager
fake_ep = mock.Mock()
fake_ep.module_name = 'fake'
fake_ep.attrs = ['fake attrs']
Enforce all flake8 rules except E129 Bring ironic-python-agent in line with the other ironic projects. Stop ignoring all E12* errors except E129 Stop ignoring E711 Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
ext1 = extension.Extension(
'fake_generic0', fake_ep, None,
Fix HardwareManager loading order bug The agent attempts to sort the HardwareManager implementations found by stevedore in order of suitability. This patch fixes a bug in the sorting order that caused the GenericHardwareManager to always be loaded. Fixes bug 1327923 Change-Id: I1cc20e5351a704325f0cf41dab205c9164799f6a
2014-06-08 18:45:40 -07:00
FakeHardwareManager(hardware.HardwareSupport.GENERIC))
Enforce all flake8 rules except E129 Bring ironic-python-agent in line with the other ironic projects. Stop ignoring all E12* errors except E129 Stop ignoring E711 Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
ext2 = extension.Extension(
'fake_mainline0', fake_ep, None,
Fix HardwareManager loading order bug The agent attempts to sort the HardwareManager implementations found by stevedore in order of suitability. This patch fixes a bug in the sorting order that caused the GenericHardwareManager to always be loaded. Fixes bug 1327923 Change-Id: I1cc20e5351a704325f0cf41dab205c9164799f6a
2014-06-08 18:45:40 -07:00
FakeHardwareManager(hardware.HardwareSupport.MAINLINE))
Enforce all flake8 rules except E129 Bring ironic-python-agent in line with the other ironic projects. Stop ignoring all E12* errors except E129 Stop ignoring E711 Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
ext3 = extension.Extension(
'fake_generic1', fake_ep, None,
Fix HardwareManager loading order bug The agent attempts to sort the HardwareManager implementations found by stevedore in order of suitability. This patch fixes a bug in the sorting order that caused the GenericHardwareManager to always be loaded. Fixes bug 1327923 Change-Id: I1cc20e5351a704325f0cf41dab205c9164799f6a
2014-06-08 18:45:40 -07:00
FakeHardwareManager(hardware.HardwareSupport.GENERIC))
self.correct_hw_manager = ext2.obj
self.fake_ext_mgr = extension.ExtensionManager.make_test_instance([
ext1, ext2, ext3
])
Wait for udev to settle before listing the block devices This patch is making the list_all_block_devices() method to wait for udev to settle it's event queue prior to listing the devices. Sometimes the ironic-python-agent service may start before all devices were detected and end up erroring out because it couldn't find a suitable disk for deployment. Closes-Bug: #1551300 Change-Id: I1ae2062a711115a1ea14b79ae9ace7ddd2fff9d5
2016-02-25 16:32:47 +00:00
@mock.patch.object(hardware, '_udev_settle', lambda *_: None)
Prevent tests' unmocked access to utils.execute() This change introduces a new base test class that mocks out utils.execute and forces an exception if it gets called. This has rooted out many tests that were doing this as a side effect of calling other functions, doing things like modprobe and running iscsi on the host's actual machine. The tests are all now appropriately patched in places where this was happening, and the new base class permanently prevents this from accidentally happening again. If you really want to call utils.execute() then you need to re-mock it in your unit test. Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-04-03 15:32:44 +10:00
class TestGenericHardwareManager(base.IronicAgentTest):
add tests of hardware module
2014-01-07 16:36:59 -08:00
def setUp(self):
Adding oslotest to improve debugging oslotest includes things like showing log messages/stdout on test errors, which makes debugging much easier. Also moved mock to second group in imports, as it's a 3rd party library. Change-Id: I016ae0a376d42dec28085687ea7194df4cd44eda
2014-04-03 14:02:53 -07:00
super(TestGenericHardwareManager, self).setUp()
simplify HardwareManager comparison
2014-01-22 11:09:02 -08:00
self.hardware = hardware.GenericHardwareManager()
IPA:'shred' utility to use configured iterations Today, there is no option to configure number of iterations to be done for shred block device erasing and defaults it to 1. This patch adds a configuration option to change the number of passes to be done to erase a block device. Change-Id: I1921d33a6b364c4682b6c9baaf61ac092cfa11d7 Partial-Bug:#1465130
2015-06-15 21:36:27 +05:30
self.node = {'uuid': 'dda135fb-732d-4742-8e72-df8f3199d244',
'driver_internal_info': {}}
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
CONF.clear_override('disk_wait_attempts')
CONF.clear_override('disk_wait_delay')
add tests of hardware module
2014-01-07 16:36:59 -08:00
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
def test_get_clean_steps(self):
expected_clean_steps = [
{
'step': 'erase_devices',
'priority': 10,
'interface': 'deploy',
'reboot_requested': False,
'abortable': True
},
{
'step': 'erase_devices_metadata',
'priority': 99,
'interface': 'deploy',
'reboot_requested': False,
'abortable': True
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
},
{
'step': 'delete_configuration',
'priority': 0,
'interface': 'raid',
'reboot_requested': False,
'abortable': True
},
{
'step': 'create_configuration',
'priority': 0,
'interface': 'raid',
'reboot_requested': False,
'abortable': True
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
}
]
clean_steps = self.hardware.get_clean_steps(self.node, [])
self.assertEqual(expected_clean_steps, clean_steps)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch('binascii.hexlify', autospec=True)
@mock.patch('ironic_python_agent.netutils.get_lldp_info', autospec=True)
Add a new Hardware Manager for CNA network card This patch adds a new hardware manager, which will disable the embedded LLDP agent on Intel CNA network cards in order to allow the gathering of LLDP data during the inspection process. Change-Id: I572756ac6a7bf67a7f446738ba9d145e1c1bdc48 Closes-Bug: #1623659
2016-11-23 17:14:15 +01:00
def test_collect_lldp_data(self, mock_lldp_info, mock_hexlify):
if_names = ['eth0', 'lo']
mock_lldp_info.return_value = {if_names[0]: [
(0, b''),
(1, b'foo\x01'),
(2, b'\x02bar')],
}
mock_hexlify.side_effect = [
b'',
b'666f6f01',
b'02626172'
]
expected_lldp_data = {
'eth0': [
(0, ''),
(1, '666f6f01'),
(2, '02626172')],
}
result = self.hardware.collect_lldp_data(if_names)
Replace assertEqual(True/False, expr) with assertTrue/assertFalse in tests In some cases, If the result of expr is a boolen value, we shoud use assertTrue/assertFalse to instead. Beacause it is clear and simple. Change-Id: Ie61369f6335a90b09bb24192282d33da5272c13f Story: #2003785 Task: #26490
2018-09-17 11:10:40 +08:00
self.assertIn(if_names[0], result)
Add a new Hardware Manager for CNA network card This patch adds a new hardware manager, which will disable the embedded LLDP agent on Intel CNA network cards in order to allow the gathering of LLDP data during the inspection process. Change-Id: I572756ac6a7bf67a7f446738ba9d145e1c1bdc48 Closes-Bug: #1623659
2016-11-23 17:14:15 +01:00
self.assertEqual(expected_lldp_data, result)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch('ironic_python_agent.netutils.get_lldp_info', autospec=True)
Add a new Hardware Manager for CNA network card This patch adds a new hardware manager, which will disable the embedded LLDP agent on Intel CNA network cards in order to allow the gathering of LLDP data during the inspection process. Change-Id: I572756ac6a7bf67a7f446738ba9d145e1c1bdc48 Closes-Bug: #1623659
2016-11-23 17:14:15 +01:00
def test_collect_lldp_data_netutils_exception(self, mock_lldp_info):
if_names = ['eth0', 'lo']
mock_lldp_info.side_effect = Exception('fake error')
result = self.hardware.collect_lldp_data(if_names)
expected_lldp_data = {}
self.assertEqual(expected_lldp_data, result)
@mock.patch.object(hardware, 'LOG', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch('binascii.hexlify', autospec=True)
@mock.patch('ironic_python_agent.netutils.get_lldp_info', autospec=True)
Add a new Hardware Manager for CNA network card This patch adds a new hardware manager, which will disable the embedded LLDP agent on Intel CNA network cards in order to allow the gathering of LLDP data during the inspection process. Change-Id: I572756ac6a7bf67a7f446738ba9d145e1c1bdc48 Closes-Bug: #1623659
2016-11-23 17:14:15 +01:00
def test_collect_lldp_data_decode_exception(self, mock_lldp_info,
mock_hexlify, mock_log):
if_names = ['eth0', 'lo']
mock_lldp_info.return_value = {if_names[0]: [
(0, b''),
(1, b'foo\x01'),
(2, b'\x02bar')],
}
mock_hexlify.side_effect = [
b'',
b'666f6f01',
binascii.Error('fake_error')
]
expected_lldp_data = {
'eth0': [
(0, ''),
(1, '666f6f01')],
}
result = self.hardware.collect_lldp_data(if_names)
mock_log.warning.assert_called_once()
Replace assertEqual(True/False, expr) with assertTrue/assertFalse in tests In some cases, If the result of expr is a boolen value, we shoud use assertTrue/assertFalse to instead. Beacause it is clear and simple. Change-Id: Ie61369f6335a90b09bb24192282d33da5272c13f Story: #2003785 Task: #26490
2018-09-17 11:10:40 +08:00
self.assertIn(if_names[0], result)
Add a new Hardware Manager for CNA network card This patch adds a new hardware manager, which will disable the embedded LLDP agent on Intel CNA network cards in order to allow the gathering of LLDP data during the inspection process. Change-Id: I572756ac6a7bf67a7f446738ba9d145e1c1bdc48 Closes-Bug: #1623659
2016-11-23 17:14:15 +01:00
self.assertEqual(expected_lldp_data, result)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch('ironic_python_agent.hardware._get_managers', autospec=True)
@mock.patch('netifaces.ifaddresses', autospec=True)
@mock.patch('os.listdir', autospec=True)
@mock.patch('os.path.exists', autospec=True)
@mock.patch('six.moves.builtins.open', autospec=True)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
@mock.patch.object(utils, 'execute', autospec=True)
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
@mock.patch.object(netutils, 'get_mac_addr', autospec=True)
@mock.patch.object(netutils, 'interface_has_carrier', autospec=True)
update list_network_interfaces tests
2014-01-27 18:11:44 -08:00
def test_list_network_interfaces(self,
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_has_carrier,
mock_get_mac,
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_execute,
update list_network_interfaces tests
2014-01-27 18:11:44 -08:00
mocked_open,
mocked_exists,
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
mocked_listdir,
Dispatched out network interface info to all hardware managers This patch dispatches out the network_interface_info to allow vendor hardware managers to plug the spacific implementation. It also move neworking releated methods form hardware to netutils Related-Bug: #1532534 Change-Id: Idcd25c4753c009b5ba70bea97ee4eb83391a77a9
2016-11-17 02:08:13 +02:00
mocked_ifaddresses,
mocked_get_managers):
mocked_get_managers.return_value = [hardware.GenericHardwareManager()]
add tests of listing network interfaces
2014-01-27 17:51:06 -08:00
mocked_listdir.return_value = ['lo', 'eth0']
update list_network_interfaces tests
2014-01-27 18:11:44 -08:00
mocked_exists.side_effect = [False, True]
Close file handle after use The file handle for addr_info was left open, meaning we were leaking file handles when sending system information. This ensures the handle is closed. Change-Id: I4b117d4f9ec32ad3f16f43632ffe9dc96da5ab7f
2014-04-09 17:26:54 -07:00
mocked_open.return_value.__enter__ = lambda s: s
mocked_open.return_value.__exit__ = mock.Mock()
read_mock = mocked_open.return_value.read
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
read_mock.side_effect = ['1']
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
mocked_ifaddresses.return_value = {
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
netifaces.AF_INET: [{'addr': '192.168.1.2'}],
netifaces.AF_INET6: [{'addr': 'fd00::101'}]
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
}
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_execute.return_value = ('em0\n', '')
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_get_mac.mock_has_carrier = True
mock_get_mac.return_value = '00:0c:29:8c:11:b1'
add tests of listing network interfaces
2014-01-27 17:51:06 -08:00
interfaces = self.hardware.list_network_interfaces()
Fix params order in assertEqual Fix params order to correspond to real signature: assertEqual(expected, actual) Change-Id: I129aad5d8bb7ca0541c07913cb5edd8eef9e90ba Closes-Bug: #1277104
2016-01-12 09:03:19 +00:00
self.assertEqual(1, len(interfaces))
self.assertEqual('eth0', interfaces[0].name)
self.assertEqual('00:0c:29:8c:11:b1', interfaces[0].mac_address)
self.assertEqual('192.168.1.2', interfaces[0].ipv4_address)
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
self.assertEqual('fd00::101', interfaces[0].ipv6_address)
Using assertIsNone() is preferred over assertEqual() Following OpenStack Style Guidelines[1]: http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises [H203] Unit test assertions tend to give better messages for more specific assertions. As a result, assertIsNone(...) is preferred over assertEqual(None, ...) Change-Id: I53c6d6d3e53d9fac38c50faf065c131d3aa55224
2016-08-29 10:07:14 +07:00
self.assertIsNone(interfaces[0].lldp)
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
self.assertTrue(interfaces[0].has_carrier)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
self.assertEqual('em0', interfaces[0].biosdevname)
@mock.patch('ironic_python_agent.hardware._get_managers', autospec=True)
@mock.patch('netifaces.ifaddresses', autospec=True)
@mock.patch('os.listdir', autospec=True)
@mock.patch('os.path.exists', autospec=True)
@mock.patch('six.moves.builtins.open', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
@mock.patch.object(netutils, 'get_mac_addr', autospec=True)
@mock.patch.object(netutils, 'interface_has_carrier', autospec=True)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
def test_list_network_interfaces_with_biosdevname(self,
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_has_carrier,
mock_get_mac,
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_execute,
mocked_open,
mocked_exists,
mocked_listdir,
mocked_ifaddresses,
mocked_get_managers):
mocked_get_managers.return_value = [hardware.GenericHardwareManager()]
mocked_listdir.return_value = ['lo', 'eth0']
mocked_exists.side_effect = [False, True]
mocked_open.return_value.__enter__ = lambda s: s
mocked_open.return_value.__exit__ = mock.Mock()
read_mock = mocked_open.return_value.read
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
read_mock.side_effect = ['1']
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_ifaddresses.return_value = {
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
netifaces.AF_INET: [{'addr': '192.168.1.2'}],
netifaces.AF_INET6: [{'addr': 'fd00::101'}]
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
}
mocked_execute.return_value = ('em0\n', '')
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_get_mac.return_value = '00:0c:29:8c:11:b1'
mock_has_carrier.return_value = True
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
interfaces = self.hardware.list_network_interfaces()
self.assertEqual(1, len(interfaces))
self.assertEqual('eth0', interfaces[0].name)
self.assertEqual('00:0c:29:8c:11:b1', interfaces[0].mac_address)
self.assertEqual('192.168.1.2', interfaces[0].ipv4_address)
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
self.assertEqual('fd00::101', interfaces[0].ipv6_address)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
self.assertIsNone(interfaces[0].lldp)
self.assertTrue(interfaces[0].has_carrier)
self.assertEqual('em0', interfaces[0].biosdevname)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bios_given_nic_name_ok(self, mock_execute):
interface_name = 'eth0'
mock_execute.return_value = ('em0\n', '')
result = self.hardware.get_bios_given_nic_name(interface_name)
self.assertEqual('em0', result)
mock_execute.assert_called_once_with('biosdevname', '-i',
interface_name)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bios_given_nic_name_oserror(self, mock_execute):
interface_name = 'eth0'
mock_execute.side_effect = OSError()
result = self.hardware.get_bios_given_nic_name(interface_name)
self.assertIsNone(result)
mock_execute.assert_called_once_with('biosdevname', '-i',
interface_name)
@mock.patch.object(utils, 'execute', autospec=True)
@mock.patch.object(hardware, 'LOG', autospec=True)
def test_get_bios_given_nic_name_process_exec_err4(self, mock_log,
mock_execute):
interface_name = 'eth0'
mock_execute.side_effect = [
processutils.ProcessExecutionError(exit_code=4)]
result = self.hardware.get_bios_given_nic_name(interface_name)
mock_log.info.assert_called_once_with(
'The system is a virtual machine, so biosdevname utility does '
'not provide names for virtual NICs.')
self.assertIsNone(result)
mock_execute.assert_called_once_with('biosdevname', '-i',
interface_name)
@mock.patch.object(utils, 'execute', autospec=True)
@mock.patch.object(hardware, 'LOG', autospec=True)
def test_get_bios_given_nic_name_process_exec_err3(self, mock_log,
mock_execute):
interface_name = 'eth0'
mock_execute.side_effect = [
processutils.ProcessExecutionError(exit_code=3)]
result = self.hardware.get_bios_given_nic_name(interface_name)
mock_log.warning.assert_called_once_with(
'Biosdevname returned exit code %s', 3)
self.assertIsNone(result)
mock_execute.assert_called_once_with('biosdevname', '-i',
interface_name)
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch('ironic_python_agent.hardware._get_managers', autospec=True)
@mock.patch('ironic_python_agent.netutils.get_lldp_info', autospec=True)
@mock.patch('netifaces.ifaddresses', autospec=True)
@mock.patch('os.listdir', autospec=True)
@mock.patch('os.path.exists', autospec=True)
@mock.patch('six.moves.builtins.open', autospec=True)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
@mock.patch.object(utils, 'execute', autospec=True)
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
@mock.patch.object(netutils, 'get_mac_addr', autospec=True)
@mock.patch.object(netutils, 'interface_has_carrier', autospec=True)
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
def test_list_network_interfaces_with_lldp(self,
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_has_carrier,
mock_get_mac,
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_execute,
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
mocked_open,
mocked_exists,
mocked_listdir,
mocked_ifaddresses,
Dispatched out network interface info to all hardware managers This patch dispatches out the network_interface_info to allow vendor hardware managers to plug the spacific implementation. It also move neworking releated methods form hardware to netutils Related-Bug: #1532534 Change-Id: Idcd25c4753c009b5ba70bea97ee4eb83391a77a9
2016-11-17 02:08:13 +02:00
mocked_lldp_info,
mocked_get_managers):
mocked_get_managers.return_value = [hardware.GenericHardwareManager()]
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
CONF.set_override('collect_lldp', True)
mocked_listdir.return_value = ['lo', 'eth0']
mocked_exists.side_effect = [False, True]
mocked_open.return_value.__enter__ = lambda s: s
mocked_open.return_value.__exit__ = mock.Mock()
read_mock = mocked_open.return_value.read
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
read_mock.side_effect = ['1']
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
mocked_ifaddresses.return_value = {
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
netifaces.AF_INET: [{'addr': '192.168.1.2'}],
netifaces.AF_INET6: [{'addr': 'fd00::101'}]
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
}
mocked_lldp_info.return_value = {'eth0': [
(0, b''),
(1, b'\x04\x88Z\x92\xecTY'),
(2, b'\x05Ethernet1/18'),
(3, b'\x00x')]
}
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_has_carrier.return_value = True
mock_get_mac.return_value = '00:0c:29:8c:11:b1'
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_execute.return_value = ('em0\n', '')
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
interfaces = self.hardware.list_network_interfaces()
self.assertEqual(1, len(interfaces))
self.assertEqual('eth0', interfaces[0].name)
self.assertEqual('00:0c:29:8c:11:b1', interfaces[0].mac_address)
self.assertEqual('192.168.1.2', interfaces[0].ipv4_address)
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
self.assertEqual('fd00::101', interfaces[0].ipv6_address)
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
expected_lldp_info = [
(0, ''),
(1, '04885a92ec5459'),
(2, '0545746865726e6574312f3138'),
(3, '0078'),
]
self.assertEqual(expected_lldp_info, interfaces[0].lldp)
self.assertTrue(interfaces[0].has_carrier)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
self.assertEqual('em0', interfaces[0].biosdevname)
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
@mock.patch.object(netutils, 'interface_has_carrier', autospec=True)
@mock.patch.object(netutils, 'get_mac_addr', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch('ironic_python_agent.hardware._get_managers', autospec=True)
@mock.patch('ironic_python_agent.netutils.get_lldp_info', autospec=True)
@mock.patch('netifaces.ifaddresses', autospec=True)
@mock.patch('os.listdir', autospec=True)
@mock.patch('os.path.exists', autospec=True)
@mock.patch('six.moves.builtins.open', autospec=True)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
@mock.patch.object(utils, 'execute', autospec=True)
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
def test_list_network_interfaces_with_lldp_error(
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
self, mocked_execute, mocked_open, mocked_exists, mocked_listdir,
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mocked_ifaddresses, mocked_lldp_info, mocked_get_managers,
mock_get_mac, mock_has_carrier):
Dispatched out network interface info to all hardware managers This patch dispatches out the network_interface_info to allow vendor hardware managers to plug the spacific implementation. It also move neworking releated methods form hardware to netutils Related-Bug: #1532534 Change-Id: Idcd25c4753c009b5ba70bea97ee4eb83391a77a9
2016-11-17 02:08:13 +02:00
mocked_get_managers.return_value = [hardware.GenericHardwareManager()]
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
CONF.set_override('collect_lldp', True)
mocked_listdir.return_value = ['lo', 'eth0']
mocked_exists.side_effect = [False, True]
mocked_open.return_value.__enter__ = lambda s: s
mocked_open.return_value.__exit__ = mock.Mock()
read_mock = mocked_open.return_value.read
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
read_mock.side_effect = ['1']
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
mocked_ifaddresses.return_value = {
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
netifaces.AF_INET: [{'addr': '192.168.1.2'}],
netifaces.AF_INET6: [{'addr': 'fd00::101'}]
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
}
mocked_lldp_info.side_effect = Exception('Boom!')
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_execute.return_value = ('em0\n', '')
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_has_carrier.return_value = True
mock_get_mac.return_value = '00:0c:29:8c:11:b1'
Support LLDP data as part of interfaces in inventory To support multi-tenant networking in Ironic we need to be able to discover not just the NICs a baremetal machine has but also the physical connectivity to switches in the network. This patch collects LLDP (Link Layer Discovery Protocol) data as part of the list interfaces stage of the generic hardware manager. This information can then be processed by the ironic inspector to populate the local link information on each ironic port. The processing done on this data in ironic python agent is limited, this is to allow for server side processing hooks to process as much or as little of the data as they want. This is to allow for multi-vendor environments that might use different parts of the LLDP packet to use a generic ramdisk and configure the processing server side using inspector plugins. Reserved fields switch_port_descr and switch_chassis_descr have been deprecated for removal in Ocata in favor of passing the whole packet. Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a Partial-Bug: #1526403
2016-05-24 17:46:49 +01:00
interfaces = self.hardware.list_network_interfaces()
self.assertEqual(1, len(interfaces))
self.assertEqual('eth0', interfaces[0].name)
self.assertEqual('00:0c:29:8c:11:b1', interfaces[0].mac_address)
self.assertEqual('192.168.1.2', interfaces[0].ipv4_address)
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
self.assertEqual('fd00::101', interfaces[0].ipv6_address)
Using assertIsNone() is preferred over assertEqual() Following OpenStack Style Guidelines[1]: http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises [H203] Unit test assertions tend to give better messages for more specific assertions. As a result, assertIsNone(...) is preferred over assertEqual(None, ...) Change-Id: I53c6d6d3e53d9fac38c50faf065c131d3aa55224
2016-08-29 10:07:14 +07:00
self.assertIsNone(interfaces[0].lldp)
Wait for the interfaces to get IP addresses before inspection In the DIB build the DHCP code (provided by the dhcp-all-interfaces element) races with the service starting IPA. It does not matter for deployment itself, as we're waiting for the route to the Ironic API to appear. However, for inspection it may result in reporting back all NIC's without IP addresses. Inspection fails in this case. This change makes inspection wait for *all* NIC's to get their IP addresses up to a small timeout. The timeout is 60 seconds by default and can be changed via the new ipa-inspection-dhcp-wait-timeout kernel option (0 to not wait). After the wait inspection proceedes in any case, so the worst downside is making inspection 60 seconds longer. To avoid waiting for NIC's that are not even connected, this change extends the NetworkInterface class with 'has_carrier' field. Closes-Bug: #1564954 Change-Id: I5bf14de4c1c622f4bf6e3eadbe20c44759da5d66
2016-04-01 16:36:01 +02:00
self.assertTrue(interfaces[0].has_carrier)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
self.assertEqual('em0', interfaces[0].biosdevname)
Wait for the interfaces to get IP addresses before inspection In the DIB build the DHCP code (provided by the dhcp-all-interfaces element) races with the service starting IPA. It does not matter for deployment itself, as we're waiting for the route to the Ironic API to appear. However, for inspection it may result in reporting back all NIC's without IP addresses. Inspection fails in this case. This change makes inspection wait for *all* NIC's to get their IP addresses up to a small timeout. The timeout is 60 seconds by default and can be changed via the new ipa-inspection-dhcp-wait-timeout kernel option (0 to not wait). After the wait inspection proceedes in any case, so the worst downside is making inspection 60 seconds longer. To avoid waiting for NIC's that are not even connected, this change extends the NetworkInterface class with 'has_carrier' field. Closes-Bug: #1564954 Change-Id: I5bf14de4c1c622f4bf6e3eadbe20c44759da5d66
2016-04-01 16:36:01 +02:00
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch('ironic_python_agent.hardware._get_managers', autospec=True)
@mock.patch('netifaces.ifaddresses', autospec=True)
@mock.patch('os.listdir', autospec=True)
@mock.patch('os.path.exists', autospec=True)
@mock.patch('six.moves.builtins.open', autospec=True)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
@mock.patch.object(utils, 'execute', autospec=True)
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
@mock.patch.object(netutils, 'get_mac_addr', autospec=True)
@mock.patch.object(netutils, 'interface_has_carrier', autospec=True)
Wait for the interfaces to get IP addresses before inspection In the DIB build the DHCP code (provided by the dhcp-all-interfaces element) races with the service starting IPA. It does not matter for deployment itself, as we're waiting for the route to the Ironic API to appear. However, for inspection it may result in reporting back all NIC's without IP addresses. Inspection fails in this case. This change makes inspection wait for *all* NIC's to get their IP addresses up to a small timeout. The timeout is 60 seconds by default and can be changed via the new ipa-inspection-dhcp-wait-timeout kernel option (0 to not wait). After the wait inspection proceedes in any case, so the worst downside is making inspection 60 seconds longer. To avoid waiting for NIC's that are not even connected, this change extends the NetworkInterface class with 'has_carrier' field. Closes-Bug: #1564954 Change-Id: I5bf14de4c1c622f4bf6e3eadbe20c44759da5d66
2016-04-01 16:36:01 +02:00
def test_list_network_interfaces_no_carrier(self,
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_has_carrier,
mock_get_mac,
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_execute,
Wait for the interfaces to get IP addresses before inspection In the DIB build the DHCP code (provided by the dhcp-all-interfaces element) races with the service starting IPA. It does not matter for deployment itself, as we're waiting for the route to the Ironic API to appear. However, for inspection it may result in reporting back all NIC's without IP addresses. Inspection fails in this case. This change makes inspection wait for *all* NIC's to get their IP addresses up to a small timeout. The timeout is 60 seconds by default and can be changed via the new ipa-inspection-dhcp-wait-timeout kernel option (0 to not wait). After the wait inspection proceedes in any case, so the worst downside is making inspection 60 seconds longer. To avoid waiting for NIC's that are not even connected, this change extends the NetworkInterface class with 'has_carrier' field. Closes-Bug: #1564954 Change-Id: I5bf14de4c1c622f4bf6e3eadbe20c44759da5d66
2016-04-01 16:36:01 +02:00
mocked_open,
mocked_exists,
mocked_listdir,
Dispatched out network interface info to all hardware managers This patch dispatches out the network_interface_info to allow vendor hardware managers to plug the spacific implementation. It also move neworking releated methods form hardware to netutils Related-Bug: #1532534 Change-Id: Idcd25c4753c009b5ba70bea97ee4eb83391a77a9
2016-11-17 02:08:13 +02:00
mocked_ifaddresses,
mocked_get_managers):
mocked_get_managers.return_value = [hardware.GenericHardwareManager()]
Wait for the interfaces to get IP addresses before inspection In the DIB build the DHCP code (provided by the dhcp-all-interfaces element) races with the service starting IPA. It does not matter for deployment itself, as we're waiting for the route to the Ironic API to appear. However, for inspection it may result in reporting back all NIC's without IP addresses. Inspection fails in this case. This change makes inspection wait for *all* NIC's to get their IP addresses up to a small timeout. The timeout is 60 seconds by default and can be changed via the new ipa-inspection-dhcp-wait-timeout kernel option (0 to not wait). After the wait inspection proceedes in any case, so the worst downside is making inspection 60 seconds longer. To avoid waiting for NIC's that are not even connected, this change extends the NetworkInterface class with 'has_carrier' field. Closes-Bug: #1564954 Change-Id: I5bf14de4c1c622f4bf6e3eadbe20c44759da5d66
2016-04-01 16:36:01 +02:00
mocked_listdir.return_value = ['lo', 'eth0']
mocked_exists.side_effect = [False, True]
mocked_open.return_value.__enter__ = lambda s: s
mocked_open.return_value.__exit__ = mock.Mock()
read_mock = mocked_open.return_value.read
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
read_mock.side_effect = [OSError('boom')]
Wait for the interfaces to get IP addresses before inspection In the DIB build the DHCP code (provided by the dhcp-all-interfaces element) races with the service starting IPA. It does not matter for deployment itself, as we're waiting for the route to the Ironic API to appear. However, for inspection it may result in reporting back all NIC's without IP addresses. Inspection fails in this case. This change makes inspection wait for *all* NIC's to get their IP addresses up to a small timeout. The timeout is 60 seconds by default and can be changed via the new ipa-inspection-dhcp-wait-timeout kernel option (0 to not wait). After the wait inspection proceedes in any case, so the worst downside is making inspection 60 seconds longer. To avoid waiting for NIC's that are not even connected, this change extends the NetworkInterface class with 'has_carrier' field. Closes-Bug: #1564954 Change-Id: I5bf14de4c1c622f4bf6e3eadbe20c44759da5d66
2016-04-01 16:36:01 +02:00
mocked_ifaddresses.return_value = {
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
netifaces.AF_INET: [{'addr': '192.168.1.2'}],
netifaces.AF_INET6: [{'addr': 'fd00::101'}]
Wait for the interfaces to get IP addresses before inspection In the DIB build the DHCP code (provided by the dhcp-all-interfaces element) races with the service starting IPA. It does not matter for deployment itself, as we're waiting for the route to the Ironic API to appear. However, for inspection it may result in reporting back all NIC's without IP addresses. Inspection fails in this case. This change makes inspection wait for *all* NIC's to get their IP addresses up to a small timeout. The timeout is 60 seconds by default and can be changed via the new ipa-inspection-dhcp-wait-timeout kernel option (0 to not wait). After the wait inspection proceedes in any case, so the worst downside is making inspection 60 seconds longer. To avoid waiting for NIC's that are not even connected, this change extends the NetworkInterface class with 'has_carrier' field. Closes-Bug: #1564954 Change-Id: I5bf14de4c1c622f4bf6e3eadbe20c44759da5d66
2016-04-01 16:36:01 +02:00
}
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_execute.return_value = ('em0\n', '')
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_has_carrier.return_value = False
mock_get_mac.return_value = '00:0c:29:8c:11:b1'
Wait for the interfaces to get IP addresses before inspection In the DIB build the DHCP code (provided by the dhcp-all-interfaces element) races with the service starting IPA. It does not matter for deployment itself, as we're waiting for the route to the Ironic API to appear. However, for inspection it may result in reporting back all NIC's without IP addresses. Inspection fails in this case. This change makes inspection wait for *all* NIC's to get their IP addresses up to a small timeout. The timeout is 60 seconds by default and can be changed via the new ipa-inspection-dhcp-wait-timeout kernel option (0 to not wait). After the wait inspection proceedes in any case, so the worst downside is making inspection 60 seconds longer. To avoid waiting for NIC's that are not even connected, this change extends the NetworkInterface class with 'has_carrier' field. Closes-Bug: #1564954 Change-Id: I5bf14de4c1c622f4bf6e3eadbe20c44759da5d66
2016-04-01 16:36:01 +02:00
interfaces = self.hardware.list_network_interfaces()
self.assertEqual(1, len(interfaces))
self.assertEqual('eth0', interfaces[0].name)
self.assertEqual('00:0c:29:8c:11:b1', interfaces[0].mac_address)
self.assertEqual('192.168.1.2', interfaces[0].ipv4_address)
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
self.assertEqual('fd00::101', interfaces[0].ipv6_address)
Wait for the interfaces to get IP addresses before inspection In the DIB build the DHCP code (provided by the dhcp-all-interfaces element) races with the service starting IPA. It does not matter for deployment itself, as we're waiting for the route to the Ironic API to appear. However, for inspection it may result in reporting back all NIC's without IP addresses. Inspection fails in this case. This change makes inspection wait for *all* NIC's to get their IP addresses up to a small timeout. The timeout is 60 seconds by default and can be changed via the new ipa-inspection-dhcp-wait-timeout kernel option (0 to not wait). After the wait inspection proceedes in any case, so the worst downside is making inspection 60 seconds longer. To avoid waiting for NIC's that are not even connected, this change extends the NetworkInterface class with 'has_carrier' field. Closes-Bug: #1564954 Change-Id: I5bf14de4c1c622f4bf6e3eadbe20c44759da5d66
2016-04-01 16:36:01 +02:00
self.assertFalse(interfaces[0].has_carrier)
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
self.assertIsNone(interfaces[0].vendor)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
self.assertEqual('em0', interfaces[0].biosdevname)
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch('ironic_python_agent.hardware._get_managers', autospec=True)
@mock.patch('netifaces.ifaddresses', autospec=True)
@mock.patch('os.listdir', autospec=True)
@mock.patch('os.path.exists', autospec=True)
@mock.patch('six.moves.builtins.open', autospec=True)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
@mock.patch.object(utils, 'execute', autospec=True)
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
@mock.patch.object(netutils, 'get_mac_addr', autospec=True)
@mock.patch.object(netutils, 'interface_has_carrier', autospec=True)
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
def test_list_network_interfaces_with_vendor_info(self,
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_has_carrier,
mock_get_mac,
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_execute,
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
mocked_open,
mocked_exists,
mocked_listdir,
Dispatched out network interface info to all hardware managers This patch dispatches out the network_interface_info to allow vendor hardware managers to plug the spacific implementation. It also move neworking releated methods form hardware to netutils Related-Bug: #1532534 Change-Id: Idcd25c4753c009b5ba70bea97ee4eb83391a77a9
2016-11-17 02:08:13 +02:00
mocked_ifaddresses,
mocked_get_managers):
mocked_get_managers.return_value = [hardware.GenericHardwareManager()]
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
mocked_listdir.return_value = ['lo', 'eth0']
mocked_exists.side_effect = [False, True]
mocked_open.return_value.__enter__ = lambda s: s
mocked_open.return_value.__exit__ = mock.Mock()
read_mock = mocked_open.return_value.read
mac = '00:0c:29:8c:11:b1'
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
read_mock.side_effect = ['0x15b3\n', '0x1014\n']
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
mocked_ifaddresses.return_value = {
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
netifaces.AF_INET: [{'addr': '192.168.1.2'}],
netifaces.AF_INET6: [{'addr': 'fd00::101'}]
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
}
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
mocked_execute.return_value = ('em0\n', '')
GenericHardwareManager: get mac address using netifaces Change-Id: Ie052c596b536325cbd3d26fe27e476a4b0b1981d
2018-03-13 16:06:55 +02:00
mock_has_carrier.return_value = True
mock_get_mac.return_value = mac
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
interfaces = self.hardware.list_network_interfaces()
self.assertEqual(1, len(interfaces))
self.assertEqual('eth0', interfaces[0].name)
self.assertEqual(mac, interfaces[0].mac_address)
self.assertEqual('192.168.1.2', interfaces[0].ipv4_address)
Collect IPv6 address during introspection This patch adds support to retrieve IPv6 address. A new field ``ipv6_address`` is added to NetworkInterface and store the assigned IPv6 address (if any). Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9 Closes-Bug: #1744064 Story: #1744064 Task: #11604
2018-01-21 22:44:33 +08:00
self.assertEqual('fd00::101', interfaces[0].ipv6_address)
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
self.assertTrue(interfaces[0].has_carrier)
self.assertEqual('0x15b3', interfaces[0].vendor)
self.assertEqual('0x1014', interfaces[0].product)
Collect NIC name given by BIOS Adds an extra field ``biosdevname`` to network interface inventory collected by ``default`` inspection collector (which collects the whole inventory returned by hardware manager) of ironic-python-agent. This feature requires biosdevname utility to collect the bios given NIC names. The tooling module for tinyIPA is created for the same purpose. For CoreOS IPA pxe images, biosdevname tooling module is limited, because Docker repository is created and embedded into CoreOS pxe images. The Docker repository uses debian to download the packages. Debian does not have biosdevname package. Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this variable to ``true`` in your shell before building tinyIPA. Closes-Bug: #1635351 Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2016-11-10 21:29:40 +00:00
self.assertEqual('em0', interfaces[0].biosdevname)
add tests of listing network interfaces
2014-01-27 17:51:06 -08:00
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
@mock.patch.object(os, 'readlink', autospec=True)
@mock.patch.object(os, 'listdir', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware, 'get_cached_node', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
def test_get_os_install_device(self, mocked_execute, mock_cached_node,
mocked_listdir, mocked_readlink):
Report /dev/disk/by-path on inspection followup This is the followup patch for commit d0a53149f82a3587515a4371f0f4cad8570dc715) fixing issues with the unit tests not addressed initially. Change-Id: I7889bf908bcb64b79bf303c6ae356fd3f4e94a83
2017-09-27 15:42:13 +02:00
mocked_readlink.return_value = '../../sda'
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
mocked_listdir.return_value = ['1:0:0:0']
Get root device hints from the node object In order to support a more complex syntax for root device hints (e.g operators: greater than, less than, in, etc...) we need to stop relying on the kernel command line for passing the root device hints. This patch changes this approach by getting the root device hints from a cached node object that was set in the hardware module. Two new functions: "cache_node" and "get_cached_node" were added to the hardware module. The idea is to facilitate the access to a node object representation from the hardware extension methods without changing method signatures, which would break compatibility with out-of-tree hardware managers. Note that the new "get_cached_node" is just a guard function to facilitate the tests for the code. The function parse_root_device_hints() and its tests were removed since it's not used/needed anymore. Partial-Bug: #1561137 Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-19 10:48:02 +01:00
mock_cached_node.return_value = None
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
mocked_execute.return_value = (BLK_DEVICE_TEMPLATE, '')
Fix params order in assertEqual Fix params order to correspond to real signature: assertEqual(expected, actual) Change-Id: I129aad5d8bb7ca0541c07913cb5edd8eef9e90ba Closes-Bug: #1277104
2016-01-12 09:03:19 +00:00
self.assertEqual('/dev/sdb', self.hardware.get_os_install_device())
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
mocked_execute.assert_called_once_with(
Include empty devices when listing block devices When listing block devices, empty devices are not listed by default. In order to clean up broken software RAID devices, however, we need to list these as well. This patch adds the corresponding option to 'lsblk', makes sure the device type filtering handles the additional device type ('md' instead of 'raid*' when an md device has only spares), and takes care of lsblk reporting size '' for empty devices. Story: #2006215 Task: #35807 Change-Id: I523e2ed90f3a477b5f5f27054b65fcfd26239ba2
2019-07-12 15:53:06 +02:00
'lsblk', '-Pbia', '-oKNAME,MODEL,SIZE,ROTA,TYPE',
Fix multi-device behavior ATARAID is functionally a version of software RAID where the setup is managed by the controller and the Operating System takes over managing the RAID after boot. Most commonly this is found for mirrored boot devices. Prior to this patch, we were looking for non-dependent items (i.e. base block devices), with a type of disk. Now we will permit the "disk" to be added to the list if lsblk indicates that it is a type containing "raid". The lsblk results should not change as we explicitly look for disk objects. Change-Id: Ia4a03b33cc06ce42e1bc33026683c28b31901cb7 Story: #2003445 Task: #24647
2018-08-16 10:57:59 -07:00
check_exit_code=[0])
mock_cached_node.assert_called_once_with()
@mock.patch.object(os, 'readlink', autospec=True)
@mock.patch.object(os, 'listdir', autospec=True)
@mock.patch.object(hardware, 'get_cached_node', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_os_install_device_raid(self, mocked_execute,
mock_cached_node, mocked_listdir,
mocked_readlink):
# NOTE(TheJulia): The readlink and listdir mocks are just to satisfy
# what is functionally an available path check and that information
# is stored in the returned result for use by root device hints.
mocked_readlink.side_effect = '../../sda'
mocked_listdir.return_value = ['1:0:0:0']
mock_cached_node.return_value = None
mocked_execute.return_value = (RAID_BLK_DEVICE_TEMPLATE, '')
# This should ideally select the smallest device and in theory raid
# should always be smaller
self.assertEqual('/dev/md0', self.hardware.get_os_install_device())
mocked_execute.assert_called_once_with(
Include empty devices when listing block devices When listing block devices, empty devices are not listed by default. In order to clean up broken software RAID devices, however, we need to list these as well. This patch adds the corresponding option to 'lsblk', makes sure the device type filtering handles the additional device type ('md' instead of 'raid*' when an md device has only spares), and takes care of lsblk reporting size '' for empty devices. Story: #2006215 Task: #35807 Change-Id: I523e2ed90f3a477b5f5f27054b65fcfd26239ba2
2019-07-12 15:53:06 +02:00
'lsblk', '-Pbia', '-oKNAME,MODEL,SIZE,ROTA,TYPE',
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
check_exit_code=[0])
Get root device hints from the node object In order to support a more complex syntax for root device hints (e.g operators: greater than, less than, in, etc...) we need to stop relying on the kernel command line for passing the root device hints. This patch changes this approach by getting the root device hints from a cached node object that was set in the hardware module. Two new functions: "cache_node" and "get_cached_node" were added to the hardware module. The idea is to facilitate the access to a node object representation from the hardware extension methods without changing method signatures, which would break compatibility with out-of-tree hardware managers. Note that the new "get_cached_node" is just a guard function to facilitate the tests for the code. The function parse_root_device_hints() and its tests were removed since it's not used/needed anymore. Partial-Bug: #1561137 Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-19 10:48:02 +01:00
mock_cached_node.assert_called_once_with()
add test of list_hardware_info
2014-01-28 11:25:00 -08:00
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
@mock.patch.object(os, 'readlink', autospec=True)
@mock.patch.object(os, 'listdir', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware, 'get_cached_node', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
Get root device hints from the node object In order to support a more complex syntax for root device hints (e.g operators: greater than, less than, in, etc...) we need to stop relying on the kernel command line for passing the root device hints. This patch changes this approach by getting the root device hints from a cached node object that was set in the hardware module. Two new functions: "cache_node" and "get_cached_node" were added to the hardware module. The idea is to facilitate the access to a node object representation from the hardware extension methods without changing method signatures, which would break compatibility with out-of-tree hardware managers. Note that the new "get_cached_node" is just a guard function to facilitate the tests for the code. The function parse_root_device_hints() and its tests were removed since it's not used/needed anymore. Partial-Bug: #1561137 Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-19 10:48:02 +01:00
def test_get_os_install_device_fails(self, mocked_execute,
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
mock_cached_node,
mocked_listdir, mocked_readlink):
Fix get_os_install_device() Instead of silently failing, raise DeviceNotFound when no root device hints were provided and all found block devices are smaller than 4GB. Change-Id: Idd2e2c5905adf847f00ad15a84a817c3715225dd Closes-Bug: #1490761
2015-09-02 15:21:04 +00:00
"""Fail to find device >=4GB w/o root device hints"""
Report /dev/disk/by-path on inspection followup This is the followup patch for commit d0a53149f82a3587515a4371f0f4cad8570dc715) fixing issues with the unit tests not addressed initially. Change-Id: I7889bf908bcb64b79bf303c6ae356fd3f4e94a83
2017-09-27 15:42:13 +02:00
mocked_readlink.return_value = '../../sda'
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
mocked_listdir.return_value = ['1:0:0:0']
Get root device hints from the node object In order to support a more complex syntax for root device hints (e.g operators: greater than, less than, in, etc...) we need to stop relying on the kernel command line for passing the root device hints. This patch changes this approach by getting the root device hints from a cached node object that was set in the hardware module. Two new functions: "cache_node" and "get_cached_node" were added to the hardware module. The idea is to facilitate the access to a node object representation from the hardware extension methods without changing method signatures, which would break compatibility with out-of-tree hardware managers. Note that the new "get_cached_node" is just a guard function to facilitate the tests for the code. The function parse_root_device_hints() and its tests were removed since it's not used/needed anymore. Partial-Bug: #1561137 Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-19 10:48:02 +01:00
mock_cached_node.return_value = None
Fix get_os_install_device() Instead of silently failing, raise DeviceNotFound when no root device hints were provided and all found block devices are smaller than 4GB. Change-Id: Idd2e2c5905adf847f00ad15a84a817c3715225dd Closes-Bug: #1490761
2015-09-02 15:21:04 +00:00
mocked_execute.return_value = (BLK_DEVICE_TEMPLATE_SMALL, '')
ex = self.assertRaises(errors.DeviceNotFound,
self.hardware.get_os_install_device)
mocked_execute.assert_called_once_with(
Include empty devices when listing block devices When listing block devices, empty devices are not listed by default. In order to clean up broken software RAID devices, however, we need to list these as well. This patch adds the corresponding option to 'lsblk', makes sure the device type filtering handles the additional device type ('md' instead of 'raid*' when an md device has only spares), and takes care of lsblk reporting size '' for empty devices. Story: #2006215 Task: #35807 Change-Id: I523e2ed90f3a477b5f5f27054b65fcfd26239ba2
2019-07-12 15:53:06 +02:00
'lsblk', '-Pbia', '-oKNAME,MODEL,SIZE,ROTA,TYPE',
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
check_exit_code=[0])
Fix get_os_install_device() Instead of silently failing, raise DeviceNotFound when no root device hints were provided and all found block devices are smaller than 4GB. Change-Id: Idd2e2c5905adf847f00ad15a84a817c3715225dd Closes-Bug: #1490761
2015-09-02 15:21:04 +00:00
self.assertIn(str(4 * units.Gi), ex.details)
Get root device hints from the node object In order to support a more complex syntax for root device hints (e.g operators: greater than, less than, in, etc...) we need to stop relying on the kernel command line for passing the root device hints. This patch changes this approach by getting the root device hints from a cached node object that was set in the hardware module. Two new functions: "cache_node" and "get_cached_node" were added to the hardware module. The idea is to facilitate the access to a node object representation from the hardware extension methods without changing method signatures, which would break compatibility with out-of-tree hardware managers. Note that the new "get_cached_node" is just a guard function to facilitate the tests for the code. The function parse_root_device_hints() and its tests were removed since it's not used/needed anymore. Partial-Bug: #1561137 Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-19 10:48:02 +01:00
mock_cached_node.assert_called_once_with()
Fix get_os_install_device() Instead of silently failing, raise DeviceNotFound when no root device hints were provided and all found block devices are smaller than 4GB. Change-Id: Idd2e2c5905adf847f00ad15a84a817c3715225dd Closes-Bug: #1490761
2015-09-02 15:21:04 +00:00
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware, 'list_all_block_devices', autospec=True)
@mock.patch.object(hardware, 'get_cached_node', autospec=True)
Extend root device hints to support device name This patch is extending the root device hints to also look at the device name. This patch also refactors the tests for root device hints making it easier to test a different hint per test. Change-Id: I48d6456c75bbe6ddf16ac6561e5461ca51eb9c37 Partial-Bug: #1526732
2016-01-19 13:36:26 +00:00
def _get_os_install_device_root_device_hints(self, hints, expected_device,
Get root device hints from the node object In order to support a more complex syntax for root device hints (e.g operators: greater than, less than, in, etc...) we need to stop relying on the kernel command line for passing the root device hints. This patch changes this approach by getting the root device hints from a cached node object that was set in the hardware module. Two new functions: "cache_node" and "get_cached_node" were added to the hardware module. The idea is to facilitate the access to a node object representation from the hardware extension methods without changing method signatures, which would break compatibility with out-of-tree hardware managers. Note that the new "get_cached_node" is just a guard function to facilitate the tests for the code. The function parse_root_device_hints() and its tests were removed since it's not used/needed anymore. Partial-Bug: #1561137 Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-19 10:48:02 +01:00
mock_cached_node, mock_dev):
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
mock_cached_node.return_value = {'properties': {'root_device': hints},
'uuid': 'node1'}
Add support for root device hints This patch add support for root device hints on IPA. Instead of picking the first disk >= 4G, if the hints are specified IPA will look at it to decide which device it should pick for the deployment. The initial patch supports the following hints: Size, model, WWN, serial, vendor. Implements: blueprint ipa-as-default-ramdisk Implements: blueprint root-device-hints Change-Id: I2b00b3fb3b61001033750dd8951f9353d6f2e361
2015-03-10 11:20:40 +00:00
model = 'fastable sd131 7'
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
mock_dev.return_value = [
hardware.BlockDevice(name='/dev/sda',
model='TinyUSB Drive',
size=3116853504,
rotational=False,
vendor='Super Vendor',
wwn='wwn0',
Follow up patch for da9c3b0adc67efa916fc534d975823c0a45948a1 This patch is a follow up patch fixing some nits left by the review da9c3b0adc67efa916fc534d975823c0a45948a1, this patch adds the wwn_with_extension and wwn_vendor_extension root device hints to the "serializable_fields" list attribute of the BlockDevice class and fixes some tests. Change-Id: I6039be535988319276f9ac355c80997d34328ce8
2015-11-18 09:53:54 +00:00
wwn_with_extension='wwn0ven0',
wwn_vendor_extension='ven0',
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
serial='serial0'),
hardware.BlockDevice(name='/dev/sdb',
model=model,
size=10737418240,
Extend root device hints to support "rotational" This patch is extending the root device hints to also consider if a disk is a rotational device or not. This hint makes it easier to separate SSDs and HDDs when selecting the device to deploy the image onto. Partial-Bug: #1599517 Change-Id: I270fe57df825929bdef7911b3a6757cf7163a5f1
2016-07-06 14:16:16 +01:00
rotational=True,
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
vendor='fake-vendor',
wwn='fake-wwn',
Follow up patch for da9c3b0adc67efa916fc534d975823c0a45948a1 This patch is a follow up patch fixing some nits left by the review da9c3b0adc67efa916fc534d975823c0a45948a1, this patch adds the wwn_with_extension and wwn_vendor_extension root device hints to the "serializable_fields" list attribute of the BlockDevice class and fixes some tests. Change-Id: I6039be535988319276f9ac355c80997d34328ce8
2015-11-18 09:53:54 +00:00
wwn_with_extension='fake-wwnven0',
wwn_vendor_extension='ven0',
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
serial='fake-serial',
by_path='/dev/disk/by-path/1:0:0:0'),
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
]
Add support for root device hints This patch add support for root device hints on IPA. Instead of picking the first disk >= 4G, if the hints are specified IPA will look at it to decide which device it should pick for the deployment. The initial patch supports the following hints: Size, model, WWN, serial, vendor. Implements: blueprint ipa-as-default-ramdisk Implements: blueprint root-device-hints Change-Id: I2b00b3fb3b61001033750dd8951f9353d6f2e361
2015-03-10 11:20:40 +00:00
Extend root device hints to support device name This patch is extending the root device hints to also look at the device name. This patch also refactors the tests for root device hints making it easier to test a different hint per test. Change-Id: I48d6456c75bbe6ddf16ac6561e5461ca51eb9c37 Partial-Bug: #1526732
2016-01-19 13:36:26 +00:00
self.assertEqual(expected_device,
self.hardware.get_os_install_device())
Get root device hints from the node object In order to support a more complex syntax for root device hints (e.g operators: greater than, less than, in, etc...) we need to stop relying on the kernel command line for passing the root device hints. This patch changes this approach by getting the root device hints from a cached node object that was set in the hardware module. Two new functions: "cache_node" and "get_cached_node" were added to the hardware module. The idea is to facilitate the access to a node object representation from the hardware extension methods without changing method signatures, which would break compatibility with out-of-tree hardware managers. Note that the new "get_cached_node" is just a guard function to facilitate the tests for the code. The function parse_root_device_hints() and its tests were removed since it's not used/needed anymore. Partial-Bug: #1561137 Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-19 10:48:02 +01:00
mock_cached_node.assert_called_once_with()
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
mock_dev.assert_called_once_with()
Add support for root device hints This patch add support for root device hints on IPA. Instead of picking the first disk >= 4G, if the hints are specified IPA will look at it to decide which device it should pick for the deployment. The initial patch supports the following hints: Size, model, WWN, serial, vendor. Implements: blueprint ipa-as-default-ramdisk Implements: blueprint root-device-hints Change-Id: I2b00b3fb3b61001033750dd8951f9353d6f2e361
2015-03-10 11:20:40 +00:00
Extend root device hints to support device name This patch is extending the root device hints to also look at the device name. This patch also refactors the tests for root device hints making it easier to test a different hint per test. Change-Id: I48d6456c75bbe6ddf16ac6561e5461ca51eb9c37 Partial-Bug: #1526732
2016-01-19 13:36:26 +00:00
def test_get_os_install_device_root_device_hints_model(self):
self._get_os_install_device_root_device_hints(
{'model': 'fastable sd131 7'}, '/dev/sdb')
def test_get_os_install_device_root_device_hints_wwn(self):
self._get_os_install_device_root_device_hints(
{'wwn': 'wwn0'}, '/dev/sda')
def test_get_os_install_device_root_device_hints_serial(self):
self._get_os_install_device_root_device_hints(
{'serial': 'serial0'}, '/dev/sda')
def test_get_os_install_device_root_device_hints_size(self):
self._get_os_install_device_root_device_hints(
{'size': 10}, '/dev/sdb')
Small refactor in the root device loop matching logic When matching the root device hints passed to Ironic and the devices present in the disk, the logic to the "size" attribute was different was outside the main loop. This patch refactors that uses the same loop for matching the size attribute, improving a little the readability. This patch also makes sure that we convert the value of "size" to integer before attempting to match it. Change-Id: I619153232b9b58696369185ad5be90ccfd38a4ed
2016-07-07 09:38:17 +01:00
def test_get_os_install_device_root_device_hints_size_str(self):
self._get_os_install_device_root_device_hints(
{'size': '10'}, '/dev/sdb')
Add support for root device hints with operators This patch is updating IPA to use the match_root_device_hints() method provided by ironic-lib version 2.2.0. Partial-Bug: #1561137 Depends-On: I1d9dc7a57ea391a3419710c289242b39a4201463 Change-Id: Id93dd0360137df600f5a656348279e56c6b84bf9
2016-09-07 14:10:11 +01:00
def test_get_os_install_device_root_device_hints_size_not_int(self):
Small refactor in the root device loop matching logic When matching the root device hints passed to Ironic and the devices present in the disk, the logic to the "size" attribute was different was outside the main loop. This patch refactors that uses the same loop for matching the size attribute, improving a little the readability. This patch also makes sure that we convert the value of "size" to integer before attempting to match it. Change-Id: I619153232b9b58696369185ad5be90ccfd38a4ed
2016-07-07 09:38:17 +01:00
self.assertRaises(errors.DeviceNotFound,
self._get_os_install_device_root_device_hints,
{'size': 'not-int'}, '/dev/sdb')
Extend root device hints to support device name This patch is extending the root device hints to also look at the device name. This patch also refactors the tests for root device hints making it easier to test a different hint per test. Change-Id: I48d6456c75bbe6ddf16ac6561e5461ca51eb9c37 Partial-Bug: #1526732
2016-01-19 13:36:26 +00:00
def test_get_os_install_device_root_device_hints_vendor(self):
self._get_os_install_device_root_device_hints(
{'vendor': 'fake-vendor'}, '/dev/sdb')
def test_get_os_install_device_root_device_hints_name(self):
self._get_os_install_device_root_device_hints(
{'name': '/dev/sdb'}, '/dev/sdb')
Extend root device hints to support "rotational" This patch is extending the root device hints to also consider if a disk is a rotational device or not. This hint makes it easier to separate SSDs and HDDs when selecting the device to deploy the image onto. Partial-Bug: #1599517 Change-Id: I270fe57df825929bdef7911b3a6757cf7163a5f1
2016-07-06 14:16:16 +01:00
def test_get_os_install_device_root_device_hints_rotational(self):
for value in (True, 'true', 'on', 'y', 'yes'):
self._get_os_install_device_root_device_hints(
{'rotational': value}, '/dev/sdb')
Report /dev/disk/by-path on inspection followup This is the followup patch for commit d0a53149f82a3587515a4371f0f4cad8570dc715) fixing issues with the unit tests not addressed initially. Change-Id: I7889bf908bcb64b79bf303c6ae356fd3f4e94a83
2017-09-27 15:42:13 +02:00
def test_get_os_install_device_root_device_hints_by_path(self):
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
self._get_os_install_device_root_device_hints(
{'by_path': '/dev/disk/by-path/1:0:0:0'}, '/dev/sdb')
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware, 'list_all_block_devices', autospec=True)
@mock.patch.object(hardware, 'get_cached_node', autospec=True)
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
def test_get_os_install_device_root_device_hints_no_device_found(
Get root device hints from the node object In order to support a more complex syntax for root device hints (e.g operators: greater than, less than, in, etc...) we need to stop relying on the kernel command line for passing the root device hints. This patch changes this approach by getting the root device hints from a cached node object that was set in the hardware module. Two new functions: "cache_node" and "get_cached_node" were added to the hardware module. The idea is to facilitate the access to a node object representation from the hardware extension methods without changing method signatures, which would break compatibility with out-of-tree hardware managers. Note that the new "get_cached_node" is just a guard function to facilitate the tests for the code. The function parse_root_device_hints() and its tests were removed since it's not used/needed anymore. Partial-Bug: #1561137 Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-19 10:48:02 +01:00
self, mock_cached_node, mock_dev):
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
model = 'fastable sd131 7'
Get root device hints from the node object In order to support a more complex syntax for root device hints (e.g operators: greater than, less than, in, etc...) we need to stop relying on the kernel command line for passing the root device hints. This patch changes this approach by getting the root device hints from a cached node object that was set in the hardware module. Two new functions: "cache_node" and "get_cached_node" were added to the hardware module. The idea is to facilitate the access to a node object representation from the hardware extension methods without changing method signatures, which would break compatibility with out-of-tree hardware managers. Note that the new "get_cached_node" is just a guard function to facilitate the tests for the code. The function parse_root_device_hints() and its tests were removed since it's not used/needed anymore. Partial-Bug: #1561137 Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-19 10:48:02 +01:00
mock_cached_node.return_value = {
'properties': {
'root_device': {
'model': model,
'wwn': 'fake-wwn',
'serial': 'fake-serial',
'vendor': 'fake-vendor',
'size': 10}}}
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
# Model is different here
mock_dev.return_value = [
hardware.BlockDevice(name='/dev/sda',
model='TinyUSB Drive',
size=3116853504,
rotational=False,
vendor='Super Vendor',
wwn='wwn0',
serial='serial0'),
hardware.BlockDevice(name='/dev/sdb',
model='Another Model',
size=10737418240,
rotational=False,
vendor='fake-vendor',
wwn='fake-wwn',
serial='fake-serial'),
]
Add support for root device hints This patch add support for root device hints on IPA. Instead of picking the first disk >= 4G, if the hints are specified IPA will look at it to decide which device it should pick for the deployment. The initial patch supports the following hints: Size, model, WWN, serial, vendor. Implements: blueprint ipa-as-default-ramdisk Implements: blueprint root-device-hints Change-Id: I2b00b3fb3b61001033750dd8951f9353d6f2e361
2015-03-10 11:20:40 +00:00
self.assertRaises(errors.DeviceNotFound,
self.hardware.get_os_install_device)
Get root device hints from the node object In order to support a more complex syntax for root device hints (e.g operators: greater than, less than, in, etc...) we need to stop relying on the kernel command line for passing the root device hints. This patch changes this approach by getting the root device hints from a cached node object that was set in the hardware module. Two new functions: "cache_node" and "get_cached_node" were added to the hardware module. The idea is to facilitate the access to a node object representation from the hardware extension methods without changing method signatures, which would break compatibility with out-of-tree hardware managers. Note that the new "get_cached_node" is just a guard function to facilitate the tests for the code. The function parse_root_device_hints() and its tests were removed since it's not used/needed anymore. Partial-Bug: #1561137 Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-19 10:48:02 +01:00
mock_cached_node.assert_called_once_with()
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
mock_dev.assert_called_once_with()
Add support for root device hints This patch add support for root device hints on IPA. Instead of picking the first disk >= 4G, if the hints are specified IPA will look at it to decide which device it should pick for the deployment. The initial patch supports the following hints: Size, model, WWN, serial, vendor. Implements: blueprint ipa-as-default-ramdisk Implements: blueprint root-device-hints Change-Id: I2b00b3fb3b61001033750dd8951f9353d6f2e361
2015-03-10 11:20:40 +00:00
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
def test__get_device_info(self):
Add support for root device hints This patch add support for root device hints on IPA. Instead of picking the first disk >= 4G, if the hints are specified IPA will look at it to decide which device it should pick for the deployment. The initial patch supports the following hints: Size, model, WWN, serial, vendor. Implements: blueprint ipa-as-default-ramdisk Implements: blueprint root-device-hints Change-Id: I2b00b3fb3b61001033750dd8951f9353d6f2e361
2015-03-10 11:20:40 +00:00
fileobj = mock.mock_open(read_data='fake-vendor')
Update unit tests to use six.moves.builtins.open Change-Id: I724a42d22d4c9011feb8aa09726e5186686f3729
2016-02-17 14:08:42 +02:00
with mock.patch(
'six.moves.builtins.open', fileobj, create=True) as mock_open:
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
vendor = hardware._get_device_info(
'/dev/sdfake', 'block', 'vendor')
Add support for root device hints This patch add support for root device hints on IPA. Instead of picking the first disk >= 4G, if the hints are specified IPA will look at it to decide which device it should pick for the deployment. The initial patch supports the following hints: Size, model, WWN, serial, vendor. Implements: blueprint ipa-as-default-ramdisk Implements: blueprint root-device-hints Change-Id: I2b00b3fb3b61001033750dd8951f9353d6f2e361
2015-03-10 11:20:40 +00:00
mock_open.assert_called_once_with(
'/sys/class/block/sdfake/device/vendor', 'r')
self.assertEqual('fake-vendor', vendor)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
def test_get_cpus(self, mocked_execute):
Returns CPU flags in the CPU inventory These flags will be processed in a new ironic-inspector plugin to support setting capabilities like cpu_vt (virtualization enabled). Change-Id: I5fe9310c316841eabdd2d5e2ef2ae30afa03d29a Partial-Bug: #1571580
2016-04-20 13:58:14 +02:00
mocked_execute.side_effect = [
(LSCPU_OUTPUT, ''),
(CPUINFO_FLAGS_OUTPUT, '')
]
Implement version 2 payload with more hw info Bumping the payload version to two, and adding CPU, Memory, and Block Device information to the inventory. Added dependency on psutil for available memory and cpu_count. Requirements line copied from global requirements Change-Id: Ia39c85c91b1d60468667787a7978020084dc6c2a
2014-04-09 17:09:51 -07:00
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
cpus = self.hardware.get_cpus()
Fix params order in assertEqual Fix params order to correspond to real signature: assertEqual(expected, actual) Change-Id: I129aad5d8bb7ca0541c07913cb5edd8eef9e90ba Closes-Bug: #1277104
2016-01-12 09:03:19 +00:00
self.assertEqual('Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz',
cpus.model_name)
self.assertEqual('2400.0000', cpus.frequency)
self.assertEqual(4, cpus.count)
self.assertEqual('x86_64', cpus.architecture)
Returns CPU flags in the CPU inventory These flags will be processed in a new ironic-inspector plugin to support setting capabilities like cpu_vt (virtualization enabled). Change-Id: I5fe9310c316841eabdd2d5e2ef2ae30afa03d29a Partial-Bug: #1571580
2016-04-20 13:58:14 +02:00
self.assertEqual(['fpu', 'vme', 'de', 'pse'], cpus.flags)
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
def test_get_cpus2(self, mocked_execute):
Returns CPU flags in the CPU inventory These flags will be processed in a new ironic-inspector plugin to support setting capabilities like cpu_vt (virtualization enabled). Change-Id: I5fe9310c316841eabdd2d5e2ef2ae30afa03d29a Partial-Bug: #1571580
2016-04-20 13:58:14 +02:00
mocked_execute.side_effect = [
(LSCPU_OUTPUT_NO_MAX_MHZ, ''),
(CPUINFO_FLAGS_OUTPUT, '')
]
Implement version 2 payload with more hw info Bumping the payload version to two, and adding CPU, Memory, and Block Device information to the inventory. Added dependency on psutil for available memory and cpu_count. Requirements line copied from global requirements Change-Id: Ia39c85c91b1d60468667787a7978020084dc6c2a
2014-04-09 17:09:51 -07:00
cpus = self.hardware.get_cpus()
Fix params order in assertEqual Fix params order to correspond to real signature: assertEqual(expected, actual) Change-Id: I129aad5d8bb7ca0541c07913cb5edd8eef9e90ba Closes-Bug: #1277104
2016-01-12 09:03:19 +00:00
self.assertEqual('Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz',
cpus.model_name)
self.assertEqual('1794.433', cpus.frequency)
self.assertEqual(12, cpus.count)
self.assertEqual('x86_64', cpus.architecture)
Returns CPU flags in the CPU inventory These flags will be processed in a new ironic-inspector plugin to support setting capabilities like cpu_vt (virtualization enabled). Change-Id: I5fe9310c316841eabdd2d5e2ef2ae30afa03d29a Partial-Bug: #1571580
2016-04-20 13:58:14 +02:00
self.assertEqual(['fpu', 'vme', 'de', 'pse'], cpus.flags)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Returns CPU flags in the CPU inventory These flags will be processed in a new ironic-inspector plugin to support setting capabilities like cpu_vt (virtualization enabled). Change-Id: I5fe9310c316841eabdd2d5e2ef2ae30afa03d29a Partial-Bug: #1571580
2016-04-20 13:58:14 +02:00
def test_get_cpus_no_flags(self, mocked_execute):
mocked_execute.side_effect = [
(LSCPU_OUTPUT, ''),
processutils.ProcessExecutionError()
]
cpus = self.hardware.get_cpus()
self.assertEqual('Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz',
cpus.model_name)
self.assertEqual('2400.0000', cpus.frequency)
self.assertEqual(4, cpus.count)
self.assertEqual('x86_64', cpus.architecture)
self.assertEqual([], cpus.flags)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Returns CPU flags in the CPU inventory These flags will be processed in a new ironic-inspector plugin to support setting capabilities like cpu_vt (virtualization enabled). Change-Id: I5fe9310c316841eabdd2d5e2ef2ae30afa03d29a Partial-Bug: #1571580
2016-04-20 13:58:14 +02:00
def test_get_cpus_illegal_flags(self, mocked_execute):
mocked_execute.side_effect = [
(LSCPU_OUTPUT, ''),
('I am not a flag', '')
]
cpus = self.hardware.get_cpus()
self.assertEqual('Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz',
cpus.model_name)
self.assertEqual('2400.0000', cpus.frequency)
self.assertEqual(4, cpus.count)
self.assertEqual('x86_64', cpus.architecture)
self.assertEqual([], cpus.flags)
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
Support psutil 5.0.1 An upper-constraints update to psutil caused IPA to start using psutil 5.0.1. We had a hard-coded assumption that psutil would be major version 1 or 2. This allows us to use the updated psutil and attempts to simply fail gracefully if an unrecognized psutil version is used. Change-Id: Ibe072440159561b34a29b478d955876e5fb7f103 Closes-Bug: 1659137
2017-01-24 22:59:46 +00:00
@mock.patch('psutil.virtual_memory', autospec=True)
Remove support for older psutil versions Global requirements was recently updated to force psutil=>3.0.1. This patch removes support for older versions of psutil as well as changing to opportunistically attempt to work if a version >5 is released but doesn't change the interface we use. Change-Id: I1f7fab33fd275fb8b5cd7704dc13375402756d06 Related-bug: #1659137
2017-01-25 08:52:46 -08:00
@mock.patch.object(utils, 'execute', autospec=True)
Ironic python agent does not extract correct available memory This patch fixes parsing of memory class output of lshw output. Task: 30201 Story: 2005308 Change-Id: I0e7ff9f86a266d2f798457136fed0100c78569e2
2019-03-27 22:38:59 +11:00
def test_get_memory_psutil_v1(self, mocked_execute, mocked_psutil):
Remove support for older psutil versions Global requirements was recently updated to force psutil=>3.0.1. This patch removes support for older versions of psutil as well as changing to opportunistically attempt to work if a version >5 is released but doesn't change the interface we use. Change-Id: I1f7fab33fd275fb8b5cd7704dc13375402756d06 Related-bug: #1659137
2017-01-25 08:52:46 -08:00
mocked_psutil.return_value.total = 3952 * 1024 * 1024
Ironic python agent does not extract correct available memory This patch fixes parsing of memory class output of lshw output. Task: 30201 Story: 2005308 Change-Id: I0e7ff9f86a266d2f798457136fed0100c78569e2
2019-03-27 22:38:59 +11:00
mocked_execute.return_value = LSHW_JSON_OUTPUT_V1
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
mem = self.hardware.get_memory()
Fix params order in assertEqual Fix params order to correspond to real signature: assertEqual(expected, actual) Change-Id: I129aad5d8bb7ca0541c07913cb5edd8eef9e90ba Closes-Bug: #1277104
2016-01-12 09:03:19 +00:00
self.assertEqual(3952 * 1024 * 1024, mem.total)
self.assertEqual(4096, mem.physical_mb)
Implement version 2 payload with more hw info Bumping the payload version to two, and adding CPU, Memory, and Block Device information to the inventory. Added dependency on psutil for available memory and cpu_count. Requirements line copied from global requirements Change-Id: Ia39c85c91b1d60468667787a7978020084dc6c2a
2014-04-09 17:09:51 -07:00
Remove support for older psutil versions Global requirements was recently updated to force psutil=>3.0.1. This patch removes support for older versions of psutil as well as changing to opportunistically attempt to work if a version >5 is released but doesn't change the interface we use. Change-Id: I1f7fab33fd275fb8b5cd7704dc13375402756d06 Related-bug: #1659137
2017-01-25 08:52:46 -08:00
@mock.patch('psutil.virtual_memory', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
Ironic python agent does not extract correct available memory This patch fixes parsing of memory class output of lshw output. Task: 30201 Story: 2005308 Change-Id: I0e7ff9f86a266d2f798457136fed0100c78569e2
2019-03-27 22:38:59 +11:00
def test_get_memory_psutil_v2(self, mocked_execute, mocked_psutil):
mocked_psutil.return_value.total = 3952 * 1024 * 1024
mocked_execute.return_value = LSHW_JSON_OUTPUT_V2
mem = self.hardware.get_memory()
self.assertEqual(3952 * 1024 * 1024, mem.total)
self.assertEqual(65536, mem.physical_mb)
@mock.patch('psutil.virtual_memory', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_memory_psutil_exception_v1(self, mocked_execute,
mocked_psutil):
mocked_execute.return_value = LSHW_JSON_OUTPUT_V1
Remove support for older psutil versions Global requirements was recently updated to force psutil=>3.0.1. This patch removes support for older versions of psutil as well as changing to opportunistically attempt to work if a version >5 is released but doesn't change the interface we use. Change-Id: I1f7fab33fd275fb8b5cd7704dc13375402756d06 Related-bug: #1659137
2017-01-25 08:52:46 -08:00
mocked_psutil.side_effect = AttributeError()
mem = self.hardware.get_memory()
self.assertIsNone(mem.total)
self.assertEqual(4096, mem.physical_mb)
Ironic python agent does not extract correct available memory This patch fixes parsing of memory class output of lshw output. Task: 30201 Story: 2005308 Change-Id: I0e7ff9f86a266d2f798457136fed0100c78569e2
2019-03-27 22:38:59 +11:00
@mock.patch('psutil.virtual_memory', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_memory_psutil_exception_v2(self, mocked_execute,
mocked_psutil):
mocked_execute.return_value = LSHW_JSON_OUTPUT_V2
mocked_psutil.side_effect = AttributeError()
mem = self.hardware.get_memory()
self.assertIsNone(mem.total)
self.assertEqual(65536, mem.physical_mb)
Use lshw in place of dmidecode for the default hardware manager Currently the generic hardware manager uses dmidecode to get the total physical memory and system details. This patch switches the generic hardware manager to use lshw, as it is capable of reading more than DMI [0]. This enables systems that do not support DMI to use the generic hardware manager, such as IBM Power systems. [0] https://github.com/lyonel/lshw/blob/master/README.md Closes-Bug: #1715790 Change-Id: Ie370331df6bb5ef131c5cb60f458877e2a7ad71a Depends-On: Idaf05b8efce28cd0cbf339cf693db4f55a693d9b
2017-11-30 16:30:42 +00:00
@mock.patch('psutil.virtual_memory', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_memory_lshw_exception(self, mocked_execute, mocked_psutil):
mocked_execute.side_effect = OSError()
mocked_psutil.return_value.total = 3952 * 1024 * 1024
mem = self.hardware.get_memory()
self.assertEqual(3952 * 1024 * 1024, mem.total)
self.assertIsNone(mem.physical_mb)
Get the hostname of the introspected host This will retrieve the hostname of the host being introspected, and return it as part of the introspection data. Change-Id: I54084251e1b0f3a40fe5ac760b2a7e45199c9fdb Story: #2005867 Task: #33674
2019-06-07 12:11:08 -04:00
@mock.patch('ironic_python_agent.netutils.get_hostname', autospec=True)
def test_list_hardware_info(self, mocked_get_hostname):
add test of list_hardware_info
2014-01-28 11:25:00 -08:00
self.hardware.list_network_interfaces = mock.Mock()
self.hardware.list_network_interfaces.return_value = [
hardware.NetworkInterface('eth0', '00:0c:29:8c:11:b1'),
hardware.NetworkInterface('eth1', '00:0c:29:8c:11:b2'),
]
Implement version 2 payload with more hw info Bumping the payload version to two, and adding CPU, Memory, and Block Device information to the inventory. Added dependency on psutil for available memory and cpu_count. Requirements line copied from global requirements Change-Id: Ia39c85c91b1d60468667787a7978020084dc6c2a
2014-04-09 17:09:51 -07:00
self.hardware.get_cpus = mock.Mock()
self.hardware.get_cpus.return_value = hardware.CPU(
'Awesome CPU x14 9001',
9001,
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
14,
'x86_64')
Implement version 2 payload with more hw info Bumping the payload version to two, and adding CPU, Memory, and Block Device information to the inventory. Added dependency on psutil for available memory and cpu_count. Requirements line copied from global requirements Change-Id: Ia39c85c91b1d60468667787a7978020084dc6c2a
2014-04-09 17:09:51 -07:00
self.hardware.get_memory = mock.Mock()
self.hardware.get_memory.return_value = hardware.Memory(1017012)
self.hardware.list_block_devices = mock.Mock()
self.hardware.list_block_devices.return_value = [
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
hardware.BlockDevice('/dev/sdj', 'big', 1073741824, True),
hardware.BlockDevice('/dev/hdaa', 'small', 65535, False),
Implement version 2 payload with more hw info Bumping the payload version to two, and adding CPU, Memory, and Block Device information to the inventory. Added dependency on psutil for available memory and cpu_count. Requirements line copied from global requirements Change-Id: Ia39c85c91b1d60468667787a7978020084dc6c2a
2014-04-09 17:09:51 -07:00
]
Add boot information into the inventory Adds a new BootInfo object with 2 fields: * current_boot_mode - bios or uefi, detected from presence of /sys/firmware/efi as per the following answer: http://askubuntu.com/a/162896 This field will be used for setting the boot_mode capability in ironic-inspector * pxe_interface - PXE booting interface, if it can be detected. This fields is already used by ironic-inspector, added here for consistency. Change-Id: Ib36b592ffaba3bfa055d65c9526607867d302584 Partial-Bug: #1571580
2016-04-20 13:58:14 +02:00
self.hardware.get_boot_info = mock.Mock()
self.hardware.get_boot_info.return_value = hardware.BootInfo(
current_boot_mode='bios', pxe_interface='boot:if')
Prevent tests' unmocked access to utils.execute() This change introduces a new base test class that mocks out utils.execute and forces an exception if it gets called. This has rooted out many tests that were doing this as a side effect of calling other functions, doing things like modprobe and running iscsi on the host's actual machine. The tests are all now appropriately patched in places where this was happening, and the new base class permanently prevents this from accidentally happening again. If you really want to call utils.execute() then you need to re-mock it in your unit test. Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-04-03 15:32:44 +10:00
self.hardware.get_bmc_address = mock.Mock()
Discover IPv6 BMC address Change-Id: Ia7f24c1df782de4809a5195876bbf0309b481fd7 Story: #2005133 Task: #29817
2019-03-04 15:31:14 +03:00
self.hardware.get_bmc_v6address = mock.Mock()
Prevent tests' unmocked access to utils.execute() This change introduces a new base test class that mocks out utils.execute and forces an exception if it gets called. This has rooted out many tests that were doing this as a side effect of calling other functions, doing things like modprobe and running iscsi on the host's actual machine. The tests are all now appropriately patched in places where this was happening, and the new base class permanently prevents this from accidentally happening again. If you really want to call utils.execute() then you need to re-mock it in your unit test. Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-04-03 15:32:44 +10:00
self.hardware.get_system_vendor_info = mock.Mock()
Get the hostname of the introspected host This will retrieve the hostname of the host being introspected, and return it as part of the introspection data. Change-Id: I54084251e1b0f3a40fe5ac760b2a7e45199c9fdb Story: #2005867 Task: #33674
2019-06-07 12:11:08 -04:00
mocked_get_hostname.return_value = 'mock_hostname'
add test of list_hardware_info
2014-01-28 11:25:00 -08:00
hardware_info = self.hardware.list_hardware_info()
Fix params order in assertEqual Fix params order to correspond to real signature: assertEqual(expected, actual) Change-Id: I129aad5d8bb7ca0541c07913cb5edd8eef9e90ba Closes-Bug: #1277104
2016-01-12 09:03:19 +00:00
self.assertEqual(self.hardware.get_memory(), hardware_info['memory'])
self.assertEqual(self.hardware.get_cpus(), hardware_info['cpu'])
self.assertEqual(self.hardware.list_block_devices(),
hardware_info['disks'])
self.assertEqual(self.hardware.list_network_interfaces(),
hardware_info['interfaces'])
Add boot information into the inventory Adds a new BootInfo object with 2 fields: * current_boot_mode - bios or uefi, detected from presence of /sys/firmware/efi as per the following answer: http://askubuntu.com/a/162896 This field will be used for setting the boot_mode capability in ironic-inspector * pxe_interface - PXE booting interface, if it can be detected. This fields is already used by ironic-inspector, added here for consistency. Change-Id: Ib36b592ffaba3bfa055d65c9526607867d302584 Partial-Bug: #1571580
2016-04-20 13:58:14 +02:00
self.assertEqual(self.hardware.get_boot_info(),
hardware_info['boot'])
Get the hostname of the introspected host This will retrieve the hostname of the host being introspected, and return it as part of the introspection data. Change-Id: I54084251e1b0f3a40fe5ac760b2a7e45199c9fdb Story: #2005867 Task: #33674
2019-06-07 12:11:08 -04:00
self.assertEqual('mock_hostname', hardware_info['hostname'])
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware, 'list_all_block_devices', autospec=True)
Refactor list_block_devices to its own function This function is useful in any HardwareManager that interacts with disks. Subclassing GenericHardwareManager is not ideal for any hardware manager that interacts with only specific devices. Change-Id: Ib20e68a8916590513c0a825e44407a110cfbb441
2015-08-07 15:41:30 -07:00
def test_list_block_devices(self, list_mock):
device = hardware.BlockDevice('/dev/hdaa', 'small', 65535, False)
list_mock.return_value = [device]
devices = self.hardware.list_block_devices()
self.assertEqual([device], devices)
list_mock.assert_called_once_with()
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
@mock.patch.object(hardware, 'list_all_block_devices', autospec=True)
def test_list_block_devices_including_partitions(self, list_mock):
device = hardware.BlockDevice('/dev/hdaa', 'small', 65535, False)
partition = hardware.BlockDevice('/dev/hdaa1', '', 32767, False)
list_mock.side_effect = [[device], [partition]]
devices = self.hardware.list_block_devices(include_partitions=True)
self.assertEqual([device, partition], devices)
self.assertEqual([mock.call(), mock.call(block_type='part',
ignore_raid=True)],
list_mock.call_args_list)
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
@mock.patch.object(os, 'readlink', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(os, 'listdir', autospec=True)
@mock.patch.object(hardware, '_get_device_info', autospec=True)
Using from_device_file from Devices The from_device_file method in pyudev.Device is deprecated since version 0.18 Using the same method from pyudev.Devices is now recommended. Increasing minimum version of pyudev to 0.18 and correctly handling exception raised by from_device_file. Change-Id: I0637585241c81a36f69aef8017f1134f5c526eb1
2019-07-08 18:42:16 +02:00
@mock.patch.object(pyudev.Devices, 'from_device_file', autospec=False)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
def test_list_all_block_device(self, mocked_execute, mocked_udev,
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
mocked_dev_vendor, mock_listdir,
mock_readlink):
by_path_map = {
Report /dev/disk/by-path on inspection followup This is the followup patch for commit d0a53149f82a3587515a4371f0f4cad8570dc715) fixing issues with the unit tests not addressed initially. Change-Id: I7889bf908bcb64b79bf303c6ae356fd3f4e94a83
2017-09-27 15:42:13 +02:00
'/dev/disk/by-path/1:0:0:0': '../../dev/sda',
'/dev/disk/by-path/1:0:0:1': '../../dev/sdb',
'/dev/disk/by-path/1:0:0:2': '../../dev/sdc',
# pretend that the by-path link to ../../dev/sdd is missing
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
}
mock_readlink.side_effect = lambda x, m=by_path_map: m[x]
mock_listdir.return_value = [os.path.basename(x)
for x in sorted(by_path_map)]
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
mocked_execute.return_value = (BLK_DEVICE_TEMPLATE, '')
Using from_device_file from Devices The from_device_file method in pyudev.Device is deprecated since version 0.18 Using the same method from pyudev.Devices is now recommended. Increasing minimum version of pyudev to 0.18 and correctly handling exception raised by from_device_file. Change-Id: I0637585241c81a36f69aef8017f1134f5c526eb1
2019-07-08 18:42:16 +02:00
mocked_udev.side_effect = pyudev.DeviceNotFoundByFileError()
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
mocked_dev_vendor.return_value = 'Super Vendor'
devices = hardware.list_all_block_devices()
expected_devices = [
hardware.BlockDevice(name='/dev/sda',
model='TinyUSB Drive',
size=3116853504,
rotational=False,
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
vendor='Super Vendor',
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
hctl='1:0:0:0',
by_path='/dev/disk/by-path/1:0:0:0'),
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
hardware.BlockDevice(name='/dev/sdb',
model='Fastable SD131 7',
size=10737418240,
rotational=False,
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
vendor='Super Vendor',
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
hctl='1:0:0:0',
by_path='/dev/disk/by-path/1:0:0:1'),
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
hardware.BlockDevice(name='/dev/sdc',
model='NWD-BLP4-1600',
size=1765517033472,
rotational=False,
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
vendor='Super Vendor',
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
hctl='1:0:0:0',
by_path='/dev/disk/by-path/1:0:0:2'),
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
hardware.BlockDevice(name='/dev/sdd',
model='NWD-BLP4-1600',
size=1765517033472,
rotational=False,
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
vendor='Super Vendor',
Report /dev/disk/by-path on inspection followup This is the followup patch for commit d0a53149f82a3587515a4371f0f4cad8570dc715) fixing issues with the unit tests not addressed initially. Change-Id: I7889bf908bcb64b79bf303c6ae356fd3f4e94a83
2017-09-27 15:42:13 +02:00
hctl='1:0:0:0'),
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
]
pyudev exception has changed for from_device_file Now pyudev raises DeviceNotFoundByFileError which does not inherit from EnvironmentError, so our 'except' block in hardware.py no longer catch the exception. It broke unit tests, but it can also potentially break the deploy. This patch updates hardware.py to catch both old a new exceptions. Change-Id: Iaefd6089f6f766a241054d8e132b2f3098c8130d Closes-Bug: #1522756
2015-12-04 12:33:07 +01:00
self.assertEqual(4, len(devices))
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
for expected, device in zip(expected_devices, devices):
# Compare all attrs of the objects
for attr in ['name', 'model', 'size', 'rotational',
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
'wwn', 'vendor', 'serial', 'hctl']:
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
self.assertEqual(getattr(expected, attr),
getattr(device, attr))
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
expected_calls = [mock.call('/sys/block/%s/device/scsi_device' % dev)
for dev in ('sda', 'sdb', 'sdc', 'sdd')]
mock_listdir.assert_has_calls(expected_calls)
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
expected_calls = [mock.call('/dev/disk/by-path/1:0:0:%d' % dev)
Report /dev/disk/by-path on inspection followup This is the followup patch for commit d0a53149f82a3587515a4371f0f4cad8570dc715) fixing issues with the unit tests not addressed initially. Change-Id: I7889bf908bcb64b79bf303c6ae356fd3f4e94a83
2017-09-27 15:42:13 +02:00
for dev in range(3)]
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
mock_readlink.assert_has_calls(expected_calls)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(os, 'listdir', autospec=True)
@mock.patch.object(hardware, '_get_device_info', autospec=True)
Using from_device_file from Devices The from_device_file method in pyudev.Device is deprecated since version 0.18 Using the same method from pyudev.Devices is now recommended. Increasing minimum version of pyudev to 0.18 and correctly handling exception raised by from_device_file. Change-Id: I0637585241c81a36f69aef8017f1134f5c526eb1
2019-07-08 18:42:16 +02:00
@mock.patch.object(pyudev.Devices, 'from_device_file', autospec=False)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
def test_list_all_block_device_hctl_fail(self, mocked_execute, mocked_udev,
mocked_dev_vendor,
mocked_listdir):
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
mocked_listdir.side_effect = (OSError, OSError, IndexError)
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
mocked_execute.return_value = (BLK_DEVICE_TEMPLATE_SMALL, '')
mocked_dev_vendor.return_value = 'Super Vendor'
devices = hardware.list_all_block_devices()
self.assertEqual(2, len(devices))
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
expected_calls = [
mock.call('/dev/disk/by-path'),
mock.call('/sys/block/sda/device/scsi_device'),
mock.call('/sys/block/sdb/device/scsi_device')
]
self.assertEqual(expected_calls, mocked_listdir.call_args_list)
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
@mock.patch.object(os, 'readlink', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(os, 'listdir', autospec=True)
@mock.patch.object(hardware, '_get_device_info', autospec=True)
Using from_device_file from Devices The from_device_file method in pyudev.Device is deprecated since version 0.18 Using the same method from pyudev.Devices is now recommended. Increasing minimum version of pyudev to 0.18 and correctly handling exception raised by from_device_file. Change-Id: I0637585241c81a36f69aef8017f1134f5c526eb1
2019-07-08 18:42:16 +02:00
@mock.patch.object(pyudev.Devices, 'from_device_file', autospec=False)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
def test_list_all_block_device_with_udev(self, mocked_execute, mocked_udev,
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
mocked_dev_vendor, mocked_listdir,
mocked_readlink):
Report /dev/disk/by-path on inspection followup This is the followup patch for commit d0a53149f82a3587515a4371f0f4cad8570dc715) fixing issues with the unit tests not addressed initially. Change-Id: I7889bf908bcb64b79bf303c6ae356fd3f4e94a83
2017-09-27 15:42:13 +02:00
mocked_readlink.return_value = '../../sda'
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
mocked_listdir.return_value = ['1:0:0:0']
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
mocked_execute.return_value = (BLK_DEVICE_TEMPLATE, '')
mocked_udev.side_effect = iter([
Extend root device hints for different types of WWN This patch is extending the root device hints to also look at ID_WWN_WITH_EXTENSION and ID_WWN_VENDOR_EXTENSION from udev. Prior to this patch the IPA ramdisk only cared about ID_WWN but in some systems in some platforms with a RAID controller, this ID can be same even if they are different disks (see bug 1516641). Closes-Bug: #1516641 Change-Id: Ic3e9a1111dfcc99702190c173562a0dccf5f94c4
2015-11-16 14:50:55 +00:00
{'ID_WWN': 'wwn%d' % i, 'ID_SERIAL_SHORT': 'serial%d' % i,
'ID_WWN_WITH_EXTENSION': 'wwn-ext%d' % i,
'ID_WWN_VENDOR_EXTENSION': 'wwn-vendor-ext%d' % i}
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
for i in range(4)
])
mocked_dev_vendor.return_value = 'Super Vendor'
Refactor list_block_devices to its own function This function is useful in any HardwareManager that interacts with disks. Subclassing GenericHardwareManager is not ideal for any hardware manager that interacts with only specific devices. Change-Id: Ib20e68a8916590513c0a825e44407a110cfbb441
2015-08-07 15:41:30 -07:00
devices = hardware.list_all_block_devices()
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
expected_devices = [
hardware.BlockDevice(name='/dev/sda',
model='TinyUSB Drive',
size=3116853504,
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
rotational=False,
vendor='Super Vendor',
wwn='wwn0',
Extend root device hints for different types of WWN This patch is extending the root device hints to also look at ID_WWN_WITH_EXTENSION and ID_WWN_VENDOR_EXTENSION from udev. Prior to this patch the IPA ramdisk only cared about ID_WWN but in some systems in some platforms with a RAID controller, this ID can be same even if they are different disks (see bug 1516641). Closes-Bug: #1516641 Change-Id: Ic3e9a1111dfcc99702190c173562a0dccf5f94c4
2015-11-16 14:50:55 +00:00
wwn_with_extension='wwn-ext0',
wwn_vendor_extension='wwn-vendor-ext0',
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
serial='serial0',
hctl='1:0:0:0'),
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
hardware.BlockDevice(name='/dev/sdb',
model='Fastable SD131 7',
Add support for root device hints This patch add support for root device hints on IPA. Instead of picking the first disk >= 4G, if the hints are specified IPA will look at it to decide which device it should pick for the deployment. The initial patch supports the following hints: Size, model, WWN, serial, vendor. Implements: blueprint ipa-as-default-ramdisk Implements: blueprint root-device-hints Change-Id: I2b00b3fb3b61001033750dd8951f9353d6f2e361
2015-03-10 11:20:40 +00:00
size=10737418240,
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
rotational=False,
vendor='Super Vendor',
wwn='wwn1',
Extend root device hints for different types of WWN This patch is extending the root device hints to also look at ID_WWN_WITH_EXTENSION and ID_WWN_VENDOR_EXTENSION from udev. Prior to this patch the IPA ramdisk only cared about ID_WWN but in some systems in some platforms with a RAID controller, this ID can be same even if they are different disks (see bug 1516641). Closes-Bug: #1516641 Change-Id: Ic3e9a1111dfcc99702190c173562a0dccf5f94c4
2015-11-16 14:50:55 +00:00
wwn_with_extension='wwn-ext1',
wwn_vendor_extension='wwn-vendor-ext1',
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
serial='serial1',
hctl='1:0:0:0'),
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
hardware.BlockDevice(name='/dev/sdc',
model='NWD-BLP4-1600',
size=1765517033472,
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
rotational=False,
vendor='Super Vendor',
wwn='wwn2',
Extend root device hints for different types of WWN This patch is extending the root device hints to also look at ID_WWN_WITH_EXTENSION and ID_WWN_VENDOR_EXTENSION from udev. Prior to this patch the IPA ramdisk only cared about ID_WWN but in some systems in some platforms with a RAID controller, this ID can be same even if they are different disks (see bug 1516641). Closes-Bug: #1516641 Change-Id: Ic3e9a1111dfcc99702190c173562a0dccf5f94c4
2015-11-16 14:50:55 +00:00
wwn_with_extension='wwn-ext2',
wwn_vendor_extension='wwn-vendor-ext2',
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
serial='serial2',
hctl='1:0:0:0'),
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
hardware.BlockDevice(name='/dev/sdd',
model='NWD-BLP4-1600',
size=1765517033472,
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
rotational=False,
vendor='Super Vendor',
wwn='wwn3',
Extend root device hints for different types of WWN This patch is extending the root device hints to also look at ID_WWN_WITH_EXTENSION and ID_WWN_VENDOR_EXTENSION from udev. Prior to this patch the IPA ramdisk only cared about ID_WWN but in some systems in some platforms with a RAID controller, this ID can be same even if they are different disks (see bug 1516641). Closes-Bug: #1516641 Change-Id: Ic3e9a1111dfcc99702190c173562a0dccf5f94c4
2015-11-16 14:50:55 +00:00
wwn_with_extension='wwn-ext3',
wwn_vendor_extension='wwn-vendor-ext3',
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
serial='serial3',
hctl='1:0:0:0')
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
]
self.assertEqual(4, len(expected_devices))
for expected, device in zip(expected_devices, devices):
# Compare all attrs of the objects
Expose serial, wwn and vendor on the BlockDevice object Currently we only use these disk properties for root device hints. However, they'll be really useful for inspector, especially for also implementing root device hints. Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-18 12:52:33 +02:00
for attr in ['name', 'model', 'size', 'rotational',
Extend root device hints for different types of WWN This patch is extending the root device hints to also look at ID_WWN_WITH_EXTENSION and ID_WWN_VENDOR_EXTENSION from udev. Prior to this patch the IPA ramdisk only cared about ID_WWN but in some systems in some platforms with a RAID controller, this ID can be same even if they are different disks (see bug 1516641). Closes-Bug: #1516641 Change-Id: Ic3e9a1111dfcc99702190c173562a0dccf5f94c4
2015-11-16 14:50:55 +00:00
'wwn', 'vendor', 'serial', 'wwn_with_extension',
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
'wwn_vendor_extension', 'hctl']:
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
self.assertEqual(getattr(expected, attr),
getattr(device, attr))
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
expected_calls = [mock.call('/sys/block/%s/device/scsi_device' % dev)
for dev in ('sda', 'sdb', 'sdc', 'sdd')]
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
mocked_listdir.assert_has_calls(expected_calls)
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware, 'dispatch_to_managers', autospec=True)
Parallel erase disk devices Currently we erase the disks one by one, which takes a long time to finish, this patch adds support to the IPA so that it can erase disks in parallel if told so. Story: 1546949 Task: 11548 Co-Authored-By: yuan liang <leetpy2@gmail.com> Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: If5cfb6ec000a654d07103c4b378d4c135249e238
2016-07-25 19:07:27 +08:00
def test_erase_devices_no_parallel_by_default(self, mocked_dispatch):
Dispatch the call to erase_block_device There is no way for two hardware managers to handle erasing two disks in two different ways. dispatch_to_managers was designed specifically for this case, and the default behavior will remain the same for the GenericHardwareManager (erase_block_device will pick up each disk). Also return the result of the dispatch calls, so they'll be logged by Ironic and give more cleaning insight. Change-Id: I19e9dc8539a0729fbb96cae92fe633e24608fc68
2015-08-17 09:51:21 -07:00
mocked_dispatch.return_value = 'erased device'
self.hardware.list_block_devices = mock.Mock()
self.hardware.list_block_devices.return_value = [
hardware.BlockDevice('/dev/sdj', 'big', 1073741824, True),
hardware.BlockDevice('/dev/hdaa', 'small', 65535, False),
]
expected = {'/dev/hdaa': 'erased device', '/dev/sdj': 'erased device'}
result = self.hardware.erase_devices({}, [])
self.assertEqual(expected, result)
Parallel erase disk devices Currently we erase the disks one by one, which takes a long time to finish, this patch adds support to the IPA so that it can erase disks in parallel if told so. Story: 1546949 Task: 11548 Co-Authored-By: yuan liang <leetpy2@gmail.com> Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: If5cfb6ec000a654d07103c4b378d4c135249e238
2016-07-25 19:07:27 +08:00
@mock.patch('multiprocessing.pool.ThreadPool.apply_async', autospec=True)
@mock.patch.object(hardware, 'dispatch_to_managers', autospec=True)
def test_erase_devices_concurrency(self, mocked_dispatch, mocked_async):
internal_info = self.node['driver_internal_info']
internal_info['disk_erasure_concurrency'] = 10
mocked_dispatch.return_value = 'erased device'
if six.PY3:
apply_result = multiprocessing.pool.ApplyResult({}, None, None)
else:
apply_result = multiprocessing.pool.ApplyResult({}, None)
apply_result._success = True
apply_result._ready = True
apply_result.get = lambda: 'erased device'
mocked_async.return_value = apply_result
Follow up to parallel disk erasure Improve test to verify apply_async is called twice as expected. Story: 1546949 Task: 11548 Change-Id: I41736dfb2932dd0036bbc4cbc51929bf61a16569
2018-10-24 10:48:27 +08:00
blkdev1 = hardware.BlockDevice('/dev/sdj', 'big', 1073741824, True)
blkdev2 = hardware.BlockDevice('/dev/hdaa', 'small', 65535, False)
Parallel erase disk devices Currently we erase the disks one by one, which takes a long time to finish, this patch adds support to the IPA so that it can erase disks in parallel if told so. Story: 1546949 Task: 11548 Co-Authored-By: yuan liang <leetpy2@gmail.com> Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: If5cfb6ec000a654d07103c4b378d4c135249e238
2016-07-25 19:07:27 +08:00
self.hardware.list_block_devices = mock.Mock()
Follow up to parallel disk erasure Improve test to verify apply_async is called twice as expected. Story: 1546949 Task: 11548 Change-Id: I41736dfb2932dd0036bbc4cbc51929bf61a16569
2018-10-24 10:48:27 +08:00
self.hardware.list_block_devices.return_value = [blkdev1, blkdev2]
Parallel erase disk devices Currently we erase the disks one by one, which takes a long time to finish, this patch adds support to the IPA so that it can erase disks in parallel if told so. Story: 1546949 Task: 11548 Co-Authored-By: yuan liang <leetpy2@gmail.com> Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: If5cfb6ec000a654d07103c4b378d4c135249e238
2016-07-25 19:07:27 +08:00
expected = {'/dev/hdaa': 'erased device', '/dev/sdj': 'erased device'}
result = self.hardware.erase_devices(self.node, [])
Follow up to parallel disk erasure Improve test to verify apply_async is called twice as expected. Story: 1546949 Task: 11548 Change-Id: I41736dfb2932dd0036bbc4cbc51929bf61a16569
2018-10-24 10:48:27 +08:00
calls = [mock.call(mock.ANY, mocked_dispatch, ('erase_block_device',),
{'node': self.node, 'block_device': dev})
for dev in (blkdev1, blkdev2)]
mocked_async.assert_has_calls(calls)
Parallel erase disk devices Currently we erase the disks one by one, which takes a long time to finish, this patch adds support to the IPA so that it can erase disks in parallel if told so. Story: 1546949 Task: 11548 Co-Authored-By: yuan liang <leetpy2@gmail.com> Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com> Change-Id: If5cfb6ec000a654d07103c4b378d4c135249e238
2016-07-25 19:07:27 +08:00
self.assertEqual(expected, result)
@mock.patch.object(hardware, 'ThreadPool', autospec=True)
def test_erase_devices_concurrency_pool_size(self, mocked_pool):
self.hardware.list_block_devices = mock.Mock()
self.hardware.list_block_devices.return_value = [
hardware.BlockDevice('/dev/sdj', 'big', 1073741824, True),
hardware.BlockDevice('/dev/hdaa', 'small', 65535, False),
]
# Test pool size 10 with 2 disks
internal_info = self.node['driver_internal_info']
internal_info['disk_erasure_concurrency'] = 10
self.hardware.erase_devices(self.node, [])
mocked_pool.assert_called_with(2)
# Test default pool size with 2 disks
internal_info = self.node['driver_internal_info']
del internal_info['disk_erasure_concurrency']
self.hardware.erase_devices(self.node, [])
mocked_pool.assert_called_with(1)
@mock.patch.object(hardware, 'dispatch_to_managers', autospec=True)
def test_erase_devices_without_disk(self, mocked_dispatch):
self.hardware.list_block_devices = mock.Mock()
self.hardware.list_block_devices.return_value = []
expected = {}
result = self.hardware.erase_devices({}, [])
self.assertEqual(expected, result)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
def test_erase_block_device_ata_success(self, mocked_execute,
mocked_raid_member):
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
mocked_execute.side_effect = [
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
(create_hdparm_info(
supported=True, enabled=False, frozen=False,
enhanced_erase=False), ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
('', ''),
('', ''),
(create_hdparm_info(
supported=True, enabled=False, frozen=False,
enhanced_erase=False), ''),
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
self.hardware.erase_block_device(self.node, block_device)
mocked_execute.assert_has_calls([
mock.call('hdparm', '-I', '/dev/sda'),
mock.call('smartctl', '-d', 'ata', '/dev/sda', '-g', 'security',
check_exit_code=[0, 127]),
mock.call('hdparm', '--user-master', 'u', '--security-set-pass',
'NULL', '/dev/sda'),
mock.call('hdparm', '--user-master', 'u', '--security-erase',
'NULL', '/dev/sda'),
mock.call('hdparm', '-I', '/dev/sda'),
])
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
def test_erase_block_device_ata_success_no_smartctl(self, mocked_execute,
mocked_raid_member):
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
mocked_execute.side_effect = [
(create_hdparm_info(
supported=True, enabled=False, frozen=False,
enhanced_erase=False), ''),
OSError('boom'),
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
('', ''),
('', ''),
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
(create_hdparm_info(
supported=True, enabled=False, frozen=False,
enhanced_erase=False), ''),
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
IPA:'shred' utility to use configured iterations Today, there is no option to configure number of iterations to be done for shred block device erasing and defaults it to 1. This patch adds a configuration option to change the number of passes to be done to erase a block device. Change-Id: I1921d33a6b364c4682b6c9baaf61ac092cfa11d7 Partial-Bug:#1465130
2015-06-15 21:36:27 +05:30
self.hardware.erase_block_device(self.node, block_device)
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
mocked_execute.assert_has_calls([
mock.call('hdparm', '-I', '/dev/sda'),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
mock.call('smartctl', '-d', 'ata', '/dev/sda', '-g', 'security',
check_exit_code=[0, 127]),
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
mock.call('hdparm', '--user-master', 'u', '--security-set-pass',
'NULL', '/dev/sda'),
mock.call('hdparm', '--user-master', 'u', '--security-erase',
'NULL', '/dev/sda'),
mock.call('hdparm', '-I', '/dev/sda'),
])
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
def test_erase_block_device_nosecurity_shred(self, mocked_execute,
mocked_raid_member):
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
hdparm_output = HDPARM_INFO_TEMPLATE.split('\nSecurity:')[0]
mocked_execute.side_effect = [
Add fallback secure erase using shred This prevents devices from failing cleaning if they don't support ATA security erase. Instead, the device will be erased by overwriting using GNU shred. Change-Id: Ie4f88b97c8c34efe2397538662622abd0b963aea
2015-03-17 16:26:06 -07:00
(hdparm_output, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_UNAVAILABLE_OUTPUT, ''),
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
(SHRED_OUTPUT_1_ITERATION_ZERO_TRUE, '')
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
IPA:'shred' utility to use configured iterations Today, there is no option to configure number of iterations to be done for shred block device erasing and defaults it to 1. This patch adds a configuration option to change the number of passes to be done to erase a block device. Change-Id: I1921d33a6b364c4682b6c9baaf61ac092cfa11d7 Partial-Bug:#1465130
2015-06-15 21:36:27 +05:30
self.hardware.erase_block_device(self.node, block_device)
Add fallback secure erase using shred This prevents devices from failing cleaning if they don't support ATA security erase. Instead, the device will be erased by overwriting using GNU shred. Change-Id: Ie4f88b97c8c34efe2397538662622abd0b963aea
2015-03-17 16:26:06 -07:00
mocked_execute.assert_has_calls([
mock.call('hdparm', '-I', '/dev/sda'),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
mock.call('smartctl', '-d', 'ata', '/dev/sda', '-g', 'security',
check_exit_code=[0, 127]),
Add fallback secure erase using shred This prevents devices from failing cleaning if they don't support ATA security erase. Instead, the device will be erased by overwriting using GNU shred. Change-Id: Ie4f88b97c8c34efe2397538662622abd0b963aea
2015-03-17 16:26:06 -07:00
mock.call('shred', '--force', '--zero', '--verbose',
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
'--iterations', '1', '/dev/sda')
Add fallback secure erase using shred This prevents devices from failing cleaning if they don't support ATA security erase. Instead, the device will be erased by overwriting using GNU shred. Change-Id: Ie4f88b97c8c34efe2397538662622abd0b963aea
2015-03-17 16:26:06 -07:00
])
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
def test_erase_block_device_notsupported_shred(self, mocked_execute,
mocked_raid_member):
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output = create_hdparm_info(
supported=False, enabled=False, frozen=False, enhanced_erase=False)
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
mocked_execute.side_effect = [
Add fallback secure erase using shred This prevents devices from failing cleaning if they don't support ATA security erase. Instead, the device will be erased by overwriting using GNU shred. Change-Id: Ie4f88b97c8c34efe2397538662622abd0b963aea
2015-03-17 16:26:06 -07:00
(hdparm_output, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_UNAVAILABLE_OUTPUT, ''),
(SHRED_OUTPUT_1_ITERATION_ZERO_TRUE, '')
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
self.hardware.erase_block_device(self.node, block_device)
mocked_execute.assert_has_calls([
mock.call('hdparm', '-I', '/dev/sda'),
mock.call('smartctl', '-d', 'ata', '/dev/sda', '-g', 'security',
check_exit_code=[0, 127]),
mock.call('shred', '--force', '--zero', '--verbose',
'--iterations', '1', '/dev/sda')
])
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
def test_erase_block_device_smartctl_unsupported_shred(self,
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_execute,
mocked_raid_member):
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
hdparm_output = create_hdparm_info(
supported=True, enabled=False, frozen=False, enhanced_erase=False)
mocked_execute.side_effect = [
(hdparm_output, ''),
(SMARTCTL_UNAVAILABLE_OUTPUT, ''),
(SHRED_OUTPUT_1_ITERATION_ZERO_TRUE, '')
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
self.hardware.erase_block_device(self.node, block_device)
mocked_execute.assert_has_calls([
mock.call('hdparm', '-I', '/dev/sda'),
mock.call('smartctl', '-d', 'ata', '/dev/sda', '-g', 'security',
check_exit_code=[0, 127]),
mock.call('shred', '--force', '--zero', '--verbose',
'--iterations', '1', '/dev/sda')
])
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
def test_erase_block_device_smartctl_fails_security_fallback_to_shred(
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
self, mocked_execute, mocked_raid_member):
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
hdparm_output = create_hdparm_info(
supported=True, enabled=False, frozen=False, enhanced_erase=False)
mocked_execute.side_effect = [
(hdparm_output, ''),
processutils.ProcessExecutionError(),
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
(SHRED_OUTPUT_1_ITERATION_ZERO_TRUE, '')
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
IPA:'shred' utility to use configured iterations Today, there is no option to configure number of iterations to be done for shred block device erasing and defaults it to 1. This patch adds a configuration option to change the number of passes to be done to erase a block device. Change-Id: I1921d33a6b364c4682b6c9baaf61ac092cfa11d7 Partial-Bug:#1465130
2015-06-15 21:36:27 +05:30
self.hardware.erase_block_device(self.node, block_device)
Add fallback secure erase using shred This prevents devices from failing cleaning if they don't support ATA security erase. Instead, the device will be erased by overwriting using GNU shred. Change-Id: Ie4f88b97c8c34efe2397538662622abd0b963aea
2015-03-17 16:26:06 -07:00
mocked_execute.assert_has_calls([
mock.call('hdparm', '-I', '/dev/sda'),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
mock.call('smartctl', '-d', 'ata', '/dev/sda', '-g', 'security',
check_exit_code=[0, 127]),
Add fallback secure erase using shred This prevents devices from failing cleaning if they don't support ATA security erase. Instead, the device will be erased by overwriting using GNU shred. Change-Id: Ie4f88b97c8c34efe2397538662622abd0b963aea
2015-03-17 16:26:06 -07:00
mock.call('shred', '--force', '--zero', '--verbose',
Convert Int to String for shred execute Was running into 'expected string, int found' when calling shred with an Int for iterations. Change-Id: Iffce247caba5b0d62ac89b6411402c8d975cfd2f Closes-Bug: #1469838
2015-07-01 15:08:18 +00:00
'--iterations', '1', '/dev/sda')
Add fallback secure erase using shred This prevents devices from failing cleaning if they don't support ATA security erase. Instead, the device will be erased by overwriting using GNU shred. Change-Id: Ie4f88b97c8c34efe2397538662622abd0b963aea
2015-03-17 16:26:06 -07:00
])
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
def test_erase_block_device_shred_uses_internal_info(self, mocked_execute,
mocked_raid_member):
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
hdparm_output = create_hdparm_info(
supported=False, enabled=False, frozen=False, enhanced_erase=False)
Replace dict.get(key) with dict[key] in tests This replaces dict.get(key) with dict[key] in the unit tests, since we should be explicit about whether we expect the key to exist or not. (These are tests, after all.) Change-Id: Ide98ceba325214e55cab2063d9ace150d63f9617 Closes-Bug: #1590808
2016-06-23 17:41:23 -04:00
info = self.node['driver_internal_info']
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
info['agent_erase_devices_iterations'] = 2
info['agent_erase_devices_zeroize'] = False
mocked_execute.side_effect = [
(hdparm_output, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
(SHRED_OUTPUT_2_ITERATIONS_ZERO_FALSE, '')
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
self.hardware.erase_block_device(self.node, block_device)
mocked_execute.assert_has_calls([
mock.call('hdparm', '-I', '/dev/sda'),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
mock.call('smartctl', '-d', 'ata', '/dev/sda', '-g', 'security',
check_exit_code=[0, 127]),
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
mock.call('shred', '--force', '--verbose',
'--iterations', '2', '/dev/sda')
])
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
def test_erase_block_device_shred_0_pass_no_zeroize(self, mocked_execute,
mocked_raid_member):
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
hdparm_output = create_hdparm_info(
supported=False, enabled=False, frozen=False, enhanced_erase=False)
Replace dict.get(key) with dict[key] in tests This replaces dict.get(key) with dict[key] in the unit tests, since we should be explicit about whether we expect the key to exist or not. (These are tests, after all.) Change-Id: Ide98ceba325214e55cab2063d9ace150d63f9617 Closes-Bug: #1590808
2016-06-23 17:41:23 -04:00
info = self.node['driver_internal_info']
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
info['agent_erase_devices_iterations'] = 0
info['agent_erase_devices_zeroize'] = False
mocked_execute.side_effect = [
(hdparm_output, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_UNAVAILABLE_OUTPUT, ''),
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
(SHRED_OUTPUT_0_ITERATIONS_ZERO_FALSE, '')
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
self.hardware.erase_block_device(self.node, block_device)
mocked_execute.assert_has_calls([
mock.call('hdparm', '-I', '/dev/sda'),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
mock.call('smartctl', '-d', 'ata', '/dev/sda', '-g', 'security',
check_exit_code=[0, 127]),
Allow shred zeroize option to be configured Introduce a new parameter in driver_internal_info called agent_erase_devices_zeroize to control the behavior of shred. This parameter controls the --zero argument used when invoking shred. Configuring this to false disabled the last pass of zeroes, leaving the device with random data. Change-Id: I7053034f5b5bc6737b535ee601e6fb71284d4a83 Partial-bug: #1568811 Depends-On: Ia7ea8d909df9ae86a6dbd68ba94746b171535eb8
2016-04-12 08:37:18 -04:00
mock.call('shred', '--force', '--verbose',
'--iterations', '0', '/dev/sda')
])
Fix inband disk erase using agent_ilo driver In-band disk erase using shred fails for agent_ilo driver as it tries to erase the virtual floppy device attached.This fix is to skip the virtual media devices and continue with other disks. Change-Id: I26745985382d440f7d4b3fbfffb14545067fcca6 Closes-Bug:#1450298
2015-04-29 22:27:46 +05:30
@mock.patch.object(hardware.GenericHardwareManager,
'_is_virtual_media_device', autospec=True)
def test_erase_block_device_virtual_media(self, vm_mock):
vm_mock.return_value = True
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
IPA:'shred' utility to use configured iterations Today, there is no option to configure number of iterations to be done for shred block device erasing and defaults it to 1. This patch adds a configuration option to change the number of passes to be done to erase a block device. Change-Id: I1921d33a6b364c4682b6c9baaf61ac092cfa11d7 Partial-Bug:#1465130
2015-06-15 21:36:27 +05:30
self.hardware.erase_block_device(self.node, block_device)
Fix inband disk erase using agent_ilo driver In-band disk erase using shred fails for agent_ilo driver as it tries to erase the virtual floppy device attached.This fix is to skip the virtual media devices and continue with other disks. Change-Id: I26745985382d440f7d4b3fbfffb14545067fcca6 Closes-Bug:#1450298
2015-04-29 22:27:46 +05:30
vm_mock.assert_called_once_with(self.hardware, block_device)
@mock.patch.object(os, 'readlink', autospec=True)
@mock.patch.object(os.path, 'exists', autospec=True)
def test__is_virtual_media_device_exists(self, mocked_exists,
mocked_link):
mocked_exists.return_value = True
mocked_link.return_value = '../../sda'
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
res = self.hardware._is_virtual_media_device(block_device)
self.assertTrue(res)
mocked_exists.assert_called_once_with('/dev/disk/by-label/ir-vfd-dev')
mocked_link.assert_called_once_with('/dev/disk/by-label/ir-vfd-dev')
@mock.patch.object(os, 'readlink', autospec=True)
@mock.patch.object(os.path, 'exists', autospec=True)
def test__is_virtual_media_device_exists_no_match(self, mocked_exists,
mocked_link):
mocked_exists.return_value = True
mocked_link.return_value = '../../sdb'
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
res = self.hardware._is_virtual_media_device(block_device)
self.assertFalse(res)
mocked_exists.assert_called_once_with('/dev/disk/by-label/ir-vfd-dev')
mocked_link.assert_called_once_with('/dev/disk/by-label/ir-vfd-dev')
@mock.patch.object(os, 'readlink', autospec=True)
@mock.patch.object(os.path, 'exists', autospec=True)
def test__is_virtual_media_device_path_doesnt_exist(self, mocked_exists,
Enforce all flake8 rules except E129 Bring ironic-python-agent in line with the other ironic projects. Stop ignoring all E12* errors except E129 Stop ignoring E711 Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
mocked_link):
Fix inband disk erase using agent_ilo driver In-band disk erase using shred fails for agent_ilo driver as it tries to erase the virtual floppy device attached.This fix is to skip the virtual media devices and continue with other disks. Change-Id: I26745985382d440f7d4b3fbfffb14545067fcca6 Closes-Bug:#1450298
2015-04-29 22:27:46 +05:30
mocked_exists.return_value = False
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
res = self.hardware._is_virtual_media_device(block_device)
self.assertFalse(res)
mocked_exists.assert_called_once_with('/dev/disk/by-label/ir-vfd-dev')
self.assertFalse(mocked_link.called)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Fix error in in-band disk erase using shred in-band disk erase using shred fails with error "'module' object has no attribute 'ProcessExecutionError'". This commit is to fix the issue. Change-Id: Ia0c426074b2f0e9d534ed96a3e213933160edc61 Closes-Bug:#144799
2015-04-24 15:08:41 +05:30
def test_erase_block_device_shred_fail_oserror(self, mocked_execute):
mocked_execute.side_effect = OSError
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
IPA:'shred' utility to use configured iterations Today, there is no option to configure number of iterations to be done for shred block device erasing and defaults it to 1. This patch adds a configuration option to change the number of passes to be done to erase a block device. Change-Id: I1921d33a6b364c4682b6c9baaf61ac092cfa11d7 Partial-Bug:#1465130
2015-06-15 21:36:27 +05:30
res = self.hardware._shred_block_device(self.node, block_device)
Fix error in in-band disk erase using shred in-band disk erase using shred fails with error "'module' object has no attribute 'ProcessExecutionError'". This commit is to fix the issue. Change-Id: Ia0c426074b2f0e9d534ed96a3e213933160edc61 Closes-Bug:#144799
2015-04-24 15:08:41 +05:30
self.assertFalse(res)
Enforce all flake8 rules except E129 Bring ironic-python-agent in line with the other ironic projects. Stop ignoring all E12* errors except E129 Stop ignoring E711 Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
mocked_execute.assert_called_once_with(
'shred', '--force', '--zero', '--verbose', '--iterations', '1',
'/dev/sda')
Fix error in in-band disk erase using shred in-band disk erase using shred fails with error "'module' object has no attribute 'ProcessExecutionError'". This commit is to fix the issue. Change-Id: Ia0c426074b2f0e9d534ed96a3e213933160edc61 Closes-Bug:#144799
2015-04-24 15:08:41 +05:30
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Fix error in in-band disk erase using shred in-band disk erase using shred fails with error "'module' object has no attribute 'ProcessExecutionError'". This commit is to fix the issue. Change-Id: Ia0c426074b2f0e9d534ed96a3e213933160edc61 Closes-Bug:#144799
2015-04-24 15:08:41 +05:30
def test_erase_block_device_shred_fail_processerror(self, mocked_execute):
mocked_execute.side_effect = processutils.ProcessExecutionError
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
IPA:'shred' utility to use configured iterations Today, there is no option to configure number of iterations to be done for shred block device erasing and defaults it to 1. This patch adds a configuration option to change the number of passes to be done to erase a block device. Change-Id: I1921d33a6b364c4682b6c9baaf61ac092cfa11d7 Partial-Bug:#1465130
2015-06-15 21:36:27 +05:30
res = self.hardware._shred_block_device(self.node, block_device)
Fix error in in-band disk erase using shred in-band disk erase using shred fails with error "'module' object has no attribute 'ProcessExecutionError'". This commit is to fix the issue. Change-Id: Ia0c426074b2f0e9d534ed96a3e213933160edc61 Closes-Bug:#144799
2015-04-24 15:08:41 +05:30
self.assertFalse(res)
Enforce all flake8 rules except E129 Bring ironic-python-agent in line with the other ironic projects. Stop ignoring all E12* errors except E129 Stop ignoring E711 Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
mocked_execute.assert_called_once_with(
'shred', '--force', '--zero', '--verbose', '--iterations', '1',
'/dev/sda')
Fix error in in-band disk erase using shred in-band disk erase using shred fails with error "'module' object has no attribute 'ProcessExecutionError'". This commit is to fix the issue. Change-Id: Ia0c426074b2f0e9d534ed96a3e213933160edc61 Closes-Bug:#144799
2015-04-24 15:08:41 +05:30
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
@mock.patch.object(utils, 'execute', autospec=True)
def test_erase_block_device_ata_security_unlock_fallback_pass(
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
self, mocked_execute, mocked_raid_member):
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
hdparm_output = create_hdparm_info(
supported=True, enabled=True, locked=True
)
hdparm_output_unlocked = create_hdparm_info(
supported=True, enabled=True, frozen=False, enhanced_erase=False)
hdparm_output_not_enabled = create_hdparm_info(
supported=True, enabled=False, frozen=False, enhanced_erase=False)
mocked_execute.side_effect = [
(hdparm_output, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
processutils.ProcessExecutionError(), # NULL fails to unlock
(hdparm_output, ''), # recheck security lines
None, # security unlock with ""
(hdparm_output_unlocked, ''),
'',
(hdparm_output_not_enabled, '')
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
self.hardware.erase_block_device(self.node, block_device)
mocked_execute.assert_any_call('hdparm', '--user-master', 'u',
'--security-unlock', '', '/dev/sda')
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware.GenericHardwareManager, '_shred_block_device',
autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
def test_erase_block_device_ata_security_enabled(
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
self, mocked_execute, mock_shred, mocked_raid_member):
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
# Tests that an exception is thrown if all of the recovery passwords
# fail to unlock the device without throwing exception
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output = create_hdparm_info(
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
supported=True, enabled=True, locked=True)
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
mocked_execute.side_effect = [
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
(hdparm_output, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
None,
(hdparm_output, ''),
None,
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
(hdparm_output, ''),
None,
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
(hdparm_output, '')
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
self.assertRaises(
errors.IncompatibleHardwareMethodError,
self.hardware.erase_block_device,
self.node,
block_device)
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
mocked_execute.assert_any_call('hdparm', '--user-master', 'u',
'--security-unlock', '', '/dev/sda')
mocked_execute.assert_any_call('hdparm', '--user-master', 'u',
'--security-unlock', 'NULL', '/dev/sda')
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
self.assertFalse(mock_shred.called)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware.GenericHardwareManager, '_shred_block_device',
autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
def test_erase_block_device_ata_security_enabled_unlock_attempt(
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
self, mocked_execute, mock_shred, mocked_raid_member):
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output = create_hdparm_info(
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
supported=True, enabled=True, locked=True)
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output_not_enabled = create_hdparm_info(
supported=True, enabled=False, frozen=False, enhanced_erase=False)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
mocked_execute.side_effect = [
(hdparm_output, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
'',
(hdparm_output_not_enabled, ''),
'',
'',
(hdparm_output_not_enabled, '')
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
self.hardware.erase_block_device(self.node, block_device)
self.assertFalse(mock_shred.called)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
def test__ata_erase_security_enabled_unlock_exception(
self, mocked_execute):
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
# test that an exception is thrown when security unlock fails with
# ProcessExecutionError
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output = create_hdparm_info(
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
supported=True, enabled=True, locked=True)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
mocked_execute.side_effect = [
(hdparm_output, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
processutils.ProcessExecutionError(),
(hdparm_output, ''),
processutils.ProcessExecutionError(),
(hdparm_output, ''),
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
]
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
self.assertRaises(errors.BlockDeviceEraseError,
self.hardware._ata_erase,
block_device)
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
mocked_execute.assert_any_call('hdparm', '--user-master', 'u',
'--security-unlock', '', '/dev/sda')
mocked_execute.assert_any_call('hdparm', '--user-master', 'u',
'--security-unlock', 'NULL', '/dev/sda')
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
def test__ata_erase_security_enabled_set_password_exception(
self, mocked_execute):
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output = create_hdparm_info(
supported=True, enabled=False, frozen=False, enhanced_erase=False)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
mocked_execute.side_effect = [
(hdparm_output, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
processutils.ProcessExecutionError()
]
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
self.assertRaises(errors.BlockDeviceEraseError,
self.hardware._ata_erase,
block_device)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
def test__ata_erase_security_erase_exec_exception(
self, mocked_execute):
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
# Exception on security erase
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output = create_hdparm_info(
supported=True, enabled=False, frozen=False, enhanced_erase=False)
Try to unlock failed device before proceeding When a hard error has occured with secure erase, we should attempt an unlock of the device becuase the current mode can prevent disk IO. This may upset some things like raid controllers even if they are in a pass-through mode. Change-Id: I32e1d962fbbb4a305d5dbebea92ac48ebd9b67ca Story: #2002546 Task: #22107
2018-06-13 11:58:27 -07:00
hdparm_unlocked_output = create_hdparm_info(
supported=True, locked=True, frozen=False, enhanced_erase=False)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
mocked_execute.side_effect = [
(hdparm_output, '', '-1'),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
'', # security-set-pass
Try to unlock failed device before proceeding When a hard error has occured with secure erase, we should attempt an unlock of the device becuase the current mode can prevent disk IO. This may upset some things like raid controllers even if they are in a pass-through mode. Change-Id: I32e1d962fbbb4a305d5dbebea92ac48ebd9b67ca Story: #2002546 Task: #22107
2018-06-13 11:58:27 -07:00
processutils.ProcessExecutionError(), # security-erase
(hdparm_unlocked_output, '', '-1'),
'', # attempt security unlock
(hdparm_output, '', '-1')
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
]
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
self.assertRaises(errors.BlockDeviceEraseError,
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
self.hardware._ata_erase,
block_device)
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware.GenericHardwareManager, '_shred_block_device',
autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
def test_erase_block_device_ata_frozen(self, mocked_execute, mock_shred,
mocked_raid_member):
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output = create_hdparm_info(
supported=True, enabled=False, frozen=True, enhanced_erase=False)
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
mocked_execute.side_effect = [
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(hdparm_output, ''),
(SMARTCTL_NORMAL_OUTPUT, '')
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
self.assertRaises(
errors.IncompatibleHardwareMethodError,
self.hardware.erase_block_device,
self.node,
block_device)
self.assertFalse(mock_shred.called)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware.GenericHardwareManager, '_shred_block_device',
autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
def test_erase_block_device_ata_failed(self, mocked_execute, mock_shred,
mocked_raid_member):
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output_before = create_hdparm_info(
supported=True, enabled=False, frozen=False, enhanced_erase=False)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
# If security mode remains enabled after the erase, it is indicative
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
# of a failed erase.
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output_after = create_hdparm_info(
supported=True, enabled=True, frozen=False, enhanced_erase=False)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
mocked_execute.side_effect = [
(hdparm_output_before, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
('', ''),
('', ''),
(hdparm_output_after, ''),
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
self.assertRaises(
errors.IncompatibleHardwareMethodError,
self.hardware.erase_block_device,
self.node,
block_device)
self.assertFalse(mock_shred.called)
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(hardware.GenericHardwareManager, '_shred_block_device',
autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
def test_erase_block_device_ata_failed_continued(
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
self, mocked_execute, mock_shred, mocked_raid_member):
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
Replace dict.get(key) with dict[key] in tests This replaces dict.get(key) with dict[key] in the unit tests, since we should be explicit about whether we expect the key to exist or not. (These are tests, after all.) Change-Id: Ide98ceba325214e55cab2063d9ace150d63f9617 Closes-Bug: #1590808
2016-06-23 17:41:23 -04:00
info = self.node['driver_internal_info']
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
info['agent_continue_if_ata_erase_failed'] = True
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output_before = create_hdparm_info(
supported=True, enabled=False, frozen=False, enhanced_erase=False)
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
# If security mode remains enabled after the erase, it is indicative
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
# of a failed erase.
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
hdparm_output_after = create_hdparm_info(
supported=True, enabled=True, frozen=False, enhanced_erase=False)
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
mocked_execute.side_effect = [
(hdparm_output_before, ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
('', ''),
('', ''),
(hdparm_output_after, ''),
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Add a HardwareManager method to erase devices Add erase_devices method to the HardwareManager class. By default this method iterates block devices, and calls a new abstract erase_block_device method for each device. This patch includes a simple implementation of erase_block_device on the GenericHardwareManager which attempts to issue an ATA secure erase on supported devices. Change-Id: I81da065395b8785f636f1b0a0d60c9f1c045441e
2014-06-04 10:44:25 -07:00
Improve Disk Detection The previous implementation of list_block_devices used blockdev, which would list partitions, software RAID and other devices as block devices. By switching to lsblk, the agent can filter down to only physical block devices, which is all the agent cares about for any of its operations. This change adds two new fields to the BlockDevice class: model, a string of the block devices reported model, and rotational, a boolean representing a spinning disk (True) or a solid state disk (False). This data can be useful for vendor hardware managers. Change-Id: I385c3bb378c2c49385bca14a1d7efa074933becf Closes-Bug: 1344351
2014-07-18 08:13:12 -07:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
Provide fallback from ATA erase to shredding Presently should the ATA erasure operation fails, IPA halts the cleaning process and the node goes to CLEANFAIL state as a result. This failure could be the result of a previous cleaning failure that left drive security enabled, for which code has been added in an attempt to address this case by attempting to unlock the the drive. In the event that an operator wishes to automatically fallback to disk scrubbing operations, the capability has been added through a driver_internal_info field "agent_continue_if_ata_erase_failed" that can be set to True, however defaults to False keeping the same behavior that IPA presently exhibits in the event of ATA erase operations failing. Partial-Bug: #1536695 Change-Id: I88edd9477f4f05aa55b2fe8efa4bbff1c5573bb1
2016-01-21 11:26:44 -05:00
self.hardware.erase_block_device(self.node, block_device)
self.assertTrue(mock_shred.called)
Added support for secure erase enhanced Added support for secure erase enhanced option when doing an ATA secure erase. Also made it prefer the enhanced version if hdparm claims the device supports it. Change-Id: I2ce04d0b967d6c176acd40ea10a05bcab5dbf2d0
2015-02-05 17:03:33 -06:00
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Provide knob to disable ata secure erase We need to allow the operator to able to explicitly disable secure erase, in case it is problematic in their environment or hardware. Change-Id: I4c68efa65cdd7f88f54f8dd9a8bcbeee9e8124a8 Story: #2002546 Task: #22108
2018-06-13 12:20:22 -07:00
@mock.patch.object(hardware.GenericHardwareManager, '_shred_block_device',
autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_erase_block_device_ata_erase_disabled(
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
self, mocked_execute, mock_shred, mocked_raid_member):
Provide knob to disable ata secure erase We need to allow the operator to able to explicitly disable secure erase, in case it is problematic in their environment or hardware. Change-Id: I4c68efa65cdd7f88f54f8dd9a8bcbeee9e8124a8 Story: #2002546 Task: #22108
2018-06-13 12:20:22 -07:00
info = self.node['driver_internal_info']
info['agent_enable_ata_secure_erase'] = False
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Provide knob to disable ata secure erase We need to allow the operator to able to explicitly disable secure erase, in case it is problematic in their environment or hardware. Change-Id: I4c68efa65cdd7f88f54f8dd9a8bcbeee9e8124a8 Story: #2002546 Task: #22108
2018-06-13 12:20:22 -07:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
self.hardware.erase_block_device(self.node, block_device)
self.assertTrue(mock_shred.called)
self.assertFalse(mocked_execute.called)
Added support for secure erase enhanced Added support for secure erase enhanced option when doing an ATA secure erase. Also made it prefer the enhanced version if hdparm claims the device supports it. Change-Id: I2ce04d0b967d6c176acd40ea10a05bcab5dbf2d0
2015-02-05 17:03:33 -06:00
def test_normal_vs_enhanced_security_erase(self):
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Added support for secure erase enhanced Added support for secure erase enhanced option when doing an ATA secure erase. Also made it prefer the enhanced version if hdparm claims the device supports it. Change-Id: I2ce04d0b967d6c176acd40ea10a05bcab5dbf2d0
2015-02-05 17:03:33 -06:00
def test_security_erase_option(test_case,
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
enhanced_erase,
Added support for secure erase enhanced Added support for secure erase enhanced option when doing an ATA secure erase. Also made it prefer the enhanced version if hdparm claims the device supports it. Change-Id: I2ce04d0b967d6c176acd40ea10a05bcab5dbf2d0
2015-02-05 17:03:33 -06:00
expected_option,
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_execute,
mocked_raid_member):
Added support for secure erase enhanced Added support for secure erase enhanced option when doing an ATA secure erase. Also made it prefer the enhanced version if hdparm claims the device supports it. Change-Id: I2ce04d0b967d6c176acd40ea10a05bcab5dbf2d0
2015-02-05 17:03:33 -06:00
mocked_execute.side_effect = [
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
(create_hdparm_info(
supported=True, enabled=False, frozen=False,
enhanced_erase=enhanced_erase), ''),
Refuse secure erase if ATA command does not work Adds dependency upon smartmontools's binary smartctl to query the block devices via ATA mode which fails on pass-thru buses such as ATA over SCSI and ATA over USB, in an effort to prevent the initiation of ATA secure erase with one of these interfaces in place which may render the disk unreachable after security options are enabled for ATA Secure Erase or upon the Secure Erase command being sent to the Hard Disk. Change-Id: I7635a197eb000650e919fac386b38ac15ef17041 Story: #2002546 Task: #22109 Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-13 12:13:20 -07:00
(SMARTCTL_NORMAL_OUTPUT, ''),
Added support for secure erase enhanced Added support for secure erase enhanced option when doing an ATA secure erase. Also made it prefer the enhanced version if hdparm claims the device supports it. Change-Id: I2ce04d0b967d6c176acd40ea10a05bcab5dbf2d0
2015-02-05 17:03:33 -06:00
('', ''),
('', ''),
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
(create_hdparm_info(
supported=True, enabled=False, frozen=False,
enhanced_erase=enhanced_erase), ''),
Added support for secure erase enhanced Added support for secure erase enhanced option when doing an ATA secure erase. Also made it prefer the enhanced version if hdparm claims the device supports it. Change-Id: I2ce04d0b967d6c176acd40ea10a05bcab5dbf2d0
2015-02-05 17:03:33 -06:00
]
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mocked_raid_member.return_value = False
Added support for secure erase enhanced Added support for secure erase enhanced option when doing an ATA secure erase. Also made it prefer the enhanced version if hdparm claims the device supports it. Change-Id: I2ce04d0b967d6c176acd40ea10a05bcab5dbf2d0
2015-02-05 17:03:33 -06:00
block_device = hardware.BlockDevice('/dev/sda', 'big', 1073741824,
True)
IPA:'shred' utility to use configured iterations Today, there is no option to configure number of iterations to be done for shred block device erasing and defaults it to 1. This patch adds a configuration option to change the number of passes to be done to erase a block device. Change-Id: I1921d33a6b364c4682b6c9baaf61ac092cfa11d7 Partial-Bug:#1465130
2015-06-15 21:36:27 +05:30
test_case.hardware.erase_block_device(self.node, block_device)
Added support for secure erase enhanced Added support for secure erase enhanced option when doing an ATA secure erase. Also made it prefer the enhanced version if hdparm claims the device supports it. Change-Id: I2ce04d0b967d6c176acd40ea10a05bcab5dbf2d0
2015-02-05 17:03:33 -06:00
mocked_execute.assert_any_call('hdparm', '--user-master', 'u',
expected_option,
'NULL', '/dev/sda')
Enforce all flake8 rules except E129 Bring ironic-python-agent in line with the other ironic projects. Stop ignoring all E12* errors except E129 Stop ignoring E711 Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
test_security_erase_option(
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
self, True, '--security-erase-enhanced')
Enforce all flake8 rules except E129 Bring ironic-python-agent in line with the other ironic projects. Stop ignoring all E12* errors except E129 Stop ignoring E711 Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
test_security_erase_option(
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
self, False, '--security-erase')
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(utils, 'execute', autospec=True)
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_virtual_media_device', autospec=True)
@mock.patch.object(hardware.GenericHardwareManager,
'list_block_devices', autospec=True)
@mock.patch.object(disk_utils, 'destroy_disk_metadata', autospec=True)
def test_erase_devices_metadata(
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
self, mock_metadata, mock_list_devs, mock__is_vmedia,
mock_execute):
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
block_devices = [
hardware.BlockDevice('/dev/sr0', 'vmedia', 12345, True),
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
hardware.BlockDevice('/dev/sdb2', 'raid-member', 32767, False),
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
hardware.BlockDevice('/dev/sda', 'small', 65535, False),
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
hardware.BlockDevice('/dev/sda1', '', 32767, False),
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
hardware.BlockDevice('/dev/sda2', 'raid-member', 32767, False),
hardware.BlockDevice('/dev/md0', 'raid-device', 32767, False)
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
]
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
# NOTE(coreywright): Don't return the list, but a copy of it, because
# we depend on its elements' order when referencing it later during
# verification, but the method under test sorts the list changing it.
mock_list_devs.return_value = list(block_devices)
mock__is_vmedia.side_effect = lambda _, dev: dev.name == '/dev/sr0'
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mock_execute.side_effect = [
('sdb2 linux_raid_member host:1 f9978968', ''),
('sda2 linux_raid_member host:1 f9978969', ''),
('sda1', ''), ('sda', ''), ('md0', '')]
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
self.hardware.erase_devices_metadata(self.node, [])
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
self.assertEqual([mock.call('/dev/sda1', self.node['uuid']),
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mock.call('/dev/sda', self.node['uuid']),
mock.call('/dev/md0', self.node['uuid'])],
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
mock_metadata.call_args_list)
mock_list_devs.assert_called_once_with(self.hardware,
include_partitions=True)
self.assertEqual([mock.call(self.hardware, block_devices[0]),
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mock.call(self.hardware, block_devices[1]),
mock.call(self.hardware, block_devices[4]),
mock.call(self.hardware, block_devices[3]),
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
mock.call(self.hardware, block_devices[2]),
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mock.call(self.hardware, block_devices[5])],
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
mock__is_vmedia.call_args_list)
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_linux_raid_member', autospec=True)
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
@mock.patch.object(hardware.GenericHardwareManager,
'_is_virtual_media_device', autospec=True)
@mock.patch.object(hardware.GenericHardwareManager,
'list_block_devices', autospec=True)
@mock.patch.object(disk_utils, 'destroy_disk_metadata', autospec=True)
def test_erase_devices_metadata_error(
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
self, mock_metadata, mock_list_devs, mock__is_vmedia,
mock__is_raid_member):
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
block_devices = [
hardware.BlockDevice('/dev/sda', 'small', 65535, False),
hardware.BlockDevice('/dev/sdb', 'big', 10737418240, True),
]
mock__is_vmedia.return_value = False
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
mock__is_raid_member.return_value = False
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
# NOTE(coreywright): Don't return the list, but a copy of it, because
# we depend on its elements' order when referencing it later during
# verification, but the method under test sorts the list changing it.
mock_list_devs.return_value = list(block_devices)
# Simulate first call to destroy_disk_metadata() failing, which is for
# /dev/sdb due to erase_devices_metadata() reverse sorting block
# devices by name, and second call succeeding, which is for /dev/sda
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
error_output = 'Booo00000ooommmmm'
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
error_regex = '(?s)/dev/sdb.*' + error_output
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
mock_metadata.side_effect = (
processutils.ProcessExecutionError(error_output),
None,
)
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
self.assertRaisesRegex(errors.BlockDeviceEraseError, error_regex,
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
self.hardware.erase_devices_metadata,
self.node, [])
# Assert all devices are erased independent if one of them
# failed previously
Allow erasing metadata from disk partitions Modify the metadata erasing call chain to retrieve a list of devices that includes partitions in addition to disks so it can erase metadata from all of them, otherwise incidentally recreating disk partitions causes the Linux kernel to discover and automatically recreate some types of storage entities (eg LVM PVs, VGs, & LVs, RAID members & devices). Change-Id: If8f47a083966051856439e3291a6872929b93e3b Story: #2003673 Task: #26192
2018-08-29 22:57:02 -05:00
self.assertEqual([mock.call('/dev/sdb', self.node['uuid']),
mock.call('/dev/sda', self.node['uuid'])],
mock_metadata.call_args_list)
mock_list_devs.assert_called_once_with(self.hardware,
include_partitions=True)
self.assertEqual([mock.call(self.hardware, block_devices[1]),
mock.call(self.hardware, block_devices[0])],
mock__is_vmedia.call_args_list)
Add erase_devices_metadata cleaning step This patch is adding a new cleaning step called "erase_devices_metadata" to the GenericHardwareManager. This step is responsible for erasing the metadata of the disks present in the node (partition tables, signatures, filesystem identifiers etc...). It's important to note that the "erase_devices" cleaning step will also remove all these metadatas (because it will zero/shred the whole disk) but, it takes a lot of time to run and for some usages of Ironic only cleaning the device metadata and leaving the data from previous tenants on the disk after the machine is recycled is fine. That's the use case for systems using Ironic just to install the same base image onto many nodes which will run another cloud on top afterwards (TripleO). The new cleaning step has a default priority of 99, so it should run before the "erase_devices" cleaning step so that we can guarantee that the metadata was removed even in case of a failure when cleaning the disks. The version of the GenericHardwareManager was bumped to "1.1" with the addition of this new clean step. This patch make use of the "destroy_disk_metadata" method from ironic-lib to get rid of the metadata. Closes-Bug: #1603411 Change-Id: I3d7b39d5ee3e03ce63185e4168b1ac954a896c93
2016-07-18 17:18:14 +01:00
Software RAID: Protect RAID metadata upon cleaning During automatic cleaning, the generic hardware manager erases the data and the metadata on all block devices. If this is done on RAID members, the RAID device is destroyed. This patch proposes to exclude RAID members from (meta)data cleaning. Change-Id: I9ec7674925d981062c311cf34eadfe4cb891f077 Story: #2004581 Task: #33640
2019-06-11 09:33:59 +02:00
@mock.patch.object(utils, 'execute', autospec=True)
def test__is_linux_raid_member(self, mocked_execute):
raid_member = hardware.BlockDevice('/dev/sda1', 'small', 65535, False)
mocked_execute.return_value = ('linux_raid_member host.domain:0 '
'85fa41e4-e0ae'), ''
self.assertTrue(self.hardware._is_linux_raid_member(raid_member))
@mock.patch.object(utils, 'execute', autospec=True)
def test__is_linux_raid_member_false(self, mocked_execute):
raid_member = hardware.BlockDevice('/dev/md0', 'small', 65535, False)
mocked_execute.return_value = 'md0', ''
self.assertFalse(self.hardware._is_linux_raid_member(raid_member))
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
def test_get_bmc_address(self, mocked_execute):
mocked_execute.return_value = '192.1.2.3\n', ''
self.assertEqual('192.1.2.3', self.hardware.get_bmc_address())
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Extend hardware manager with data needed for inspector * Added NetworkInterface.ip4_address * Added HardwareManager.get_bmc_address() * Added Memory.physical_mb This is total memory as reported by dmidecode, and yes, it's different from total, as it includes kernel reserved space. * Added CPU.architecture As a side effect, get_cpus was switched to lscpu. Also fixes problem when get_cpus reported the current frequency instead of maximum one. Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-06 13:03:27 +02:00
def test_get_bmc_address_virt(self, mocked_execute):
mocked_execute.side_effect = processutils.ProcessExecutionError()
self.assertIsNone(self.hardware.get_bmc_address())
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
BMC can be configured using different lan channel It may happen that BMC is configured to use non-zero channel. In this case we should iterate across all of them as long as we get a correct IP address (in this case different than "0.0.0.0" which is a placeholder for not configured console). Change-Id: I3c351af1882b24c8f56e4363249b19b5c3a4a446 Closes-Bug: #1702514
2017-07-05 18:20:43 +02:00
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_address_zeroed(self, mocked_execute):
mocked_execute.return_value = '0.0.0.0\n', ''
self.assertEqual('0.0.0.0', self.hardware.get_bmc_address())
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_address_invalid(self, mocked_execute):
# In case of invalid lan channel, stdout is empty and the error
# on stderr is "Invalid channel"
mocked_execute.return_value = '\n', 'Invalid channel: 55'
self.assertEqual('0.0.0.0', self.hardware.get_bmc_address())
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_address_random_error(self, mocked_execute):
mocked_execute.return_value = '192.1.2.3\n', 'Random error message'
self.assertEqual('192.1.2.3', self.hardware.get_bmc_address())
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_address_iterate_channels(self, mocked_execute):
# For channel 1 we simulate unconfigured IP
# and for any other we return a correct IP address
def side_effect(*args, **kwargs):
if args[0].startswith("ipmitool lan print 1"):
Remove assumption that a valid IPMI channel cannot follow an invalid one It seems to be incorrect at least for some iLO machines. Also harden the code against invalid output from ipmitool. Change-Id: I733785e9c7d86eadca963f0776910504bf91bcfe Closes-Bug: #1714944
2017-09-11 16:08:33 +02:00
return '', 'Invalid channel 1\n'
elif args[0].startswith("ipmitool lan print 2"):
BMC can be configured using different lan channel It may happen that BMC is configured to use non-zero channel. In this case we should iterate across all of them as long as we get a correct IP address (in this case different than "0.0.0.0" which is a placeholder for not configured console). Change-Id: I3c351af1882b24c8f56e4363249b19b5c3a4a446 Closes-Bug: #1702514
2017-07-05 18:20:43 +02:00
return '0.0.0.0\n', ''
Remove assumption that a valid IPMI channel cannot follow an invalid one It seems to be incorrect at least for some iLO machines. Also harden the code against invalid output from ipmitool. Change-Id: I733785e9c7d86eadca963f0776910504bf91bcfe Closes-Bug: #1714944
2017-09-11 16:08:33 +02:00
elif args[0].startswith("ipmitool lan print 3"):
return 'meow', ''
BMC can be configured using different lan channel It may happen that BMC is configured to use non-zero channel. In this case we should iterate across all of them as long as we get a correct IP address (in this case different than "0.0.0.0" which is a placeholder for not configured console). Change-Id: I3c351af1882b24c8f56e4363249b19b5c3a4a446 Closes-Bug: #1702514
2017-07-05 18:20:43 +02:00
else:
return '192.1.2.3\n', ''
mocked_execute.side_effect = side_effect
self.assertEqual('192.1.2.3', self.hardware.get_bmc_address())
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_address_not_available(self, mocked_execute):
mocked_execute.return_value = '', ''
self.assertEqual('0.0.0.0', self.hardware.get_bmc_address())
Discover IPv6 BMC address Change-Id: Ia7f24c1df782de4809a5195876bbf0309b481fd7 Story: #2005133 Task: #29817
2019-03-04 15:31:14 +03:00
@mock.patch.object(utils, 'try_execute', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_v6address_not_enabled(self, mocked_execute, mte):
Add more channel number for detecting BMC IPv6 address This is a missing fix for patch.[1] [1] https://review.opendev.org/#/c/656275/ Change-Id: I8443ae0c08487afea383e111b6c2b6d5941d4024 Task: 30654 Story: 2005528
2019-05-20 11:08:31 +00:00
mocked_execute.side_effect = [('ipv4\n', '')] * 11
Discover IPv6 BMC address Change-Id: Ia7f24c1df782de4809a5195876bbf0309b481fd7 Story: #2005133 Task: #29817
2019-03-04 15:31:14 +03:00
self.assertEqual('::/0', self.hardware.get_bmc_v6address())
@mock.patch.object(utils, 'try_execute', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_v6address_dynamic_address(self, mocked_execute, mte):
mocked_execute.side_effect = [
('ipv6\n', ''),
(IPMITOOL_LAN6_PRINT_DYNAMIC_ADDR, '')
]
self.assertEqual('2001:1234:1234:1234:1234:1234:1234:1234',
self.hardware.get_bmc_v6address())
@mock.patch.object(utils, 'try_execute', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_v6address_static_address_both(self, mocked_execute, mte):
dynamic_disabled = \
IPMITOOL_LAN6_PRINT_DYNAMIC_ADDR.replace('active', 'disabled')
mocked_execute.side_effect = [
('both\n', ''),
(dynamic_disabled, ''),
(IPMITOOL_LAN6_PRINT_STATIC_ADDR, '')
]
self.assertEqual('2001:5678:5678:5678:5678:5678:5678:5678',
self.hardware.get_bmc_v6address())
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_v6address_virt(self, mocked_execute):
mocked_execute.side_effect = processutils.ProcessExecutionError()
self.assertIsNone(self.hardware.get_bmc_v6address())
@mock.patch.object(utils, 'try_execute', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_v6address_invalid_enables(self, mocked_execute, mte):
def side_effect(*args, **kwargs):
if args[0].startswith('ipmitool lan6 print'):
return '', 'Failed to get IPv6/IPv4 Addressing Enables'
mocked_execute.side_effect = side_effect
self.assertEqual('::/0', self.hardware.get_bmc_v6address())
@mock.patch.object(utils, 'try_execute', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_v6address_invalid_get_address(self, mocked_execute, mte):
def side_effect(*args, **kwargs):
if args[0].startswith('ipmitool lan6 print'):
if args[0].endswith('dynamic_addr') \
or args[0].endswith('static_addr'):
raise processutils.ProcessExecutionError()
return 'ipv6', ''
mocked_execute.side_effect = side_effect
self.assertEqual('::/0', self.hardware.get_bmc_v6address())
@mock.patch.object(hardware, 'LOG', autospec=True)
@mock.patch.object(utils, 'try_execute', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
Add more channel number for detecting BMC IPv6 address This is a missing fix for patch.[1] [1] https://review.opendev.org/#/c/656275/ Change-Id: I8443ae0c08487afea383e111b6c2b6d5941d4024 Task: 30654 Story: 2005528
2019-05-20 11:08:31 +00:00
def test_get_bmc_v6address_ipmitool_invalid_stdout_format(
Discover IPv6 BMC address Change-Id: Ia7f24c1df782de4809a5195876bbf0309b481fd7 Story: #2005133 Task: #29817
2019-03-04 15:31:14 +03:00
self, mocked_execute, mte, mocked_log):
def side_effect(*args, **kwargs):
if args[0].startswith('ipmitool lan6 print'):
if args[0].endswith('dynamic_addr') \
or args[0].endswith('static_addr'):
return 'Invalid\n\tyaml', ''
return 'ipv6', ''
mocked_execute.side_effect = side_effect
self.assertEqual('::/0', self.hardware.get_bmc_v6address())
one_call = mock.call('Cannot process output of "%(cmd)s" '
'command: %(e)s', mock.ANY)
mocked_log.warning.assert_has_calls([one_call] * 14)
@mock.patch.object(utils, 'try_execute', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_bmc_v6address_channel_7(self, mocked_execute, mte):
def side_effect(*args, **kwargs):
if not args[0].startswith('ipmitool lan6 print 7'):
# ipv6 is not enabled for channels 1-6
if 'enables |' in args[0]:
return '', ''
else:
if 'enables |' in args[0]:
return 'ipv6', ''
if args[0].endswith('dynamic_addr'):
raise processutils.ProcessExecutionError()
elif args[0].endswith('static_addr'):
return IPMITOOL_LAN6_PRINT_STATIC_ADDR, ''
mocked_execute.side_effect = side_effect
self.assertEqual('2001:5678:5678:5678:5678:5678:5678:5678',
self.hardware.get_bmc_v6address())
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
@mock.patch.object(utils, 'execute', autospec=True)
def test_validate_configuration_no_configuration(self, mocked_execute):
self.assertRaises(errors.SoftwareRAIDError,
self.hardware.validate_configuration,
self.node, [])
@mock.patch.object(utils, 'execute', autospec=True)
def test_create_configuration(self, mocked_execute):
raid_config = {
"logical_disks": [
{
"size_gb": "100",
"raid_level": "1",
"controller": "software",
},
{
"size_gb": "MAX",
"raid_level": "0",
"controller": "software",
},
]
}
self.node['target_raid_config'] = raid_config
device1 = hardware.BlockDevice('/dev/sda', 'sda', 1073741824, True)
device2 = hardware.BlockDevice('/dev/sdb', 'sdb', 1073741824, True)
self.hardware.list_block_devices = mock.Mock()
self.hardware.list_block_devices.return_value = [device1, device2]
result = self.hardware.create_configuration(self.node, [])
mocked_execute.assert_has_calls([
mock.call('parted', '/dev/sda', '-s', '--', 'mklabel', 'msdos'),
mock.call('parted', '/dev/sdb', '-s', '--', 'mklabel', 'msdos'),
mock.call('parted', '/dev/sda', '-s', '-a', 'optimal', '--',
'mkpart', 'primary', '2048s', 102400),
mock.call('parted', '/dev/sdb', '-s', '-a', 'optimal', '--',
'mkpart', 'primary', '2048s', 102400),
mock.call('parted', '/dev/sda', '-s', '-a', 'optimal', '--',
'mkpart', 'primary', 102400, '-1'),
mock.call('parted', '/dev/sdb', '-s', '-a', 'optimal', '--',
'mkpart', 'primary', 102400, '-1'),
Software RAID: Fix shell commands Pass an array to utils.execute() or enforce string usage via 'shell=True', but don't mix. Follow-up to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: I2e6c628360aecf81039089af78b19fe6a956e564
2019-06-05 16:42:37 +02:00
mock.call('mdadm', '--create', '/dev/md0', '--force', '--run',
'--metadata=1', '--level', '1', '--raid-devices', 2,
'/dev/sda1', '/dev/sdb1'),
mock.call('mdadm', '--create', '/dev/md1', '--force', '--run',
'--metadata=1', '--level', '0', '--raid-devices', 2,
'/dev/sda2', '/dev/sdb2')])
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
self.assertEqual(raid_config, result)
@mock.patch.object(utils, 'execute', autospec=True)
def test_create_configuration_invalid_raid_config(self, mocked_execute):
raid_config = {
"logical_disks": [
{
"size_gb": "MAX",
"raid_level": "1",
"controller": "software",
},
{
"size_gb": "MAX",
"raid_level": "0",
"controller": "software",
},
]
}
self.node['target_raid_config'] = raid_config
self.assertRaises(errors.SoftwareRAIDError,
self.hardware.create_configuration,
self.node, [])
@mock.patch.object(utils, 'execute', autospec=True)
def test_create_configuration_partitions_detected(self, mocked_execute):
raid_config = {
"logical_disks": [
{
"size_gb": "100",
"raid_level": "1",
"controller": "software",
},
{
"size_gb": "MAX",
"raid_level": "0",
"controller": "software",
},
]
}
self.node['target_raid_config'] = raid_config
device1 = hardware.BlockDevice('/dev/sda', 'sda', 1073741824, True)
device2 = hardware.BlockDevice('/dev/sdb', 'sdb', 1073741824, True)
partition1 = hardware.BlockDevice('/dev/sdb1', 'sdb1', 268435456, True)
self.hardware.list_block_devices = mock.Mock()
self.hardware.list_block_devices.side_effect = [
[device1, device2],
[device1, device2, partition1]]
self.assertRaises(errors.SoftwareRAIDError,
self.hardware.create_configuration,
self.node, [])
@mock.patch.object(utils, 'execute', autospec=True)
def test_create_configuration_device_handling_failures(self,
mocked_execute):
raid_config = {
"logical_disks": [
{
"size_gb": "100",
"raid_level": "1",
"controller": "software",
},
{
"size_gb": "MAX",
"raid_level": "0",
"controller": "software",
},
]
}
self.node['target_raid_config'] = raid_config
device1 = hardware.BlockDevice('/dev/sda', 'sda', 1073741824, True)
device2 = hardware.BlockDevice('/dev/sdb', 'sdb', 1073741824, True)
self.hardware.list_block_devices = mock.Mock()
self.hardware.list_block_devices.side_effect = [
[device1, device2],
[device1, device2],
[device1, device2],
[device1, device2],
[device1, device2],
[device1, device2]]
# partition table creation
error_regex = "Failed to create partition table on /dev/sda"
mocked_execute.side_effect = [
processutils.ProcessExecutionError]
self.assertRaisesRegex(errors.SoftwareRAIDError, error_regex,
self.hardware.create_configuration,
self.node, [])
# partition creation
error_regex = "Failed to create partitions on /dev/sda"
mocked_execute.side_effect = [
None, None, # partition tables on sd{a,b}
processutils.ProcessExecutionError]
self.assertRaisesRegex(errors.SoftwareRAIDError, error_regex,
self.hardware.create_configuration,
self.node, [])
# raid device creation
error_regex = ("Failed to create md device /dev/md0 "
"on /dev/sda1 /dev/sdb1")
mocked_execute.side_effect = [
None, None, # partition tables on sd{a,b}
None, None, # RAID-1 partitions on sd{a,b}
None, None, # RAID-N partitions on sd{a,b}
processutils.ProcessExecutionError]
self.assertRaisesRegex(errors.SoftwareRAIDError, error_regex,
self.hardware.create_configuration,
self.node, [])
@mock.patch.object(utils, 'execute', autospec=True)
def test__get_component_devices(self, mocked_execute):
mocked_execute.side_effect = [(MDADM_DETAIL_OUTPUT, '')]
raid_device = hardware.BlockDevice('/dev/md0', 'RAID-1',
1073741824, True)
component_devices = hardware._get_component_devices(raid_device.name)
self.assertEqual(['/dev/vde1', '/dev/vdf1'], component_devices)
Software RAID: Detect component & holder disks in broken RAIDs This patch amends the component and holder disk detection code to be able to also extract information from inactive/broken md devices. This may be required in case of broken RAID-0 (which will show up as an empty inactive device after a reboot which none the less keeps its spare devices 'in-use' and prevents RAID re-creation). Story: #2006215 Task: #35808 Change-Id: I4ce41b335843e18c67668192d2fccff1c6cdfb92
2019-07-12 14:50:14 +02:00
@mock.patch.object(utils, 'execute', autospec=True)
def test__get_component_devices_broken_raid0(self, mocked_execute):
mocked_execute.side_effect = [(MDADM_DETAIL_OUTPUT_BROKEN_RAID0, '')]
raid_device = hardware.BlockDevice('/dev/md126', 'RAID-0',
1073741824, True)
component_devices = hardware._get_component_devices(raid_device.name)
self.assertEqual(['/dev/sda2'], component_devices)
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
@mock.patch.object(utils, 'execute', autospec=True)
Software RAID: Add IPA deploy support This patch proposes to extend the IPA to be able to deploy on software RAID devices. The IPA needs to be able to detect an md device, find the underlying holder disks and configure grub on them. Change-Id: Ieb2c95ff130b5cc1e643fcde500066d9458ddbec Story: #2004581 Task: #29102
2019-02-26 17:56:20 +01:00
def test_get_holder_disks(self, mocked_execute):
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
mocked_execute.side_effect = [(MDADM_DETAIL_OUTPUT, '')]
raid_device = hardware.BlockDevice('/dev/md0', 'RAID-1',
1073741824, True)
Software RAID: Add IPA deploy support This patch proposes to extend the IPA to be able to deploy on software RAID devices. The IPA needs to be able to detect an md device, find the underlying holder disks and configure grub on them. Change-Id: Ieb2c95ff130b5cc1e643fcde500066d9458ddbec Story: #2004581 Task: #29102
2019-02-26 17:56:20 +01:00
holder_disks = hardware.get_holder_disks(raid_device.name)
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
self.assertEqual(['/dev/vde', '/dev/vdf'], holder_disks)
Software RAID: Detect component & holder disks in broken RAIDs This patch amends the component and holder disk detection code to be able to also extract information from inactive/broken md devices. This may be required in case of broken RAID-0 (which will show up as an empty inactive device after a reboot which none the less keeps its spare devices 'in-use' and prevents RAID re-creation). Story: #2006215 Task: #35808 Change-Id: I4ce41b335843e18c67668192d2fccff1c6cdfb92
2019-07-12 14:50:14 +02:00
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_holder_disks_broken_raid0(self, mocked_execute):
mocked_execute.side_effect = [(MDADM_DETAIL_OUTPUT_BROKEN_RAID0, '')]
raid_device = hardware.BlockDevice('/dev/md126', 'RAID-0',
1073741824, True)
holder_disks = hardware.get_holder_disks(raid_device.name)
self.assertEqual(['/dev/sda'], holder_disks)
Software RAID: Fix race in test for get_holder_disks The explicit creation of a mock object in one test disturbed the execution of another test (which mock'ed the same object). Use a decorator instead. Followup to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: Id54d0ade7084363cb3e53752205db9a9d99d0195
2019-06-07 14:22:38 +02:00
@mock.patch.object(hardware, 'get_holder_disks', autospec=True)
@mock.patch.object(hardware, '_get_component_devices', autospec=True)
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
@mock.patch.object(hardware, 'list_all_block_devices', autospec=True)
@mock.patch.object(utils, 'execute', autospec=True)
Software RAID: Fix race in test for get_holder_disks The explicit creation of a mock object in one test disturbed the execution of another test (which mock'ed the same object). Use a decorator instead. Followup to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: Id54d0ade7084363cb3e53752205db9a9d99d0195
2019-06-07 14:22:38 +02:00
def test_delete_configuration(self, mocked_execute, mocked_list,
mocked_get_component, mocked_get_holder):
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
raid_device1 = hardware.BlockDevice('/dev/md0', 'RAID-1',
1073741824, True)
raid_device2 = hardware.BlockDevice('/dev/md1', 'RAID-0',
2147483648, True)
hardware.list_all_block_devices.side_effect = [
[raid_device1, raid_device2]]
Software RAID: Fix race in test for get_holder_disks The explicit creation of a mock object in one test disturbed the execution of another test (which mock'ed the same object). Use a decorator instead. Followup to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: Id54d0ade7084363cb3e53752205db9a9d99d0195
2019-06-07 14:22:38 +02:00
mocked_get_component.side_effect = [
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
["/dev/sda1", "/dev/sda2"],
["/dev/sdb1", "/dev/sdb2"]]
Software RAID: Fix race in test for get_holder_disks The explicit creation of a mock object in one test disturbed the execution of another test (which mock'ed the same object). Use a decorator instead. Followup to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: Id54d0ade7084363cb3e53752205db9a9d99d0195
2019-06-07 14:22:38 +02:00
mocked_get_holder.side_effect = [
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
["/dev/sda", "/dev/sdb"],
["/dev/sda", "/dev/sdb"]]
mocked_execute.side_effect = [
None, None, None,
['_', 'mdadm --examine output for sda1'],
None,
['_', 'mdadm --examine output for sdb1'],
None, None, None,
None, None, None,
['_', 'mdadm --examine output for sda2'],
None,
['_', 'mdadm --examine output for sdb2'],
None, None, None]
self.hardware.delete_configuration(self.node, [])
mocked_execute.assert_has_calls([
mock.call('wipefs', '-af', '/dev/md0'),
mock.call('mdadm', '--stop', '/dev/md0'),
Software RAID: Don't preformat log strings Remove the preformatting of log messages to avoid any formatting is done when the corresponding log level not enabled. Additional nits addressed: - make the supported RAID levels a top-level frozenset - remove unused return value from validate_configuration Follow-up to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: If6484847a16f938c80523f0b7ec6ae52ea9cf598
2019-06-06 17:31:59 +02:00
mock.call('mdadm', '--examine', '/dev/sda1',
use_standard_locale=True),
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
mock.call('mdadm', '--zero-superblock', '/dev/sda1'),
Software RAID: Don't preformat log strings Remove the preformatting of log messages to avoid any formatting is done when the corresponding log level not enabled. Additional nits addressed: - make the supported RAID levels a top-level frozenset - remove unused return value from validate_configuration Follow-up to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: If6484847a16f938c80523f0b7ec6ae52ea9cf598
2019-06-06 17:31:59 +02:00
mock.call('mdadm', '--examine', '/dev/sda2',
use_standard_locale=True),
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
mock.call('mdadm', '--zero-superblock', '/dev/sda2'),
mock.call('wipefs', '-af', '/dev/sda'),
mock.call('wipefs', '-af', '/dev/sdb'),
mock.call('wipefs', '-af', '/dev/md1'),
mock.call('mdadm', '--stop', '/dev/md1'),
Software RAID: Don't preformat log strings Remove the preformatting of log messages to avoid any formatting is done when the corresponding log level not enabled. Additional nits addressed: - make the supported RAID levels a top-level frozenset - remove unused return value from validate_configuration Follow-up to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: If6484847a16f938c80523f0b7ec6ae52ea9cf598
2019-06-06 17:31:59 +02:00
mock.call('mdadm', '--examine', '/dev/sdb1',
use_standard_locale=True),
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
mock.call('mdadm', '--zero-superblock', '/dev/sdb1'),
Software RAID: Don't preformat log strings Remove the preformatting of log messages to avoid any formatting is done when the corresponding log level not enabled. Additional nits addressed: - make the supported RAID levels a top-level frozenset - remove unused return value from validate_configuration Follow-up to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: If6484847a16f938c80523f0b7ec6ae52ea9cf598
2019-06-06 17:31:59 +02:00
mock.call('mdadm', '--examine', '/dev/sdb2',
use_standard_locale=True),
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
mock.call('mdadm', '--zero-superblock', '/dev/sdb2'),
mock.call('wipefs', '-af', '/dev/sda'),
mock.call('wipefs', '-af', '/dev/sdb')])
@mock.patch.object(utils, 'execute', autospec=True)
def test_validate_configuration_valid_raid1(self, mocked_execute):
raid_config = {
"logical_disks": [
{
"size_gb": "MAX",
"raid_level": "1",
"controller": "software",
},
]
}
Software RAID: Don't preformat log strings Remove the preformatting of log messages to avoid any formatting is done when the corresponding log level not enabled. Additional nits addressed: - make the supported RAID levels a top-level frozenset - remove unused return value from validate_configuration Follow-up to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: If6484847a16f938c80523f0b7ec6ae52ea9cf598
2019-06-06 17:31:59 +02:00
self.assertIsNone(self.hardware.validate_configuration(raid_config,
self.node))
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
@mock.patch.object(utils, 'execute', autospec=True)
def test_validate_configuration_valid_raid1_raidN(self, mocked_execute):
raid_config = {
"logical_disks": [
{
"size_gb": "100",
"raid_level": "1",
"controller": "software",
},
{
"size_gb": "MAX",
"raid_level": "0",
"controller": "software",
},
]
}
Software RAID: Don't preformat log strings Remove the preformatting of log messages to avoid any formatting is done when the corresponding log level not enabled. Additional nits addressed: - make the supported RAID levels a top-level frozenset - remove unused return value from validate_configuration Follow-up to Id20302537f7994982c7584af546a7e7520e9612b Change-Id: If6484847a16f938c80523f0b7ec6ae52ea9cf598
2019-06-06 17:31:59 +02:00
self.assertIsNone(self.hardware.validate_configuration(raid_config,
self.node))
Software RAID: Create/delete configurations This patch proposes to extend the IPA to be able to configure software RAID devices. For this, the {create,delete}_configuration methods of the GenericHardwareManager are implemented. Change-Id: Id20302537f7994982c7584af546a7e7520e9612b Story: #2004581 Task: #29101
2019-02-04 13:17:23 +01:00
@mock.patch.object(utils, 'execute', autospec=True)
def test_validate_configuration_invalid_MAX_MAX(self, mocked_execute):
raid_config = {
"logical_disks": [
{
"size_gb": "MAX",
"raid_level": "1",
"controller": "software",
},
{
"size_gb": "MAX",
"raid_level": "0",
"controller": "software",
},
]
}
self.assertRaises(errors.SoftwareRAIDError,
self.hardware.validate_configuration,
raid_config, self.node)
@mock.patch.object(utils, 'execute', autospec=True)
def test_validate_configuration_invalid_raid_level(self, mocked_execute):
raid_config = {
"logical_disks": [
{
"size_gb": "MAX",
"raid_level": "1",
"controller": "software",
},
{
"size_gb": "MAX",
"raid_level": "42",
"controller": "software",
},
]
}
self.assertRaises(errors.SoftwareRAIDError,
self.hardware.validate_configuration,
raid_config, self.node)
@mock.patch.object(utils, 'execute', autospec=True)
def test_validate_configuration_invalid_no_of_raids(self, mocked_execute):
raid_config = {
"logical_disks": [
{
"size_gb": "MAX",
"raid_level": "1",
"controller": "software",
},
{
"size_gb": "MAX",
"raid_level": "0",
"controller": "software",
},
{
"size_gb": "MAX",
"raid_level": "1+0",
"controller": "software",
},
]
}
self.assertRaises(errors.SoftwareRAIDError,
self.hardware.validate_configuration,
raid_config, self.node)
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
@mock.patch.object(utils, 'execute', autospec=True)
Add 'system_vendor' information to data This patch set add hardware vendor information to data. By using this data, we can get hints to detect driver. Change-Id: I39385fd5d616edfad719c255f22642f215bfb532
2016-02-01 17:18:30 +09:00
def test_get_system_vendor_info(self, mocked_execute):
Ironic python agent does not extract correct available memory This patch fixes parsing of memory class output of lshw output. Task: 30201 Story: 2005308 Change-Id: I0e7ff9f86a266d2f798457136fed0100c78569e2
2019-03-27 22:38:59 +11:00
mocked_execute.return_value = LSHW_JSON_OUTPUT_V1
Use lshw in place of dmidecode for the default hardware manager Currently the generic hardware manager uses dmidecode to get the total physical memory and system details. This patch switches the generic hardware manager to use lshw, as it is capable of reading more than DMI [0]. This enables systems that do not support DMI to use the generic hardware manager, such as IBM Power systems. [0] https://github.com/lyonel/lshw/blob/master/README.md Closes-Bug: #1715790 Change-Id: Ie370331df6bb5ef131c5cb60f458877e2a7ad71a Depends-On: Idaf05b8efce28cd0cbf339cf693db4f55a693d9b
2017-11-30 16:30:42 +00:00
vendor_info = self.hardware.get_system_vendor_info()
self.assertEqual('ABC123 (GENERIC_SERVER)', vendor_info.product_name)
self.assertEqual('1234567', vendor_info.serial_number)
self.assertEqual('GENERIC', vendor_info.manufacturer)
@mock.patch.object(utils, 'execute', autospec=True)
def test_get_system_vendor_info_failure(self, mocked_execute):
mocked_execute.side_effect = processutils.ProcessExecutionError()
vendor_info = self.hardware.get_system_vendor_info()
self.assertEqual('', vendor_info.product_name)
self.assertEqual('', vendor_info.serial_number)
self.assertEqual('', vendor_info.manufacturer)
Add 'system_vendor' information to data This patch set add hardware vendor information to data. By using this data, we can get hints to detect driver. Change-Id: I39385fd5d616edfad719c255f22642f215bfb532
2016-02-01 17:18:30 +09:00
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
@mock.patch.object(hardware.GenericHardwareManager,
'get_os_install_device', autospec=True)
Handle diskless hardware connected to remote iscsi It is required to issue iscsistart -b on those hardware in order for /dev/sda /dev/sdb etc to be exposed to the OS. On this type of hw, without the fix introspection fails with - missing: ['local_gb']' err. Change-Id: If6667b123e9b890d97c03612596fb78ee2b0ae85 Closes-Bug: 1590606
2016-06-09 09:23:29 -07:00
@mock.patch.object(hardware, '_check_for_iscsi', autospec=True)
Wait for at least one suitable disk to appear on start up Some kernel modules take substantial time to initialize. For example, with mpt2sas RAID driver inspection and deployment randomly fail due to IPA starting before the driver finishes initialization. This problem is probably impossible to solve in a generic case, as modern Linux environment do not have a notion of "hardware is fully initialized" moment. All hardware is essentially hotplug. To solve it at least for the simplest case, this patch adds a wait loop on start up waiting for at least one suitable disk to appear in inventory. Note that root device hints are not considered, as the node might not be known at that moment yet. Change-Id: Id163ca28f7c140c302ea04947ded3f3c58b284de Partial-Bug: #1582797
2016-05-24 10:04:12 +02:00
@mock.patch.object(time, 'sleep', autospec=True)
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
def test_evaluate_hw_waits_for_disks(
self, mocked_sleep, mocked_check_for_iscsi, mocked_get_inst_dev):
mocked_get_inst_dev.side_effect = [
Wait for at least one suitable disk to appear on start up Some kernel modules take substantial time to initialize. For example, with mpt2sas RAID driver inspection and deployment randomly fail due to IPA starting before the driver finishes initialization. This problem is probably impossible to solve in a generic case, as modern Linux environment do not have a notion of "hardware is fully initialized" moment. All hardware is essentially hotplug. To solve it at least for the simplest case, this patch adds a wait loop on start up waiting for at least one suitable disk to appear in inventory. Note that root device hints are not considered, as the node might not be known at that moment yet. Change-Id: Id163ca28f7c140c302ea04947ded3f3c58b284de Partial-Bug: #1582797
2016-05-24 10:04:12 +02:00
errors.DeviceNotFound('boom'),
None
]
result = self.hardware.evaluate_hardware_support()
Handle diskless hardware connected to remote iscsi It is required to issue iscsistart -b on those hardware in order for /dev/sda /dev/sdb etc to be exposed to the OS. On this type of hw, without the fix introspection fails with - missing: ['local_gb']' err. Change-Id: If6667b123e9b890d97c03612596fb78ee2b0ae85 Closes-Bug: 1590606
2016-06-09 09:23:29 -07:00
self.assertTrue(mocked_check_for_iscsi.called)
Wait for at least one suitable disk to appear on start up Some kernel modules take substantial time to initialize. For example, with mpt2sas RAID driver inspection and deployment randomly fail due to IPA starting before the driver finishes initialization. This problem is probably impossible to solve in a generic case, as modern Linux environment do not have a notion of "hardware is fully initialized" moment. All hardware is essentially hotplug. To solve it at least for the simplest case, this patch adds a wait loop on start up waiting for at least one suitable disk to appear in inventory. Note that root device hints are not considered, as the node might not be known at that moment yet. Change-Id: Id163ca28f7c140c302ea04947ded3f3c58b284de Partial-Bug: #1582797
2016-05-24 10:04:12 +02:00
self.assertEqual(hardware.HardwareSupport.GENERIC, result)
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
mocked_get_inst_dev.assert_called_with(mock.ANY)
self.assertEqual(2, mocked_get_inst_dev.call_count)
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
mocked_sleep.assert_called_once_with(CONF.disk_wait_delay)
Fix off-by-one error in warning This fixes an off-by-one error in a warning message. This is a follow-up to 3189c16a5e95ade468fa8bc37302eb9979f5a8c9. Change-Id: I89b56974c1b919f4c03498873d3ce9860d5644c5 Related-Bug: #1670916
2017-10-31 10:14:25 -04:00
@mock.patch.object(hardware, 'LOG', autospec=True)
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
@mock.patch.object(hardware.GenericHardwareManager,
'get_os_install_device', autospec=True)
@mock.patch.object(hardware, '_check_for_iscsi', autospec=True)
@mock.patch.object(time, 'sleep', autospec=True)
def test_evaluate_hw_no_wait_for_disks(
Fix off-by-one error in warning This fixes an off-by-one error in a warning message. This is a follow-up to 3189c16a5e95ade468fa8bc37302eb9979f5a8c9. Change-Id: I89b56974c1b919f4c03498873d3ce9860d5644c5 Related-Bug: #1670916
2017-10-31 10:14:25 -04:00
self, mocked_sleep, mocked_check_for_iscsi, mocked_get_inst_dev,
mocked_log):
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
CONF.set_override('disk_wait_attempts', '0')
result = self.hardware.evaluate_hardware_support()
self.assertTrue(mocked_check_for_iscsi.called)
self.assertEqual(hardware.HardwareSupport.GENERIC, result)
self.assertFalse(mocked_get_inst_dev.called)
self.assertFalse(mocked_sleep.called)
Fix off-by-one error in warning This fixes an off-by-one error in a warning message. This is a follow-up to 3189c16a5e95ade468fa8bc37302eb9979f5a8c9. Change-Id: I89b56974c1b919f4c03498873d3ce9860d5644c5 Related-Bug: #1670916
2017-10-31 10:14:25 -04:00
self.assertFalse(mocked_log.called)
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
Fix off-by-one error in warning This fixes an off-by-one error in a warning message. This is a follow-up to 3189c16a5e95ade468fa8bc37302eb9979f5a8c9. Change-Id: I89b56974c1b919f4c03498873d3ce9860d5644c5 Related-Bug: #1670916
2017-10-31 10:14:25 -04:00
@mock.patch.object(hardware, 'LOG', autospec=True)
Prevent tests' unmocked access to utils.execute() This change introduces a new base test class that mocks out utils.execute and forces an exception if it gets called. This has rooted out many tests that were doing this as a side effect of calling other functions, doing things like modprobe and running iscsi on the host's actual machine. The tests are all now appropriately patched in places where this was happening, and the new base class permanently prevents this from accidentally happening again. If you really want to call utils.execute() then you need to re-mock it in your unit test. Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-04-03 15:32:44 +10:00
@mock.patch.object(hardware, '_check_for_iscsi', mock.Mock())
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
@mock.patch.object(hardware.GenericHardwareManager,
'get_os_install_device', autospec=True)
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
@mock.patch.object(time, 'sleep', autospec=True)
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
def test_evaluate_hw_waits_for_disks_nonconfigured(
Fix off-by-one error in warning This fixes an off-by-one error in a warning message. This is a follow-up to 3189c16a5e95ade468fa8bc37302eb9979f5a8c9. Change-Id: I89b56974c1b919f4c03498873d3ce9860d5644c5 Related-Bug: #1670916
2017-10-31 10:14:25 -04:00
self, mocked_sleep, mocked_get_inst_dev, mocked_log):
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
mocked_get_inst_dev.side_effect = [
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
None
]
self.hardware.evaluate_hardware_support()
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
mocked_get_inst_dev.assert_called_with(mock.ANY)
self.assertEqual(10, mocked_get_inst_dev.call_count)
expected_calls = [mock.call(CONF.disk_wait_delay)] * 9
mocked_sleep.assert_has_calls(expected_calls)
Fix off-by-one error in warning This fixes an off-by-one error in a warning message. This is a follow-up to 3189c16a5e95ade468fa8bc37302eb9979f5a8c9. Change-Id: I89b56974c1b919f4c03498873d3ce9860d5644c5 Related-Bug: #1670916
2017-10-31 10:14:25 -04:00
mocked_log.warning.assert_called_once_with(
'The root device was not detected in %d seconds',
CONF.disk_wait_delay * 9)
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
Fix off-by-one error in warning This fixes an off-by-one error in a warning message. This is a follow-up to 3189c16a5e95ade468fa8bc37302eb9979f5a8c9. Change-Id: I89b56974c1b919f4c03498873d3ce9860d5644c5 Related-Bug: #1670916
2017-10-31 10:14:25 -04:00
@mock.patch.object(hardware, 'LOG', autospec=True)
Prevent tests' unmocked access to utils.execute() This change introduces a new base test class that mocks out utils.execute and forces an exception if it gets called. This has rooted out many tests that were doing this as a side effect of calling other functions, doing things like modprobe and running iscsi on the host's actual machine. The tests are all now appropriately patched in places where this was happening, and the new base class permanently prevents this from accidentally happening again. If you really want to call utils.execute() then you need to re-mock it in your unit test. Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-04-03 15:32:44 +10:00
@mock.patch.object(hardware, '_check_for_iscsi', mock.Mock())
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
@mock.patch.object(hardware.GenericHardwareManager,
'get_os_install_device', autospec=True)
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
@mock.patch.object(time, 'sleep', autospec=True)
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
def test_evaluate_hw_waits_for_disks_configured(self, mocked_sleep,
Fix off-by-one error in warning This fixes an off-by-one error in a warning message. This is a follow-up to 3189c16a5e95ade468fa8bc37302eb9979f5a8c9. Change-Id: I89b56974c1b919f4c03498873d3ce9860d5644c5 Related-Bug: #1670916
2017-10-31 10:14:25 -04:00
mocked_get_inst_dev,
mocked_log):
CONF.set_override('disk_wait_attempts', '1')
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
mocked_get_inst_dev.side_effect = [
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
errors.DeviceNotFound('boom'),
errors.DeviceNotFound('boom'),
None
]
self.hardware.evaluate_hardware_support()
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
mocked_get_inst_dev.assert_called_with(mock.ANY)
Fix off-by-one error in warning This fixes an off-by-one error in a warning message. This is a follow-up to 3189c16a5e95ade468fa8bc37302eb9979f5a8c9. Change-Id: I89b56974c1b919f4c03498873d3ce9860d5644c5 Related-Bug: #1670916
2017-10-31 10:14:25 -04:00
self.assertEqual(1, mocked_get_inst_dev.call_count)
self.assertFalse(mocked_sleep.called)
mocked_log.warning.assert_called_once_with(
'The root device was not detected')
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
Prevent tests' unmocked access to utils.execute() This change introduces a new base test class that mocks out utils.execute and forces an exception if it gets called. This has rooted out many tests that were doing this as a side effect of calling other functions, doing things like modprobe and running iscsi on the host's actual machine. The tests are all now appropriately patched in places where this was happening, and the new base class permanently prevents this from accidentally happening again. If you really want to call utils.execute() then you need to re-mock it in your unit test. Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-04-03 15:32:44 +10:00
@mock.patch.object(hardware, '_check_for_iscsi', mock.Mock())
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
@mock.patch.object(hardware.GenericHardwareManager,
'get_os_install_device', autospec=True)
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
@mock.patch.object(time, 'sleep', autospec=True)
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
def test_evaluate_hw_disks_timeout_unconfigured(self, mocked_sleep,
mocked_get_inst_dev):
mocked_get_inst_dev.side_effect = errors.DeviceNotFound('boom')
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
self.hardware.evaluate_hardware_support()
mocked_sleep.assert_called_with(3)
Prevent tests' unmocked access to utils.execute() This change introduces a new base test class that mocks out utils.execute and forces an exception if it gets called. This has rooted out many tests that were doing this as a side effect of calling other functions, doing things like modprobe and running iscsi on the host's actual machine. The tests are all now appropriately patched in places where this was happening, and the new base class permanently prevents this from accidentally happening again. If you really want to call utils.execute() then you need to re-mock it in your unit test. Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-04-03 15:32:44 +10:00
@mock.patch.object(hardware, '_check_for_iscsi', mock.Mock())
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
@mock.patch.object(hardware.GenericHardwareManager,
'get_os_install_device', autospec=True)
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
@mock.patch.object(time, 'sleep', autospec=True)
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
def test_evaluate_hw_disks_timeout_configured(self, mocked_sleep,
mocked_root_dev):
Remove usage of parameter enforce_type Oslo.config deprecated parameter enforce_type and change its default value to True in Ifa552de0a994e40388cbc9f7dbaa55700ca276b0. Remove the usage of it to avoid DeprecationWarning: "Using the 'enforce_type' argument is deprecated in version '4.0' and will be removed in version '5.0': The argument enforce_type has changed its default value to True and then will be removed completely." Change-Id: I0f0fb540c43edde64e489915c5199da40a0da9c1 Related--Bug: #1517839
2017-06-14 13:47:29 +08:00
CONF.set_override('disk_wait_delay', '5')
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
mocked_root_dev.side_effect = errors.DeviceNotFound('boom')
self.hardware.evaluate_hardware_support()
mocked_sleep.assert_called_with(5)
Wait for at least one suitable disk to appear on start up Some kernel modules take substantial time to initialize. For example, with mpt2sas RAID driver inspection and deployment randomly fail due to IPA starting before the driver finishes initialization. This problem is probably impossible to solve in a generic case, as modern Linux environment do not have a notion of "hardware is fully initialized" moment. All hardware is essentially hotplug. To solve it at least for the simplest case, this patch adds a wait loop on start up waiting for at least one suitable disk to appear in inventory. Note that root device hints are not considered, as the node might not be known at that moment yet. Change-Id: Id163ca28f7c140c302ea04947ded3f3c58b284de Partial-Bug: #1582797
2016-05-24 10:04:12 +02:00
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
@mock.patch.object(hardware.GenericHardwareManager,
'get_os_install_device', autospec=True)
Handle diskless hardware connected to remote iscsi It is required to issue iscsistart -b on those hardware in order for /dev/sda /dev/sdb etc to be exposed to the OS. On this type of hw, without the fix introspection fails with - missing: ['local_gb']' err. Change-Id: If6667b123e9b890d97c03612596fb78ee2b0ae85 Closes-Bug: 1590606
2016-06-09 09:23:29 -07:00
@mock.patch.object(hardware, '_check_for_iscsi', autospec=True)
Wait for at least one suitable disk to appear on start up Some kernel modules take substantial time to initialize. For example, with mpt2sas RAID driver inspection and deployment randomly fail due to IPA starting before the driver finishes initialization. This problem is probably impossible to solve in a generic case, as modern Linux environment do not have a notion of "hardware is fully initialized" moment. All hardware is essentially hotplug. To solve it at least for the simplest case, this patch adds a wait loop on start up waiting for at least one suitable disk to appear in inventory. Note that root device hints are not considered, as the node might not be known at that moment yet. Change-Id: Id163ca28f7c140c302ea04947ded3f3c58b284de Partial-Bug: #1582797
2016-05-24 10:04:12 +02:00
@mock.patch.object(time, 'sleep', autospec=True)
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
def test_evaluate_hw_disks_timeout(
self, mocked_sleep, mocked_check_for_iscsi, mocked_get_inst_dev):
mocked_get_inst_dev.side_effect = errors.DeviceNotFound('boom')
Wait for at least one suitable disk to appear on start up Some kernel modules take substantial time to initialize. For example, with mpt2sas RAID driver inspection and deployment randomly fail due to IPA starting before the driver finishes initialization. This problem is probably impossible to solve in a generic case, as modern Linux environment do not have a notion of "hardware is fully initialized" moment. All hardware is essentially hotplug. To solve it at least for the simplest case, this patch adds a wait loop on start up waiting for at least one suitable disk to appear in inventory. Note that root device hints are not considered, as the node might not be known at that moment yet. Change-Id: Id163ca28f7c140c302ea04947ded3f3c58b284de Partial-Bug: #1582797
2016-05-24 10:04:12 +02:00
result = self.hardware.evaluate_hardware_support()
self.assertEqual(hardware.HardwareSupport.GENERIC, result)
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
mocked_get_inst_dev.assert_called_with(mock.ANY)
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
self.assertEqual(CONF.disk_wait_attempts,
Fix waiting for target disk to appear This patch is changing the _wait_for_disks() method behavior to wait to a specific disk if any device hints is specified. There are cases where the deployment might fail or succeed randomly depending on the order and time that the disks shows up. If no root device hints is specified, the method will just wait for any suitable disk to show up, like before. The _wait_for_disks call was made into a proper hardware manager method. It is now also called each time the cached node is updated, not only on start up. This is to ensure that we wait for the device, matching root device hints (which are part of the node). The loop was corrected to avoid redundant sleeps and warnings. Finally, this patch adds more logging around detecting the root device. Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com> Change-Id: I10ca70d6a390ed802505c0d10d440dfb52beb56c Closes-Bug: #1670916
2017-03-09 14:02:06 +00:00
mocked_get_inst_dev.call_count)
Add configuration options for DISK_WAIT https://review.openstack.org/#/c/320295/ introduced two internal variables: _DISK_WAIT_ATTEMPTS and _DISK_WAIT_DELAY. These values are hardcoded. This patch adds configuration options for these so that an operator can change them based on their own needs/fleet of hardware. Change-Id: I2ba97669ec710fb4a435307466cd8add9c2293ba Closes-Bug: #1585663
2016-06-02 13:34:10 -04:00
mocked_sleep.assert_called_with(CONF.disk_wait_delay)
Wait for at least one suitable disk to appear on start up Some kernel modules take substantial time to initialize. For example, with mpt2sas RAID driver inspection and deployment randomly fail due to IPA starting before the driver finishes initialization. This problem is probably impossible to solve in a generic case, as modern Linux environment do not have a notion of "hardware is fully initialized" moment. All hardware is essentially hotplug. To solve it at least for the simplest case, this patch adds a wait loop on start up waiting for at least one suitable disk to appear in inventory. Note that root device hints are not considered, as the node might not be known at that moment yet. Change-Id: Id163ca28f7c140c302ea04947ded3f3c58b284de Partial-Bug: #1582797
2016-05-24 10:04:12 +02:00
Add boot information into the inventory Adds a new BootInfo object with 2 fields: * current_boot_mode - bios or uefi, detected from presence of /sys/firmware/efi as per the following answer: http://askubuntu.com/a/162896 This field will be used for setting the boot_mode capability in ironic-inspector * pxe_interface - PXE booting interface, if it can be detected. This fields is already used by ironic-inspector, added here for consistency. Change-Id: Ib36b592ffaba3bfa055d65c9526607867d302584 Partial-Bug: #1571580
2016-04-20 13:58:14 +02:00
@mock.patch.object(utils, 'get_agent_params',
lambda: {'BOOTIF': 'boot:if'})
@mock.patch.object(os.path, 'isdir', autospec=True)
def test_get_boot_info_pxe_interface(self, mocked_isdir):
mocked_isdir.return_value = False
result = self.hardware.get_boot_info()
self.assertEqual(hardware.BootInfo(current_boot_mode='bios',
pxe_interface='boot:if'),
result)
@mock.patch.object(os.path, 'isdir', autospec=True)
def test_get_boot_info_bios(self, mocked_isdir):
mocked_isdir.return_value = False
result = self.hardware.get_boot_info()
self.assertEqual(hardware.BootInfo(current_boot_mode='bios'), result)
mocked_isdir.assert_called_once_with('/sys/firmware/efi')
@mock.patch.object(os.path, 'isdir', autospec=True)
def test_get_boot_info_uefi(self, mocked_isdir):
mocked_isdir.return_value = True
result = self.hardware.get_boot_info()
self.assertEqual(hardware.BootInfo(current_boot_mode='uefi'), result)
mocked_isdir.assert_called_once_with('/sys/firmware/efi')
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
Add HCTL to BlockDevices This patch is adding a "hctl" attribute to the BlockDevices. HCTL stands for: Host, Channel, Target and Lun, which is basically the SCSI address. The idea behind this patch is to allow root device hints to find the disk for deployment based on the SCSI address. Partial-Bug: #1648036 Change-Id: If8897c68609e0df0378ee919b803ca5e497def02
2016-12-07 10:50:45 +00:00
@mock.patch.object(os, 'listdir', lambda *_: [])
Wait for udev to settle before listing the block devices This patch is making the list_all_block_devices() method to wait for udev to settle it's event queue prior to listing the devices. Sometimes the ironic-python-agent service may start before all devices were detected and end up erroring out because it couldn't find a suitable disk for deployment. Closes-Bug: #1551300 Change-Id: I1ae2062a711115a1ea14b79ae9ace7ddd2fff9d5
2016-02-25 16:32:47 +00:00
@mock.patch.object(utils, 'execute', autospec=True)
Prevent tests' unmocked access to utils.execute() This change introduces a new base test class that mocks out utils.execute and forces an exception if it gets called. This has rooted out many tests that were doing this as a side effect of calling other functions, doing things like modprobe and running iscsi on the host's actual machine. The tests are all now appropriately patched in places where this was happening, and the new base class permanently prevents this from accidentally happening again. If you really want to call utils.execute() then you need to re-mock it in your unit test. Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-04-03 15:32:44 +10:00
class TestModuleFunctions(base.IronicAgentTest):
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
@mock.patch.object(os, 'readlink', autospec=True)
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
@mock.patch.object(hardware, '_get_device_info',
lambda x, y, z: 'FooTastic')
Wait for udev to settle before listing the block devices This patch is making the list_all_block_devices() method to wait for udev to settle it's event queue prior to listing the devices. Sometimes the ironic-python-agent service may start before all devices were detected and end up erroring out because it couldn't find a suitable disk for deployment. Closes-Bug: #1551300 Change-Id: I1ae2062a711115a1ea14b79ae9ace7ddd2fff9d5
2016-02-25 16:32:47 +00:00
@mock.patch.object(hardware, '_udev_settle', autospec=True)
Using from_device_file from Devices The from_device_file method in pyudev.Device is deprecated since version 0.18 Using the same method from pyudev.Devices is now recommended. Increasing minimum version of pyudev to 0.18 and correctly handling exception raised by from_device_file. Change-Id: I0637585241c81a36f69aef8017f1134f5c526eb1
2019-07-08 18:42:16 +02:00
@mock.patch.object(hardware.pyudev.Devices, "from_device_file",
Add missing 'autospec' argument to mock.patch Add missing 'autospec' keyword argument to mock.patch and mock.patch.object calls. Use 'autospec=True' except for a few cases where it fails because the mocked function is a @classmethod and it doesn't work. In that case explicity set it to 'autospec=False' Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 08:30:25 -07:00
autospec=False)
Wait for udev to settle before listing the block devices This patch is making the list_all_block_devices() method to wait for udev to settle it's event queue prior to listing the devices. Sometimes the ironic-python-agent service may start before all devices were detected and end up erroring out because it couldn't find a suitable disk for deployment. Closes-Bug: #1551300 Change-Id: I1ae2062a711115a1ea14b79ae9ace7ddd2fff9d5
2016-02-25 16:32:47 +00:00
def test_list_all_block_devices_success(self, mocked_fromdevfile,
Report /dev/disk/by-path on inspection When inspecting block devices on a node, discover and report the /dev/disk/by-path/XXX name along with the /dev/XXX block device name. The second name does not change between Linux system reboots and has greater chances to be the same across similarly configured nodes. Note: this patch depends on https://review.openstack.org/#/c/500524/ library patch, but this dependency can't be expressed with Depends-On clause. Therefore once this patch requires a followup patch to enable one currently disabled test in this patch. Change-Id: I09874f19890500d352521f89573e2aaf50a29022 Closes-Bug: #1679726
2017-08-28 16:59:39 +02:00
mocked_udev, mocked_readlink,
mocked_execute):
Report /dev/disk/by-path on inspection followup This is the followup patch for commit d0a53149f82a3587515a4371f0f4cad8570dc715) fixing issues with the unit tests not addressed initially. Change-Id: I7889bf908bcb64b79bf303c6ae356fd3f4e94a83
2017-09-27 15:42:13 +02:00
mocked_readlink.return_value = '../../sda'
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
mocked_fromdevfile.return_value = {}
mocked_execute.return_value = (BLK_DEVICE_TEMPLATE_SMALL, '')
result = hardware.list_all_block_devices()
mocked_execute.assert_called_once_with(
Include empty devices when listing block devices When listing block devices, empty devices are not listed by default. In order to clean up broken software RAID devices, however, we need to list these as well. This patch adds the corresponding option to 'lsblk', makes sure the device type filtering handles the additional device type ('md' instead of 'raid*' when an md device has only spares), and takes care of lsblk reporting size '' for empty devices. Story: #2006215 Task: #35807 Change-Id: I523e2ed90f3a477b5f5f27054b65fcfd26239ba2
2019-07-12 15:53:06 +02:00
'lsblk', '-Pbia', '-oKNAME,MODEL,SIZE,ROTA,TYPE',
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
check_exit_code=[0])
self.assertEqual(BLK_DEVICE_TEMPLATE_SMALL_DEVICES, result)
Wait for udev to settle before listing the block devices This patch is making the list_all_block_devices() method to wait for udev to settle it's event queue prior to listing the devices. Sometimes the ironic-python-agent service may start before all devices were detected and end up erroring out because it couldn't find a suitable disk for deployment. Closes-Bug: #1551300 Change-Id: I1ae2062a711115a1ea14b79ae9ace7ddd2fff9d5
2016-02-25 16:32:47 +00:00
mocked_udev.assert_called_once_with()
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
Fix multi-device behavior ATARAID is functionally a version of software RAID where the setup is managed by the controller and the Operating System takes over managing the RAID after boot. Most commonly this is found for mirrored boot devices. Prior to this patch, we were looking for non-dependent items (i.e. base block devices), with a type of disk. Now we will permit the "disk" to be added to the list if lsblk indicates that it is a type containing "raid". The lsblk results should not change as we explicitly look for disk objects. Change-Id: Ia4a03b33cc06ce42e1bc33026683c28b31901cb7 Story: #2003445 Task: #24647
2018-08-16 10:57:59 -07:00
@mock.patch.object(os, 'readlink', autospec=True)
@mock.patch.object(hardware, '_get_device_info',
lambda x, y, z: 'FooTastic')
@mock.patch.object(hardware, '_udev_settle', autospec=True)
Using from_device_file from Devices The from_device_file method in pyudev.Device is deprecated since version 0.18 Using the same method from pyudev.Devices is now recommended. Increasing minimum version of pyudev to 0.18 and correctly handling exception raised by from_device_file. Change-Id: I0637585241c81a36f69aef8017f1134f5c526eb1
2019-07-08 18:42:16 +02:00
@mock.patch.object(hardware.pyudev.Devices, "from_device_file",
Fix multi-device behavior ATARAID is functionally a version of software RAID where the setup is managed by the controller and the Operating System takes over managing the RAID after boot. Most commonly this is found for mirrored boot devices. Prior to this patch, we were looking for non-dependent items (i.e. base block devices), with a type of disk. Now we will permit the "disk" to be added to the list if lsblk indicates that it is a type containing "raid". The lsblk results should not change as we explicitly look for disk objects. Change-Id: Ia4a03b33cc06ce42e1bc33026683c28b31901cb7 Story: #2003445 Task: #24647
2018-08-16 10:57:59 -07:00
autospec=False)
def test_list_all_block_devices_success_raid(self, mocked_fromdevfile,
mocked_udev, mocked_readlink,
mocked_execute):
mocked_readlink.return_value = '../../sda'
mocked_fromdevfile.return_value = {}
mocked_execute.return_value = (RAID_BLK_DEVICE_TEMPLATE, '')
result = hardware.list_all_block_devices()
mocked_execute.assert_called_once_with(
Include empty devices when listing block devices When listing block devices, empty devices are not listed by default. In order to clean up broken software RAID devices, however, we need to list these as well. This patch adds the corresponding option to 'lsblk', makes sure the device type filtering handles the additional device type ('md' instead of 'raid*' when an md device has only spares), and takes care of lsblk reporting size '' for empty devices. Story: #2006215 Task: #35807 Change-Id: I523e2ed90f3a477b5f5f27054b65fcfd26239ba2
2019-07-12 15:53:06 +02:00
'lsblk', '-Pbia', '-oKNAME,MODEL,SIZE,ROTA,TYPE',
Fix multi-device behavior ATARAID is functionally a version of software RAID where the setup is managed by the controller and the Operating System takes over managing the RAID after boot. Most commonly this is found for mirrored boot devices. Prior to this patch, we were looking for non-dependent items (i.e. base block devices), with a type of disk. Now we will permit the "disk" to be added to the list if lsblk indicates that it is a type containing "raid". The lsblk results should not change as we explicitly look for disk objects. Change-Id: Ia4a03b33cc06ce42e1bc33026683c28b31901cb7 Story: #2003445 Task: #24647
2018-08-16 10:57:59 -07:00
check_exit_code=[0])
self.assertEqual(RAID_BLK_DEVICE_TEMPLATE_DEVICES, result)
mocked_udev.assert_called_once_with()
Add vendor, product to interface information This patch updates the interfaces JSON to include vendor and product u'interfaces': [ { u'mac_address': u'00:0c:29:8c:11:b1', u'name': u'eth0', u'ipv4_address': None, u'switch_chassis_descr': None, u'switch_port_descr': None, u'lldp': None, u'vendor': u'0x15b3', u'product': u'0x1014, }] Co-Authored-By: yogananth subramanian <ysubrama@redhat.com> Closes-Bug: #1611856 Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-02-11 14:00:01 +02:00
@mock.patch.object(hardware, '_get_device_info',
lambda x, y: "FooTastic")
Wait for udev to settle before listing the block devices This patch is making the list_all_block_devices() method to wait for udev to settle it's event queue prior to listing the devices. Sometimes the ironic-python-agent service may start before all devices were detected and end up erroring out because it couldn't find a suitable disk for deployment. Closes-Bug: #1551300 Change-Id: I1ae2062a711115a1ea14b79ae9ace7ddd2fff9d5
2016-02-25 16:32:47 +00:00
@mock.patch.object(hardware, '_udev_settle', autospec=True)
def test_list_all_block_devices_wrong_block_type(self, mocked_udev,
mocked_execute):
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
mocked_execute.return_value = ('TYPE="foo" MODEL="model"', '')
result = hardware.list_all_block_devices()
mocked_execute.assert_called_once_with(
Include empty devices when listing block devices When listing block devices, empty devices are not listed by default. In order to clean up broken software RAID devices, however, we need to list these as well. This patch adds the corresponding option to 'lsblk', makes sure the device type filtering handles the additional device type ('md' instead of 'raid*' when an md device has only spares), and takes care of lsblk reporting size '' for empty devices. Story: #2006215 Task: #35807 Change-Id: I523e2ed90f3a477b5f5f27054b65fcfd26239ba2
2019-07-12 15:53:06 +02:00
'lsblk', '-Pbia', '-oKNAME,MODEL,SIZE,ROTA,TYPE',
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
check_exit_code=[0])
self.assertEqual([], result)
Wait for udev to settle before listing the block devices This patch is making the list_all_block_devices() method to wait for udev to settle it's event queue prior to listing the devices. Sometimes the ironic-python-agent service may start before all devices were detected and end up erroring out because it couldn't find a suitable disk for deployment. Closes-Bug: #1551300 Change-Id: I1ae2062a711115a1ea14b79ae9ace7ddd2fff9d5
2016-02-25 16:32:47 +00:00
mocked_udev.assert_called_once_with()
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
Wait for udev to settle before listing the block devices This patch is making the list_all_block_devices() method to wait for udev to settle it's event queue prior to listing the devices. Sometimes the ironic-python-agent service may start before all devices were detected and end up erroring out because it couldn't find a suitable disk for deployment. Closes-Bug: #1551300 Change-Id: I1ae2062a711115a1ea14b79ae9ace7ddd2fff9d5
2016-02-25 16:32:47 +00:00
@mock.patch.object(hardware, '_udev_settle', autospec=True)
def test_list_all_block_devices_missing(self, mocked_udev,
mocked_execute):
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
"""Test for missing values returned from lsblk"""
mocked_execute.return_value = ('TYPE="disk" MODEL="model"', '')
Replace assertRaisesRegexp with assertRaisesRegex This patch replace assertRaisesRegexp with assertRaisesRegex which is deprecated in python3 https://docs.python.org/3.2/library/unittest.html Also it update the base tests to be oslotest BaseTestCase for python2.7 and python3 compatibility Change-Id: I02571946f0643247e208d98dc91ea78cd9d351ee
2016-06-21 19:56:11 +03:00
self.assertRaisesRegex(
Refactor list_all_block_devices & add block_type param Put the columns to retrieve from lsblk into a list so that future modifications to columns will require fewer code changes. Also add a 'block_type' parameter which defaults to 'disk'. To make the function more flexible if callers wanted a different block type. Update and add unit tests Change-Id: If06460e13a5b56dc8d6efca9ff5b58ac6ba1f357
2015-08-28 11:14:52 -07:00
errors.BlockDeviceError,
r'^Block device caused unknown error: KNAME, ROTA, SIZE must be '
r'returned by lsblk.$',
hardware.list_all_block_devices)
Wait for udev to settle before listing the block devices This patch is making the list_all_block_devices() method to wait for udev to settle it's event queue prior to listing the devices. Sometimes the ironic-python-agent service may start before all devices were detected and end up erroring out because it couldn't find a suitable disk for deployment. Closes-Bug: #1551300 Change-Id: I1ae2062a711115a1ea14b79ae9ace7ddd2fff9d5
2016-02-25 16:32:47 +00:00
mocked_udev.assert_called_once_with()
def test__udev_settle(self, mocked_execute):
hardware._udev_settle()
mocked_execute.assert_called_once_with('udevadm', 'settle')
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
Handle diskless hardware connected to remote iscsi It is required to issue iscsistart -b on those hardware in order for /dev/sda /dev/sdb etc to be exposed to the OS. On this type of hw, without the fix introspection fails with - missing: ['local_gb']' err. Change-Id: If6667b123e9b890d97c03612596fb78ee2b0ae85 Closes-Bug: 1590606
2016-06-09 09:23:29 -07:00
def test__check_for_iscsi(self, mocked_execute):
hardware._check_for_iscsi()
mocked_execute.assert_has_calls([
mock.call('iscsistart', '-f'),
mock.call('iscsistart', '-b')])
def test__check_for_iscsi_no_iscsi(self, mocked_execute):
mocked_execute.side_effect = processutils.ProcessExecutionError()
hardware._check_for_iscsi()
mocked_execute.assert_has_calls([
mock.call('iscsistart', '-f')])
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
def create_hdparm_info(supported=False, enabled=False, locked=False,
frozen=False, enhanced_erase=False):
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
def update_values(values, state, key):
if not state:
values[key] = 'not' + values[key]
values = {
'supported': '\tsupported',
'enabled': '\tenabled',
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
'locked': '\tlocked',
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
'frozen': '\tfrozen',
'enhanced_erase': '\tsupported: enhanced erase',
}
update_values(values, supported, 'supported')
update_values(values, enabled, 'enabled')
rework ATA secure erase hdparm versions prior to 9.51 interpret the value, NULL, as a password with string value: "NULL". Example output of hdparm with NULL password: [root@localhost ~]# hdparm --user-master u --security-unlock NULL /dev/sda security_password="NULL" /dev/sda: Issuing SECURITY_UNLOCK command, password="NULL", user=user SECURITY_UNLOCK: Input/output error Example output of hdparm with "" as password: [root@localhost ~]# hdparm --user-master u --security-unlock "" /dev/sda security_password="" /dev/sda: Issuing SECURITY_UNLOCK command, password="", user=user Note the values of security_password in the output above. The output was observed on a CentOS 7 system, which ships hdparm 9.43 in the offical repositories. This change attempts to unlock the drive with the empty string if an unlock with NULL was unsucessful. Issuing a security-unlock will cause a state transition from SEC4 (security enabled, locked, not frozen) to SEC5 (security enabled, unlocked, not frozen). In order to check that a password unlock attempt was successful it makes sense to check that the drive is in the unlocked state (a necessary condition for SEC5). Only after all unlock attempts fail, do we consider the drive out of our control. The conditions to check the drive is in the right state have been adjusted to ensure that the drive is in the SEC5 state prior to issuing a secure erase. Previously, on the "recovery from previous fail" path, the security state was asserted to be "not enabled" after an unlock - this could never have been the case. A good overview of the ATA security states can be found here: http://www.admin-magazine.com/Archive/2014/19/Using-the-ATA-security-features-of-modern-hard-disks-and-SSDs Change-Id: Ic24b706a04ff6c08d750b9e3d79eb79eab2952ad Story: 2001762 Task: 12161 Story: 2001763 Task: 12162
2018-05-10 21:53:44 +01:00
update_values(values, locked, 'locked')
Create helper function: create_hdparm_info() Create a helper function: create_hdparm_info() and use it in the unit tests. Also fix mispelling of 'indicative' Change-Id: Ifa15fde72bc0ca6d925408c1dcafce85c192abb7
2016-04-08 14:10:52 -07:00
update_values(values, frozen, 'frozen')
update_values(values, enhanced_erase, 'enhanced_erase')
return HDPARM_INFO_TEMPLATE % values
Copy Permalink