Currently list_partitions method fails if partition
name is defined on any GPT partition, it also only
extract one partition flag.
Updated _PARTED_PRINT_RE is now set to capture
partition name/description and partition flags.
Change-Id: I40410622829453a485aa33047848922ab134d725
Task: 30250
Story: 2005322
We're hitting a problem in the CI where /dev/sda2 does not exist after
being created. The logs suggest running partprobe may fix the problem,
so let's try it.
With this, we can also encounter a case where there refresh will fail
because the partition was created using fsync calls to force the write
of the partition table. However, filesystem VFS layer does impact this
behavior, and fsync on files in a ramdisk is a noop, so we must force
everything to be flushed to disk before attempting to execute partprobe.
Also run 'sgdisk -v' after partitioning to leave some traces in the logs
for debugging when something goes wrong.
Story: #2004744
Task: #28815
Change-Id: I9db48a3462422749290bbb887c14816734ab0478
Depends-On: I8d6f8c99e1ab5f26e3537630d0f6086e02a2b1ec
To support streaming raw partition images we need to fetch the image
directly from HTTP. To avoid copying this logic from IPA to ironic-lib,
enable work_on_disk to skip flashing the image by passing image_path=None.
To simplify life for the caller, return partitions dictionary in result.
Change-Id: I3189a2de115ae4bf47ad911161c2f5f3f062ef3c
Story: #2003809
Task: #26760
Currently some of our tests set overrides, but nothing restores them.
This changes adds a config fixture to the bass class and also switches
some tests to using set_override instead of direct assignment.
Change-Id: I178061608e407047fb87d547c277ea38db353c26
Story: #2003891
Task: #26758
When using dd command, it may be needed to specify conversion
options, such as sparse for optimization. This change allows to
accept this option, in preparation for ironic to sent it when
enabled on configuration.
Change-Id: I166abbf513557a836de3cce4fc9ea6b7fedbb562
Story: #2003755
Task: #26443
Currently the search for config drive is just done for config-2
in lowercase. If we pass the label with vfat, the label will
be in uppercase by convention. So the comparison will fail, and
this needs to be case insensitive to work.
Change-Id: I1dd196841fc3599333d68459eac606bd3b104b7b
Story: #2003547
Task: #24834
On centos7.2, it doesn't happen every time, but sometimes, partprobe
fails with error `Device or resource busy` in _fix_gpt_structs. In fact,
this is not a problem, I tried ignoring this error and ran remain code.
Things went well. The target of running `partprobe` is to check GPT
table, we can run `sgdisk -v` instead.
This patch is to replace `partprobe` command in _fix_gpt_structs to
avoid `Device or resource busy` error.
Change-Id: I43182f17b1c6229132814313f7582ab30245f6bb
blkid does not always return a UUID when querying partitions (sda2 in my
case). It does seem to return a PARTUUID though, so use that if needed.
Change-Id: I6f8dbd8fb8a02f6246b7ab00801159544f3288d6
Story: 2002052
Task: 19701
As suggested by TheJulia, the `_fix_gpt_structs` code
which is currently not accessible/callable from outside of ironic-lib
should be exposed so that other code like ironic-python-agent
can use it such as https://review.openstack.org/#/c/568524/
This patchset now exposes this code as `fix_gpt_structs`
(no leading underscore).
Change-Id: Icd824982d23966605fb179270bbf428cab23a41d
There's a lambda here that seems to just return the addition
of two ints. Replacing it with a simple addition seems to produce
the same result.
Change-Id: I4b188986407c4ab83848d392e76ea71c95740737
The partprobe command may fail because the device is busy some
times. Add retry attempts to resolve this problem.
Closes-Bug: #1756760
Change-Id: Ic888e7b6c93e83511a76bf8000619531b1eec9e1
Previously it was using functools.wraps() to create the decorators.
The problem is that we can't use the
oslo_utils.reflection.get_signature() function to get the correct
signature.
Change it to use six.wraps() which will add the '__wrapped__'
attribute which will be used when calling get_signature() and return
the signature of the decorated function.
Added unit tests to show we are able to see the signature of a wrapped
function.
Closes-Bug: #1746730
Change-Id: I75428e948b64b3b6758d31678a80d158a11c6beb
The 'self' parameter was missing in the class methods of the
DoNotCallPopen class. Add the 'self' parameter to the class methods.
Change-Id: I29bed962a97c8b28b3feb266d76243da4b4cab7a
* Simplify the logic in utils.py/wait_for_disk_to_become_available
* Correct unit tests to not use None as a return value for calls to
utils.execute() that return no output. In reality utils.execute()
will return empty strings for example: ('', '')
* Add unit tests to test 'psmisc' and 'busybox' versions of output
data from 'fuser'
* Add a shorter check device interval as test was taking over 2
seconds and showing up in longest test run time list.
Change-Id: Ibc481e4a76c63612e1774b5c7a8144ee934eb2f5
* Move _wait_for_disk_to_become_available function
from DiskPartitioner class to utils.py
* Call the wait_for_disk_to_become_available during
the destroy_disk_metadata metadata to ensure that
nothing is using disk and preventing metadata
from being destroyed.
Change-Id: I9134b4cfdf15ba9610b4d1f844e64858d69713a5
Closes-Bug: #1739421
An issue was discovered if we mock an already mock-ed function. This
was happening in our execute() detection code. Change it to not use a
mock and instead a function.
Add test cases to ensure it works as expected.
Change-Id: If58071b41c7e69bc182ea033bf647924f996f1f3
Use latest version of flake8-import-order package, like we do in
openstack/ironic
Fix two issues detected by new version.
Change-Id: I3d2f5499f3a863167419469d313e06be79f51841
With the introduction of the `by_path` introspection
attribute at the IPA [1], it is now possible to leverage
the stable block device names for root device hinting.
This patch adds the `by_path` attribute to the collection
of valid root device hints.
1. https://review.openstack.org/#/c/498489/
Change-Id: I6f1c6b5ec5face7180aeb2d1aeebe50a6eb33f24
Config option 'backend' with parameter choices =['noop', 'statsd'],
that means oslo.config will ensure value is one of them, otherwise
a ValueError will be raised, so don't need handle and test config
option 'backend' with invalid value.
Change-Id: Ibda809416c565d57f19179d7f13f4cee8f8145ff
NVMe drives use different device naming convention for partitions:
> lsblk /dev/nvme0n1
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259:0 0 1.8T 0 disk
|-nvme0n1p1 259:2 0 1.3G 0 part
`-nvme0n1p2 259:1 0 64M 0 part
The ironic-python-agent fails to provision NVMe drives,
because it does not expect the 'p' letter before the partition number.
This patch fixes that.
Change-Id: I5061f6d5b00f404d4aaadc6b18675039ba2ed854
Closes-Bug: #1695265
In testcase, setUp will be called automatically. This patch used to
remove setUp functions that do nothing. Besides, it will keep code clean.
Change-Id: I015e4faa2e185a49609078c281f449c65a0d8b27
Apparently, wipefs still leaves some metadata around, which can break ceph.
Use sgdisk's zapping feature to be absolutely sure.
Change-Id: I51aea8fb5ba02d63a05a3794405caf000ff28e75
Closes-Bug: #1690458
This change introduces a new base test class that mocks out
utils.execute() and similar functions and forces an exception if it
gets called. If utils.execute() is mocked by the test case then it
will work. What this does is prevent un-mocked access to
utils.execute() / processutils.execute() and similar subprocess
library functions.
Inspired by Julian Edwards' patch to ironic-python-agent
Change-Id: Ie5bda8f4c65cf4dfb40b91c11b61485c954addde
Remove the unit tests that were testing the functionality of
oslo_concurrency.processutils.execute. It is not our job to unit test
3rd party libraries.
Change-Id: I1516af623346a590e25f4f09226c784031ca78a7
Config option metrics/backend only accepts values "noop" or "statsd" [1],
however test_nonexisting_backend() checks for a different value.
oslo.config 4.0 will check overridden value type by default[2], so we
need to set enforce_type in CONF.set_override to allow unit tests to
pass.
[1] 36109e58f2/ironic_lib/metrics_utils.py (L25-L28)
[2] https://review.openstack.org/#/c/328692/
Change-Id: Ia597da0d03cfcc5f34b271a12cc42600c6a6a2d3
We waiting for device is ready by using 'udevadm settle
--exit-if-exist=/dev/blah' which should wait for disk before
start writing image. In the bug attached we can see that,
dd complains that target device doesn't exit.
When tested locally udevadm returns 0 when target device doesn't
exist:
$ udevadm settle --exit-if-exists=/dev/blah; echo $?
0
$ ls /dev/blah
ls: cannot access '/dev/blah': No such file or directory
This patch replaces calls 'udevadm settle' without arguments that waits
until udev event queue is empty.
Also add simple 'test -e' check, with 15 retries and random delay
(0.2 - 2sec) between each retry [0], to wait for configrive device.
[0] https://github.com/openstack/oslo.concurrency/blob/7afbc99/oslo_concurrency/processutils.py#L439
Change-Id: I6d2d485dd543fa8975187862d3e4b8d6567e0224
Closes-Bug: #1673731
Add missing 'autospec' statements to unit test mocks. This helps to make
sure we don't accidentally call invalid assert methods.
Change-Id: I8dd2a3165b983dcc105e412717453b3bf3293546
Use the flake8 plugin flake8-import-order to check import ordering. It
can do it automatically and don't need reviewers to check it.
Change-Id: Ib3fde54d8baa2c6e3567ff3f1a902de4b25f9f70
With multiple qemu-img processes running at the same time the host machine
can quickly run out of memory. This patch is limiting the memory space
to 1 GiB when invoking qemu-img.
Change-Id: If791452edc38c4732ba2b96220e7969c72b234dd
Closes-Bug: #1657808
This fixes the exception message that is raised when there are
multiple devices with the same label. Instead of including the
original device path, it was showing one of the device's name
& label.
The message now includes the original device path along with
the two devices that have the same label.
Change-Id: Id114b25e95492686bc6679e217d4963d6a3ed0c4
Related-Bug: #1654269
Previously it was checked using blkid -t LABEL=config-2, but
this was not finding the partition, causing that images with a
pre-existing config-drive partition had an extra one created
by ironic.
Switched to using the 'lsblk' command and parsing the output of that to
get the information on the partition.
Change-Id: Ic5228ce91d1a133fd846ceb7be547a227292dd38
Closes-Bug: #1654269
Depends-On: I3676cd8e5c5e61fe4e3f67ab9e121c4cd0cd599b
This fixes a bug where ironic-lib would create configdrive partitions as
ext2, which is incompatible with any OS but Linux and goes against the
configdrive standard as written in cloud-init docs.
Change-Id: Ib4ad8e0b4d6dbef9d441d9ae13fd82f8f51957e6
Closes-bug: #1655389
This patch replaces python standard base64 library call to
oslo_serialization.base64 to follow OpenStack Python3 porting
standard [1].
Use base64 encoding takes 8-bit binary byte data and encodes it. On
Python3, A string is a sequence of Unicode characters thus base64 has
no idea what to do with Unicode data, it's not 8-bit[2]. We use
oslo_serialization.base64 for python2 and python3.
[1] https://wiki.openstack.org/wiki/Python3
[2] http://stackoverflow.com/questions/8908287/base64-encoding-in-python-3
Change-Id: I9d453ab6ebc826cad60c89d2a74c5764626eba01
This patch is adding support for comparing the HCTL (Host, Channel,
Target, Lun) attribute of a block device as part of the root device
hints mechanism.
Partial-Bug: #1648036
Depends-On: If8897c68609e0df0378ee919b803ca5e497def02
Change-Id: I2d4a8b2dcef5cab95c27be12ddb8d08b62896869
A failure may occur if the dd to config drive partition happens
before it actually becomes available. This change adds a udevadm
settle call that ensures that config drive device is present in
/dev folder.
Closes-Bug: #1633063
Change-Id: I5526f791371e7a6f4cdad151bb163707504e76cf
ironic has some test cases that check the internals of what ironic-lib
does. This adds some test cases to ironic-lib's work_on_disk method to
ensure coverage will stay up to par when those tests are purged from
ironic proper.
Change-Id: If9499982156e349056cffd87d62d1da172494204
Related-Bug: 1552843
This patch is adding a new method called count_msdos_partitions()
responsible for counting the number of primary and logical partitions
in a MBR partition table.
Prior to this patch, the code checking if there were more than 4 primary
partitions present in the partiton table was using the list_partitions()
method which do not distinguish between the types*. So, even if the disk
had 2 primary and 5 logical partitions the check would fail thinking
that 7 primary partitions were present (which is impossible btw).
This patch make use of "partprobe" to count the partitions, which is
already required in ironic-lib.
* Not distinguishing types is a pro on the list_partitions() method
because primary and logical does not apply for others disk labels such
as GPT.
This patch also fix a small typo in a non-related string :-)
Change-Id: I7042cd12775b0d715c15f466db2d4c5dc9e1a60a
Closes-Bug: #1629926