After merging change I0b60f9078f23f8464d8234841645ed520e8ba655, we
noticed an issue with existing unit tests which started failing.
The reason is 'nvme_hostid' was an additional parameter returned
in the response while fetching connector properties from nvme
connector.
This is environment specific and won't occur in environments where
'/etc/nvme/hostid' file doesn't exist due to which these tests
passed in gate but failed in the local run when hostid file
was present.
This patch mocks the get_nvme_host_id method for tests so the
hostid is never returned irrespective of the environment.
Closes-Bug: #2032941
Change-Id: I8b1aaedfdb9bef6e34813e39dede9afe98371d2b
In a multipath enabled deployment, when we try to extend a volume
and some paths are down, we fail to extend the multipath device and
leave the environment in an inconsistent state. See LP Bug #2032177
for more details.
To handle this, we check if all the paths are up before trying to
extend the device and fail fast if any path is down. This ensures
we don't partially extend some paths and leave the other to the
original size leading to inconsistent state in the environment.
Closes-Bug: 2032177
Change-Id: I5fc02efc5e9657821a1335f1c1ac5fe036e9329a
The NVMe-oF connector currently create the `/etc/nvme/hostnqn` file if
it doesn't exist, but it may still be missing the `/etc/nvme/hostid`
value.
Some distribution packages create the file on installation but others
may not.
It is recommended for the file to be present so that nvme doesn't
randomly generate it.
Randomly generating it means that the value will be different for the
same storage array and the same volume if we connect, disconnect, and
connect the volume again.
This patch ensures that the file will exist and will try to use the
system's UUID as reported by DMI or a randomly generated one.
Closes-Bug: #2016029
Change-Id: I0b60f9078f23f8464d8234841645ed520e8ba655
On Change-Id Ib3b066a7da071b1c2de78a1a4e569676539bd335 we improved the
RBDVolumeIOWrapper's flush and close methods, and this patch improves
them even further.
If the IOWrapper's close is not explicitly called and it's just
dereferenced (happens in unit tests) then during Garbage Collection the
wrapped image may be destroyed before the wrapper, which would trigger
the image being closed without the wrapper knowing, so when the wrapper
gets destroyed it will fail because it calls its close method, which
calls its flush, which calls the underlying image's flush, which will
fail because the underlying image was already closed.
We need to check if the underlying image has already being flushed
before calling the flush.
Calling the underlying close method for the Image or IOWrapper classes
is not a problem because they are idempotent.
Change-Id: Ib5a517d58427df0d1d1b22ad3dc66f673da147fe
This patch adds non SAM addressing modes for LUNs, specifically for
SAM-2 and SAM-3 flat addressing.
Code has been manually verified on Pure storage systems that uses SAM-2
addressing mode, because it's unusual for CI jobs to have more than
256 LUNs on a single target.
Closes-Bug: #2006960
Change-Id: If32d054e8f944f162bdc8700d842134a80049877
When multipath is enabled and friendly names are ON and we try
to extend a volume, we pass the SCSI WWID to the multipath_resize_map
method which is not correct.
There are 2 things we can pass to the multipathd resize map command:
1) Multipath device (eg: dm-0)
2) Alias (eg: mpath1) or UUID (eg: 36005076303ffc56200000000000010aa)
The value should be an alias (mpath1) when friendly names are ON
and UUID (36005076303ffc56200000000000010aa) when friendly names
are OFF. However, we only pass the UUID irrespective of the value
set for friendly names.
This patch passes the multipath device path (to multipathd resize
map command) which is the real path of the multipath device (/dev/dm-*).
This fixes the issue as it passes the same value irrespective of if
the friendly names are ON or OFF.
-> self.multipath_resize_map(os.path.realpath(mpath_device))
(Pdb) mpath_device
'/dev/disk/by-id/dm-uuid-mpath-3600140522774ce73be84f9cb9537e0c9'
(Pdb) os.path.realpath(mpath_device)
'/dev/dm-5'
Closes-Bug: 1609753
Change-Id: I1c60af19c2ebaa9de878cd07cfc0077c5ea56fe3
The -v arg suppresses printing of [/dir] with the
device for bind mounts and btrfs volumes, which is
what we want for this usage.
This fixes _get_host_uuid() failing when using
a btrfs rootfs.
Closes-Bug: #2026257
Change-Id: I2d8f24193ecf821843bf8f4ea14b445561d6225c
This reverts commit 8070ac3bd903a443fbfd02abf8b0554d5d05cac1.
Reason for revert: This requires some more discussion, I should not have ninja-approved it.
Change-Id: I25917b95a32da4fd831d669cd21988f400f258e0
This patch adds support for the force and ignore_errors on the
disconnect_volume of the FC connector like we have in the iSCSI
connector.
Related-Bug: #2004555
Change-Id: Ia74ecfba03ba23de9d30eb33706245a7f85e1d66
The purpose of providing force=True and ignore_errors=True
is to tell os-brick that we want to disconnect the volume
even if flush fails (force) and not raise any exceptions
(ignore_errors). Currently, in an iSCSI multipath environment,
disconnect_volume can fail when both parameters are True.
The current flow when disconnecting an iSCSI volume is
that if flushing a multipath device fails, we manually
remove the device, logout from the target portals,
and try the flush again.
There are two problems here:
1) code problem: The second flush is not wrapped by
ExceptionChainer. This causes it to raise the exception
immediately after flush fails irrespective of the value
of the ignore_errors flag.
2) conceptual problem: In this situation, there is no point
in making the second flush attempt. Instead, we should just
remove the multipath map from multipathd monitoring since
we have already removed the paths manually.
This patch fixes the conceptual problem, as we don't make a second
flush call and ignore any errors on the execution ``multipathd del map``
thereby also fixing the code problem.
Closes-Bug: #2012251
Change-Id: I828911495a2de550ea997e6f51cc039a7b7fa8cd
-l100%FREE was implemented years ago and is
already used by our code when the LVM version
meets the supports_full_pool_create check.
Change-Id: I8f26e3880429e648e7e985f9c6cb7dff56715727
mypy detected this problem -- the
supports_full_pool_create property will always
return True when referenced via the class instead
of via an instance.
Change-Id: Iefc691ae8d5eb001c9abbf554600a8f8132e038c
Currently we don't have os_brick DEBUG log levels in Nova when setting
the service to debug mode.
That happens because Nova is forcefully setting oslo.privsep.daemon
levels to INFO to prevent leaking instance XML details (bug #1784062).
Oslo Privsep now supports per-context debug log levels, so this patch
sets the log level name for its only os_brick privsep context to
"os_brick.privileged" to differentiate it from the service it runs under
which uses the default "oslo_privsep.daemon".
This way even though Nova is still disabling its own privileged calls it
won't affect os-brick privileged calls, allowing us to properly debug
block device attach/detach operations.
Closes-Bug: #1922052
Related-Bug: #1784062
Change-Id: I0de32021eb90ca045845a6c7c7e3d27e52895948
On some platforms there is no 'dmidecode' program (e.g. s390x), when
privsep attempts to run a program that's not found, it raises a
FileNotFoundError instead of ProcessExecutionError.
Closes-Bug: #1994083
Change-Id: I6fc43ab7e243d50b74036d1af5f9e8f880401cc6
The RBDVolumeIOWrapper class inherits from io.RawIOBase but its close
method doesn't flush and after being called the "closed" attribute still
returns False.
Trying to set the "closed" attribute ourselves in
RBDVolumeIOWrapper.close results in an error, so we call the parent's
implementation to do it.
The parent's close method will also call the flush method.
This change ensures that rbd_image.close is only called once, that the
closed attribute is correct, and that the data is flushed on close.
Without these changes we could have, unintentionally, multiple calls to
the rbd_image.close method. If we called close on the wrapper
ourselves, then when the Garbage Collector is cleaning up the wrapper it
will call the io.RawIOBase finalizer that will see that the file is not
closed and call close again.
Now that we correctly set the closed flag on the instance, the flush
method should also behave like the parent implementation and fail if it
has already been closed.
Change-Id: Ib3b066a7da071b1c2de78a1a4e569676539bd335
Move the _get_system_uuid method from the NVMeOFConnector connector to
os_brick/privsep/nvmeof.py renaming it to get_system_uuid and running it
as privileged.
This allows us to read sys/class/dmi/id/product_uuid file in Python
instead of running a subprocess to execute cat.
It also allows the connector to have one less privsep calls if the file
doesn't exist, because it can execute the dmidecode command directly
without making another request to the privsep daemon.
Change-Id: I8f2edef6fda97af0ff3f92e39c8b24a85b6c3402
Use dict[], list[] etc. instead of Dict[], List[].
This works on all versions of Python that we support.
Change-Id: If9c737277c041ffd2d3bca1863d300ae7d9bd52e
This coverage is somewhat limited because this
uses classes (context, keymgr) that live outside
of os-brick and therefore don't have type definitons
here.
Change-Id: I7827a870c04a7a6e02510b7909430feebb70e6f1
The get_md_name uses the contents of mdstat using cat and piping it to
grep and then to awk using a privileged shell, but there's no need to do
that.
Actually the mdstat file has 0444 access, so we can read it in Python
directly instead of having to call the privsep daemon to create the
subprocess with the shell.
This patch changes the current implementation of the method to do the
reading and searching in Python.
Change-Id: Idbba6f9a6d928aa94d2920fc1bf8b6e2a0626a07
