After some changes to the FC connector we have introduced a regression
on the way we do the scans, and we end up scanning using wildcards even
though we shouldn't.
The targets in the "initiator_target_map" don't mean that they are all
connected, so we must take that into account.
With the current code, if we have the following connections:
HBA host7 ---- SWITCH ---- port W (channel 0, target 2)
\--- SWITCH ---- port X (channel 0, target 3)
HBA host8 ---- SWITCH ---- port Y (channel 0, target 2)
\--- SWITCH ---- port Z (channel 0, target 3)
We will end up with the following scans 8 scans for LUN L:
- - L > host7
- - L > host7
0 2 L > host7
0 3 L > host7
0 2 L > host8
0 3 L > host8
- - L > host8
- - L > host8
Which correspond to the responses from _get_hba_channel_scsi_target like
this:
port Y port Z port W port X
host7 ... ['-','-',L] ['-','-',L] ['0','2',L] ['0','3',L]
host8 ... ['0','2',L] ['0','3',L] ['-','-',L] ['-','-',L]
And we should only be doing 4 scans:
0 2 L > host7
0 3 L > host7
0 2 L > host8
0 3 L > host8
Most storage arrays get their target ports automatically detected by the
Linux FC initiator and sysfs gets populated with that information, but
there are some that don't.
We'll do a narrow scan using the channel, target, and LUN for the former
and a wider scan for the latter.
If all paths to a former type of array were down on the system boot the
array could look like it's of the latter type and make us bring us
unwanted volumes into the system by doing a broad scan.
To prevent this from happening Cinder drivers can use the
"enable_wildcard_scan" key in the connection information to let us know
they don't want us to do broad scans even if no target ports are found
(because they know the cause is there's no connection).
Close-Bug: #1849504
Related-Bug: #1765000
Related-Bug: #1828440
Change-Id: I5dbefaff43fb902b15117b443fc92f7b6a6ad8c9
The nvme connector in os-brick is actually for nvme over fabrics, so
to avoid future confusion we have renamed the nvme connector object
to NVMeOF to better reflect it's capability. This patch keeps the
backwards compatibility of the mapping of the initiator.NVME to
the renamed nvmeof object.
Change-Id: I97b41139f2e67ab42e2aa8075c51ef939b3cde18
Fix a call to _iscsiadm_update() in which a list was being passed as
the target_iqn connection property. This property is used directly as
an argument to the iscsiadm -T option, so it must be a plain string.
Change-Id: I9c2ff1de1f89fb49dd6c5a90679d5c4238d5476a
Closes-bug: 1838820
Method _discover_mpath_device isn't overwrited in FibreChannelConnector,
just use it by self._discover_mpath_device.
Change-Id: I3d5a6e0010e56d256da17e4b20be3db0569c6702
This patch relaxes the single-pathed connections and allows them to
complete even when we cannot detect the WWN on the sysfs, just like
multipath connections do.
Closes-Bug: #1834443
Change-Id: Iae5a304329a2b172bc6b7f310623fad18956ae45
In FC environment, when disconnect volume,
if the first path in the loop is failed,
it will lead get scsi wwn to failure.
So I think we need to add path validity check
before get scsi wwn
Change-Id: I9d3a9dac13dcd585330e7b891c61b2626e5aabec
Closes-Bug: 1831621
Previously linuxscsi.extend_volume would always attempt to wait for
multipath devices to appear regardless of multipath actually being used
by the connector.
This change corrects this by passing down the existing use_multipath
attribute from the iSCSI and FC connectors into linuxscsi.extend_volume.
The same attribute is introduced to the NVMe connector to also allow it
to skip this search for multipath devices.
Change-Id: I29d65ae036957f3a63cba93dd330b14e3361a1b9
Closes-bug: #1832247
Current FC OS-Brick code only checks for single WWNN to exclude some
HBAs from scanning when we don't receive an initiator_target_map from
the backend.
There are storage arrays,like XtremIO, that due to their architecture
and design have all target ports automatically mapped to LUNs and are
returning the initiator_target_map, but some of those ports may not be
connected to our HBAs, so we end up with another case of bug #1765000.
This patch makes sure that we always check if the target implements
single WWNN, not only when we don't receive the initiator_target_map.
With this we decrease the likelihood of ending with unexpected devices
in our system, because now we will ignore unconnected HBAs (even if
reported in the initiator_target_map) if we are working with single WWNN
target.
Related-Bug: #1765000
Closes-Bug: #1828440
Change-Id: I02c208142f5b342f894666831449243863eccbfe
Starting with py36, DeprecationWarnings are raised for special
characters in strings that are not being used as valid escape sequences.
This addresses a couple instances we had for string replacement by
changing the replacement strings to be raw strings.
Change-Id: I93291e978aacaf3f639f32174605528f0038f13a
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
A long time ago the driver was changed to store volume_id
in the nova database and use that. Previously it would use the
volume_name and make a call to get the volume_id.
Both should be supported, but as the code path currently throws
a KeyError it will never reach the backwards compatible code.
This makes sure that the driver works even after an upgrade from Mitaka.
Depends-On: https://review.openstack.org/#/c/643130/
Change-Id: Iab6a3ca6bf92dc5606c7723dc6c62bf07fa1d934
Closes-Bug: #1648629
This reverts commit a7f7abc5b8d0e230bb9b586422df0438358b42d1.
The Cinder ScaleIO driver has been rebranded to VxFlexOS, which is fine.
What shouldn't have happened is that the rebranding made it's way
into os-brick, resulting in a new connector protocol and mapping.
The new VxFlexOS driver in cinder should just use the existing
scaleio connector protocol, as it's not customer/user facing, and
ensures backwards and forwards compatibility.
Change-Id: Ia1e119c16091bbf6ff79e0acea8c1a7c656d6dd1
This reverts commit 6b60614afaf88819ce92a2d0ed269035a3c8b261.
The Cinder ScaleIO driver has been rebranded to VxFlexOS, which is fine.
What shouldn't have happened is that the rebranding made it's way
into os-brick, resulting in a new connector protocol and mapping.
The new VxFlexOS driver in cinder should just use the existing
scaleio connector protocol, as it's not customer/user facing, and
ensures backwards and forwards compatibility.
Change-Id: Icf7063eb47368cb7b0ffb2e3f5e30903797c7b34
Trailing newline character is present in UUID returned by
_get_system_uuid method. This causes problems when the returned UUID
is not checked for newline. Remove trailing newline before returning
UUID.
Change-Id: Ibe37a14f67ce7452f088d7913a76c6233202ccea
Closes-Bug: #1819062
A long time ago the driver was changed to store volume_id in the
nova database and use that. Previously it would use the volume_name
and make a call to get the volume_id.
Both should be supported, but as the code path currently
throws a KeyError it will never reach the backwardscompatible
code.
This makes sure that the driver works even after an upgrade
from Mitaka.
Change-Id: Ic389e1a8f0f43ef410eee89202fef19d554fce90
Closes-Bug: #1648629
When ceph auth diasbled, both 'user' and 'keyring' will be 'None',
and the method _check_or_get_keyring_contents must return strings
for ceph conf file writelines() generation.
Change-Id: Ieafd40ace4f50c1a966c9097c5de270c68b0666b
Closes-Bug: #1817271
Dell EMC ScaleIO has been rebranded to VxFlex OS.
To follow this changes "scaleio" protocol renamed to "vxflexos",
"ScaleIOConnector" class renamed to "VxFlexOsConnector".
Old names will continue to work but will be removed in the Train
release.
Change-Id: I3bb1c985c64007c6960b6fa6f4cd893760a9a342
For FC connections there are multiple places where we check the
initiator target map provided by the backend against the port names of
the HBAs on the system.
Currently this check is case sensitive, but some backends are returning
the port names in the initiator target map upper cased, which usually
results in attach failures.
Some of the reasons for the attach failures is that os-brick will not
issue scan requests.
Example from a 3PAR backend from a specific system:
Connector properties:
{
'wwpns': ['10001409dcd71ff6', '10001409dcd71ff7'],
'wwnns': ['20001409dcd71ff6', '20001409dcd71ff7'],
...
}
Connection properties:
{
'initiator_target_map': {
'10001409DCD71FF6': ['20320002AC01E166', '21420002AC01E166'],
'10001409DCD71FF7': ['20410002AC01E166', '21410002AC01E166']
}
...
}
This patch converts to lower case the
initiator_target_map and the target_wwn/target_wwns.
Closes-Bug: #1775677
Change-Id: I12b9535d8a9969356394e406a1ed5ac4a5f1f959
Should check not only if 'initiator_target_map' is in conn_props but
also its value is not None.
Change-Id: I7ca0bf5fac20a370097f1c39ce005a1b23814459
Closes-Bug: #1746218
Missing dmidecode is only relevant for RSD backend, but the warning will
appear on all systems since this is logged on the get_connector_properties.
We don't really need dmidecode to get the UUID, we can just go and read
the contents of /sys/class/dmi/id/product_uuid
To better support systems that don't have dmidecode in the system, we
will check sysfs first, and then try dmidecode (in case it covers some
additional cases), and log it as a debug level message if dmidecode is
not present.
Closes-Bug: #1811822
Change-Id: Ibfaed6503bd91ad64987deecdc2e9a1471441005
If an exception occurs in _get_ips_iqns_luns(), a
TargetPortalsNotFound exception is raised, but nothing is logged. It
would be useful for debugging to know what led to the exception.
Change-Id: I2ed7f9641acc9d8d506c368a44d789950a4f25dd
Closes-Bug: 1814512
During restore, the backing vmdk is overwritten using backup
data. If the restored vmdk has a different uuid or size than the
original one, it will not be reflected in vSphere. Due to this
stale metadata, there is no way we can fix the uuid or backing
vmdk size if it changes after restore. The recommended way is to
detach the backing vmdk from the backend volume (shadow VM),
delete it and attach the restored vmdk. This will ensure update
of current vmdk size and uuid in vSphere.
Change-Id: Ib3d5c115f9076c19c3e33eaf6cccf630439a4c26
Partial-bug: 1814346
Partial-bug: 1814347
If a WWN has been provided in connection_properties, verify that the
found devices have that expected WWN. Fail the attachment if not. This
prevents cases where a device already exists on the host from an old
attachment that wasn't fully cleaned up, and the new attachment got
the same LUN. Using this old device could lead to data corruption.
Change-Id: I45a2221c0518213dc8132831c0bde9db4734da2b
Closes-Bug: 1813609
It is noticed that there could be some delays for a successful `nvme
connect` after "initialize_connection()" is completed in Cinder RSD
driver.
So add retries here, while keep the original logic intact.
Change-Id: I049f1d701c99612959d8262342aa63a043a97db0
* When execute `nvme connect`, "host-nqn" is needed in RSD environment.
So add it as an optional parameter.
* It is observed that the first `nvme list` can miss the newly connected
nvme device, right after a success call of `nvme connect`. The
solution is to add the retry&sleep logic back.
* Sometimes there can be more than 10 nvme devices, e.g.
"/dev/nvme10n10". Need to update the device re pattern.
* In RSD environment, the connector needs additional information like
"system uuid", to let cinder-volume know the nova-cpu service is
running on which RSD node.
* Add a new protocol "NVMEOF" mapping to allow both Cinder and Nova to
identify and use the nvme connector.
Implements: blueprint nvme-volume-driver
Change-Id: I26e3dc140b2cf30a97665679fdc4d2f897cd4872
When using Scale-Out File Server SMB shares, we have to use the local
share paths instead of the UNC paths if the host is part of the
cluster.
There's an os-win method that allows fetching the shared path but
it expects the share name, while we're passing subdirs as well,
for which reason it fails.
We'll have to make sure we're passing the share name only, appending
the subdir afterwards.
Closes-Bug: #1805341
Change-Id: I47d7e7afefe050cd4b25435cfcfc3e8963bd04bc
We introduced "shared_targets" and "service_uuid" fields in volumes to
allow volume consumers to protect themselves from unintended leftover
devices when handling iSCSI connections with shared targets.
The way they protect themselves from the automatic scans that happen on
detach/map race conditions is by locking and only allowing one attach or
one detach operation for each server to happen at a given time.
When using an up to date Open iSCSI initiator we don't need to use
locks, as it has the possibility to disable automatic LUN scans (which
are the real cause of the leftover devices), and OS-Brick already
supports this feature.
This is currently not the case, since Nova is blindly locking whenever
"shared_targets" is set to True.
Thanks to the context manager introduced in this patch we can improve
our separation of concerns (Nova doesn't have to care when we have to
lock), and we can fine tune the locking to only lock when the iSCSI
initiator doesn't support disabling automatic scans.
The context manager "guard_connection" lives in
"os_brick.initiator.utils" and needs a cinder volume. The volume can be
passed as a dictionary or OVO instance.
Change-Id: I4970363301d5d1f4e7d0f07e09b34d15ee6884c3
Related-Bug: #1800515
