749 Commits

Author SHA1 Message Date
Sean McGinnis
5dc5a55d33 Provide setting to ignore lvm descriptor leak warnings
For some reason the leaked descriptor warning message coming
from LVM is causing Cinder to fail startup and it appears to be
masking out the vg response in vgs calls.

We typically don't hit this, but due to the nature of Kolla and
I guess going through the different processes via the containers
this gets logged every time vgs is called.  Eric Harney rightly
pointed out that rather than use exception handling and such
that we should use the LVM env variable mechanism we already have
in place in Cinder.

This was added to the LVM local_dev code in the openstack/cinder repo,
but was not also added to os-brick. The long term goal is to have all of
this handling in os-brick, so this replicates the changes from cinder
done in I85612fa49475beea58d30330c8fe8352a2f91123.

Change-Id: I9d5aaad0e6213535afc1ce071f12190cc6aa02d0
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-05-07 21:53:37 +00:00
614b2cb1d5 Update master for stable/stein
Add file to the reno documentation build to show release notes for
stable/stein.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/stein.

Change-Id: I54570e4d32416def24d0f73364a3e415085724b3
Sem-Ver: feature
2019-03-18 14:41:42 +00:00
Yury Kulazhenkov
05b6c9f9fa Fix ScaleIO KeyError after upgrade
A long time ago the driver was changed to store volume_id
in the nova database and use that. Previously it would use the
volume_name and make a call to get the volume_id.

Both should be supported, but as the code path currently throws
a KeyError it will never reach the backwards compatible code.
This makes sure that the driver works even after an upgrade from Mitaka.

Depends-On: https://review.openstack.org/#/c/643130/
Change-Id: Iab6a3ca6bf92dc5606c7723dc6c62bf07fa1d934
Closes-Bug: #1648629
2.8.1
2019-03-14 05:41:24 +00:00
Walter A. Boring IV
3cfdf89837 Revert "rename ScaleIO connector to VxFlex OS"
This reverts commit a7f7abc5b8d0e230bb9b586422df0438358b42d1.

The Cinder ScaleIO driver has been rebranded to VxFlexOS, which is fine.
What shouldn't have happened is that the rebranding made it's way
into os-brick, resulting in a new connector protocol and mapping.
The new VxFlexOS driver in cinder should just use the existing
scaleio connector protocol, as it's not customer/user facing, and
ensures backwards and forwards compatibility.

Change-Id: Ia1e119c16091bbf6ff79e0acea8c1a7c656d6dd1
2019-03-12 18:41:45 +00:00
Walter A. Boring IV
ba2168d8fe Revert "Fix VxFlexOs KeyError after upgrade"
This reverts commit 6b60614afaf88819ce92a2d0ed269035a3c8b261.

The Cinder ScaleIO driver has been rebranded to VxFlexOS, which is fine.
What shouldn't have happened is that the rebranding made it's way
into os-brick, resulting in a new connector protocol and mapping.
The new VxFlexOS driver in cinder should just use the existing
scaleio connector protocol, as it's not customer/user facing, and
ensures backwards and forwards compatibility.

Change-Id: Icf7063eb47368cb7b0ffb2e3f5e30903797c7b34
2019-03-12 18:38:26 +00:00
Avishay Traeger
6adbce5ef7 Revert "Verify WWN of connected iSCSI devices if passed"
This reverts commit 14be08d0b53836963a530e2990a6db73901cbdac.

Change-Id: I1c99cdff5074180a09c315954c743b2bb0565f53
Related-Bug: #1819577
2019-03-12 12:03:57 +02:00
Zuul
a22d316f79 Merge "Fix VxFlexOs KeyError after upgrade" 2019-03-11 12:22:04 +00:00
Zuul
71cc39438f Merge "Remove py35 from setup.cfg" 2019-03-11 12:22:03 +00:00
Tejdeep Kautharam
a2b72a967f Remove trailing newline character in UUID
Trailing newline character is present in UUID returned by
_get_system_uuid method. This causes problems when the returned UUID
is not checked for newline. Remove trailing newline before returning
UUID.

Change-Id: Ibe37a14f67ce7452f088d7913a76c6233202ccea
Closes-Bug: #1819062
2019-03-07 17:06:18 -08:00
Erik Olof Gunnar Andersson
6b60614afa Fix VxFlexOs KeyError after upgrade
A long time ago the driver was changed to store volume_id in the
nova database and use that. Previously it would use the volume_name
and make a call to get the volume_id.

Both should be supported, but as the code path currently
throws a KeyError it will never reach the backwardscompatible
code.

This makes sure that the driver works even after an upgrade
from Mitaka.

Change-Id: Ic389e1a8f0f43ef410eee89202fef19d554fce90
Closes-Bug: #1648629
2019-03-04 21:33:52 -08:00
whoami-rajat
b6de2cce49 Remove py35 from setup.cfg
This is a followup to Drop py35 jobs from gate and tox env[1].

[1] https://review.openstack.org/#/c/639387/

Change-Id: I9e27e95a4493d9887a7dd8c7d7c7c5b511277f95
2019-02-28 11:02:40 +05:30
Sean McGinnis
c88071dc19
Drop py35 jobs
Python 3.5 was the target runtime for the Rocky release. The current
target py3 runtime for Stein is Python 3.6, so there is no reason to
keep testing against the older version.

https://governance.openstack.org/tc/reference/runtimes/stein.html#python-runtime-for-stein

Change-Id: I3a8d2137a1d3158a476a6c23b8a6229b1dfc20f6
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2.8.0
2019-02-26 14:24:46 -06:00
Zuul
961e906849 Merge "add python 3.7 unit test job" 2019-02-26 20:11:26 +00:00
Zuul
eaff50d9a0 Merge "Fix get keyring content failed when ceph auth disabled" 2019-02-26 18:55:38 +00:00
Zuul
04c6c0cc29 Merge "rename ScaleIO connector to VxFlex OS" 2019-02-26 18:49:14 +00:00
Sahid Orentino Ferdjaoui
ddab8ea8ad add python 3.7 unit test job
See ML discussion here [1] for context.

[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135626.html

Story: #2004073
Co-authored-by: Charles Short <chucks@redhat.com>
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
Change-Id: Ic6d01c35f04482acfdab975a118ffa9178b906b0
2019-02-26 16:40:04 +01:00
Rui Yuan Dou
a6dff156d7 Fix get keyring content failed when ceph auth disabled
When ceph auth diasbled, both 'user' and 'keyring' will be 'None',
and the method _check_or_get_keyring_contents must return strings
for ceph conf file writelines() generation.

Change-Id: Ieafd40ace4f50c1a966c9097c5de270c68b0666b
Closes-Bug: #1817271
2019-02-26 09:07:48 +08:00
Zuul
318a5cf33d Merge "Py3: Fix invalid escape sequencees" 2019-02-25 19:59:42 +00:00
Yury Kulazhenkov
a7f7abc5b8 rename ScaleIO connector to VxFlex OS
Dell EMC ScaleIO has been rebranded to VxFlex OS.
To follow this changes "scaleio" protocol renamed to "vxflexos",
"ScaleIOConnector" class renamed to "VxFlexOsConnector".
Old names will continue to work but will be removed in the Train
release.

Change-Id: I3bb1c985c64007c6960b6fa6f4cd893760a9a342
2019-02-25 11:12:03 +03:00
Eric Harney
8938733ba8 Py3: Fix invalid escape sequencees
These throw warnings on Python 3.6.

Change-Id: If46e59792423b384bbb2d46c05ac0b57df06254b
2019-02-21 14:36:29 -05:00
Gorka Eguileor
037f9fe667 Fix FC case sensitive scanning
For FC connections there are multiple places where we check the
initiator target map provided by the backend against the port names of
the HBAs on the system.

Currently this check is case sensitive, but some backends are returning
the port names in the initiator target map upper cased, which usually
results in attach failures.

Some of the reasons for the attach failures is that os-brick will not
issue scan requests.

Example from a 3PAR backend from a specific system:

  Connector properties:
    {
     'wwpns': ['10001409dcd71ff6', '10001409dcd71ff7'],
     'wwnns': ['20001409dcd71ff6', '20001409dcd71ff7'],
     ...
    }

  Connection properties:
    {
     'initiator_target_map': {
       '10001409DCD71FF6': ['20320002AC01E166', '21420002AC01E166'],
       '10001409DCD71FF7': ['20410002AC01E166', '21410002AC01E166']
     }
     ...
    }

This patch converts to lower case the
initiator_target_map and the target_wwn/target_wwns.

Closes-Bug: #1775677
Change-Id: I12b9535d8a9969356394e406a1ed5ac4a5f1f959
2019-02-21 10:27:50 +01:00
Zuul
fb5ebc2f9e Merge "Make sure looping calls are properly mocked" 2019-02-20 18:25:30 +00:00
Zuul
6d88dbc7f2 Merge "Update hacking version" 2019-02-20 17:06:14 +00:00
Zuul
ae0a489b13 Merge "Handle None value 'inititator_target_map'" 2019-02-20 16:45:47 +00:00
Zuul
953cd147a1 Merge "VMware: Detach backing vmdk during disconnect" 2019-02-20 16:33:45 +00:00
Zuul
d9d278b6f9 Merge "iSCSI: log exception if portals not found" 2019-02-15 18:25:29 +00:00
Zuul
54bc00cc5f Merge "Remove time checks from test_custom_execute_timeout_* tests" 2019-02-15 18:22:52 +00:00
Zuul
e2cbf3f851 Merge "Add missing params in NoOpEncryptor" 2019-02-15 18:22:51 +00:00
Zuul
97c02d2bb8 Merge "Don't warn on missing dmidecode" 2019-02-15 18:02:17 +00:00
Sean McGinnis
4df3ef8f33
Make sure looping calls are properly mocked
The test test_connect_volume_device_not_valid in
FibreChannelConnectorTestCase was taking over 6 seconds to complete
since the FixedIntervalLoopingCall in the method under test was not
being mocked out, making the test hang until the timeout was reached.

This fixes this and any other connector test case by moving the mocking
with a test fake into the base connector test class setUp call.

Change-Id: I6b174d94b5518d6267c1fa75528a90d95a312419
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-02-08 09:52:02 -06:00
Sean McGinnis
959ec5d54d
Add slowest test output to end of test run
Change-Id: Ie74a663b147bde12456d9f64edca05cfb4a29136
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-02-07 14:43:32 -06:00
Sam Wan
53877a95bc
Handle None value 'inititator_target_map'
Should check not only if 'initiator_target_map' is in conn_props but
also its value is not None.

Change-Id: I7ca0bf5fac20a370097f1c39ce005a1b23814459
Closes-Bug: #1746218
2019-02-07 13:17:32 -06:00
Gorka Eguileor
08c8ef4ccf Don't warn on missing dmidecode
Missing dmidecode is only relevant for RSD backend, but the warning will
appear on all systems since this is logged on the get_connector_properties.

We don't really need dmidecode to get the UUID, we can just go and read
the contents of /sys/class/dmi/id/product_uuid

To better support systems that don't have dmidecode in the system, we
will check sysfs first, and then try dmidecode (in case it covers some
additional cases), and log it as a debug level message if dmidecode is
not present.

Closes-Bug: #1811822
Change-Id: Ibfaed6503bd91ad64987deecdc2e9a1471441005
2019-02-07 09:50:01 +01:00
Avishay Traeger
f3ad8c08fe iSCSI: log exception if portals not found
If an exception occurs in _get_ips_iqns_luns(), a
TargetPortalsNotFound exception is raised, but nothing is logged. It
would be useful for debugging to know what led to the exception.

Change-Id: I2ed7f9641acc9d8d506c368a44d789950a4f25dd
Closes-Bug: 1814512
2019-02-04 11:28:33 +02:00
Vipin Balachandran
9cf03af278 VMware: Detach backing vmdk during disconnect
During restore, the backing vmdk is overwritten using backup
data. If the restored vmdk has a different uuid or size than the
original one, it will not be reflected in vSphere. Due to this
stale metadata, there is no way we can fix the uuid or backing
vmdk size if it changes after restore. The recommended way is to
detach the backing vmdk from the backend volume (shadow VM),
delete it and attach the restored vmdk. This will ensure update
of current vmdk size and uuid in vSphere.

Change-Id: Ib3d5c115f9076c19c3e33eaf6cccf630439a4c26
Partial-bug: 1814346
Partial-bug: 1814347
2019-02-01 15:05:07 -08:00
Avishay Traeger
14be08d0b5 Verify WWN of connected iSCSI devices if passed
If a WWN has been provided in connection_properties, verify that the
found devices have that expected WWN. Fail the attachment if not. This
prevents cases where a device already exists on the host from an old
attachment that wasn't fully cleaned up, and the new attachment got
the same LUN. Using this old device could lead to data corruption.

Change-Id: I45a2221c0518213dc8132831c0bde9db4734da2b
Closes-Bug: 1813609
2019-01-30 08:49:52 +02:00
yenai
b92dd00535 Add missing params in NoOpEncryptor
Change-Id: Id8b6c3d0ef7f4c8d37232a81cab1e50458d32709
Closes-Bug: #1812790
2019-01-22 14:23:43 +08:00
ZhijunWei
e6ddc92efb Update hacking version
Use latest release 1.1.0 and compatible changes w.r.t pep8

Change-Id: I44910cd34930799dc04166a4e57498c84210f14d
2019-01-18 21:30:00 -05:00
Zuul
322554c386 Merge "Remove unused connection properties" 2019-01-11 00:52:36 +00:00
Yingxin
a0cb8779ce Add retry to nvme connect in nvme connector
It is noticed that there could be some delays for a successful `nvme
connect` after "initialize_connection()" is completed in Cinder RSD
driver.

So add retries here, while keep the original logic intact.

Change-Id: I049f1d701c99612959d8262342aa63a043a97db0
2019-01-09 12:52:44 +08:00
Zuul
d7ebf977bb Merge "Support RSD scenario of nvme connector" 2019-01-05 00:09:07 +00:00
Yingxin
891abdff2f Support RSD scenario of nvme connector
* When execute `nvme connect`, "host-nqn" is needed in RSD environment.
  So add it as an optional parameter.

* It is observed that the first `nvme list` can miss the newly connected
  nvme device, right after a success call of `nvme connect`. The
  solution is to add the retry&sleep logic back.

* Sometimes there can be more than 10 nvme devices, e.g.
  "/dev/nvme10n10". Need to update the device re pattern.

* In RSD environment, the connector needs additional information like
  "system uuid", to let cinder-volume know the nova-cpu service is
  running on which RSD node.

* Add a new protocol "NVMEOF" mapping to allow both Cinder and Nova to
  identify and use the nvme connector.

Implements: blueprint nvme-volume-driver
Change-Id: I26e3dc140b2cf30a97665679fdc4d2f897cd4872
2018-12-24 13:51:20 +08:00
Zuul
7fde113dff Merge "Windows SMBFS: fix using share subdirs" 2018-12-19 18:04:58 +00:00
Ivan Kolodyazhny
e40d94c69d Remove time checks from test_custom_execute_timeout_* tests
Time checks were introduced in I058ff0a0e5ad517507dc3cda39087c913558561d
and adjusted in I3c2874855c2f061e93ecd088c516696366350833 commits. They
are still unstable on same environments. Let's disable these checks
until we'll have a better solution with a propper mocking.

Change-Id: Ia9054d7516909f68b89b647b743ca1d3ddef954e
2018-12-17 19:15:05 +02:00
Rui Yuan Dou
881c958272 Fix create ceph conf failed when cephx disable
Change-Id: Ia7f5a7b90d8fd9812aa2b244bec06a076ca39fdd
Closes-Bug: #1808322
2018-12-13 19:48:22 +08:00
Zuul
83f263ccf1 Merge "Tests: Fix PrivRootwrapTestCase failure" 2018-12-05 21:23:52 +00:00
Zuul
96b993c1b2 Merge "Fix NFS "already mounted" detection" 2018-12-05 19:35:59 +00:00
Eric Harney
4635996f87 Tests: Fix PrivRootwrapTestCase failure
On a slow enough machine, this can take more than 0.3
seconds to execute.

Change-Id: I3c2874855c2f061e93ecd088c516696366350833
2018-12-05 10:31:18 -05:00
qingszhao
48097057bb Change openstack-dev to openstack-discuss
Mailinglists have been updated. Openstack-discuss replaces openstack-dev.

Change-Id: Ic8589bef7e15316e1d1aa036b2a7c002eb5897ef
2018-12-04 17:15:35 +00:00
Eric Harney
360a20c7c7 Fix NFS "already mounted" detection
Previous fix

2f32c98a RemoteFS: don't fail in do_mount if already mounted

was made with the assumption that "already mounted" appearing
in stderr from an NFS mount means that the NFS share was already
mounted.

However, the NFS client can fail with "busy or already mounted"
in cases where it fails to mount the share as well.

This results in the Cinder NFS backup driver believing that the
NFS mount has succeeded when it didn't, and as a result, data
can be written to the local disk instead of the NFS target.

Fail if the share isn't actually mounted rather than succeeding.

Related-Bug: #1780813
Closes-Bug: #1797233
Change-Id: Iebd1afb3340fcaeb1969784966c4f9be35a28417
2018-12-04 13:38:15 +00:00