Commit Graph

4353 Commits

Author SHA1 Message Date
Goutham Pacha Ravi
148e473006 [CI] Add bindep.txt
Bindep is a tool for checking the presence of binary
packages needed to use an application / library. It
started life as a way to make it easier to set up a
development environment for OpenStack projects.

Infra has been looking to drop the global
bindep_fallback.txt [1][2] file for all
OpenStack projects, they recommend that projects
maintain their own with packages as necessary.

This commit is not a clean cherry-pick.
DevStack for CentOS is currently broken. See
bug 1833696 [3]. The LVM job will be fixed
by a subsequent commit to this branch.

[1] http://lists.openstack.org/pipermail/openstack-infra/2018-April/005926.html
[2] http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006888.html
[3] https://launchpad.net/bugs/1833696
Change-Id: I1824f132b0ffadc6c9179582d37295f3abed355f
(cherry picked from commit 1709fd7fc2)
(cherry picked from commit 6ac1730127)
(cherry picked from commit 4fbe70fbb6)
2019-07-04 00:16:05 +00:00
OpenDev Sysadmins
390aa443fe OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:27:41 +00:00
Goutham Pacha Ravi
72fb56c5c4 Fix tls-proxy issues with the devstack plugin
Enabling tls-proxy allows devstack to
set up a tls proxy server that front-ends
interactions with the manila-api and
terminates tls connections.

Also enable tls-proxy in dummy and lvm
jobs. The dummy driver job is configured
to run the in-built wsgi server, the lvm
job is configured to use mod-wsgi.

Closes-Bug: #1816836
Change-Id: I48b0ccc082604d78242ba61bee94a45efeb2467b
(cherry picked from commit 8f1c7dc91f)
(cherry picked from commit edc60f76c2)
2019-04-15 19:58:34 +00:00
Tom Barron
625fce5bb0 Only allow IP access type for CephFS NFS
For the CephFS NFS back end only ``IP`` access type
is valid so enforce this in the driver.

Also validate access level since there is a utility
routine to check both access type and access level.

Closes-bug: #1816420
Change-Id: I6c96f861b30ef7ccac05a7c199a62f0d69044c3a
(cherry picked from commit 9df750fd19)
(cherry picked from commit 08148bba49)
2019-04-03 21:30:29 +00:00
Ian Wienand
6820efaafd Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I325f3118749f4c17155650cf059132635b62b0d3
2019-03-24 20:34:09 +00:00
Helen Walsh
011662c3bf VMAX manila - deprecate old tags correctly
Aligning with VNX/Unity terminology in
966b1621d3
Old tags were replaced without deprecating correctly.  Also adding a
release note for this change

Change-Id: If180f1a2ea8c307c8a40f2a17b35796357306c03
Closes-Bug: #1798393
(cherry picked from commit ff3857b563)
2019-03-21 10:30:11 +00:00
Zuul
28b43ebcaf Merge "Manila VMAX docs - differences between quotas" into stable/queens 2019-03-20 20:05:39 +00:00
Zuul
fed9d4ca5a Merge "Manila VMAX docs - improve pre-configurations on VMAX section" into stable/queens 2019-03-20 20:05:38 +00:00
Zuul
9d25c33271 Merge "Manila VMAX docs - clarify snapshot support" into stable/queens 2019-03-20 20:05:36 +00:00
Zuul
499537105b Merge "Manila VMAX docs - clarify driver_handles_share_servers" into stable/queens 2019-03-20 14:45:03 +00:00
Zuul
6511d2075b Merge "VMAX manila doc - SSL Support" into stable/queens 2019-03-20 14:45:02 +00:00
Zuul
b1821892db Merge "VMAX manila doc - use of correct VMAX tags" into stable/queens 2019-03-20 14:45:00 +00:00
Zuul
95dde5ea48 Merge "Destroy type quotas when a share type is deleted" into stable/queens 2019-03-20 14:14:04 +00:00
Helen Walsh
7c14283837 Manila VMAX docs - differences between quotas
Highlight the difference between nas_quotas and OpenStack manila
quotas.

Change-Id: I9eb2eb48b377c707191dc2f4e0a8a6f2dc606fe6
(cherry picked from commit 9ab3b8e80b)
2019-03-20 11:52:37 +00:00
Helen Walsh
988c63ec29 Manila VMAX docs - improve pre-configurations on VMAX section
Add storage pool and license configuration to the Pre-configurations
on VMAX section.

Change-Id: I9f598c6918455de7344d80d2663d9fd3a4cc3ed9
(cherry picked from commit 2528762a69)
2019-03-20 11:22:46 +00:00
Helen Walsh
9ffe3901c7 Manila VMAX docs - clarify snapshot support
As snapshot support is disabled by default, this clarifies how to enable
it for both create snapshot of share and create share from snapshot.

Change-Id: I40a7b8c72b8ce67689c6b9a769cdb3399a3c6242
(cherry picked from commit 6bc9b90cab)
2019-03-20 11:18:47 +00:00
Helen Walsh
e229873328 Manila VMAX docs - clarify driver_handles_share_servers
Clarify that only driver_handles_share_servers = True is supported in
the VMAX manila driver.

Change-Id: Iac40b4d250a87d8c993ec2db5ad1d7e03eab189b
(cherry picked from commit 9aac703672)
2019-03-20 10:34:07 +00:00
Helen Walsh
9f9878718c VMAX manila doc - SSL Support
Documenting the steps for setting up the Secure Socket Layer for
trusted communication between the manila host and the eNas Control
Station.

Change-Id: Ia2efb004125acea08d8bf72d99958c9a4f512dfd
(cherry picked from commit c8a0d1715f)
2019-03-20 10:31:38 +00:00
Helen Walsh
d10334e146 VMAX manila doc - use of correct VMAX tags
https://review.openstack.org/#/c/608725/ now aligns terminology with VNX and
Unity.  Updating he documentation accordingly.

Change-Id: I9dc91a48daed9779b02068b4fdd05a07227135dc
(cherry picked from commit d21179e48a)
2019-03-20 10:29:53 +00:00
snpd
c75d48c067 Fix driver filter to not check share_backend_name
Fixed the bug stating: Driver filter disallows using
queries with share_backend_name. The driver filter was
checking for hard equality between the share_backend_name
specified in the share type and the name reported by the
host [1]. This defeats the purpose of the capabilities
filter giving the ability to use "<in>" (selection)
operator in the extra-spec. Thus this commit fixes it by
updating the required files according to
openstack/cinder/commit/b32011 as now the driver does not
check for shared_backend_name.

Closes-Bug: #1815700

Change-Id: If384392d0f140922bb85a1dc50b0c4494c231e2f
(cherry picked from commit 86c1576110)
(cherry picked from commit ce370a1892)
2019-03-20 03:54:23 +00:00
Zuul
15a47ef681 Merge "Extend remove_version_from_href support" into stable/queens 2019-03-19 20:14:28 +00:00
Goutham Pacha Ravi
c9f9541193 Destroy type quotas when a share type is deleted
Upon share type deletion, we were leaving behind
share type quotas, usages and reservations. These
couldn't be cleaned up without resorting to manual
intervention.

Cleanup quotas when a share type is being destroyed,
since the resources that they pertained to are gone
too.

Closes-Bug: #1811680
Change-Id: I04b1fe88808596aa8c05429b964190d654587f9a
(cherry picked from commit 4b6cfcf690)
(cherry picked from commit aae7bc7235)
2019-03-19 14:34:27 +00:00
Goutham Pacha Ravi
07c4809940 Fix API version inferred w/ un-versioned URLs
With [1], we fixed the issue with v1 API URLs when
configuring manila with a reverse proxy, like uwsgi.
However, the version discovery can be made against a
"unversioned" URL, i.e, the "/" endpoint.

Example version discovery target when there's no reverse
proxy:

   curl -i -X GET www.openstack-overcloud.com:8786/

Example version discovery target when manila is configured
with a web proxy:

   curl -i -X GET www.openstack-overcloud.com/shared-file-system/

Currently, the API assumes that a v2 endpoint is requested
and hence sets a default API version in the request, which
results in the wrong headers communicated to the client.

Fix this issue. The release note added with [1] should
suffice for this fix.

[1] I0363d7174f3d7ddefa8ced59b182faed665e9c36
Change-Id: I50d1024ee8485b8290c24fa850e60755b518fff3
Closes-Bug: #1818081
(Cherry-picked from commit 5a3be01535)
(Cherry-picked from commit 4b0c953f7f)
2019-03-18 12:10:23 +00:00
Goutham Pacha Ravi
587a1b2ad3 Fix version selector when for proxy-style URLs
When manila API is served behind a proxy, the
"script_name" in the request can have the proxy
component in it. So, this patch fixes the version
selection logic by looking for the version in the
script name string instead of equivalence.

In addition, this patch adds some missing unit
tests and fixes tests that invoke a mocked
wsgi app for testing request context.

Change-Id: I0363d7174f3d7ddefa8ced59b182faed665e9c36
Partial-Bug: #1815038
Closes-Bug: #1818081
(cherry picked from commit 0d8310ec7a)
(cherry picked from commit 6b39562c0e)
2019-03-17 13:56:22 -07:00
Victoria Martinez de la Cruz
4bfff6055b Extend remove_version_from_href support
Improve remove_version_from_href so that it supports URLs for which
the version does not directly trail the hostname

Given: 'http://manila.example.com/share/v1.1/123'
Returns: 'http://manila.example.com/share/123'

Based on I8ccc449116ff164aacc0aefca3a0ec2ac8f73aa8

Change-Id: I13010d3ee0a83a67204145837503abe7126263cd
Closes-Bug: #1815038
(cherry picked from commit 6d970b7375)
(cherry picked from commit 670449f3e1)
2019-03-17 20:55:34 +00:00
Goutham Pacha Ravi
534aed13d7 Return request-id to APIs that don't respond with a body
Request-IDs help track actions initiated via the API.
With this patch, we can supply the request ID in the
following APIs:
- POST {resource}/action APIs for actions such as
  'reset-state', 'force-delete', 'shrink',
  'extend', 'manage', 'deny_access', 'addProjectAccess',
  'removeProjectAccess', 'promote', 'resync', 'unmanage',
  'revert', 'migration_start', 'migration_complete',
  'migration_cancel'
- POST /share_unmanage/{id}
- DELETE /{resource}/{id} APIs for
  shares, share networks, share groups, share replicas,
  security services, share servers, share snapshots,
  extra specs, messages, quota sets, share-group-snapshots,
  group specs, share-group-types, share types

Change-Id: I64ed14b5a44d6e1b37cfc9321b25bdf4e0aabea5
Closes-Bug: #1815532
(cherry picked from commit 52db0754fe)
(cherry picked from commit ce111ed14e)
2019-03-15 17:42:56 +00:00
Tom Barron
00bd9230de Port dummy driver manage/unmanage changes to stable
Branchless manila-tempest-plugin had changes for
the manage share server feature that required changes in
the dummy driver for its manage and unmanage routines.

Here we test porting the dummy driver changes for the
manage and unmanage share and snapshot methods, omitting
the manage and unmanage methods for share servers since
these are not supported in the stable branches.

Note that backporting such changes to a stable branch
for a regular driver would likely be unacceptable given
stable policy but the dummy driver is test-only code that
never runs outside of tests.

Change-Id: Ib9e5dd3ad6b745ea1dfe442d9357a70ca4c148ec
(cherry picked from commit 6b25b0a407)
2019-03-13 21:30:12 +00:00
Zuul
1cb508a941 Merge "add tripleo scenario004 to experimental queue" into stable/queens 2019-02-19 12:06:41 +00:00
Zuul
8e6cd32a90 Merge "Set mode for CephFS volumes and snapshots" into stable/queens 2019-02-19 03:51:29 +00:00
Zuul
4877b2139b Merge "[LVM] Run filesystem check before assigning UUID" into stable/queens 2019-02-18 20:38:18 +00:00
Goutham Pacha Ravi
e9e4a3a6e3 [LVM] Run filesystem check before assigning UUID
We assign a random UUID with tune2fs to snapshots
and shares created from snapshots so that they
don't conflict with the parent shares/snapshots
when both of them are being mounted.

tune2fs requires that a filesystem check be
performed "recently" before UUID assignments.
So, perform a filesystem check right away to
allow tune2fs to assign a random UUID.

Depends-on: https://review.openstack.org/#/c/637565
Change-Id: I858a318f7a83e033cc3f2699859e38b6b74c8d24
Related-Bug: #1645751
Closes-Bug: #1798219
(cherry picked from commit 817cce347a)
(cherry picked from commit 55507cc603)
2019-02-18 16:56:56 +00:00
Tom Barron
5bfe7f335e Fix service image boot issues
We are getting:

403 errors 'Only volume-backed servers are allowed for flavors
with zero disk' on scenario tests.

Appears to be due to this change [1] which just merged to
nova master branch.  Ubuntu bionic server doc says the root disk
needs 1.5 GB so we are setting the flavor definition to require
2GB.

[1] c8e65a5eb1

Closes-bug: #1816050
Change-Id: Iba0a15b78bf75a04c1ac0e64e70634772b2dca5c
(cherry picked from commit 68b79e2cea)
(cherry picked from commit fcb40df777)
2019-02-18 15:09:21 +00:00
Tom Barron
1fa04f15a6 add tripleo scenario004 to experimental queue
A recent change [1] in master added a tripleo scenario004
job to the experimental queue so we can run it before merging
changes that might affect downstream tripleo CI.  We can't
cherry-pick that change because that scenario isn't supported
on single node CI in stable/rocky.

Here we add a multinode version of that scenario.

Change-Id: I2743ce3c472d940c23baf3955e09a81427e381e1
(cherry picked from commit 6510f79111)
2019-02-18 07:33:47 -05:00
Tom Barron
cf98b95973 Set mode for CephFS volumes and snapshots
Corresponding ceph_volume_client PR [1]


[1] https://github.com/ceph/ceph/pull/24839

Depends-on: https://review.openstack.org/#/c/634028/
Change-Id: Iad78b8cc25c34d3675cd38ce9c757ad0e411b613
(cherry picked from commit 721bb70a81)
(cherry picked from commit 4831c624ee)
2019-02-13 22:46:34 +00:00
Tom Barron
4de3117566 Bump timeout on dsvm jobs
Many time out even when all tests are succeeding and
we will save more resources not having to recheck than by
keeping the timeout where it is.

Change-Id: I2a84e7a1340ccccb79170196b810aeda7d12914d
(cherry picked from commit 1471376131)
(cherry picked from commit 94ee6dfde8)
2019-02-13 14:55:10 -05:00
Zuul
db125e1dd6 Merge "Fix ganesha for 0.0.0.0/0 access" into stable/queens 2019-02-12 14:04:50 +00:00
Tom Barron
40792c69a7 Fix ganesha for 0.0.0.0/0 access
Translate '0.0.0.0/0' to '0.0.0.0' when allowing IP
access with the ganesha driver since the ganesha back end
cannot handle the former expression and the latter has
the desired effect of allowing access to all IPs.

Depends-on: https://review.openstack.org/#/c/626879/
Closes-bug: #1800627
Change-Id: Ica698b6a70a128522c2c2de76a69e59207fd60ac
(cherry picked from commit 67f2f61395)
(cherry picked from commit a65c2b0974)
2019-02-08 11:01:48 +00:00
Tom Barron
13f121ad20 Only run the needed services for CephFS jobs
These jobs often fail due to timeouts so only enable the services
required for devstack tests with these back ends.

Change-Id: Ib620ec3c2a9999ea9908a7c23aa7172c40a4cbd9
(cherry picked from commit f096d1145c)
(cherry picked from commit a950ce0da5)
2019-01-29 17:27:15 +00:00
Zuul
2049332391 Merge "Set paramiko logging to DEBUG level" into stable/queens 2019-01-11 20:44:30 +00:00
Tom Barron
674ee3fe03 Change ssh_utils parameter to correctly send keepalive packets
The current implementation of ssh_utils never sends keepalive packets.
In ssh_utils.SSHPool, the socket timeout parameter is set to None
intending to keep ssh connections open.
However, when the parameter is set to None, ssh_utils does not run the
code to compare idle duration and keepalive interval.

This patch reverts the socket timeout parameter to default (0.1 sec).
The ssh_utils compares them every 0.1 seconds, and sends a keepalive
packet if idle duration > keepalive interval (= self.conn_timeout).

See cinder change: I8234083107207b9ebc0849947e8de92b5cf3e36e

Change-Id: Ib13e5c6246412d667554cb0bd9c419b513af70c7
Related-Bug: #1673662
(cherry picked from commit 1814ad4113)
(cherry picked from commit 742e30922b)
2019-01-10 22:37:40 +00:00
Tom Barron
0b47c687e0 Set paramiko logging to DEBUG level
Connections to backends via paramiko often fail for obscure
reasons.

If the service log level is DEBUG, increase paramiko logging
to DEBUG level as well so that we have more information when issues
of this sort occur.

Change-Id: I44d8902d58c2ad48b8a37dfbff96c5b9471651e4
(cherry picked from commit 575d6ae621)
(cherry picked from commit 0cde7b098d)
2019-01-10 21:01:41 +00:00
Tom Barron
041f60a7b0 Improve service instance module debug logging
Change-Id: Ib3a9bebe362609d7198e053afebc6004f3d94baf
(cherry picked from commit 2117632c50)
(cherry picked from commit bb6256b5e8)
2019-01-09 22:16:00 +00:00
Zuul
6043e91e4a Merge "Adjust ssh timeouts" into stable/queens 2019-01-09 15:49:11 +00:00
Zuul
aedbc54c66 Merge "Deprecate old keystone session config opts" into stable/queens 2019-01-08 19:57:03 +00:00
Tom Barron
2abde73d1e Adjust ssh timeouts
Generic driver jobs are failing because of timeouts when
establishing the initial ssh connection from manila-share
to the service VM.

Bump up the default value of the connection timeout for paramiko
client and also set the banner timeout since the failure occurred
during banner exchange.  Set the two timeouts to the same value
for now.  This ensures that the connection timeout is at least as
long as the banner timeout and there is no current need in manila
to control these independently.

This is more of a workaround than a real fix since a real fix
would remove the delay during banner exchange.  I suspect that
the real fix will need to be in neutron/ovs though.

Change-Id: Ib5e59faaf9667b9cb5e7d4072531b7d6c3d4da39
Partial-bug: #1807216
(cherry picked from commit 7548706b09)
(cherry picked from commit 8919db0186)
2019-01-07 13:42:31 +00:00
Goutham Pacha Ravi
71fede769c Deprecate old keystone session config opts
In the past, the options ca_certificates_file,
nova_ca_certificates_file, cinder_ca_certificates_file,
api_insecure, nova_api_insecure, cinder_api_insecure
were supplied to instantiate nova, neutron and cinder
clients. These options have now been subsumed in a more
generic way into the Keystone session logic as 'cafile'
and 'insecure'. Deprecate the older options in Stein so
that we can remove them in a future release.

This deprecation began many releases ago when we switched
to using keystone sessions [1]. However, we were still
overriding the values of "insecure" and "cafile" if provided,
forcing users to continue using deprecated parameters
"api_insecure" and "ca_certificates_file". So despite
this fix originating in the Stein release, it would be
prudent to backport it to all maintained releases and
remove support for these older options in/beyond Train
release (9.0.0).

[1] Ic211a11308a3295409467efd88bff413482ee58d
Change-Id: I148e9079c7c1ab119f519f727d4ad97758473325
Related-Bug: #1802393
Closes-Bug: #1809318
(cherry picked from commit 198bea78ac)
(cherry picked from commit 4947cb0f6c)
2019-01-03 11:53:36 -08:00
Tom Barron
68f6578adc Fix image_name retrieval in custom-image jobs
And use a 'file://' url for the custom manila image.

See also https://review.openstack.org/#/c/623330.  These
are interdependent patches but no harm is done by merging
this one first and having 623330 depend on it since retrieval
of the image name is not working correctly anyways.

Partial-Bug: #1807969

Change-Id: Ib3a74d170f159dfbc38b5302f00800f3530f4921
(cherry picked from commit 6299ed5ed0)
(cherry picked from commit 758eaac8cc)
2018-12-28 17:52:18 +00:00
Tom Barron
8e930cafe3 remove glusterfs-nfs job from check queue
The job never passes, is unmaintained, and
wastes CI resources.

Change-Id: I05d7ee7f8ed01b98f2ec6cf629af1d5002b2cdbb
(cherry picked from commit fe3c1f5df3)
(cherry picked from commit d1b195c81e)
2018-12-27 09:32:02 +00:00
Silvan Kaiser
d5ff8bb949 Fix mutable default argument in Quobyte jsonrpc
Change-Id: I03d86c45aed3238cfa82de3eea5bc44c285cc118
Closes-Bug: #1773929
(cherry picked from commit 5deab9b9cf)
2018-10-29 14:58:56 +00:00
Zuul
36068e0223 Merge "change tox envlist from 3.5 to 3" into stable/queens 2018-10-22 21:47:11 +00:00