180 Commits

Author SHA1 Message Date
Ghanshyam Mann
65b5986a9e Set up the doc infra and add doc job
devstack-plugins-ceph does not have the doc infra set
up and no doc publish job.

This is need to add the contributor doc as per the ussuri
community goal.

Change-Id: Ib62c34b52b28b753f7f3046d51492fd8d9fb6193
2020-04-22 00:54:27 +00:00
Lee Yarwood
a7876566b4 zuul: Switch devstack-plugin-ceph-tempest-py3 to voting
While the job itself is not currently voting within openstack/nova the
use of the plugin essentially is within the nova-live-migration job.

As such we should really enable voting of this job within this project
with the eventual goal of also enabling it within openstack/nova and
other projects using it.

Change-Id: Id243b92e800e2d6e8b2228fadbd1c72b194bbfb9
2020-03-13 14:08:07 +00:00
Ghanshyam Mann
dd2db52ac8 Stop running devstack-plugin-ceph-tempest on master
devstack-plugin-ceph-tempest is py2 job and not supposed
to run on master gate. devstack-plugin-ceph master gate already
run its py3 version devstack-plugin-ceph-tempest-py3.

We need to keep the devstack-plugin-ceph-tempest definition
for stable branches gate.

Change-Id: I6c512b29a0346db1f92de34b1dbfb56958fa2cad
2020-02-18 21:04:10 -06:00
Lee Yarwood
299e958635 Enable extend_attached_volume feature
I5698e451861828a8b1240d046d1610d8d37ca5a2 introduced support for this
test in Stein so enable the compute feature from Stein onwards.

Blueprint: extend-in-use-rbd-volumes
Change-Id: I6b2575b12fdb9636f106b3ca28681e29ad3eb8ac
2019-12-06 14:56:37 +00:00
Stephen Finucane
856dbe5280 Only install python3 packages if necessary
In change I43de36ab570af40bc837d859568f5428c167f083, we started
installing the 'python3-' variants of the 'python-' packages installed
by the 'ceph-common' package in 'install_ceph_remote'. In that change,
we made the assertion that we didn't need to "both hiding this behind
the 'python3_enabled' function or 'USE_PYTHON3' variable since (a) these
packages can happily coexist, (b) projects should be switching to Python
3 now, and (c) using these would require sourcing additional, otherwise
unnecessary devstack libraries just to use this simple function".
However, this didn't take into account the fact that this plugin is not
versioned. This means *any* job that uses this plugin, including things
like stable/pike, will now attempt to install the 'python3-' packages.
Since the base OS on stable/pike, Xenial (16.04), doesn't actually
provide these 'python3-' variants, the command will always fail.

Start hiding the command behind a 'use_python3' function call. Hopefully
by time we drop Python 2 support here we won't have maintained branches
using Xenial. If we do, we'll need to branch or something like that.

Change-Id: I890a1e673fde212218eff4fcf82f7941b3283e5f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-Bug: #1853280
2019-11-20 13:19:23 +00:00
Zuul
ee8cc0e9d4 Merge "Use vercmp" 2019-11-18 16:10:53 +00:00
Zuul
ee65cca853 Merge "Allow choosing Ceph release on CentOS" 2019-11-18 15:55:02 +00:00
Stephen Finucane
3a36c1c6d2 Install 'python3-' variants of packages in 'install_ceph_remote'
Previously, this command would simply install the 'ceph-common' package,
which includes a number of Python dependencies. Unfortunately, these
packages are the 'python-' variants [1] and there doesn't appear to be a
'ceph-common-python3' package or similar to install the 'python3-'
variants. As a result, we need to install the 'python3-' variants
manually to ensure consumers of the plugin (like nova) can switch to
Python 3 in their tests.

We don't bother hiding this behind the 'python3_enabled' function or
'USE_PYTHON3' variable since (a) these packages can happily coexist, (b)
projects should be switching to Python 3 now, and (c) using these would
require sourcing additional, otherwise unnecessary devstack libraries
just to use this simple function.

[1] https://packages.ubuntu.com/bionic/ceph-common

Change-Id: I43de36ab570af40bc837d859568f5428c167f083
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2019-11-14 15:27:16 +00:00
Tom Barron
1c9e3e772e Manila: clean pools unconditionally
Cleanup of CephFS data and metadata pools for manila was only
being done for external CephFS installs.

Do it whenever manila is enabled.

Change-Id: Iee238f91b04231d1e41774a49615023fe26e56f3
2019-09-22 20:47:47 +00:00
Lee Yarwood
04871bee0c tempest: re-enable shelve tests for stable queens
Related-Bug: #1653953
Depends-On: https://review.opendev.org/#/c/668123/
Change-Id: I3e1f9683a28a2e2e67a0327031f0670f33391126
2019-08-13 09:58:36 +00:00
Eric Harney
874e54abeb Use vercmp
Use devstack's vercmp function instead of
bc for version comparisons.

Change-Id: I6436a6f836ad1ab9594858986bfdb8d6d6dbc392
2019-08-09 10:16:16 -04:00
Zuul
248df63d78 Merge "Update syntax for cephfs snapshot enablement" 2019-08-08 17:18:14 +00:00
Zuul
b2c9af8247 Merge "Update call for keys creation" 2019-08-08 17:15:55 +00:00
Zuul
a7b863d7db Merge "Add stable versions of devstack-plugin-ceph-tempest-py3" 2019-08-08 17:15:54 +00:00
Eric Harney
1413355064 Allow choosing Ceph release on CentOS
Change-Id: I293ccf2111e2983465fcd8c12b3ea931ad446344
2019-08-06 09:52:35 -04:00
vkmc
ae9d524b79 Update syntax for cephfs snapshot enablement
We should be using "ceph fs set <fs_name> allow_new_snaps true"
instead of using the mds call

Change-Id: I7a709d49ec4c93b2a06e961ca8ca1414cf4c1918
Closes-Bug: #1838790
2019-08-02 13:31:40 -03:00
vkmc
cbdcb8c10a Update call for keys creation
ceph-create-keys was being used to create keys.
This command has been deprecated and it does nothing.
For this reason, keys were not available when starting
daemons and this made the deployment to fail.

This change adds the correct key creation using ceph-authtool.

Change-Id: Ie1a41f67bbf1c7a58e89168b0c8ad63f101648b4
Co-Authored-By: Giulio Fidente <gfidente@redhat.com>
2019-08-02 13:16:47 -03:00
Tom Barron
945f41b398 run IPv6 scenario tests with CephFS/NFS back end
Depends-on: https://review.opendev.org/#/c/671134/
Depends-on: https://review.opendev.org/#/c/670510/
Change-Id: Ib812d2b960b1375129c7ff74c63c1acd2649102f
2019-07-24 18:50:35 +00:00
Ghanshyam Mann
0a8488b071 Add stable versions of devstack-plugin-ceph-tempest-py3
devstack-plugin-ceph is branchless and its jobs run on
stable branches. To tests the devstack-plugin-ceph changes
for stable branches, let' add the stable version of
devstack-plugin-ceph-tempest-py3 job in experimental pipeline.

Adding in experimental pipeline as we do not really have to run
those all the time on check gate but can be run on demand.

Change-Id: I9010f0d6226304537adcbe4e3f97250c754a7de9
2019-07-04 04:48:42 +00:00
Zuul
1221819b6c Merge "Disable [validation]/run_validation in tempest.conf" 2019-07-03 20:17:18 +00:00
Zuul
f371e892b1 Merge "Change experimental pipeline jobs" 2019-07-02 19:16:44 +00:00
Lee Yarwood
ffcba4bb55 tempest: re-enable shelve tests for stable rocky
Related-Bug: #1653953
Depends-On: https://review.opendev.org/#/c/668119/
Change-Id: I1aa87fc16a37b6051e55c35ae69ef911a87d4c11
2019-06-28 10:31:17 +01:00
Lee Yarwood
2590c305f1 tempest: re-enable shelve tests for stable stein
Related-Bug: #1653953
Depends-On: https://review.opendev.org/668118/
Change-Id: I132e26486782918d31a9ac10332f2802342883a9
2019-06-28 10:27:32 +01:00
Lee Yarwood
fdb1efbe0a tempest: re-enable shelve tests for train
The dependent nova fix resolves the issue with rbd and
shelve/unshelve testing in tempest. Re-enabling the test
needs to be branch-specific since devstack-plugin-ceph
is branch-less but the fixes in nova will be branch
specific so we don't want to enable this on any nova
branch that does not have the fix since it would break
the job on those branches.

Related-Bug: #1653953

Depends-On: https://review.opendev.org/457886/

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Change-Id: Ifbfc15033020c430a93574f2726b2279c9be130d
2019-06-27 21:49:12 -04:00
Walter A. Boring IV
ff1b423178 Add ceph-iscsi support
This patch adds support for installing, configuring all required
ceph-iscsi related packages and daemons. ceph-iscsi requires kernel
version 4.16 or greater.

To enable set
ENABLE_CEPH_ISCSI=True

Adds new config options
CEPH_ISCSI_TARGET_IQN - the rbd-target-gw iqn to use to export volumes
CEPH_ISCSI_API_HOST - the rbd-target-api ip to use to listen on
CEPH_ISCSI_API_PORT - the rbd-target-api port
CEPH_ISCSI_API_USER - the rbd-target-api user for auth against the api
CEPH_ISCSI_API_PASSWORD - the rbd-target-api password for auth against the api

The following packages are required and their deps
tcmu-runner - this requires kernel version 4.16 and > to work.  Ubuntu
doesn't have a prebuilt package for this, so we pull it down and compile it.
rtslib-fb - has a pypi package
configshell-fb - has a pypi package
targetcli-fb - doesn't have a pypi package.
ceph-iscsi - doesn't have a pypi package.

The reference documentaion to setup ceph-iscsi targets and daemons:
http://docs.ceph.com/docs/master/rbd/iscsi-target-cli-manual-install

Change-Id: Idfac4883b9d31e80124b8f2f1cc60dd9ce18c594
2019-06-27 11:30:57 -07:00
Goutham Pacha Ravi
0ba1a7215c Change experimental pipeline jobs
The manila CentOS jobs break often, and are not
maintained. Let's move them to experimental
so they don't gate every patch. The other manila
jobs currently provide necessary test coverage
to cephfs-native and cephfs-nfs protocols/drivers.
These jobs also run with python3, and we don't
need to support python3.5 any longer.

Change-Id: Ibb752a1d9aa818dd6862bcbbce4d6b6e366b4800
2019-06-12 17:30:03 -07:00
Zuul
90f1d21a83 Merge "Enable scenario tests with CEPHFS" 2019-06-10 12:29:20 +00:00
Zuul
a40357d4aa Merge "Skip tempest.api.network tests" 2019-05-29 14:12:46 +00:00
Goutham Pacha Ravi
5ca1f397ba Enable scenario tests with CEPHFS
Allow running api and scenario tests
together in the manila test jobs.

Change-Id: Ifbf998d7c8a27ad22a39f46547d1548c052614d0
2019-05-16 14:07:55 -07:00
Matt Riedemann
4a44819953 Disable [validation]/run_validation in tempest.conf
Devstack and tempest default to enable ssh validation in
a lot of API tests which can fail intermittently:

  http://status.openstack.org/elastic-recheck/#1808010

Also see bug 1802971.

Since a ceph backend for storage shouldn't really have
any impact on whether or not ssh is going to work in a
tempest run, this change disables it in the
devstack-plugin-ceph-tempest jobs which should make the
job faster and more stable.

Change-Id: Iea948bcc43315286e5c130485728152d4710bfcb
2019-04-24 11:28:43 -04:00
Matt Riedemann
5e11880882 Skip tempest.api.network tests
There is really no point in running the tempest.api.network
tests in the devstack-plugin-ceph-tempest job since a ceph
backend shouldn't make any difference to those tests.

This should speed up the job slightly and more importantly
avoid unnecessarily random failures from tests that don't matter
for this setup.

Change-Id: Icf8d776a648b93b232b83fd4493e00b5f76b3ebf
2019-04-24 10:11:20 -04:00
Le Hou
07e83e50b2 Update opendev repository
Change-Id: I63c00c728033649be33511a068b52b9f15cfa5d4
2019-04-23 17:52:51 +08:00
OpenDev Sysadmins
29510701eb OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:40:27 +00:00
Zuul
51f8550b02 Merge "Only import libvirt secret when nova is enabled" 2019-04-01 17:51:17 +00:00
Goutham Pacha Ravi
2a2fb1aeee Fix readme for manila/ubuntu support
TrivialFix

Change-Id: Icd481b55e7684a67175fe6e5c2e2ef62d6e7046c
2019-03-25 12:52:21 -07:00
Ian Wienand
182a33f440 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I0ebfde9478ec07f9f8649e239adfccbf6f7431c6
2019-03-24 20:33:29 +00:00
ghanshyam
4fbdae805f Disable multiattach feature for stable branches before stein
Below patch enabled the multiattach feature in devstack ceph job
without branch variant. Multiattach feature is only available
from stein.
- https://review.openstack.org/#/c/638499/2

This commit makes two variant of job with multiattach enable from
stein  onwards only.

Change-Id: Ib0a5667a6125fa5ae430086db93baf5a53be4877
Closes-Bug: #1820098
2019-03-14 17:29:32 +00:00
Jon Bernard
b69c941d5c Enable multiattach tests in gate jobs
Depends-On: Ie3945427b54544a3b411c23bffdad1acb5e508e1
Change-Id: I520b3db106f72cddabfde4d71eee67607d7a4558
2019-02-27 19:38:39 +00:00
Eric Harney
13d81f499b Trim tempest scenario test list
We are running a handful of scenarios that aren't related
to storage here.

Change-Id: I8a22fcfb0454e2267179d07fe67b9e04927145ff
2019-02-21 14:13:58 -05:00
Zuul
f61f618a69 Merge "Disable volume multiattach testing" 2019-02-08 16:11:22 +00:00
Tom Barron
74a88c0d3b Always use luminous for manila
Bring CentOS in line with ubuntu and use luminous rather than
jewel for all manila builds.

Change-Id: Ib0924e42551d25b68ee22ffb9b52caba302ac19c
2019-02-07 14:48:47 -05:00
Matt Riedemann
f4e44320a9 Disable volume multiattach testing
Tempest change I522a15ba3dbfee5d8ef417e43288a12319abf6ff is
enabling volume multiattach testing in the tempest-full and
tempest-full-py3 job configurations which we inherit for
devstack-plugin-ceph-tempest and devstack-plugin-ceph-tempest-py3
respectively. Since the rbd driver in cinder does not support
multiattach yet we need to disable multiattach testing for these
jobs.

Change-Id: Ia9bd6a42f33a7d9fe3041cf5c751ab691df19cb7
2019-02-05 11:48:53 -05:00
Tom Barron
2badb448e3 Use luminous for xenial and after
Change-Id: I82cc5232870a820c4c8a0526eb44de45cd67a369
2019-01-31 14:51:08 -05:00
Tom Barron
d7878f2cba Use shaman again for manila
Builds for ubuntu bionic are working now.
We need newer features not yet packaged for ubuntu bionic
for python3 and ceph volume chmod support.

Change-Id: I20fa99c6f6da7d4042b50020ab0e9f34e7ac2de6
2019-01-29 08:50:20 -05:00
Zuul
ed50be6dd4 Merge "Support manila with python3" 2019-01-29 11:52:38 +00:00
Zuul
9e1de83bf0 Merge "Allow setting test API microversions in gate tests" 2019-01-29 11:32:10 +00:00
Tom Barron
424375c739 Allow setting test API microversions in gate tests
Add MANILA_TEMPEST_MIN_API_MICROVERSION and
MANILA_TEMPEST_MAX_API_MICROVERSION to the ci
post_test_hook. These options default to
the values specified in the api_version_request.py
file.  On master branch don't set these as we
want to run with the latest.

Related-bug: #1781671
Change-Id: I085ed175dfbe132c80a6a86ab34a281b3c94f5f0
2019-01-28 14:08:44 -05:00
Tom Barron
a8be9e71a6 Support manila with python3
Use python3 compatible tempest test discovery and
install the relevant python3 libraries.

Change-Id: Iabd1952ddaba85316dec5cf60c89122dc3825912
2019-01-28 11:06:12 -05:00
Tom Barron
59edb045f2 add bionic jobs for manila
Change-Id: I893ec0c97540e76d368a3cbfd8ebe63a0685a16b
2019-01-22 12:18:48 -05:00
Tom Barron
c669989cd8 Use official bionic repos for manila
The shaman ceph and ganesha repos for ubuntu bionic are less
stable than those available with the bionic distro itself so
use the latter.

Also refactor the ceph/ganesha repo configuration so that it
is easier to read and maintain, and since bionic does not ship
with knfs installed make the service stop/disable for these
services robust in that case.

Change-Id: If00cc82cfb076c75a11738dcaca372b0cc7a1848
2019-01-03 06:17:22 -05:00