openstack/manila
Code Issues Proposed changes

Update micversion to API2.69, Manila share support Recycle Bin

Browse Source 
Add support share Recycle Bin, the end user can soft delete
share to Recycle Bin, and can restore the share within 7 days,
otherwise the share will be deleted automatically.

DocImpact
APIImpact
Partially-Implements: blueprint manila-share-support-recycle-bin

Change-Id: Ic838eec5fea890be6513514053329b1d2d86b3ba
This commit is contained in:
haixin 2021-07-14 16:34:28 +08:00 committed by haixin
parent 7c04fcb904
commit d51eb05c05
37 changed files with 1138 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
api-ref/source/parameters.yaml
View File

@ -326,6 +326,15 @@ is_public_query:
in: query
required: false
type: boolean
is_soft_deleted_query:
description: |
A boolean query parameter that, when set to True, will return all shares
in recycle bin. Default is False, will return all shares not in recycle
bin.
in: query
required: false
type: boolean
min_version: 2.69
limit:
description: |
The maximum number of shares to return.
@ -1390,6 +1399,13 @@ is_public_shares_response:
in: body
required: true
type: boolean
is_soft_deleted_response:
description: |
Whether the share has been soft deleted to recycle bin or not.
in: body
required: false
type: boolean
min_version: 2.69
links:
description: |
Pagination and bookmark links for the resource.
@ -2321,6 +2337,14 @@ revert_to_snapshot_support_share_capability:
required: true
type: boolean
min_version: 2.27
scheduled_to_be_deleted_at_response:
description: |
Estimated time at which the share in the recycle bin will be deleted
automatically.
in: body
required: false
type: string
min_version: 2.69
scheduler_hints:
description: |
One or more scheduler_hints key and value pairs as a dictionary of

3
api-ref/source/samples/share-actions-restore-request.json Normal file
View File

@ -0,0 +1,3 @@
{
"restore": null
}

3
api-ref/source/samples/share-actions-soft-delete-request.json Normal file
View File

@ -0,0 +1,3 @@
{
"soft_delete": null
}

94
api-ref/source/share-actions.inc
View File

@ -500,3 +500,97 @@ Request example
.. literalinclude:: samples/share-actions-revert-to-snapshot-request.json
:language: javascript
Soft delete share (since API v2.69)
===================================
.. rest_method:: POST /v2/shares/{share_id}/action
.. versionadded:: 2.69
Soft delete a share to recycle bin.
Preconditions
- Share status must be ``available``, ``error`` or ``inactive``
- Share can't have any snapshot.
- Share can't have a share group snapshot.
- Share can't have dependent replicas.
- You cannot soft delete share that already is in the Recycle Bin..
- You cannot soft delete a share that doesn't belong to your project.
- You cannot soft delete a share is busy with an active task.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 202
.. rest_status_code:: error status.yaml
- 400
- 401
- 403
- 404
- 409
Request
-------
.. rest_parameters:: parameters.yaml
- project_id: project_id_path
- share_id: share_id
Request example
---------------
.. literalinclude:: samples/share-actions-soft-delete-request.json
:language: javascript
Restore share (since API v2.69)
===============================
.. rest_method:: POST /v2/shares/{share_id}/action
.. versionadded:: 2.69
Restore a share from recycle bin.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 202
.. rest_status_code:: error status.yaml
- 401
- 403
- 404
Request
-------
.. rest_parameters:: parameters.yaml
- project_id: project_id_path
- share_id: share_id
Request example
---------------
.. literalinclude:: samples/share-actions-restore-request.json
:language: javascript

4
api-ref/source/shares.inc
View File

@ -130,6 +130,7 @@ Request
- name~: name_inexact_query
- description~: description_inexact_query
- with_count: with_count_query
- is_soft_deleted: is_soft_deleted_query
- limit: limit
- offset: offset
- sort_key: sort_key
@ -198,6 +199,7 @@ Request
- name~: name_inexact_query
- description~: description_inexact_query
- with_count: with_count_query
- is_soft_deleted: is_soft_deleted_query
- limit: limit
- offset: offset
- sort_key: sort_key
@ -242,6 +244,8 @@ Response parameters
- volume_type: volume_type_shares_response
- export_location: export_location
- export_locations: export_locations
- is_soft_deleted: is_soft_deleted_response
- scheduled_to_be_deleted_at: scheduled_to_be_deleted_at_response
Response example

8
manila/api/openstack/api_version_request.py
View File

@ -170,19 +170,25 @@ REST_API_VERSION_HISTORY = """
actions on the share network's endpoint:
'update_security_service', 'update_security_service_check' and
'add_security_service_check'.
* 2.64 - Added 'force' field to extend share api, which can extend share
directly without validation through share scheduler.
* 2.65 - Added ability to set affinity scheduler hints via the share
create API.
* 2.66 - Added filter search by group spec for share group type list.
* 2.67 - Added ability to set 'only_host' scheduler hint for the share
create and share replica create API.
* 2.68 - Added admin only capabilities to share metadata API
* 2.69 - Added new share action to soft delete share to recycle bin or
restore share from recycle bin. Also, a new parameter called
`is_soft_deleted` was added so users can filter out
shares in the recycle bin while listing shares.
"""
# The minimum and maximum versions of the API supported
# The default api version request is defined to be the
# minimum version of the API supported.
_MIN_API_VERSION = "2.0"
_MAX_API_VERSION = "2.68"
_MAX_API_VERSION = "2.69"
DEFAULT_API_VERSION = _MIN_API_VERSION

9
manila/api/openstack/rest_api_version_history.rst
View File

@ -376,4 +376,11 @@ ____
2.68
----
Added admin only capabilities to share metadata API
Added admin only capabilities to share metadata API.
2.69
----
Manila support Recycle Bin. Soft delete share to Recycle Bin: ``POST
/v2/shares/{share_id}/action {"soft_delete": null}``. List shares in
Recycle Bin: `` GET /v2/shares?is_soft_deleted=true``. Restore share from
Recycle Bin: `` POST /v2/shares/{share_id}/action {'restore': null}``.

8
manila/api/v1/share_snapshots.py
View File

@ -190,6 +190,14 @@ class ShareSnapshotMixin(object):
LOG.error(msg)
raise exc.HTTPUnprocessableEntity(explanation=msg)
# we do not allow soft delete share with snapshot, and also
# do not allow create snapshot for shares in recycle bin,
# since it will lead to auto delete share failed.
if share['is_soft_deleted']:
msg = _("Snapshots cannot be created for share '%s' "
"since it has been soft deleted.") % share_id
raise exc.HTTPForbidden(explanation=msg)
LOG.info("Create snapshot from share %s",
share_id, context=context)

4
manila/api/v1/share_unmanage.py
View File

@ -38,6 +38,10 @@ class ShareUnmanageMixin(object):
try:
share = self.share_api.get(context, id)
if share.get('is_soft_deleted'):
msg = _("Share '%s cannot be unmanaged, "
"since it has been soft deleted.") % share['id']
raise exc.HTTPForbidden(explanation=msg)
if share.get('has_replicas'):
msg = _("Share %s has replicas. It cannot be unmanaged "
"until all replicas are removed.") % share['id']

32
manila/api/v1/shares.py
View File

@ -135,6 +135,11 @@ class ShareMixin(object):
'with_count', search_opts)
search_opts.pop('with_count')
if 'is_soft_deleted' in search_opts:
is_soft_deleted = utils.get_bool_from_api_params(
'is_soft_deleted', search_opts)
search_opts['is_soft_deleted'] = is_soft_deleted
# Deserialize dicts
if 'metadata' in search_opts:
search_opts['metadata'] = ast.literal_eval(search_opts['metadata'])
@ -192,7 +197,7 @@ class ShareMixin(object):
'is_public', 'metadata', 'extra_specs', 'sort_key', 'sort_dir',
'share_group_id', 'share_group_snapshot_id', 'export_location_id',
'export_location_path', 'display_name~', 'display_description~',
'display_description', 'limit', 'offset')
'display_description', 'limit', 'offset', 'is_soft_deleted')
@wsgi.Controller.authorize
def update(self, req, id, body):
@ -218,6 +223,11 @@ class ShareMixin(object):
except exception.NotFound:
raise exc.HTTPNotFound()
if share.get('is_soft_deleted'):
msg = _("Share '%s cannot be updated, "
"since it has been soft deleted.") % share['id']
raise exc.HTTPForbidden(explanation=msg)
update_dict = common.validate_public_share_policy(
context, update_dict, api='update')
@ -443,6 +453,10 @@ class ShareMixin(object):
access_data.pop('metadata', None)
share = self.share_api.get(context, id)
if share.get('is_soft_deleted'):
msg = _("Cannot allow access for share '%s' "
"since it has been soft deleted.") % id
raise exc.HTTPForbidden(explanation=msg)
share_network_id = share.get('share_network_id')
if share_network_id:
share_network = db.share_network_get(context, share_network_id)
@ -490,6 +504,12 @@ class ShareMixin(object):
'deny_access', body.get('os-deny_access'))['access_id']
share = self.share_api.get(context, id)
if share.get('is_soft_deleted'):
msg = _("Cannot deny access for share '%s' "
"since it has been soft deleted.") % id
raise exc.HTTPForbidden(explanation=msg)
share_network_id = share.get('share_network_id', None)
if share_network_id:
@ -521,6 +541,11 @@ class ShareMixin(object):
share, size, force = self._get_valid_extend_parameters(
context, id, body, 'os-extend')
if share.get('is_soft_deleted'):
msg = _("Cannot extend share '%s' "
"since it has been soft deleted.") % id
raise exc.HTTPForbidden(explanation=msg)
try:
self.share_api.extend(context, share, size, force=force)
except (exception.InvalidInput, exception.InvalidShare) as e:
@ -536,6 +561,11 @@ class ShareMixin(object):
share, size = self._get_valid_shrink_parameters(
context, id, body, 'os-shrink')
if share.get('is_soft_deleted'):
msg = _("Cannot shrink share '%s' "
"since it has been soft deleted.") % id
raise exc.HTTPForbidden(explanation=msg)
try:
self.share_api.shrink(context, share, size)
except (exception.InvalidInput, exception.InvalidShare) as e:

20
manila/api/v2/share_instances.py
View File

@ -21,6 +21,7 @@ from manila.api.views import share_instance as instance_view
from manila import db
from manila import exception
from manila import share
from manila import utils
class ShareInstancesController(wsgi.Controller, wsgi.AdminActionsMixin):
@ -72,7 +73,7 @@ class ShareInstancesController(wsgi.Controller, wsgi.AdminActionsMixin):
instances = db.share_instances_get_all(context)
return self._view_builder.detail_list(req, instances)
@wsgi.Controller.api_version("2.35") # noqa
@wsgi.Controller.api_version("2.35", "2.68") # noqa
@wsgi.Controller.authorize
def index(self, req): # pylint: disable=function-redefined # noqa F811
context = req.environ['manila.context']
@ -84,6 +85,23 @@ class ShareInstancesController(wsgi.Controller, wsgi.AdminActionsMixin):
instances = db.share_instances_get_all(context, filters)
return self._view_builder.detail_list(req, instances)
@wsgi.Controller.api_version("2.69") # noqa
@wsgi.Controller.authorize
def index(self, req): # pylint: disable=function-redefined # noqa F811
context = req.environ['manila.context']
filters = {}
filters.update(req.GET)
common.remove_invalid_options(
context, filters, ('export_location_id', 'export_location_path',
'is_soft_deleted'))
if 'is_soft_deleted' in filters:
is_soft_deleted = utils.get_bool_from_api_params(
'is_soft_deleted', filters)
filters['is_soft_deleted'] = is_soft_deleted
instances = db.share_instances_get_all(context, filters)
return self._view_builder.detail_list(req, instances)
@wsgi.Controller.api_version("2.3")
@wsgi.Controller.authorize
def show(self, req, id):

5
manila/api/v2/share_replicas.py
View File

@ -169,6 +169,11 @@ class ShareReplicationController(wsgi.Controller, wsgi.AdminActionsMixin):
msg = _("No share exists with ID %s.")
raise exc.HTTPNotFound(explanation=msg % share_id)
if share_ref.get('is_soft_deleted'):
msg = _("Replica cannot be created for share '%s' "
"since it has been soft deleted.") % share_id
raise exc.HTTPForbidden(explanation=msg)
share_network_id = share_ref.get('share_network_id', None)
if share_network_id:

15
manila/api/v2/share_snapshots.py
View File

@ -116,18 +116,29 @@ class ShareSnapshotsController(share_snapshots.ShareSnapshotMixin,
description = snapshot_data.get(
'display_description', snapshot_data.get('description'))
share_id = snapshot_data['share_id']
snapshot = {
'share_id': snapshot_data['share_id'],
'share_id': share_id,
'provider_location': snapshot_data['provider_location'],
'display_name': name,
'display_description': description,
}
try:
share_ref = self.share_api.get(context, share_id)
except exception.NotFound:
raise exception.ShareNotFound(share_id=share_id)
if share_ref.get('is_soft_deleted'):
msg = _("Can not manage snapshot for share '%s' "
"since it has been soft deleted.") % share_id
raise exc.HTTPForbidden(explanation=msg)
driver_options = snapshot_data.get('driver_options', {})
try:
snapshot_ref = self.share_api.manage_snapshot(context, snapshot,
driver_options)
driver_options,
share=share_ref)
except (exception.ShareNotFound, exception.ShareSnapshotNotFound) as e:
raise exc.HTTPNotFound(explanation=e.msg)
except (exception.InvalidShare,

98
manila/api/v2/shares.py
View File

@ -66,6 +66,11 @@ class ShareController(shares.ShareMixin,
share = self.share_api.get(context, share_id)
snapshot = self.share_api.get_snapshot(context, snapshot_id)
if share.get('is_soft_deleted'):
msg = _("Share '%s cannot revert to snapshot, "
"since it has been soft deleted.") % share_id
raise exc.HTTPForbidden(explanation=msg)
# Ensure share supports reverting to a snapshot
if not share['revert_to_snapshot_support']:
msg_args = {'share_id': share_id, 'snap_id': snapshot_id}
@ -219,11 +224,29 @@ class ShareController(shares.ShareMixin,
@wsgi.Controller.api_version('2.0', '2.6')
@wsgi.action('os-reset_status')
def share_reset_status_legacy(self, req, id, body):
context = req.environ['manila.context']
try:
share = self.share_api.get(context, id)
except exception.NotFound:
raise exception.ShareNotFound(share_id=id)
if share.get('is_soft_deleted'):
msg = _("status cannot be reset for share '%s' "
"since it has been soft deleted.") % id
raise exc.HTTPForbidden(explanation=msg)
return self._reset_status(req, id, body)
@wsgi.Controller.api_version('2.7')
@wsgi.action('reset_status')
def share_reset_status(self, req, id, body):
context = req.environ['manila.context']
try:
share = self.share_api.get(context, id)
except exception.NotFound:
raise exception.ShareNotFound(share_id=id)
if share.get('is_soft_deleted'):
msg = _("status cannot be reset for share '%s' "
"since it has been soft deleted.") % id
raise exc.HTTPForbidden(explanation=msg)
return self._reset_status(req, id, body)
@wsgi.Controller.api_version('2.0', '2.6')
@ -236,6 +259,60 @@ class ShareController(shares.ShareMixin,
def share_force_delete(self, req, id, body):
return self._force_delete(req, id, body)
@wsgi.Controller.api_version('2.69')
@wsgi.action('soft_delete')
def share_soft_delete(self, req, id, body):
"""Soft delete a share."""
context = req.environ['manila.context']
LOG.debug("Soft delete share with id: %s", id, context=context)
try:
share = self.share_api.get(context, id)
self.share_api.soft_delete(context, share)
except exception.NotFound:
raise exc.HTTPNotFound()
except exception.InvalidShare as e:
raise exc.HTTPForbidden(explanation=e.msg)
except exception.ShareBusyException as e:
raise exc.HTTPForbidden(explanation=e.msg)
except exception.Conflict as e:
raise exc.HTTPConflict(explanation=e.msg)
return webob.Response(status_int=http_client.ACCEPTED)
@wsgi.Controller.api_version('2.69')
@wsgi.action('restore')
def share_restore(self, req, id, body):
"""Restore a share from recycle bin."""
context = req.environ['manila.context']
LOG.debug("Restore share with id: %s", id, context=context)
try:
share = self.share_api.get(context, id)
except exception.NotFound:
msg = _("No share exists with ID %s.")
raise exc.HTTPNotFound(explanation=msg % id)
# If the share not exist in Recycle Bin, the API will return
# success directly.
is_soft_deleted = share.get('is_soft_deleted')
if not is_soft_deleted:
return webob.Response(status_int=http_client.OK)
# If the share has reached the expired time, and is been deleting,
# it too late to restore the share.
if share['status'] in [constants.STATUS_DELETING,
constants.STATUS_ERROR_DELETING]:
msg = _("Share %s is being deleted or error deleted, "
"cannot be restore.")
raise exc.HTTPForbidden(explanation=msg % id)
self.share_api.restore(context, share)
return webob.Response(status_int=http_client.ACCEPTED)
@wsgi.Controller.api_version('2.29', experimental=True)
@wsgi.action("migration_start")
@wsgi.Controller.authorize
@ -247,6 +324,12 @@ class ShareController(shares.ShareMixin,
except exception.NotFound:
msg = _("Share %s not found.") % id
raise exc.HTTPNotFound(explanation=msg)
if share.get('is_soft_deleted'):
msg = _("Migration cannot start for share '%s' "
"since it has been soft deleted.") % id
raise exception.InvalidShare(reason=msg)
params = body.get('migration_start')
if not params:
@ -355,6 +438,15 @@ class ShareController(shares.ShareMixin,
@wsgi.action("reset_task_state")
@wsgi.Controller.authorize
def reset_task_state(self, req, id, body):
context = req.environ['manila.context']
try:
share = self.share_api.get(context, id)
except exception.NotFound:
raise exception.ShareNotFound(share_id=id)
if share.get('is_soft_deleted'):
msg = _("task state cannot be reset for share '%s' "
"since it has been soft deleted.") % id
raise exc.HTTPForbidden(explanation=msg)
return self._reset_status(req, id, body, status_attr='task_state')
@wsgi.Controller.api_version('2.0', '2.6')
@ -482,6 +574,9 @@ class ShareController(shares.ShareMixin,
if req.api_version_request < api_version.APIVersionRequest("2.42"):
req.GET.pop('with_count', None)
if req.api_version_request < api_version.APIVersionRequest("2.69"):
req.GET.pop('is_soft_deleted', None)
return self._get_shares(req, is_detail=False)
@wsgi.Controller.api_version("2.0")
@ -496,6 +591,9 @@ class ShareController(shares.ShareMixin,
req.GET.pop('description~', None)
req.GET.pop('description', None)
if req.api_version_request < api_version.APIVersionRequest("2.69"):
req.GET.pop('is_soft_deleted', None)
return self._get_shares(req, is_detail=True)

7
manila/api/views/shares.py
View File

@ -36,6 +36,7 @@ class ViewBuilder(common.ViewBuilder):
"add_mount_snapshot_support_field",
"add_progress_field",
"translate_creating_from_snapshot_status",
"add_share_recycle_bin_field",
]
def summary_list(self, request, shares, count=None):
@ -197,3 +198,9 @@ class ViewBuilder(common.ViewBuilder):
@common.ViewBuilder.versioned_method("2.54")
def add_progress_field(self, context, share_dict, share):
share_dict['progress'] = share.get('progress')
@common.ViewBuilder.versioned_method("2.69")
def add_share_recycle_bin_field(self, context, share_dict, share):
share_dict['is_soft_deleted'] = share.get('is_soft_deleted')
share_dict['scheduled_to_be_deleted_at'] = share.get(
'scheduled_to_be_deleted_at')

5
manila/common/config.py
View File

@ -127,6 +127,11 @@ global_opts = [
help="Specify list of protocols to be allowed for share "
"creation. Available values are '%s'" %
list(constants.SUPPORTED_SHARE_PROTOCOLS)),
cfg.IntOpt('soft_deleted_share_retention_time',
default=604800,
help='Maximum time (in seconds) to keep a share in the recycle '
'bin, it will be deleted automatically after this amount '
'of time has elapsed.'),
]
CONF.register_opts(global_opts)

32
manila/db/api.py
View File

@ -451,6 +451,15 @@ def share_get_all_by_share_server(context, share_server_id, filters=None,
sort_dir=sort_dir)
def get_shares_in_recycle_bin_by_share_server(
context, share_server_id, filters=None,
sort_key=None, sort_dir=None):
"""Returns all shares in recycle bin with given share server ID."""
return IMPL.get_shares_in_recycle_bin_by_share_server(
context, share_server_id, filters=filters, sort_key=sort_key,
sort_dir=sort_dir)
def share_get_all_by_share_server_with_count(
context, share_server_id, filters=None, sort_key=None, sort_dir=None):
"""Returns all shares with given share server ID."""
@ -459,11 +468,29 @@ def share_get_all_by_share_server_with_count(
sort_dir=sort_dir)
def get_shares_in_recycle_bin_by_network(
context, share_network_id, filters=None,
sort_key=None, sort_dir=None):
"""Returns all shares in recycle bin with given share network ID."""
return IMPL.get_shares_in_recycle_bin_by_network(
context, share_network_id, filters=filters, sort_key=sort_key,
sort_dir=sort_dir)
def share_delete(context, share_id):
"""Delete share."""
return IMPL.share_delete(context, share_id)
def share_soft_delete(context, share_id):
"""Soft delete share."""
return IMPL.share_soft_delete(context, share_id)
def share_restore(context, share_id):
"""Restore share."""
return IMPL.share_restore(context, share_id)
###################
@ -1077,6 +1104,11 @@ def share_server_get_all_unused_deletable(context, host, updated_before):
updated_before)
def get_all_expired_shares(context):
"""Get all expired share DB records."""
return IMPL.get_all_expired_shares(context)
def share_server_backend_details_set(context, share_server_id, server_details):
"""Create DB record with backend details."""
return IMPL.share_server_backend_details_set(context, share_server_id,

56
manila/db/migrations/alembic/versions/1946cb97bb8d_add_is_soft_deleted_and_scheduled_to_be_deleted_at_to_shares_table.py Normal file
View File

@ -0,0 +1,56 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""add is_soft_deleted and scheduled_to_be_deleted_at to shares table
Revision ID: 1946cb97bb8d
Revises: fbdfabcba377
Create Date: 2021-07-14 14:41:58.615439
"""
# revision identifiers, used by Alembic.
revision = '1946cb97bb8d'
down_revision = 'fbdfabcba377'
from alembic import op
from oslo_log import log
import sqlalchemy as sa
LOG = log.getLogger(__name__)
def upgrade():
try:
op.add_column('shares', sa.Column(
'is_soft_deleted', sa.Boolean,
nullable=False, server_default=sa.sql.false()))
op.add_column('shares', sa.Column(
'scheduled_to_be_deleted_at', sa.DateTime))
except Exception:
LOG.error("Columns shares.is_soft_deleted "
"and/or shares.scheduled_to_be_deleted_at not created!")
raise
def downgrade():
try:
op.drop_column('shares', 'is_soft_deleted')
op.drop_column('shares', 'scheduled_to_be_deleted_at')
LOG.warning("All shares in recycle bin will automatically be "
"restored, need to be manually identified and deleted "
"again.")
except Exception:
LOG.error("Column shares.is_soft_deleted and/or "
"shares.scheduled_to_be_deleted_at not dropped!")
raise

97
manila/db/sqlalchemy/api.py
View File

@ -47,6 +47,7 @@ from sqlalchemy import MetaData
from sqlalchemy import or_
from sqlalchemy.orm import joinedload
from sqlalchemy.orm import subqueryload
from sqlalchemy.sql.expression import false
from sqlalchemy.sql.expression import literal
from sqlalchemy.sql.expression import true
from sqlalchemy.sql import func
@ -1652,6 +1653,16 @@ def share_instances_get_all(context, filters=None, session=None):
models.ShareInstanceExportLocations.uuid ==
export_location_id)
query = query.join(
models.Share,
models.Share.id ==
models.ShareInstance.share_id)
is_soft_deleted = filters.get('is_soft_deleted')
if is_soft_deleted:
query = query.filter(models.Share.is_soft_deleted == true())
else:
query = query.filter(models.Share.is_soft_deleted == false())
instance_ids = filters.get('instance_ids')
if instance_ids:
query = query.filter(models.ShareInstance.id.in_(instance_ids))
@ -1987,7 +1998,7 @@ def _process_share_filters(query, filters, project_id=None, is_public=False):
if filters is None:
filters = {}
share_filter_keys = ['share_group_id', 'snapshot_id']
share_filter_keys = ['share_group_id', 'snapshot_id', 'is_soft_deleted']
instance_filter_keys = ['share_server_id', 'status', 'share_type_id',
'host', 'share_network_id']
share_filters = {}
@ -2196,6 +2207,11 @@ def _share_get_all_with_filters(context, project_id=None, share_server_id=None,
if share_server_id:
filters['share_server_id'] = share_server_id
# if not specified is_soft_deleted filter, default is False, to get
# shares not in recycle bin.
if 'is_soft_deleted' not in filters:
filters['is_soft_deleted'] = False
query = _process_share_filters(
query, filters, project_id, is_public=is_public)
@ -2228,6 +2244,25 @@ def _share_get_all_with_filters(context, project_id=None, share_server_id=None,
return query
@require_admin_context
def get_all_expired_shares(context):
query = (
_share_get_query(context).join(
models.ShareInstance,
models.ShareInstance.share_id == models.Share.id
)
)
filters = {"is_soft_deleted": True}
query = _process_share_filters(query, filters=filters)
scheduled_deleted_attr = getattr(models.Share,
'scheduled_to_be_deleted_at', None)
now_time = timeutils.utcnow()
query = query.filter(scheduled_deleted_attr.op('<=')(now_time))
result = query.all()
return result
@require_admin_context
def share_get_all(context, filters=None, sort_key=None, sort_dir=None):
project_id = filters.pop('project_id', None) if filters else None
@ -2302,6 +2337,19 @@ def share_get_all_by_share_server(context, share_server_id, filters=None,
return query
@require_context
def get_shares_in_recycle_bin_by_share_server(
context, share_server_id, filters=None, sort_key=None, sort_dir=None):
"""Returns list of shares in recycle bin with given share server."""
if filters is None:
filters = {}
filters["is_soft_deleted"] = True
query = _share_get_all_with_filters(
context, share_server_id=share_server_id, filters=filters,
sort_key=sort_key, sort_dir=sort_dir)
return query
@require_context
def share_get_all_by_share_server_with_count(
context, share_server_id, filters=None, sort_key=None, sort_dir=None):
@ -2312,6 +2360,19 @@ def share_get_all_by_share_server_with_count(
return count, query
@require_context
def get_shares_in_recycle_bin_by_network(
context, share_network_id, filters=None, sort_key=None, sort_dir=None):
"""Returns list of shares in recycle bin with given share network."""
if filters is None:
filters = {}
filters["share_network_id"] = share_network_id
filters["is_soft_deleted"] = True
query = _share_get_all_with_filters(context, filters=filters,
sort_key=sort_key, sort_dir=sort_dir)
return query
@require_context
def share_delete(context, share_id):
session = get_session()
@ -2330,6 +2391,40 @@ def share_delete(context, share_id):
filter_by(share_id=share_id).soft_delete())
@require_context
@oslo_db_api.wrap_db_retry(max_retries=5, retry_on_deadlock=True)
def share_soft_delete(context, share_id):
session = get_session()
now_time = timeutils.utcnow()
time_delta = datetime.timedelta(
seconds=CONF.soft_deleted_share_retention_time)
scheduled_to_be_deleted_at = now_time + time_delta
update_values = {
'is_soft_deleted': True,
'scheduled_to_be_deleted_at': scheduled_to_be_deleted_at
}
with session.begin():
share_ref = share_get(context, share_id, session=session)
share_ref.update(update_values)
share_ref.save(session=session)
@require_context
@oslo_db_api.wrap_db_retry(max_retries=5, retry_on_deadlock=True)
def share_restore(context, share_id):
session = get_session()
update_values = {
'is_soft_deleted': False,
'scheduled_to_be_deleted_at': None
}
with session.begin():
share_ref = share_get(context, share_id, session=session)
share_ref.update(update_values)
share_ref.save(session=session)
###################

2
manila/db/sqlalchemy/models.py
View File

@ -315,6 +315,8 @@ class Share(BASE, ManilaBase):
source_share_group_snapshot_member_id = Column(String(36), nullable=True)
task_state = Column(String(255))
is_soft_deleted = Column(Boolean, default=False)
scheduled_to_be_deleted_at = Column(DateTime)
instances = orm.relationship(
"ShareInstance",
lazy='subquery',

24
manila/policies/shares.py
View File

@ -316,6 +316,30 @@ shares_policies = [
],
deprecated_rule=deprecated_share_delete
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'soft_delete',
check_str=base.SYSTEM_ADMIN_OR_PROJECT_MEMBER,
scope_types=['system', 'project'],
description="Soft Delete a share.",
operations=[
{
'method': 'POST',
'path': '/shares/{share_id}/action',
}
],
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'restore',
check_str=base.SYSTEM_ADMIN_OR_PROJECT_MEMBER,
scope_types=['system', 'project'],
description="Restore a share.",
operations=[
{
'method': 'POST',
'path': '/shares/{share_id}/action',
}
],
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'force_delete',
check_str=base.SYSTEM_ADMIN_OR_PROJECT_ADMIN,

72
manila/share/api.py
View File

@ -988,11 +988,14 @@ class API(base.Base):
# share server here, when manage/unmanage operations will be supported
# for driver_handles_share_servers=True mode
def manage_snapshot(self, context, snapshot_data, driver_options):
def manage_snapshot(self, context, snapshot_data, driver_options,
share=None):
if not share:
try:
share = self.db.share_get(context, snapshot_data['share_id'])
except exception.NotFound:
raise exception.ShareNotFound(share_id=snapshot_data['share_id'])
raise exception.ShareNotFound(
share_id=snapshot_data['share_id'])
if share['has_replicas']:
msg = (_("Share %s has replicas. Snapshots of this share cannot "
@ -1158,6 +1161,52 @@ class API(base.Base):
self.share_rpcapi.revert_to_snapshot(
context, share, snapshot, active_replica['host'], reservations)
@policy.wrap_check_policy('share')
def soft_delete(self, context, share):
"""Soft delete share."""
share_id = share['id']
if share['is_soft_deleted']:
msg = _("The share has been soft deleted already")
raise exception.InvalidShare(reason=msg)
statuses = (constants.STATUS_AVAILABLE, constants.STATUS_ERROR,
constants.STATUS_INACTIVE)
if share['status'] not in statuses:
msg = _("Share status must be one of %(statuses)s") % {
"statuses": statuses}
raise exception.InvalidShare(reason=msg)
# If the share has more than one replica,
# it can't be soft deleted until the additional replicas are removed.
if share.has_replicas:
msg = _("Share %s has replicas. Remove the replicas before "
"soft deleting the share.") % share_id
raise exception.Conflict(err=msg)
snapshots = self.db.share_snapshot_get_all_for_share(context, share_id)
if len(snapshots):
msg = _("Share still has %d dependent snapshots.") % len(snapshots)
raise exception.InvalidShare(reason=msg)
share_group_snapshot_members_count = (
self.db.count_share_group_snapshot_members_in_share(
context, share_id))
if share_group_snapshot_members_count:
msg = (
_("Share still has %d dependent share group snapshot "
"members.") % share_group_snapshot_members_count)
raise exception.InvalidShare(reason=msg)
self._check_is_share_busy(share)
self.db.share_soft_delete(context, share_id)
@policy.wrap_check_policy('share')
def restore(self, context, share):
"""Restore share."""
share_id = share['id']
self.db.share_restore(context, share_id)
@policy.wrap_check_policy('share')
def delete(self, context, share, force=False):
"""Delete share."""
@ -1859,7 +1908,7 @@ class API(base.Base):
'display_description', 'display_description~', 'snapshot_id',
'status', 'share_type_id', 'project_id', 'export_location_id',
'export_location_path', 'limit', 'offset', 'host',
'share_network_id']
'share_network_id', 'is_soft_deleted']
for key in filter_keys:
if key in search_opts:
@ -2516,11 +2565,20 @@ class API(base.Base):
shares = self.db.share_get_all_by_share_server(
context, share_server['id'])
shares_in_recycle_bin = (
self.db.get_shares_in_recycle_bin_by_share_server(
context, share_server['id']))
if len(shares) == 0:
msg = _("Share server %s does not have shares."
% share_server['id'])
raise exception.InvalidShareServer(reason=msg)
if shares_in_recycle_bin:
msg = _("Share server %s has at least one share that has "
"been soft deleted." % share_server['id'])
raise exception.InvalidShareServer(reason=msg)
# We only handle "active" share servers for now
if share_server['status'] != constants.STATUS_ACTIVE:
msg = _('Share server %(server_id)s status must be active, '
@ -2984,6 +3042,14 @@ class API(base.Base):
# Make sure the host is in the list of available hosts
utils.validate_service_host(admin_ctx, backend_host)
shares_in_recycle_bin = (
self.db.get_shares_in_recycle_bin_by_network(
context, share_network['id']))
if shares_in_recycle_bin:
msg = _("Some shares with share network %(sn_id)s have "
"been soft deleted.") % {'sn_id': share_network['id']}
raise exception.InvalidShareNetwork(reason=msg)
shares = self.get_all(
context, search_opts={'share_network_id': share_network['id']})
shares_not_available = [

16
manila/share/manager.py
View File

@ -131,6 +131,11 @@ share_manager_opts = [
default=False,
help='Offload pending share ensure during '
'share service startup'),
cfg.IntOpt('check_for_expired_shares_in_recycle_bin_interval',
default=3600,
help='This value, specified in seconds, determines how often '
'the share manager will check for expired shares and '
'delete them from the Recycle bin.'),
]
CONF = cfg.CONF
@ -3483,6 +3488,17 @@ class ShareManager(manager.SchedulerDependentManager):
for server in servers:
self.delete_share_server(ctxt, server)
@periodic_task.periodic_task(
spacing=CONF.check_for_expired_shares_in_recycle_bin_interval)
@utils.require_driver_initialized
def delete_expired_share(self, ctxt):
LOG.debug("Check for expired share in recycle bin to delete.")
expired_shares = self.db.get_all_expired_shares(ctxt)
for share in expired_shares:
LOG.debug("share %s has expired, will be deleted", share['id'])
self.share_api.delete(ctxt, share, force=True)
@add_hooks
@utils.require_driver_initialized
def create_snapshot(self, context, share_id, snapshot_id):

9
manila/tests/api/contrib/stubs.py
View File

@ -46,6 +46,7 @@ def stub_share(id, **kwargs):
'mount_snapshot_support': False,
'replication_type': None,
'has_replicas': False,
'is_soft_deleted': False,
}
share_instance = {
@ -149,6 +150,14 @@ def stub_share_delete(self, context, *args, **param):
pass
def stub_share_soft_delete(self, context, *args, **param):
pass
def stub_share_restore(self, context, *args, **param):
pass
def stub_share_update(self, context, *args, **param):
share = stub_share('1')
return share

23
manila/tests/api/v1/test_share_snapshots.py
View File

@ -112,6 +112,29 @@ class ShareSnapshotAPITest(test.TestCase):
self.assertFalse(share_api.API.create_snapshot.called)
def test_snapshot_create_in_recycle_bin(self):
self.mock_object(share_api.API, 'create_snapshot')
self.mock_object(
share_api.API,
'get',
mock.Mock(return_value={'snapshot_support': True,
'is_soft_deleted': True}))
body = {
'snapshot': {
'share_id': 200,
'force': False,
'name': 'fake_share_name',
'description': 'fake_share_description',
}
}
req = fakes.HTTPRequest.blank('/fake/snapshots')
self.assertRaises(
webob.exc.HTTPForbidden,
self.controller.create, req, body)
self.assertFalse(share_api.API.create_snapshot.called)
def test_snapshot_create_no_body(self):
body = {}
req = fakes.HTTPRequest.blank('/fake/snapshots')

22
manila/tests/api/v1/test_share_unmanage.py
View File

@ -121,6 +121,28 @@ class ShareUnmanageTest(test.TestCase):
self.mock_policy_check.assert_called_once_with(
self.context, self.resource_name, 'unmanage')
def test_unmanage_share_that_has_been_soft_deleted(self):
share = dict(status=constants.STATUS_AVAILABLE, id='foo_id',
instance={}, is_soft_deleted=True)
mock_api_unmanage = self.mock_object(self.controller.share_api,
'unmanage')
mock_db_snapshots_get = self.mock_object(
self.controller.share_api.db, 'share_snapshot_get_all_for_share')
self.mock_object(
self.controller.share_api, 'get',
mock.Mock(return_value=share))
self.assertRaises(
webob.exc.HTTPForbidden,
self.controller.unmanage, self.request, share['id'])
self.assertFalse(mock_api_unmanage.called)
self.assertFalse(mock_db_snapshots_get.called)
self.controller.share_api.get.assert_called_once_with(
self.request.environ['manila.context'], share['id'])
self.mock_policy_check.assert_called_once_with(
self.context, self.resource_name, 'unmanage')
def test_unmanage_share_based_on_share_server(self):
share = dict(instance=dict(share_server_id='foo_id'), id='bar_id')
self.mock_object(

15
manila/tests/api/v2/test_share_instances.py
View File

@ -67,19 +67,25 @@ class ShareInstancesAPITest(test.TestCase):
self.assertEqual([i['id'] for i in expected],
[i['id'] for i in actual])
@ddt.data("2.3", "2.34", "2.35")
@ddt.data("2.3", "2.34", "2.35", "2.69")
def test_index(self, version):
url = '/share_instances'
if (api_version_request.APIVersionRequest(version) >=
api_version_request.APIVersionRequest('2.35')):
url += "?export_location_path=/admin/export/location"
if (api_version_request.APIVersionRequest(version) >=
api_version_request.APIVersionRequest('2.69')):
url += "&is_soft_deleted=true"
req = self._get_request(url, version=version)
req_context = req.environ['manila.context']
last_instance = [db_utils.create_share(size=1,
is_soft_deleted=True).instance]
share_instances_count = 3
test_instances = [
other_instances = [
db_utils.create_share(size=s + 1).instance
for s in range(0, share_instances_count)
]
test_instances = other_instances + last_instance
db.share_export_locations_update(
self.admin_context, test_instances[0]['id'],
@ -88,8 +94,13 @@ class ShareInstancesAPITest(test.TestCase):
actual_result = self.controller.index(req)
if (api_version_request.APIVersionRequest(version) >=
api_version_request.APIVersionRequest('2.69')):
test_instances = []
elif (api_version_request.APIVersionRequest(version) >=
api_version_request.APIVersionRequest('2.35')):
test_instances = test_instances[:1]
else:
test_instances = other_instances
self._validate_ids_in_share_instances_list(
test_instances, actual_result['share_instances'])
self.mock_policy_check.assert_called_once_with(

21
manila/tests/api/v2/test_share_replicas.py
View File

@ -375,6 +375,27 @@ class ShareReplicasApiTest(test.TestCase):
self.mock_policy_check.assert_called_once_with(
self.member_context, self.resource_name, 'create')
def test_create_has_been_soft_deleted(self):
share_ref = fake_share.fake_share(is_soft_deleted=True)
body = {
'share_replica': {
'share_id': 'FAKE_SHAREID',
'availability_zone': 'FAKE_AZ'
}
}
mock__view_builder_call = self.mock_object(
share_replicas.replication_view.ReplicationViewBuilder,
'detail_list')
self.mock_object(share_replicas.db, 'share_get',
mock.Mock(return_value=share_ref))
self.assertRaises(exc.HTTPForbidden,
self.controller.create,
self.replicas_req, body)
self.assertFalse(mock__view_builder_call.called)
self.mock_policy_check.assert_called_once_with(
self.member_context, self.resource_name, 'create')
@ddt.data(exception.AvailabilityZoneNotFound,
exception.ReplicationException, exception.ShareBusyException)
def test_create_exception_path(self, exception_type):

32
manila/tests/api/v2/test_share_snapshots.py
View File

@ -731,9 +731,14 @@ class ShareSnapshotAdminActionsAPITest(test.TestCase):
data['snapshot']['share_id'] = 'fake'
data['snapshot']['provider_location'] = 'fake_volume_snapshot_id'
data['snapshot']['driver_options'] = {}
return_share = fake_share.fake_share(is_soft_deleted=False,
id='fake')
return_snapshot = fake_share.fake_snapshot(
create_instance=True, id='fake_snap',
provider_location='fake_volume_snapshot_id')
self.mock_object(
share_api.API, 'get', mock.Mock(
return_value=return_share))
self.mock_object(
share_api.API, 'manage_snapshot', mock.Mock(
return_value=return_snapshot))
@ -752,7 +757,8 @@ class ShareSnapshotAdminActionsAPITest(test.TestCase):
actual_snapshot = actual_result['snapshot']
share_api.API.manage_snapshot.assert_called_once_with(
mock.ANY, share_snapshot, data['snapshot']['driver_options'])
mock.ANY, share_snapshot, data['snapshot']['driver_options'],
share=return_share)
self.assertEqual(return_snapshot['id'],
actual_result['snapshot']['id'])
self.assertEqual('fake_volume_snapshot_id',
@ -781,6 +787,11 @@ class ShareSnapshotAdminActionsAPITest(test.TestCase):
body = get_fake_manage_body(
share_id='fake', provider_location='fake_volume_snapshot_id',
driver_options={})
return_share = fake_share.fake_share(is_soft_deleted=False,
id='fake')
self.mock_object(
share_api.API, 'get', mock.Mock(
return_value=return_share))
self.mock_object(
share_api.API, 'manage_snapshot', mock.Mock(
side_effect=exception_type))
@ -798,6 +809,25 @@ class ShareSnapshotAdminActionsAPITest(test.TestCase):
self.manage_request.environ['manila.context'],
self.resource_name, 'manage_snapshot')
def test_manage_share_has_been_soft_deleted(self):
self.mock_policy_check = self.mock_object(
policy, 'check_policy', mock.Mock(return_value=True))
body = get_fake_manage_body(
share_id='fake', provider_location='fake_volume_snapshot_id',
driver_options={})
return_share = fake_share.fake_share(is_soft_deleted=True,
id='fake')
self.mock_object(
share_api.API, 'get', mock.Mock(
return_value=return_share))
self.assertRaises(webob.exc.HTTPForbidden,
self.controller.manage,
self.manage_request, body)
self.mock_policy_check.assert_called_once_with(
self.manage_request.environ['manila.context'],
self.resource_name, 'manage_snapshot')
@ddt.data('1.0', '2.6', '2.11')
def test_manage_version_not_found(self, version):
body = get_fake_manage_body(

152
manila/tests/api/v2/test_shares.py
View File

@ -63,12 +63,25 @@ class ShareAPITest(test.TestCase):
stubs.stub_share_get)
self.mock_object(share_api.API, 'update', stubs.stub_share_update)
self.mock_object(share_api.API, 'delete', stubs.stub_share_delete)
self.mock_object(share_api.API, 'soft_delete',
stubs.stub_share_soft_delete)
self.mock_object(share_api.API, 'restore', stubs.stub_share_restore)
self.mock_object(share_api.API, 'get_snapshot',
stubs.stub_snapshot_get)
self.mock_object(share_types, 'get_share_type',
stubs.stub_share_type_get)
self.maxDiff = None
self.share = {
"id": "1",
"size": 100,
"display_name": "Share Test Name",
"display_description": "Share Test Desc",
"share_proto": "fakeproto",
"availability_zone": "zone1:host1",
"is_public": False,
"task_state": None
}
self.share_in_recycle_bin = {
"id": "1",
"size": 100,
"display_name": "Share Test Name",
@ -77,6 +90,20 @@ class ShareAPITest(test.TestCase):
"availability_zone": "zone1:host1",
"is_public": False,
"task_state": None,
"is_soft_deleted": True,
"status": "available"
}
self.share_in_recycle_bin_is_deleting = {
"id": "1",
"size": 100,
"display_name": "Share Test Name",
"display_description": "Share Test Desc",
"share_proto": "fakeproto",
"availability_zone": "zone1:host1",
"is_public": False,
"task_state": None,
"is_soft_deleted": True,
"status": "deleting"
}
self.create_mock = mock.Mock(
return_value=stubs.stub_share(
@ -234,6 +261,23 @@ class ShareAPITest(test.TestCase):
mock_revert_to_snapshot.assert_called_once_with(
utils.IsAMatcher(context.RequestContext), share, snapshot)
def test__revert_share_has_been_soft_deleted(self):
snapshot = copy.deepcopy(self.snapshot)
body = {'revert': {'snapshot_id': '2'}}
req = fakes.HTTPRequest.blank('/v2/fake/shares/1/action',
use_admin_context=False,
version='2.27')
self.mock_object(
self.controller, '_validate_revert_parameters',
mock.Mock(return_value=body['revert']))
self.mock_object(share_api.API, 'get',
mock.Mock(return_value=self.share_in_recycle_bin))
self.mock_object(
share_api.API, 'get_snapshot', mock.Mock(return_value=snapshot))
self.assertRaises(
webob.exc.HTTPForbidden, self.controller._revert,
req, 1, body)
def test__revert_not_supported(self):
share = copy.deepcopy(self.share)
@ -1100,6 +1144,24 @@ class ShareAPITest(test.TestCase):
db.share_update.assert_called_once_with(utils.IsAMatcher(
context.RequestContext), share['id'], update)
def test_reset_task_state_share_has_been_soft_deleted(self):
share = self.share_in_recycle_bin
req = fakes.HTTPRequest.blank(
'/v2/fake/shares/%s/action' % share['id'],
use_admin_context=True,
version='2.22')
req.method = 'POST'
req.headers['content-type'] = 'application/json'
req.api_version_request.experimental = True
update = {'task_state': constants.TASK_STATE_MIGRATION_ERROR}
body = {'reset_task_state': update}
self.mock_object(share_api.API, 'get',
mock.Mock(return_value=share))
self.assertRaises(webob.exc.HTTPForbidden,
self.controller.reset_task_state, req, share['id'],
body)
def test_migration_complete(self):
share = db_utils.create_share()
req = fakes.HTTPRequest.blank(
@ -1524,6 +1586,60 @@ class ShareAPITest(test.TestCase):
self.assertEqual(expected, res_dict['share']['access_rules_status'])
def test_share_soft_delete(self):
req = fakes.HTTPRequest.blank('/v2/fake/shares/1/action',
version='2.69')
body = {"soft_delete": None}
resp = self.controller.share_soft_delete(req, 1, body)
self.assertEqual(202, resp.status_int)
def test_share_soft_delete_has_been_soft_deleted_already(self):
req = fakes.HTTPRequest.blank('/v2/fake/shares/1/action',
version='2.69')
body = {"soft_delete": None}
self.mock_object(share_api.API, 'get',
mock.Mock(return_value=self.share_in_recycle_bin))
self.mock_object(share_api.API, 'soft_delete',
mock.Mock(
side_effect=exception.InvalidShare(reason='err')))
self.assertRaises(
webob.exc.HTTPForbidden, self.controller.share_soft_delete,
req, 1, body)
def test_share_soft_delete_has_replicas(self):
req = fakes.HTTPRequest.blank('/v2/fake/shares/1/action',
version='2.69')
body = {"soft_delete": None}
self.mock_object(share_api.API, 'get',
mock.Mock(return_value=self.share))
self.mock_object(share_api.API, 'soft_delete',
mock.Mock(side_effect=exception.Conflict(err='err')))
self.assertRaises(
webob.exc.HTTPConflict, self.controller.share_soft_delete,
req, 1, body)
def test_share_restore(self):
req = fakes.HTTPRequest.blank('/v2/fake/shares/1/action',
version='2.69')
body = {"restore": None}
self.mock_object(share_api.API, 'get',
mock.Mock(return_value=self.share_in_recycle_bin))
resp = self.controller.share_restore(req, 1, body)
self.assertEqual(202, resp.status_int)
def test_share_restore_with_deleting_status(self):
req = fakes.HTTPRequest.blank('/v2/fake/shares/1/action',
version='2.69')
body = {"restore": None}
self.mock_object(
share_api.API, 'get',
mock.Mock(return_value=self.share_in_recycle_bin_is_deleting))
self.assertRaises(
webob.exc.HTTPForbidden, self.controller.share_restore,
req, 1, body)
def test_share_delete(self):
req = fakes.HTTPRequest.blank('/v2/fake/shares/1')
resp = self.controller.delete(req, 1)
@ -1614,7 +1730,9 @@ class ShareAPITest(test.TestCase):
{'use_admin_context': True, 'version': '2.36'},
{'use_admin_context': False, 'version': '2.36'},
{'use_admin_context': True, 'version': '2.42'},
{'use_admin_context': False, 'version': '2.42'})
{'use_admin_context': False, 'version': '2.42'},
{'use_admin_context': False, 'version': '2.69'},
{'use_admin_context': True, 'version': '2.69'})
@ddt.unpack
def test_share_list_summary_with_search_opts(self, use_admin_context,
version):
@ -1640,6 +1758,9 @@ class ShareAPITest(test.TestCase):
search_opts.update(
{'display_name~': 'fake',
'display_description~': 'fake'})
if (api_version.APIVersionRequest(version) >=
api_version.APIVersionRequest('2.69')):
search_opts.update({'is_soft_deleted': True})
method = 'get_all'
shares = [
{'id': 'id1', 'display_name': 'n1'},
@ -1658,7 +1779,7 @@ class ShareAPITest(test.TestCase):
# fake_key should be filtered for non-admin
url = '/v2/fake/shares?fake_key=fake_value'
for k, v in search_opts.items():
url = url + '&' + k + '=' + v
url = url + '&' + k + '=' + str(v)
req = fakes.HTTPRequest.blank(url, version=version,
use_admin_context=use_admin_context)
@ -1691,6 +1812,10 @@ class ShareAPITest(test.TestCase):
search_opts_expected.update(
{'display_name~': search_opts['display_name~'],
'display_description~': search_opts['display_description~']})
if (api_version.APIVersionRequest(version) >=
api_version.APIVersionRequest('2.69')):
search_opts_expected['is_soft_deleted'] = (
search_opts['is_soft_deleted'])
if use_admin_context:
search_opts_expected.update({'fake_key': 'fake_value'})
@ -1778,7 +1903,9 @@ class ShareAPITest(test.TestCase):
{'use_admin_context': True, 'version': '2.35'},
{'use_admin_context': False, 'version': '2.35'},
{'use_admin_context': True, 'version': '2.42'},
{'use_admin_context': False, 'version': '2.42'})
{'use_admin_context': False, 'version': '2.42'},
{'use_admin_context': True, 'version': '2.69'},
{'use_admin_context': False, 'version': '2.69'})
@ddt.unpack
def test_share_list_detail_with_search_opts(self, use_admin_context,
version):
@ -1812,6 +1939,8 @@ class ShareAPITest(test.TestCase):
'share_type_id': 'fake_share_type_id',
},
'has_replicas': False,
'is_soft_deleted': True,
'scheduled_to_be_deleted_at': 'fake_datatime',
},
{'id': 'id3', 'display_name': 'n3'},
]
@ -1823,12 +1952,15 @@ class ShareAPITest(test.TestCase):
search_opts.update({'with_count': 'true'})
method = 'get_all_with_count'
mock_action = {'side_effect': [(1, [shares[1]])]}
if (api_version.APIVersionRequest(version) >=
api_version.APIVersionRequest('2.69')):
search_opts.update({'is_soft_deleted': True})
if use_admin_context:
search_opts['host'] = 'fake_host'
# fake_key should be filtered for non-admin
url = '/v2/fake/shares/detail?fake_key=fake_value'
for k, v in search_opts.items():
url = url + '&' + k + '=' + v
url = url + '&' + k + '=' + str(v)
req = fakes.HTTPRequest.blank(url, version=version,
use_admin_context=use_admin_context)
@ -1857,6 +1989,10 @@ class ShareAPITest(test.TestCase):
search_opts['export_location_id'])
search_opts_expected['export_location_path'] = (
search_opts['export_location_path'])
if (api_version.APIVersionRequest(version) >=
api_version.APIVersionRequest('2.69')):
search_opts_expected['is_soft_deleted'] = (
search_opts['is_soft_deleted'])
if use_admin_context:
search_opts_expected.update({'fake_key': 'fake_value'})
@ -1889,6 +2025,11 @@ class ShareAPITest(test.TestCase):
if (api_version.APIVersionRequest(version) >=
api_version.APIVersionRequest('2.42')):
self.assertEqual(1, result['count'])
if (api_version.APIVersionRequest(version) >=
api_version.APIVersionRequest('2.69')):
self.assertEqual(
shares[1]['scheduled_to_be_deleted_at'],
result['shares'][0]['scheduled_to_be_deleted_at'])
def _list_detail_common_expected(self, admin=False):
share_dict = {
@ -2530,6 +2671,7 @@ class ShareAdminActionsAPITest(test.TestCase):
req.headers['X-Openstack-Manila-Api-Version'] = version
req.body = jsonutils.dumps(body).encode("utf-8")
req.environ['manila.context'] = ctxt
self.mock_object(share_api.API, 'get', mock.Mock(return_value=model))
resp = req.get_response(fakes.app())
@ -2565,7 +2707,7 @@ class ShareAdminActionsAPITest(test.TestCase):
@ddt.data('2.6', '2.7')
def test_share_reset_status_for_missing(self, version):
fake_share = {'id': 'missing-share-id'}
fake_share = {'id': 'missing-share-id', 'is_soft_deleted': False}
req = fakes.HTTPRequest.blank(
'/v2/fake/shares/%s/action' % fake_share['id'], version=version)

6
manila/tests/api/views/test_shares.py
View File

@ -55,13 +55,15 @@ class ViewBuilderTestCase(test.TestCase):
'create_share_from_snapshot_support': True,
'revert_to_snapshot_support': True,
'progress': '100%',
'scheduled_to_be_deleted_at': 'fake_datetime',
}
return stubs.stub_share('fake_id', **fake_share)
def test__collection_name(self):
self.assertEqual('shares', self.builder._collection_name)
@ddt.data('2.6', '2.9', '2.10', '2.11', '2.16', '2.24', '2.27', '2.54')
@ddt.data('2.6', '2.9', '2.10', '2.11', '2.16',
'2.24', '2.27', '2.54', '2.69')
def test_detail(self, microversion):
req = fakes.HTTPRequest.blank('/shares', version=microversion)
@ -91,6 +93,8 @@ class ViewBuilderTestCase(test.TestCase):
expected['revert_to_snapshot_support'] = True
if self.is_microversion_ge(microversion, '2.54'):
expected['progress'] = '100%'
if self.is_microversion_ge(microversion, '2.69'):
expected['scheduled_to_be_deleted_at'] = 'fake_datetime'
self.assertSubDictMatch(expected, result['share'])

40
manila/tests/db/migrations/alembic/migrations_data_checks.py
View File

@ -3044,3 +3044,43 @@ class AddUpdateSecurityServiceControlFields(BaseMigrationChecks):
self.test_case.assertRaises(
sa_exc.NoSuchTableError,
utils.load_table, 'async_operation_data', engine)
@map_to_migration('1946cb97bb8d')
class ShareIsSoftDeleted(BaseMigrationChecks):
def setup_upgrade_data(self, engine):
# Setup shares
share_fixture = [{'id': 'foo_share_id1'}, {'id': 'bar_share_id1'}]
share_table = utils.load_table('shares', engine)
for fixture in share_fixture:
engine.execute(share_table.insert(fixture))
# Setup share instances
si_fixture = [
{'id': 'foo_share_instance_id_oof1',
'share_id': share_fixture[0]['id'],
'cast_rules_to_readonly': False},
{'id': 'bar_share_instance_id_rab1',
'share_id': share_fixture[1]['id'],
'cast_rules_to_readonly': False},
]
si_table = utils.load_table('share_instances', engine)
for fixture in si_fixture:
engine.execute(si_table.insert(fixture))
def check_upgrade(self, engine, data):
s_table = utils.load_table('shares', engine)
for s in engine.execute(s_table.select()):
self.test_case.assertTrue(hasattr(s, 'is_soft_deleted'))
self.test_case.assertTrue(hasattr(s,
'scheduled_to_be_deleted_at'))
self.test_case.assertIn(s['is_soft_deleted'], (0, False))
self.test_case.assertIsNone(s['scheduled_to_be_deleted_at'])
def check_downgrade(self, engine):
s_table = utils.load_table('shares', engine)
for s in engine.execute(s_table.select()):
self.test_case.assertFalse(hasattr(s, 'is_soft_deleted'))
self.test_case.assertFalse(hasattr(s,
'scheduled_to_be_deleted_at'))

85
manila/tests/db/sqlalchemy/test_api.py
View File

@ -377,6 +377,34 @@ class ShareDatabaseAPITestCase(test.TestCase):
self.assertEqual(1, len(actual_result))
self.assertEqual(share['id'], actual_result[0].id)
def test_share_in_recycle_bin_filter_all_by_share_server(self):
share_network = db_utils.create_share_network()
share_server = db_utils.create_share_server(
share_network_id=share_network['id'])
share = db_utils.create_share(share_server_id=share_server['id'],
share_network_id=share_network['id'],
is_soft_deleted=True)
actual_result = db_api.get_shares_in_recycle_bin_by_share_server(
self.ctxt, share_server['id'])
self.assertEqual(1, len(actual_result))
self.assertEqual(share['id'], actual_result[0].id)
def test_share_in_recycle_bin_filter_all_by_share_network(self):
share_network = db_utils.create_share_network()
share_server = db_utils.create_share_server(
share_network_id=share_network['id'])
share = db_utils.create_share(share_server_id=share_server['id'],
share_network_id=share_network['id'],
is_soft_deleted=True)
actual_result = db_api.get_shares_in_recycle_bin_by_network(
self.ctxt, share_network['id'])
self.assertEqual(1, len(actual_result))
self.assertEqual(share['id'], actual_result[0].id)
def test_share_filter_all_by_share_group(self):
group = db_utils.create_share_group()
share = db_utils.create_share(share_group_id=group['id'])
@ -506,6 +534,18 @@ class ShareDatabaseAPITestCase(test.TestCase):
self.assertEqual('share-%s' % instance['id'], instance['name'])
def test_share_instance_get_all_by_is_soft_deleted(self):
db_utils.create_share()
db_utils.create_share(is_soft_deleted=True)
instances = db_api.share_instances_get_all(
self.ctxt, filters={'is_soft_deleted': True})
self.assertEqual(1, len(instances))
instance = instances[0]
self.assertEqual('share-%s' % instance['id'], instance['name'])
def test_share_instance_get_all_by_ids(self):
fake_share = db_utils.create_share()
expected_share_instance = db_utils.create_share_instance(
@ -653,6 +693,25 @@ class ShareDatabaseAPITestCase(test.TestCase):
self.assertEqual(shares[0]['id'], result[0]['id'])
self.assertEqual(1, len(result))
def test_share_get_all_expired(self):
now_time = timeutils.utcnow()
time_delta = datetime.timedelta(seconds=3600)
time1 = now_time + time_delta
time2 = now_time - time_delta
share1 = db_utils.create_share(status=constants.STATUS_AVAILABLE,
is_soft_deleted=False,
scheduled_to_be_deleted_at=None)
share2 = db_utils.create_share(status=constants.STATUS_AVAILABLE,
is_soft_deleted=True,
scheduled_to_be_deleted_at=time1)
share3 = db_utils.create_share(status=constants.STATUS_AVAILABLE,
is_soft_deleted=True,
scheduled_to_be_deleted_at=time2)
shares = [share1, share2, share3]
result = db_api.get_all_expired_shares(self.ctxt)
self.assertEqual(1, len(result))
self.assertEqual(shares[2]['id'], result[0]['id'])
@ddt.data(
({'status': constants.STATUS_AVAILABLE}, 'status',
[constants.STATUS_AVAILABLE, constants.STATUS_ERROR]),
@ -669,7 +728,9 @@ class ShareDatabaseAPITestCase(test.TestCase):
({'display_name': 'fake_share_name'}, 'display_name',
['fake_share_name', 'share_name']),
({'display_description': 'fake description'}, 'display_description',
['fake description', 'description'])
['fake description', 'description']),
({'is_soft_deleted': True}, 'is_soft_deleted',
[True, False])
)
@ddt.unpack
def test_share_get_all_with_filters(self, filters, key, share_values):
@ -1047,6 +1108,20 @@ class ShareDatabaseAPITestCase(test.TestCase):
db_api.share_instance_access_get(
self.ctxt, rule_id, instance['id']))
def test_share_soft_delete(self):
share = db_utils.create_share()
db_api.share_soft_delete(self.ctxt, share['id'])
share = db_api.share_get(self.ctxt, share['id'])
self.assertEqual(share['is_soft_deleted'], True)
def test_share_restore(self):
share = db_utils.create_share(is_soft_deleted=True)
db_api.share_restore(self.ctxt, share['id'])
share = db_api.share_get(self.ctxt, share['id'])
self.assertEqual(share['is_soft_deleted'], False)
@ddt.ddt
class ShareGroupDatabaseAPITestCase(test.TestCase):
@ -4293,6 +4368,10 @@ class ShareResourcesAPITestCase(test.TestCase):
else:
new_host = 'new-controller-X'
resources = [ # noqa
# share
db_utils.create_share_without_instance(
id=share_id,
status=constants.STATUS_AVAILABLE),
# share instances
db_utils.create_share_instance(
share_id=share_id,
@ -4382,6 +4461,10 @@ class ShareResourcesAPITestCase(test.TestCase):
+ expected_updates['groups']
+ expected_updates['servers'])
resources = [ # noqa
# share
db_utils.create_share_without_instance(
id=share_id,
status=constants.STATUS_AVAILABLE),
# share instances
db_utils.create_share_instance(
share_id=share_id,

6
manila/tests/db_utils.py
View File

@ -91,7 +91,8 @@ def create_share(**kwargs):
'metadata': {'fake_key': 'fake_value'},
'availability_zone': 'fake_availability_zone',
'status': constants.STATUS_CREATING,
'host': 'fake_host'
'host': 'fake_host',
'is_soft_deleted': False
}
return _create_db_row(db.share_create, share, kwargs)
@ -108,7 +109,8 @@ def create_share_without_instance(**kwargs):
'metadata': {},
'availability_zone': 'fake_availability_zone',
'status': constants.STATUS_CREATING,
'host': 'fake_host'
'host': 'fake_host',
'is_soft_deleted': False
}
share.update(copy.deepcopy(kwargs))
return db.share_create(context.get_admin_context(), share, False)

78
manila/tests/share/test_api.py
View File

@ -4663,6 +4663,9 @@ class ShareAPITestCase(test.TestCase):
mock_shares_get_all = self.mock_object(
db_api, 'share_get_all_by_share_server',
mock.Mock(return_value=[fake_share]))
mock_shares_in_recycle_bin_get_all = self.mock_object(
db_api, 'get_shares_in_recycle_bin_by_share_server',
mock.Mock(return_value=[]))
mock_get_type = self.mock_object(
share_types, 'get_share_type', mock.Mock(return_value=share_type))
mock_validate_service = self.mock_object(
@ -4691,6 +4694,8 @@ class ShareAPITestCase(test.TestCase):
mock_shares_get_all.assert_has_calls([
mock.call(self.context, fake_share_server['id']),
mock.call(self.context, fake_share_server['id'])])
mock_shares_in_recycle_bin_get_all.assert_has_calls([
mock.call(self.context, fake_share_server['id'])])
mock_get_type.assert_called_once_with(self.context, share_type['id'])
mock_validate_service.assert_called_once_with(self.context, fake_host)
mock_service_get.assert_called_once_with(
@ -6279,6 +6284,79 @@ class ShareAPITestCase(test.TestCase):
new_sec_service_id,
current_security_service_id=curr_sec_service_id)
def test_soft_delete_share_already_soft_deleted(self):
share = fakes.fake_share(id='fake_id',
status=constants.STATUS_AVAILABLE,
is_soft_deleted=True)
self.assertRaises(exception.InvalidShare,
self.api.soft_delete, self.context, share)
def test_soft_delete_invalid_status(self):
invalid_status = 'fake'
share = fakes.fake_share(id='fake_id',
status=invalid_status,
is_soft_deleted=False)
self.assertRaises(exception.InvalidShare,
self.api.soft_delete, self.context, share)
def test_soft_delete_share_with_replicas(self):
share = fakes.fake_share(id='fake_id',
has_replicas=True,
status=constants.STATUS_AVAILABLE,
is_soft_deleted=False)
self.assertRaises(exception.Conflict,
self.api.soft_delete, self.context, share)
def test_soft_delete_share_with_snapshot(self):
share = fakes.fake_share(id='fake_id',
status=constants.STATUS_AVAILABLE,
has_replicas=False,
is_soft_deleted=False)
snapshot = fakes.fake_snapshot(create_instance=True, as_primitive=True)
mock_db_snapshot_call = self.mock_object(
db_api, 'share_snapshot_get_all_for_share', mock.Mock(
return_value=[snapshot]))
self.assertRaises(exception.InvalidShare,
self.api.soft_delete, self.context, share)
mock_db_snapshot_call.assert_called_once_with(
self.context, share['id'])
@mock.patch.object(db_api, 'count_share_group_snapshot_members_in_share',
mock.Mock(return_value=2))
def test_soft_delete_share_with_group_snapshot_members(self):
share = fakes.fake_share(id='fake_id',
status=constants.STATUS_AVAILABLE,
has_replicas=False,
is_soft_deleted=False)
self.assertRaises(exception.InvalidShare,
self.api.soft_delete, self.context, share)
def test_soft_delete_share(self):
share = fakes.fake_share(id='fake_id',
status=constants.STATUS_AVAILABLE,
has_replicas=False,
is_soft_deleted=False)
self.mock_object(db_api, 'share_snapshot_get_all_for_share',
mock.Mock(return_value=[]))
self.mock_object(db_api, 'count_share_group_snapshot_members_in_share',
mock.Mock(return_value=0))
self.mock_object(db_api, 'share_soft_delete')
self.mock_object(self.api, '_check_is_share_busy')
self.api.soft_delete(self.context, share)
self.api._check_is_share_busy.assert_called_once_with(share)
def test_restore_share(self):
share = fakes.fake_share(id='fake_id',
status=constants.STATUS_AVAILABLE,
is_soft_deleted=True)
self.mock_object(db_api, 'share_restore')
self.api.restore(self.context, share)
class OtherTenantsShareActionsTestCase(test.TestCase):
def setUp(self):

13
manila/tests/share/test_manager.py
View File

@ -208,6 +208,7 @@ class ShareManagerTestCase(test.TestCase):
"unmanage_share",
"delete_share_instance",
"delete_free_share_servers",
"delete_expired_share",
"create_snapshot",
"delete_snapshot",
"update_access",
@ -3938,6 +3939,18 @@ class ShareManagerTestCase(test.TestCase):
'server1')
timeutils.utcnow.assert_called_once_with()
@mock.patch.object(db, 'get_all_expired_shares',
mock.Mock(return_value=[{"id": "share1"}, ]))
@mock.patch.object(api.API, 'delete',
mock.Mock())
def test_delete_expired_share(self):
self.share_manager.delete_expired_share(self.context)
db.get_all_expired_shares.assert_called_once_with(
self.context)
share1 = {"id": "share1"}
api.API.delete.assert_called_once_with(
self.context, share1, force=True)
@mock.patch('manila.tests.fake_notifier.FakeNotifier._notify')
def test_extend_share_invalid(self, mock_notify):
share = db_utils.create_share()

17
releasenotes/notes/manila-share-support-recycle-bin-1cc7859affaf8887.yaml Normal file
View File

@ -0,0 +1,17 @@
---
features:
- |
Manila now supports a "recycle bin" for shares. End users can soft-delete
their shares and have the ability to restore them for a specified interval.
This interval defaults to 7 days and is configurable via
"soft_deleted_share_retention_time". After this time has elapsed,
soft-deleted shares are automatically cleaned up.
upgrade:
- |
The share entity now contains two new fields: ``is_soft_deleted`` and
``scheduled_to_be_deleted_at``. The ``is_soft_deleted`` will be used to
identify shares in the recycle bin.. The ``scheduled_to_be_deleted_at``
field to show when the share will be deleted automatically. A new parameter
called ``is_soft_deleted`` was added to the share list API, and users will
be able to query shares and filter out the ones that are currently in the
recycle bin.