Update micversion to 2.77, support share transfer between project

user can create a transfer for a share. will return transfer id
and auth_key. another user can use transfer_id and auth_key to
accept this share. salt of transfer and auth_key compute crypt_hash
by sha1. then compare with crypt_hash in db.

APIImpact
DocImpact

Partially-Implements: blueprint transfer-share-between-project

Change-Id: I8facf9112a6b09e6b7aed9956c0a87fb5f1fc31f
This commit is contained in:
haixin 2022-05-30 17:56:14 +08:00
parent d5792fe218
commit b4a0fd9af0
40 changed files with 2607 additions and 23 deletions

View File

@ -52,6 +52,7 @@ shared file system storage resources.
.. include:: share-groups.inc
.. include:: share-group-types.inc
.. include:: share-group-snapshots.inc
.. include:: share-transfers.inc
======================================
Shared File Systems API (EXPERIMENTAL)

View File

@ -159,6 +159,12 @@ snapshot_instance_id_path:
in: path
required: true
type: string
transfer_id:
description: |
The unique identifier for a transfer.
in: path
required: true
type: string
# variables in query
action_id:
@ -389,7 +395,7 @@ metadata_query:
name_inexact_query:
description: |
The name pattern that can be used to filter shares,
share snapshots, share networks or share groups.
share snapshots, share networks, transfers or share groups.
in: query
required: false
type: string
@ -463,6 +469,12 @@ resource_type:
in: query
required: false
type: string
resource_type_query:
description: |
The type of the resource for which the transfer was created.
in: query
required: false
type: string
security_service_query:
description: |
The security service ID to filter out share networks.
@ -581,7 +593,7 @@ snapshot_id_query:
type: string
sort_dir:
description: |
The direction to sort a list of shares. A valid
The direction to sort a list of resources. A valid
value is ``asc``, or ``desc``.
in: query
required: false
@ -606,6 +618,15 @@ sort_key_messages:
in: query
required: false
type: string
sort_key_transfer:
description: |
The key to sort a list of transfers. A valid value
is ``id``, ``name``, ``resource_type``, ``resource_id``,
``source_project_id``, ``destination_project_id``, ``created_at``,
``expires_at``.
in: query
required: false
type: string
source_share_group_snapshot_id_query:
description: |
The source share group snapshot ID to list the
@ -639,6 +660,12 @@ with_count_query:
min_version: 2.42
# variables in body
accepted:
description: |
Whether the transfer has been accepted.
in: body
required: true
type: boolean
access:
description: |
The ``access`` object.
@ -782,6 +809,12 @@ allow_access:
in: body
required: true
type: object
auth_key:
description: |
The authentication key for the transfer.
in: body
required: true
type: string
availability_zone:
description: |
The name of the availability zone the share exists within.
@ -952,6 +985,12 @@ cidr:
required: true
type: string
max_version: 2.50
clear_access_rules:
description: |
Whether clear all access rules when accept share.
in: body
required: false
type: boolean
compatible:
description: |
Whether the destination backend can or can't handle the share server
@ -1045,6 +1084,12 @@ description_request:
in: body
required: false
type: string
destination_project_id:
description: |
UUID of the destination project to accept transfer resource.
in: body
required: true
type: string
destination_share_server_id:
description: |
UUID of the share server that was created in the destination backend during
@ -2757,6 +2802,12 @@ share_group_type_name_request:
in: body
required: false
type: string
share_id_request:
description: |
The UUID of the share.
in: body
required: true
type: string
share_id_response:
description: |
The UUID of the share.
@ -3593,6 +3644,61 @@ totalSnapshotGigabytesUsed:
in: body
required: true
type: integer
transfer:
description: |
The transfer object.
in: body
required: true
type: object
transfer_expires_at_body:
description: |
The date and time stamp when the resource transfer will expire.
After transfer expired, will be automatically deleted.
The date and time stamp format is `ISO 8601
<https://en.wikipedia.org/wiki/ISO_8601>`_:
::
CCYY-MM-DDThh:mm:ss±hh:mm
The ``±hh:mm`` value, if included, returns the time zone as an
offset from UTC.
For example, ``2016-12-31T13:14:15-05:00``.
in: body
required: true
type: string
transfer_id_in_body:
description: |
The transfer UUID.
in: body
required: true
type: string
transfer_name:
description: |
The transfer display name.
in: body
required: false
type: string
transfer_resource_id:
description: |
The UUID of the resource for the transfer.
in: body
required: true
type: string
transfer_resource_type:
description: |
The type of the resource for the transfer.
in: body
required: true
type: string
transfers:
description: |
List of transfers.
in: body
required: true
type: array
unit:
description: |
The time interval during which a number of API

View File

@ -0,0 +1,6 @@
{
"accept": {
"auth_key": "d7ef426932068a33",
"clear_access_rules": true
}
}

View File

@ -0,0 +1,6 @@
{
"transfer": {
"share_id": "29476819-28a9-4b1a-a21d-3b2d203025a0",
"name": "test_transfer"
}
}

View File

@ -0,0 +1,24 @@
{
"transfer": {
"id": "f21c72c4-2b77-445b-aa12-e8d1b44163a2",
"created_at": "2022-09-06T08:17:43.629495",
"name": "test_transfer",
"resource_type": "share",
"resource_id": "29476819-28a9-4b1a-a21d-3b2d203025a0",
"auth_key": "406a2d67cdb09afe",
"source_project_id": "714198c7ac5e45a4b785de732ea4695d",
"destination_project_id": null,
"accepted": false,
"expires_at": "2022-09-06T08:22:43.629495",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/f21c72c4-2b77-445b-aa12-e8d1b44163a2"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/f21c72c4-2b77-445b-aa12-e8d1b44163a2"
}
]
}
}

View File

@ -0,0 +1,23 @@
{
"transfer": {
"id": "d2035732-d0c0-4380-a44c-f978a264ab1a",
"created_at": "2022-09-07T01:12:29.000000",
"name": "transfer1",
"resource_type": "share",
"resource_id": "29476819-28a9-4b1a-a21d-3b2d203025a0",
"source_project_id": "714198c7ac5e45a4b785de732ea4695d",
"destination_project_id": null,
"accepted": false,
"expires_at": "2022-09-07T01:17:29.000000",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/d2035732-d0c0-4380-a44c-f978a264ab1a"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/d2035732-d0c0-4380-a44c-f978a264ab1a"
}
]
}
}

View File

@ -0,0 +1,46 @@
{
"transfers": [
{
"id": "42b0fab4-df77-4f25-a958-5370e1c95ed2",
"created_at": "2022-09-07T01:52:39.000000",
"name": "transfer2",
"resource_type": "share",
"resource_id": "0fe7cf64-b879-4902-9d86-f80aeff12b06",
"source_project_id": "714198c7ac5e45a4b785de732ea4695d",
"destination_project_id": null,
"accepted": false,
"expires_at": "2022-09-07T01:57:39.000000",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/42b0fab4-df77-4f25-a958-5370e1c95ed2"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/42b0fab4-df77-4f25-a958-5370e1c95ed2"
}
]
},
{
"id": "506a7e77-42e7-4f33-ac36-1d1dd7f2b9af",
"created_at": "2022-09-07T01:52:30.000000",
"name": "transfer1",
"resource_type": "share",
"resource_id": "29476819-28a9-4b1a-a21d-3b2d203025a0",
"source_project_id": "714198c7ac5e45a4b785de732ea4695d",
"destination_project_id": null,
"accepted": false,
"expires_at": "2022-09-07T01:57:30.000000",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/506a7e77-42e7-4f33-ac36-1d1dd7f2b9af"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/506a7e77-42e7-4f33-ac36-1d1dd7f2b9af"
}
]
}
]
}

View File

@ -0,0 +1,36 @@
{
"transfers": [
{
"id": "02a948b4-671b-4c62-b13a-18d613cb4576",
"resource_type": "share",
"resource_id": "0fe7cf64-b879-4902-9d86-f80aeff12b06",
"name": "transfer2",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/02a948b4-671b-4c62-b13a-18d613cb4576"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/02a948b4-671b-4c62-b13a-18d613cb4576"
}
]
},
{
"id": "a10209ff-b55d-4fed-9f63-abea53b6f107",
"resource_type": "share",
"resource_id": "29476819-28a9-4b1a-a21d-3b2d203025a0",
"name": "transfer1",
"links": [
{
"rel": "self",
"href": "http://192.168.48.129/shar/v2/share-transfer/a10209ff-b55d-4fed-9f63-abea53b6f107"
},
{
"rel": "bookmark",
"href": "http://192.168.48.129/shar/share-transfer/a10209ff-b55d-4fed-9f63-abea53b6f107"
}
]
}
]
}

View File

@ -0,0 +1,286 @@
.. -*- rst -*-
Share transfer (since API v2.77)
================================
Transfers a share across projects.
Create a share transfer
~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: POST /v2/share-transfers
Initiates a share transfer from a source project namespace to a destination
project namespace.
**Preconditions**
* The share ``status`` must be ``available``
* If the share has snapshots, those snapshots must be ``available``
* The share can not belong to share group
Response codes
--------------
.. rest_status_code:: success status.yaml
- 202
.. rest_status_code:: error status.yaml
- 400
- 403
- 404
Request
-------
.. rest_parameters:: parameters.yaml
- transfer: transfer
- name: transfer_name
- share_id: share_id_request
Request Example
---------------
.. literalinclude:: ./samples/share-transfer-create-request.json
:language: javascript
Response Parameters
-------------------
.. rest_parameters:: parameters.yaml
- id: transfer_id_in_body
- created_at: created_at
- name: transfer_name
- resource_type: transfer_resource_type
- resource_id: transfer_resource_id
- auth_key: auth_key
- source_project_id: project_id
- destination_project_id: destination_project_id
- accepted: accepted
- expires_at: transfer_expires_at_body
- links: links
Response Example
----------------
.. literalinclude:: ./samples/share-transfer-create-response.json
:language: javascript
Accept a share transfer in the destination project namespace
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: POST /v2/share-transfers/{transfer_id}/accept
Accepts a share transfer.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 202
.. rest_status_code:: error status.yaml
- 400
- 403
- 404
- 413
Request
-------
.. rest_parameters:: parameters.yaml
- transfer_id: transfer_id
- auth_key: auth_key
- clear_access_rules: clear_access_rules
Request Example
---------------
.. literalinclude:: ./samples/share-transfer-accept-request.json
:language: javascript
List share transfers for a project
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: GET /v2/share-transfers
Lists share transfers.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 200
Request
-------
.. rest_parameters:: parameters.yaml
- all_tenants: all_tenants_query
- limit: limit_query
- offset: offset
- sort_key: sort_key_transfer
- sort_dir: sort_dir
- name: name_query
- name~: name_inexact_query
- resource_type: resource_type_query
Response Parameters
-------------------
.. rest_parameters:: parameters.yaml
- transfers: transfers
- id: transfer_id_in_body
- resource_type: transfer_resource_type
- resource_id: transfer_resource_id
- name: transfer_name
- links: links
Response Example
----------------
.. literalinclude:: ./samples/share-transfers-list-response.json
:language: javascript
List share transfers and details
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: GET /v2/share-transfers/detail
Lists share transfers, with details.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 200
Request
-------
.. rest_parameters:: parameters.yaml
- all_tenants: all_tenants_query
- limit: limit_query
- offset: offset
- sort_key: sort_key_transfer
- sort_dir: sort_dir
Response Parameters
-------------------
.. rest_parameters:: parameters.yaml
- transfers: transfers
- id: transfer_id_in_body
- created_at: created_at
- name: transfer_name
- resource_type: transfer_resource_type
- resource_id: transfer_resource_id
- source_project_id: project_id
- destination_project_id: destination_project_id
- accepted: accepted
- expires_at: transfer_expires_at_body
- links: links
Response Example
----------------
.. literalinclude:: ./samples/share-transfers-list-detailed-response.json
:language: javascript
Show share transfer detail
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: GET /v2/share-transfers/{transfer_id}
Shows details for a share transfer.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 200
.. rest_status_code:: error status.yaml
- 404
Request
-------
.. rest_parameters:: parameters.yaml
- transfer_id: transfer_id
Response Parameters
-------------------
.. rest_parameters:: parameters.yaml
- id: transfer_id_in_body
- created_at: created_at
- name: transfer_name
- resource_type: transfer_resource_type
- resource_id: transfer_resource_id
- source_project_id: project_id
- destination_project_id: destination_project_id
- accepted: accepted
- expires_at: transfer_expires_at_body
- links: links
Response Example
----------------
.. literalinclude:: ./samples/share-transfer-show-response.json
:language: javascript
Delete a share transfer
~~~~~~~~~~~~~~~~~~~~~~~
.. rest_method:: DELETE /v2/share-transfers/{transfer_id}
Deletes a share transfer.
Response codes
--------------
.. rest_status_code:: success status.yaml
- 202
Request
-------
.. rest_parameters:: parameters.yaml
- transfer_id: transfer_id

View File

@ -83,6 +83,9 @@ A share has one of these status values:
+----------------------------------------+--------------------------------------------------------+
| ``reverting_error`` | Share revert to snapshot failed. |
+----------------------------------------+--------------------------------------------------------+
| ``awaiting_transfer`` | Share is being transferred to a different project's |
| | namespace. |
+----------------------------------------+--------------------------------------------------------+
List shares

View File

@ -193,6 +193,7 @@ REST_API_VERSION_HISTORY = """
* 2.75 - Added option to specify quiesce wait time in share replica
promote API.
* 2.76 - Added 'default_ad_site' field in security service object.
* 2.77 - Added support for share transfer between different projects.
"""
@ -200,7 +201,7 @@ REST_API_VERSION_HISTORY = """
# The default api version request is defined to be the
# minimum version of the API supported.
_MIN_API_VERSION = "2.0"
_MAX_API_VERSION = "2.76"
_MAX_API_VERSION = "2.77"
DEFAULT_API_VERSION = _MIN_API_VERSION

View File

@ -418,3 +418,7 @@ ____
2.76
----
Added 'default_ad_site' field in security service object.
2.77
----
Added support for share transfer between different projects.

View File

@ -51,6 +51,7 @@ from manila.api.v2 import share_snapshot_export_locations
from manila.api.v2 import share_snapshot_instance_export_locations
from manila.api.v2 import share_snapshot_instances
from manila.api.v2 import share_snapshots
from manila.api.v2 import share_transfer
from manila.api.v2 import share_types
from manila.api.v2 import shares
from manila.api import versions
@ -544,6 +545,13 @@ class APIRouter(manila.api.openstack.APIRouter):
collection={'detail': 'GET'},
member={'action': 'POST'})
self.resources['share_transfers'] = (
share_transfer.create_resource())
mapper.resource("share-transfer", "share-transfers",
controller=self.resources['share_transfers'],
collection={'detail': 'GET'},
member={'accept': 'POST'})
self.resources["share-replica-export-locations"] = (
share_replica_export_locations.create_resource())
for path_prefix in ['/{project_id}', '']:

View File

@ -0,0 +1,201 @@
# Copyright (c) 2022 China Telecom Digital Intelligence.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""The share transfer api."""
from http import client as http_client
from oslo_log import log as logging
from oslo_utils import strutils
from oslo_utils import uuidutils
import webob
from webob import exc
from manila.api import common
from manila.api.openstack import wsgi
from manila.api.views import transfers as transfer_view
from manila import exception
from manila.i18n import _
from manila.transfer import api as transfer_api
LOG = logging.getLogger(__name__)
SHARE_TRANSFER_VERSION = "2.77"
class ShareTransferController(wsgi.Controller):
"""The Share Transfer API controller for the OpenStack API."""
resource_name = 'share_transfer'
_view_builder_class = transfer_view.ViewBuilder
def __init__(self):
self.transfer_api = transfer_api.API()
super(ShareTransferController, self).__init__()
@wsgi.Controller.authorize('get')
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
def show(self, req, id):
"""Return data about active transfers."""
context = req.environ['manila.context']
# Not found exception will be handled at the wsgi level
transfer = self.transfer_api.get(context, transfer_id=id)
return self._view_builder.detail(req, transfer)
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
def index(self, req):
"""Returns a summary list of transfers."""
return self._get_transfers(req, is_detail=False)
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
def detail(self, req):
"""Returns a detailed list of transfers."""
return self._get_transfers(req, is_detail=True)
@wsgi.Controller.authorize('get_all')
def _get_transfers(self, req, is_detail):
"""Returns a list of transfers, transformed through view builder."""
context = req.environ['manila.context']
params = req.params.copy()
pagination_params = common.get_pagination_params(req)
limit, offset = [pagination_params.pop('limit', None),
pagination_params.pop('offset', None)]
sort_key, sort_dir = common.get_sort_params(params)
filters = params
key_map = {'name': 'display_name', 'name~': 'display_name~'}
for k in key_map:
if k in filters:
filters[key_map[k]] = filters.pop(k)
LOG.debug('Listing share transfers.')
transfers = self.transfer_api.get_all(context,
limit=limit,
sort_key=sort_key,
sort_dir=sort_dir,
filters=filters,
offset=offset)
if is_detail:
transfers = self._view_builder.detail_list(req, transfers)
else:
transfers = self._view_builder.summary_list(req, transfers)
return transfers
@wsgi.response(http_client.ACCEPTED)
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
@wsgi.Controller.authorize('create')
def create(self, req, body):
"""Create a new share transfer."""
LOG.debug('Creating new share transfer %s', body)
context = req.environ['manila.context']
if not self.is_valid_body(body, 'transfer'):
msg = _("'transfer' is missing from the request body.")
raise exc.HTTPBadRequest(explanation=msg)
transfer = body.get('transfer', {})
share_id = transfer.get('share_id')
if not share_id:
msg = _("Must supply 'share_id' attribute.")
raise exc.HTTPBadRequest(explanation=msg)
if not uuidutils.is_uuid_like(share_id):
msg = _("The 'share_id' attribute must be a uuid.")
raise exc.HTTPBadRequest(explanation=msg)
transfer_name = transfer.get('name')
if transfer_name is not None:
transfer_name = transfer_name.strip()
LOG.debug("Creating transfer of share %s", share_id)
try:
new_transfer = self.transfer_api.create(context, share_id,
transfer_name)
except exception.Invalid as error:
raise exc.HTTPBadRequest(explanation=error.msg)
transfer = self._view_builder.create(req,
dict(new_transfer))
return transfer
@wsgi.response(http_client.ACCEPTED)
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
@wsgi.Controller.authorize('accept')
def accept(self, req, id, body):
"""Accept a new share transfer."""
transfer_id = id
LOG.debug('Accepting share transfer %s', transfer_id)
context = req.environ['manila.context']
if not self.is_valid_body(body, 'accept'):
msg = _("'accept' is missing from the request body.")
raise exc.HTTPBadRequest(explanation=msg)
accept = body.get('accept', {})
auth_key = accept.get('auth_key')
if not auth_key:
msg = _("Must supply 'auth_key' while accepting a "
"share transfer.")
raise exc.HTTPBadRequest(explanation=msg)
clear_rules = accept.get('clear_access_rules', False)
if clear_rules:
try:
clear_rules = strutils.bool_from_string(clear_rules,
strict=True)
except (ValueError, TypeError):
msg = (_('Invalid boolean clear_access_rules : %(value)s') %
{'value': accept['clear_access_rules']})
raise exc.HTTPBadRequest(explanation=msg)
LOG.debug("Accepting transfer %s", transfer_id)
try:
self.transfer_api.accept(
context, transfer_id, auth_key, clear_rules=clear_rules)
except (exception.ShareSizeExceedsLimit,
exception.ShareLimitExceeded,
exception.ShareSizeExceedsAvailableQuota,
exception.ShareReplicasLimitExceeded,
exception.ShareReplicaSizeExceedsAvailableQuota,
exception.SnapshotSizeExceedsAvailableQuota,
exception.SnapshotLimitExceeded) as e:
raise exc.HTTPRequestEntityTooLarge(explanation=e.msg,
headers={'Retry-After': '0'})
except (exception.InvalidShare,
exception.InvalidSnapshot,
exception.InvalidAuthKey,
exception.TransferNotFound) as error:
raise exc.HTTPBadRequest(explanation=error.msg)
@wsgi.Controller.api_version(SHARE_TRANSFER_VERSION)
@wsgi.Controller.authorize('delete')
def delete(self, req, id):
"""Delete a transfer."""
context = req.environ['manila.context']
LOG.debug("Delete transfer with id: %s", id)
# Not found exception will be handled at the wsgi level
self.transfer_api.delete(context, transfer_id=id)
return webob.Response(status_int=http_client.OK)
def create_resource():
return wsgi.Resource(ShareTransferController())

View File

@ -0,0 +1,86 @@
# Copyright (C) 2022 China Telecom Digital Intelligence.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from manila.api import common
class ViewBuilder(common.ViewBuilder):
"""Model transfer API responses as a python dictionary."""
_collection_name = "share-transfer"
def __init__(self):
"""Initialize view builder."""
super(ViewBuilder, self).__init__()
def summary_list(self, request, transfers,):
"""Show a list of transfers without many details."""
return self._list_view(self.summary, request, transfers)
def detail_list(self, request, transfers):
"""Detailed view of a list of transfers ."""
return self._list_view(self.detail, request, transfers)
def summary(self, request, transfer):
"""Generic, non-detailed view of a transfer."""
return {
'transfer': {
'id': transfer['id'],
'name': transfer['display_name'],
'resource_type': transfer['resource_type'],
'resource_id': transfer['resource_id'],
'links': self._get_links(request,
transfer['id']),
},
}
def detail(self, request, transfer):
"""Detailed view of a single transfer."""
detail_body = {
'transfer': {
'id': transfer.get('id'),
'created_at': transfer.get('created_at'),
'name': transfer.get('display_name'),
'resource_type': transfer['resource_type'],
'resource_id': transfer['resource_id'],
'source_project_id': transfer['source_project_id'],
'destination_project_id': transfer.get(
'destination_project_id'),
'accepted': transfer['accepted'],
'expires_at': transfer.get('expires_at'),
'links': self._get_links(request, transfer['id']),
}
}
return detail_body
def create(self, request, transfer):
"""Detailed view of a single transfer when created."""
create_body = self.detail(request, transfer)
create_body['transfer']['auth_key'] = transfer.get('auth_key')
return create_body
def _list_view(self, func, request, transfers):
"""Provide a view for a list of transfers."""
transfers_list = [func(request, transfer)['transfer'] for transfer in
transfers]
transfers_links = self._get_collection_links(request,
transfers,
self._collection_name)
transfers_dict = dict(transfers=transfers_list)
if transfers_links:
transfers_dict['transfers_links'] = transfers_links
return transfers_dict

View File

@ -132,6 +132,11 @@ global_opts = [
help='Maximum time (in seconds) to keep a share in the recycle '
'bin, it will be deleted automatically after this amount '
'of time has elapsed.'),
cfg.IntOpt('transfer_retention_time',
default=300,
help='Maximum time (in seconds) to keep a share in '
'awaiting_transfer state, after timeout, the share will '
'automatically be rolled back to the available state'),
]
CONF.register_opts(global_opts)

View File

@ -44,6 +44,10 @@ STATUS_REPLICATION_CHANGE = 'replication_change'
STATUS_RESTORING = 'restoring'
STATUS_REVERTING = 'reverting'
STATUS_REVERTING_ERROR = 'reverting_error'
STATUS_AWAITING_TRANSFER = 'awaiting_transfer'
# Transfer resource type
SHARE_RESOURCE_TYPE = 'share'
# Access rule states
ACCESS_STATE_QUEUED_TO_APPLY = 'queued_to_apply'

View File

@ -494,6 +494,58 @@ def share_restore(context, share_id):
###################
def share_transfer_get(context, transfer_id):
"""Get a share transfer record or raise if it does not exist."""
return IMPL.share_transfer_get(context, transfer_id)
def transfer_get_all(context, limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
"""Get all share transfer records."""
return IMPL.transfer_get_all(context, limit=limit,
sort_key=sort_key, sort_dir=sort_dir,
filters=filters, offset=offset)
def transfer_get_all_by_project(context, project_id,
limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
"""Get all share transfer records for specified project."""
return IMPL.transfer_get_all_by_project(context, project_id,
limit=limit, sort_key=sort_key,
sort_dir=sort_dir,
filters=filters, offset=offset)
def transfer_create(context, values):
"""Create an entry in the transfers table."""
return IMPL.transfer_create(context, values)
def transfer_destroy(context, transfer_id, update_share_status=True):
"""Destroy a record in the share transfer table."""
return IMPL.transfer_destroy(context, transfer_id,
update_share_status=update_share_status)
def transfer_accept(context, transfer_id, user_id, project_id,
accept_snapshots=False):
"""Accept a share transfer."""
return IMPL.transfer_accept(context, transfer_id, user_id, project_id,
accept_snapshots=accept_snapshots)
def transfer_accept_rollback(context, transfer_id, user_id,
project_id, rollback_snap=False):
"""Rollback a share transfer."""
return IMPL.transfer_accept_rollback(context, transfer_id,
user_id, project_id,
rollback_snap=rollback_snap)
###################
def share_access_create(context, values):
"""Allow access to share."""
return IMPL.share_access_create(context, values)
@ -1177,6 +1229,11 @@ def get_all_expired_shares(context):
return IMPL.get_all_expired_shares(context)
def get_all_expired_transfers(context):
"""Get all expired transfers DB records."""
return IMPL.get_all_expired_transfers(context)
def share_server_backend_details_set(context, share_server_id, server_details):
"""Create DB record with backend details."""
return IMPL.share_server_backend_details_set(context, share_server_id,

View File

@ -0,0 +1,68 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""add_transfers
Revision ID: 1e2d600bf972
Revises: c476aeb186ec
Create Date: 2022-05-30 16:37:18.325464
"""
# revision identifiers, used by Alembic.
revision = '1e2d600bf972'
down_revision = 'c476aeb186ec'
from alembic import op
from oslo_log import log
import sqlalchemy as sa
LOG = log.getLogger(__name__)
def upgrade():
context = op.get_context()
mysql_dl = context.bind.dialect.name == 'mysql'
datetime_type = (sa.dialects.mysql.DATETIME(fsp=6)
if mysql_dl else sa.DateTime)
try:
op.create_table(
'transfers',
sa.Column('id', sa.String(36), primary_key=True, nullable=False),
sa.Column('created_at', datetime_type),
sa.Column('updated_at', datetime_type),
sa.Column('deleted_at', datetime_type),
sa.Column('deleted', sa.String(36), default='False'),
sa.Column('resource_id', sa.String(36), nullable=False),
sa.Column('resource_type', sa.String(255), nullable=False),
sa.Column('display_name', sa.String(255)),
sa.Column('salt', sa.String(255)),
sa.Column('crypt_hash', sa.String(255)),
sa.Column('expires_at', datetime_type),
sa.Column('source_project_id', sa.String(255), nullable=True),
sa.Column('destination_project_id', sa.String(255), nullable=True),
sa.Column('accepted', sa.Boolean, default=False),
mysql_engine='InnoDB',
mysql_charset='utf8',
)
except Exception:
LOG.error("Table |%s| not created!", 'transfers')
raise
def downgrade():
try:
op.drop_table('transfers')
except Exception:
LOG.error("transfers table not dropped")
raise

View File

@ -2506,6 +2506,189 @@ def share_restore(context, share_id):
###################
@context_manager.reader
def _transfer_get(context, transfer_id, resource_type='share',
session=None, read_deleted=False):
"""resource_type can be share or network(TODO network transfer)"""
query = model_query(context, models.Transfer,
session=session,
read_deleted=read_deleted).filter_by(id=transfer_id)
if not is_admin_context(context):
if resource_type == 'share':
share = models.Share
query = query.filter(models.Transfer.resource_id == share.id,
share.project_id == context.project_id)
result = query.first()
if not result:
raise exception.TransferNotFound(transfer_id=transfer_id)
return result
@context_manager.reader
def share_transfer_get(context, transfer_id, read_deleted=False):
return _transfer_get(context, transfer_id, read_deleted=read_deleted)
def _transfer_get_all(context, limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
session = get_session()
sort_key = sort_key or 'created_at'
sort_dir = sort_dir or 'desc'
with session.begin():
query = model_query(context, models.Transfer, session=session)
if filters:
legal_filter_keys = ('display_name', 'display_name~',
'id', 'resource_type', 'resource_id',
'source_project_id', 'destination_project_id')
query = exact_filter(query, models.Transfer,
filters, legal_filter_keys)
query = utils.paginate_query(query, models.Transfer, limit,
sort_key=sort_key,
sort_dir=sort_dir,
offset=offset)
return query.all()
@require_admin_context
def transfer_get_all(context, limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
return _transfer_get_all(context, limit=limit,
sort_key=sort_key, sort_dir=sort_dir,
filters=filters, offset=offset)
@require_context
def transfer_get_all_by_project(context, project_id,
limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
filters = filters.copy() if filters else {}
filters['source_project_id'] = project_id
return _transfer_get_all(context, limit=limit,
sort_key=sort_key, sort_dir=sort_dir,
filters=filters, offset=offset)
@require_context
@handle_db_data_error
def transfer_create(context, values):
if not values.get('id'):
values['id'] = uuidutils.generate_uuid()
resource_id = values['resource_id']
now_time = timeutils.utcnow()
time_delta = datetime.timedelta(
seconds=CONF.transfer_retention_time)
transfer_timeout = now_time + time_delta
values['expires_at'] = transfer_timeout
session = get_session()
with session.begin():
transfer = models.Transfer()
transfer.update(values)
transfer.save(session=session)
update = {'status': constants.STATUS_AWAITING_TRANSFER}
if values['resource_type'] == 'share':
share_update(context, resource_id, update)
return transfer
@require_context
@oslo_db_api.wrap_db_retry(max_retries=5, retry_on_deadlock=True)
def transfer_destroy(context, transfer_id,
update_share_status=True):
session = get_session()
with session.begin():
update = {'status': constants.STATUS_AVAILABLE}
transfer = share_transfer_get(context, transfer_id)
if transfer['resource_type'] == 'share':
if update_share_status:
share_update(context, transfer['resource_id'], update)
transfer_query = model_query(context, models.Transfer,
session=session).filter_by(id=transfer_id)
transfer_query.soft_delete()
@require_context
@oslo_db_api.wrap_db_retry(max_retries=5, retry_on_deadlock=True)
def transfer_accept(context, transfer_id, user_id, project_id,
accept_snapshots=False):
session = get_session()
with session.begin():
share_id = share_transfer_get(context, transfer_id)['resource_id']
update = {'status': constants.STATUS_AVAILABLE,
'user_id': user_id,
'project_id': project_id,
'updated_at': timeutils.utcnow()}
share_update(context, share_id, update)
# Update snapshots for transfer snapshots with share.
if accept_snapshots:
snapshots = share_snapshot_get_all_for_share(context, share_id)
for snapshot in snapshots:
LOG.debug('Begin to transfer snapshot: %s', snapshot['id'])
update = {'user_id': user_id,
'project_id': project_id,
'updated_at': timeutils.utcnow()}
share_snapshot_update(context, snapshot['id'], update)
query = session.query(models.Transfer).filter_by(id=transfer_id)
query.update({'deleted': True,
'deleted_at': timeutils.utcnow(),
'updated_at': timeutils.utcnow(),
'destination_project_id': project_id,
'accepted': True})
@require_context
def transfer_accept_rollback(context, transfer_id, user_id,
project_id, rollback_snap=False):
session = get_session()
with session.begin():
share_id = share_transfer_get(
context, transfer_id, read_deleted=True)['resource_id']
update = {'status': constants.STATUS_AWAITING_TRANSFER,
'user_id': user_id,
'project_id': project_id,
'updated_at': timeutils.utcnow()}
share_update(context, share_id, update)
# rollback snapshots for transfer snapshots with share.
if rollback_snap:
snapshots = share_snapshot_get_all_for_share(context, share_id)
for snapshot in snapshots:
LOG.debug('Begin to rollback snapshot: %s', snapshot['id'])
update = {'user_id': user_id,
'project_id': project_id,
'updated_at': timeutils.utcnow()}
share_snapshot_update(context, snapshot['id'], update)
query = session.query(models.Transfer).filter_by(id=transfer_id)
query.update({'deleted': 'False',
'deleted_at': None,
'updated_at': timeutils.utcnow(),
'destination_project_id': None,
'accepted': 0})
@require_admin_context
def get_all_expired_transfers(context):
session = get_session()
with session.begin():
query = model_query(context, models.Transfer, session=session)
expires_at_attr = getattr(models.Transfer, 'expires_at', None)
now_time = timeutils.utcnow()
query = query.filter(expires_at_attr.op('<=')(now_time))
result = query.all()
return result
###################
def _share_access_get_query(context, session, values, read_deleted='no'):
"""Get access record."""
query = (model_query(

View File

@ -1170,6 +1170,26 @@ class ShareNetworkSecurityServiceAssociation(BASE, ManilaBase):
nullable=False)
class Transfer(BASE, ManilaBase):
"""Represents a share transfer request."""
__tablename__ = 'transfers'
id = Column(String(36), primary_key=True, nullable=False)
deleted = Column(String(36), default='False')
# resource type can be "share" or "share_network"
resource_type = Column(String(36), nullable=False)
# The uuid of the related resource.
resource_id = Column(String(36), nullable=False)
display_name = Column(String(255))
salt = Column(String(255))
crypt_hash = Column(String(255))
expires_at = Column(DateTime)
source_project_id = Column(String(255), nullable=True)
destination_project_id = Column(String(255), nullable=True)
accepted = Column(Boolean, default=False)
class NetworkAllocation(BASE, ManilaBase):
"""Represents network allocation data."""
__tablename__ = 'network_allocations'

View File

@ -329,6 +329,10 @@ class HostBinaryNotFound(NotFound):
message = _("Could not find binary %(binary)s on host %(host)s.")
class TransferNotFound(NotFound):
message = _("Transfer %(transfer_id)s could not be found.")
class InvalidReservationExpiration(Invalid):
message = _("Invalid reservation expiration %(expire)s.")
@ -489,6 +493,10 @@ class InvalidShare(Invalid):
message = _("Invalid share: %(reason)s.")
class InvalidAuthKey(Invalid):
message = _("Invalid auth key: %(reason)s")
class ShareBusyException(Invalid):
message = _("Share is busy with an active task: %(reason)s.")
@ -527,6 +535,10 @@ class ShareSnapshotAccessExists(InvalidInput):
message = _("Share snapshot access %(access_type)s:%(access)s exists.")
class InvalidSnapshot(Invalid):
message = _("Invalid snapshot: %(reason)s")
class InvalidSnapshotAccess(Invalid):
message = _("Invalid access rule: %(reason)s")
@ -543,6 +555,10 @@ class InvalidShareAccessType(Invalid):
message = _("Invalid or unsupported share access type: %(type)s.")
class DriverCannotTransferShareWithRules(ManilaException):
message = _("Driver failed to transfer share with rules.")
class ShareBackendException(ManilaException):
message = _("Share backend error: %(msg)s.")

View File

@ -36,6 +36,7 @@ class Action(object):
SHRINK = ('009', _('shrink'))
UPDATE_ACCESS_RULES = ('010', _('update access rules'))
ADD_UPDATE_SECURITY_SERVICE = ('011', _('add or update security service'))
TRANSFER_ACCEPT = ('026', _('transfer accept'))
ALL = (
ALLOCATE_HOST,
CREATE,
@ -48,6 +49,7 @@ class Action(object):
SHRINK,
UPDATE_ACCESS_RULES,
ADD_UPDATE_SECURITY_SERVICE,
TRANSFER_ACCEPT,
)
@ -136,6 +138,9 @@ class Detail(object):
_("Share Driver failed to create share because a security service "
"has not been added to the share network used. Please add a "
"security service to the share network."))
DRIVER_FAILED_TRANSFER_ACCEPT = (
'026',
_("Share transfer cannot be accepted without clearing access rules."))
ALL = (
UNKNOWN_ERROR,
@ -163,6 +168,7 @@ class Detail(object):
SECURITY_SERVICE_FAILED_AUTH,
NO_DEFAULT_SHARE_TYPE,
MISSING_SECURITY_SERVICE,
DRIVER_FAILED_TRANSFER_ACCEPT,
)
# Exception and detail mappings

View File

@ -42,6 +42,7 @@ from manila.policies import share_snapshot
from manila.policies import share_snapshot_export_location
from manila.policies import share_snapshot_instance
from manila.policies import share_snapshot_instance_export_location
from manila.policies import share_transfer
from manila.policies import share_type
from manila.policies import share_types_extra_spec
from manila.policies import shares
@ -78,4 +79,5 @@ def list_rules():
message.list_rules(),
share_access.list_rules(),
share_access_metadata.list_rules(),
share_transfer.list_rules(),
)

View File

@ -0,0 +1,151 @@
# Copyright (c) 2022 China Telecom Digital Intelligence.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo_policy import policy
from manila.policies import base
BASE_POLICY_NAME = 'share_transfer:%s'
DEPRECATED_REASON = """
The transfer API now supports system scope and default roles.
"""
deprecated_share_transfer_get_all = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'get_all',
check_str=base.RULE_DEFAULT,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
deprecated_share_transfer_get_all_tenant = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'get_all_tenant',
check_str=base.RULE_ADMIN_API,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
deprecated_share_transfer_create = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'create',
check_str=base.RULE_DEFAULT,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
deprecated_share_transfer_get = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'get',
check_str=base.RULE_DEFAULT,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
deprecated_share_transfer_accept = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'accept',
check_str=base.RULE_DEFAULT,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
deprecated_share_transfer_delete = policy.DeprecatedRule(
name=BASE_POLICY_NAME % 'delete',
check_str=base.RULE_DEFAULT,
deprecated_reason=DEPRECATED_REASON,
deprecated_since="Antelope"
)
share_transfer_policies = [
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'get_all',
check_str=base.ADMIN_OR_PROJECT_READER,
description="List share transfers.",
operations=[
{
'method': 'GET',
'path': '/share-transfers'
},
{
'method': 'GET',
'path': '/share-transfers/detail'
}
],
deprecated_rule=deprecated_share_transfer_get_all
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'get_all_tenant',
check_str=base.ADMIN,
scope_types=['project'],
description="List share transfers with all tenants.",
operations=[
{
'method': 'GET',
'path': '/share-transfers'
},
{
'method': 'GET',
'path': '/share-transfers/detail'
}
],
deprecated_rule=deprecated_share_transfer_get_all_tenant
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'create',
check_str=base.ADMIN_OR_PROJECT_MEMBER,
description="Create a share transfer.",
operations=[
{
'method': 'POST',
'path': '/share-transfers'
}
],
deprecated_rule=deprecated_share_transfer_create
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'get',
check_str=base.ADMIN_OR_PROJECT_READER,
description="Show one specified share transfer.",
operations=[
{
'method': 'GET',
'path': '/share-transfers/{transfer_id}'
}
],
deprecated_rule=deprecated_share_transfer_get
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'accept',
check_str=base.ADMIN_OR_PROJECT_MEMBER,
description="Accept a share transfer.",
operations=[
{
'method': 'POST',
'path': '/share-transfers/{transfer_id}/accept'
}
],
deprecated_rule=deprecated_share_transfer_accept
),
policy.DocumentedRuleDefault(
name=BASE_POLICY_NAME % 'delete',
check_str=base.ADMIN_OR_PROJECT_MEMBER,
description="Delete share transfer.",
operations=[
{
'method': 'DELETE',
'path': '/share-transfers/{transfer_id}'
}
],
deprecated_rule=deprecated_share_transfer_delete
),
]
def list_rules():
return share_transfer_policies

View File

@ -141,7 +141,8 @@ class API(base.Base):
compatible_azs_name.append(az['name'])
return compatible_azs_name, compatible_azs_multiple
def _check_if_share_quotas_exceeded(self, context, quota_exception,
@staticmethod
def check_if_share_quotas_exceeded(context, quota_exception,
share_size, operation='create'):
overs = quota_exception.kwargs['overs']
usages = quota_exception.kwargs['usages']
@ -171,7 +172,8 @@ class API(base.Base):
'operation': operation})
raise exception.ShareLimitExceeded(allowed=quotas['shares'])
def _check_if_replica_quotas_exceeded(self, context, quota_exception,
@staticmethod
def check_if_replica_quotas_exceeded(context, quota_exception,
replica_size,
resource_type='share_replica'):
overs = quota_exception.kwargs['overs']
@ -291,7 +293,7 @@ class API(base.Base):
supported=CONF.enabled_share_protocols))
raise exception.InvalidInput(reason=msg)
self._check_is_share_size_within_per_share_quota_limit(context, size)
self.check_is_share_size_within_per_share_quota_limit(context, size)
deltas = {'shares': 1, 'gigabytes': size}
share_type_attributes = self.get_share_attributes_from_share_type(
@ -306,9 +308,9 @@ class API(base.Base):
reservations = QUOTAS.reserve(
context, share_type_id=share_type_id, **deltas)
except exception.OverQuota as e:
self._check_if_share_quotas_exceeded(context, e, size)
self.check_if_share_quotas_exceeded(context, e, size)
if share_type_supports_replication:
self._check_if_replica_quotas_exceeded(context, e, size,
self.check_if_replica_quotas_exceeded(context, e, size,
resource_type='share')
share_group = None
@ -702,7 +704,7 @@ class API(base.Base):
share_type_id=share_type['id']
)
except exception.OverQuota as e:
self._check_if_replica_quotas_exceeded(context, e, share['size'])
self.check_if_replica_quotas_exceeded(context, e, share['size'])
az_request_multiple_subnet_support_map = {}
if share_network_id:
@ -1437,6 +1439,12 @@ class API(base.Base):
self.share_rpcapi.unmanage_share_server(
context, share_server, force=force)
def transfer_accept(self, context, share, new_user,
new_project, clear_rules=False):
self.share_rpcapi.transfer_accept(context, share,
new_user, new_project,
clear_rules=clear_rules)
def create_snapshot(self, context, share, name, description,
force=False, metadata=None):
policy.check_policy(context, 'share', 'create_snapshot', share)
@ -2390,7 +2398,8 @@ class API(base.Base):
}
raise exception.ShareBusyException(reason=msg)
def _check_is_share_size_within_per_share_quota_limit(self, context, size):
@staticmethod
def check_is_share_size_within_per_share_quota_limit(context, size):
"""Raises an exception if share size above per share quota limit."""
try:
values = {'per_share_gigabytes': size}
@ -2442,7 +2451,7 @@ class API(base.Base):
'size': share['size']})
raise exception.InvalidInput(reason=msg)
self._check_is_share_size_within_per_share_quota_limit(context,
self.check_is_share_size_within_per_share_quota_limit(context,
new_size)
# ensure we pass the share_type provisioning filter on size
@ -2479,12 +2488,12 @@ class API(base.Base):
reservations = QUOTAS.reserve(context, **deltas)
except exception.OverQuota as exc:
# Check if the exceeded quota was 'gigabytes'
self._check_if_share_quotas_exceeded(context, exc, share['size'],
self.check_if_share_quotas_exceeded(context, exc, share['size'],
operation='extend')
# NOTE(carloss): Check if the exceeded quota is
# 'replica_gigabytes'. If so the failure could be caused due to
# lack of quotas to extend the share's replicas, then the
# '_check_if_replica_quotas_exceeded' method can't be reused here
# 'check_if_replica_quotas_exceeded' method can't be reused here
# since the error message must be different from the default one.
if supports_replication:
overs = exc.kwargs['overs']

View File

@ -588,6 +588,19 @@ class ShareDriver(object):
"""
raise NotImplementedError()
def transfer_accept(self, context, share, new_user, new_project,
access_rules=None, share_server=None):
"""Backend update project and user info if stored on the backend.
:param context: The 'context.RequestContext' object for the request.
:param share: Share instance model.
:param access_rules: A list of access rules for given share.
:param new_user: the share will be updated with the new user id .
:param new_project: the share will be updated with the new project id.
:param share_server: share server for given share.
"""
pass
def migration_get_progress(
self, context, source_share, destination_share, source_snapshots,
snapshot_mappings, share_server=None,

View File

@ -764,6 +764,13 @@ class CephFSDriver(driver.ExecuteMixin, driver.GaneshaMixin,
def get_configured_ip_versions(self):
return self.protocol_helper.get_configured_ip_versions()
def transfer_accept(self, context, share, new_user, new_project,
access_rules=None, share_server=None):
# CephFS driver cannot transfer shares by preserving access rules
same_project = share["project_id"] == new_project
if access_rules and not same_project:
raise exception.DriverCannotTransferShareWithRules()
class NativeProtocolHelper(ganesha.NASHelperBase):
"""Helper class for native CephFS protocol"""

View File

@ -53,6 +53,7 @@ from manila.share import rpcapi as share_rpcapi
from manila.share import share_types
from manila.share import snapshot_access
from manila.share import utils as share_utils
from manila.transfer import api as transfer_api
from manila import utils
profiler = importutils.try_import('osprofiler.profiler')
@ -136,6 +137,11 @@ share_manager_opts = [
help='This value, specified in seconds, determines how often '
'the share manager will check for expired shares and '
'delete them from the Recycle bin.'),
cfg.IntOpt('check_for_expired_transfers',
default=300,
help='This value, specified in seconds, determines how often '
'the share manager will check for expired transfers and '
'destroy them and roll back share state.'),
]
CONF = cfg.CONF
@ -243,7 +249,7 @@ def add_hooks(f):
class ShareManager(manager.SchedulerDependentManager):
"""Manages NAS storages."""
RPC_API_VERSION = '1.24'
RPC_API_VERSION = '1.25'
def __init__(self, share_driver=None, service_name=None, *args, **kwargs):
"""Load the driver from args, or from flags."""
@ -286,6 +292,7 @@ class ShareManager(manager.SchedulerDependentManager):
self.message_api = message_api.API()
self.share_api = api.API()
self.transfer_api = transfer_api.API()
if CONF.profiler.enabled and profiler is not None:
self.driver = profiler.trace_cls("driver")(self.driver)
self.hooks = []
@ -3557,6 +3564,79 @@ class ShareManager(manager.SchedulerDependentManager):
LOG.info("share %s has expired, will be deleted", share['id'])
self.share_api.delete(ctxt, share)
@periodic_task.periodic_task(
spacing=CONF.check_for_expired_transfers)
def delete_expired_transfers(self, ctxt):
LOG.info("Checking for expired transfers.")
expired_transfers = self.db.get_all_expired_transfers(ctxt)
for transfer in expired_transfers:
LOG.debug("Transfer %s has expired, will be destroyed.",
transfer['id'])
self.transfer_api.delete(ctxt, transfer_id=transfer['id'])
@utils.require_driver_initialized
def transfer_accept(self, context, share_id, new_user,
new_project, clear_rules):
# need elevated context as we haven't "given" the share yet
elevated_context = context.elevated()
share_ref = self.db.share_get(elevated_context, share_id)
access_rules = self.db.share_access_get_all_for_share(
elevated_context, share_id)
share_instances = self.db.share_instances_get_all_by_share(
elevated_context, share_id)
share_server = self._get_share_server(context, share_ref)
for share_instance in share_instances:
share_instance = self.db.share_instance_get(context,
share_instance['id'],
with_share_data=True)
if clear_rules and access_rules:
try:
self.access_helper.update_access_rules(
context,
share_instance['id'],
delete_all_rules=True
)
access_rules = []
except Exception:
with excutils.save_and_reraise_exception():
msg = (
"Can not remove access rules for share "
"instance %(si)s belonging to share %(shr)s.")
msg_payload = {
'si': share_instance['id'],
'shr': share_id,
}
LOG.error(msg, msg_payload)
try:
self.driver.transfer_accept(context, share_instance,
new_user,
new_project,
access_rules=access_rules,
share_server=share_server)
except exception.DriverTransferShareWithRules as e:
with excutils.save_and_reraise_exception():
self.message_api.create(
context,
message_field.Action.TRANSFER_ACCEPT,
new_project,
resource_type=message_field.Resource.SHARE,
resource_id=share_id,
detail=(message_field.Detail.
DRIVER_FAILED_TRANSFER_ACCEPT))
msg = _("The backend failed to accept the share: %s.")
LOG.error(msg, e)
msg = ('Share %(share_id)s has transfer from %(old_project_id)s to '
'%(new_project_id)s completed successfully.')
msg_args = {
"share_id": share_id,
"old_project_id": share_ref['project_id'],
"new_project_id": context.project_id
}
LOG.info(msg, msg_args)
@add_hooks
@utils.require_driver_initialized
def create_snapshot(self, context, share_id, snapshot_id):

View File

@ -84,6 +84,7 @@ class ShareAPI(object):
1.23 - Add update_share_server_network_allocations() and
check_update_share_server_network_allocations()
1.24 - Add quiesce_wait_time paramater to promote_share_replica()
1.25 - Add transfer_accept()
"""
BASE_RPC_API_VERSION = '1.0'
@ -92,7 +93,7 @@ class ShareAPI(object):
super(ShareAPI, self).__init__()
target = messaging.Target(topic=CONF.share_topic,
version=self.BASE_RPC_API_VERSION)
self.client = rpc.get_client(target, version_cap='1.24')
self.client = rpc.get_client(target, version_cap='1.25')
def create_share_instance(self, context, share_instance, host,
request_spec, filter_properties,
@ -311,6 +312,18 @@ class ShareAPI(object):
call_context = self.client.prepare(fanout=True, version='1.0')
call_context.cast(context, 'publish_service_capabilities')
def transfer_accept(self, ctxt, share, new_user,
new_project, clear_rules=False):
msg_args = {
'share_id': share['id'],
'new_user': new_user,
'new_project': new_project,
'clear_rules': clear_rules
}
host = utils.extract_host(share['instance']['host'])
call_context = self.client.prepare(server=host, version='1.25')
call_context.call(ctxt, 'transfer_accept', **msg_args)
def extend_share(self, context, share, new_size, reservations):
host = utils.extract_host(share['instance']['host'])
call_context = self.client.prepare(server=host, version='1.2')

View File

@ -0,0 +1,440 @@
# Copyright (c) 2022 China Telecom Digital Intelligence.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import http.client as http_client
from unittest import mock
import ddt
from oslo_serialization import jsonutils
import webob
from manila.api.v2 import share_transfer
from manila import context
from manila import db
from manila import exception
from manila import quota
from manila.share import api as share_api
from manila.share import rpcapi as share_rpcapi
from manila.share import share_types
from manila import test
from manila.tests.api import fakes
from manila.tests import db_utils
from manila.transfer import api as transfer_api
SHARE_TRANSFER_VERSION = "2.77"
@ddt.ddt
class ShareTransferAPITestCase(test.TestCase):
"""Test Case for transfers V3 API."""
microversion = SHARE_TRANSFER_VERSION
def setUp(self):
super(ShareTransferAPITestCase, self).setUp()
self.share_transfer_api = transfer_api.API()
self.v2_controller = share_transfer.ShareTransferController()
self.ctxt = context.RequestContext(
'fake_user_id', 'fake_project_id', auth_token=True, is_admin=True)
def _create_transfer(self, share_id='fake_share_id',
display_name='test_transfer'):
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, display_name)
return transfer
def _create_share(self, display_name='test_share',
display_description='this is a test share',
status='available',
size=1,
project_id='fake_project_id',
user_id='fake_user_id',
share_type_id='fake_type_id',
share_network_id=None):
"""Create a share object."""
share = db_utils.create_share(display_name=display_name,
display_description=display_description,
status=status, size=size,
project_id=project_id,
user_id=user_id,
share_type_id=share_type_id,
share_network_id=share_network_id
)
share_id = share['id']
return share_id
def test_show_transfer(self):
share_id = self._create_share(size=5)
transfer = self._create_transfer(share_id)
path = '/v2/fake_project_id/share-transfers/%s' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res_dict = self.v2_controller.show(req, transfer['id'])
self.assertEqual('test_transfer', res_dict['transfer']['name'])
self.assertEqual(transfer['id'], res_dict['transfer']['id'])
self.assertEqual(share_id, res_dict['transfer']['resource_id'])
def test_list_transfers(self):
share_id_1 = self._create_share(size=5)
share_id_2 = self._create_share(size=5)
transfer1 = self._create_transfer(share_id_1)
transfer2 = self._create_transfer(share_id_2)
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res_dict = self.v2_controller.index(req)
self.assertEqual(transfer1['id'], res_dict['transfers'][1]['id'])
self.assertEqual('test_transfer', res_dict['transfers'][1]['name'])
self.assertEqual(transfer2['id'], res_dict['transfers'][0]['id'])
self.assertEqual('test_transfer', res_dict['transfers'][0]['name'])
def test_list_transfers_with_all_tenants(self):
share_id_1 = self._create_share(size=5)
share_id_2 = self._create_share(size=5, project_id='fake_project_id2',
user_id='fake_user_id2')
self._create_transfer(share_id_1)
self._create_transfer(share_id_2)
path = '/v2/fake_project_id/share-transfers?all_tenants=true'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = context.get_admin_context()
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res_dict = self.v2_controller.index(req)
self.assertEqual(2, len(res_dict['transfers']))
def test_list_transfers_with_limit(self):
share_id_1 = self._create_share(size=5)
share_id_2 = self._create_share(size=5)
self._create_transfer(share_id_1)
self._create_transfer(share_id_2)
path = '/v2/fake_project_id/share-transfers?limit=1'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res_dict = self.v2_controller.index(req)
self.assertEqual(1, len(res_dict['transfers']))
@ddt.data("desc", "asc")
def test_list_transfers_with_sort(self, sort_dir):
share_id_1 = self._create_share(size=5)
share_id_2 = self._create_share(size=5)
transfer1 = self._create_transfer(share_id_1)
transfer2 = self._create_transfer(share_id_2)
path = \
'/v2/fake_project_id/share-transfers?sort_key=id&sort_dir=%s' % (
sort_dir)
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res_dict = self.v2_controller.index(req)
self.assertEqual(2, len(res_dict['transfers']))
order_ids = sorted([transfer1['id'],
transfer2['id']])
expect_result = order_ids[1] if sort_dir == "desc" else order_ids[0]
self.assertEqual(expect_result,
res_dict['transfers'][0]['id'])
def test_list_transfers_detail(self):
share_id_1 = self._create_share(size=5)
share_id_2 = self._create_share(size=5)
transfer1 = self._create_transfer(share_id_1)
transfer2 = self._create_transfer(share_id_2)
path = '/v2/fake_project_id/share-transfers/detail'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
req.headers['Accept'] = 'application/json'
res_dict = self.v2_controller.detail(req)
self.assertEqual('test_transfer',
res_dict['transfers'][1]['name'])
self.assertEqual(transfer1['id'], res_dict['transfers'][1]['id'])
self.assertEqual(share_id_1, res_dict['transfers'][1]['resource_id'])
self.assertEqual('test_transfer',
res_dict['transfers'][0]['name'])
self.assertEqual(transfer2['id'], res_dict['transfers'][0]['id'])
self.assertEqual(share_id_2, res_dict['transfers'][0]['resource_id'])
def test_create_transfer(self):
share_id = self._create_share(status='available', size=5)
body = {"transfer": {"name": "transfer1",
"share_id": share_id}}
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
res_dict = self.v2_controller.create(req, body)
self.assertIn('id', res_dict['transfer'])
self.assertIn('auth_key', res_dict['transfer'])
self.assertIn('created_at', res_dict['transfer'])
self.assertIn('name', res_dict['transfer'])
self.assertIn('resource_id', res_dict['transfer'])
@ddt.data({},
{"transfer": {"name": "transfer1"}},
{"transfer": {"name": "transfer1",
"share_id": "invalid_share_id"}})
def test_create_transfer_with_invalid_body(self, body):
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.create, req, body)
def test_create_transfer_with_invalid_share_status(self):
share_id = self._create_share()
body = {"transfer": {"name": "transfer1",
"share_id": share_id}}
db.share_update(context.get_admin_context(),
share_id, {'status': 'error'})
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.create, req, body)
def test_create_transfer_share_with_network_id(self):
share_id = self._create_share(share_network_id='fake_id')
body = {"transfer": {"name": "transfer1",
"share_id": share_id}}
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.create, req, body)
def test_create_transfer_share_with_invalid_snapshot(self):
share_id = self._create_share(share_network_id='fake_id')
db_utils.create_snapshot(share_id=share_id)
body = {"transfer": {"name": "transfer1",
"share_id": share_id}}
path = '/v2/fake_project_id/share-transfers'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.create, req, body)
def test_delete_transfer_awaiting_transfer(self):
share_id = self._create_share()
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, 'test_transfer')
path = '/v2/fake_project_id/share-transfers/%s' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'DELETE'
req.headers['Content-Type'] = 'application/json'
self.v2_controller.delete(req, transfer['id'])
# verify transfer has been deleted
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'GET'
req.headers['Content-Type'] = 'application/json'
res = req.get_response(fakes.app())
self.assertEqual(http_client.NOT_FOUND, res.status_int)
self.assertEqual(db.share_get(context.get_admin_context(),
share_id)['status'], 'available')
def test_delete_transfer_not_awaiting_transfer(self):
share_id = self._create_share()
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, 'test_transfer')
db.share_update(context.get_admin_context(),
share_id, {'status': 'available'})
path = '/v2/fake_project_id/share-transfers/%s' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'DELETE'
req.headers['Content-Type'] = 'application/json'
self.assertRaises(exception.InvalidShare,
self.v2_controller.delete, req,
transfer['id'])
def test_transfer_accept_share_id_specified(self):
share_id = self._create_share()
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, 'test_transfer')
self.mock_object(quota.QUOTAS, 'reserve', mock.Mock())
self.mock_object(quota.QUOTAS, 'commit', mock.Mock())
self.mock_object(share_api.API,
'check_is_share_size_within_per_share_quota_limit',
mock.Mock())
self.mock_object(share_rpcapi.ShareAPI,
'transfer_accept',
mock.Mock())
fake_share_type = {'id': 'fake_id',
'name': 'fake_name',
'is_public': True}
self.mock_object(share_types, 'get_share_type',
mock.Mock(return_value=fake_share_type))
self.mock_object(db, 'share_snapshot_get_all_for_share',
mock.Mock(return_value={}))
body = {"accept": {"auth_key": transfer['auth_key']}}
path = '/v2/fake_project_id/share-transfers/%s/accept' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.v2_controller.accept(req, transfer['id'], body)
def test_transfer_accept_with_not_public_share_type(self):
share_id = self._create_share()
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, 'test_transfer')
fake_share_type = {'id': 'fake_id',
'name': 'fake_name',
'is_public': False,
'projects': ['project_id1', 'project_id2']}
self.mock_object(share_types, 'get_share_type',
mock.Mock(return_value=fake_share_type))
body = {"accept": {"auth_key": transfer['auth_key']}}
path = '/v2/fake_project_id/share-transfers/%s/accept' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.accept, req,
transfer['id'], body)
@ddt.data({},
{"accept": {}},
{"accept": {"auth_key": "fake_auth_key",
"clear_access_rules": "invalid_bool"}})
def test_transfer_accept_with_invalid_body(self, body):
path = '/v2/fake_project_id/share-transfers/fake_transfer_id/accept'
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.accept, req,
'fake_transfer_id', body)
def test_transfer_accept_with_invalid_auth_key(self):
share_id = self._create_share(size=5)
transfer = self._create_transfer(share_id)
body = {"accept": {"auth_key": "invalid_auth_key"}}
path = '/v2/fake_project_id/share-transfers/%s/accept' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.accept, req,
transfer['id'], body)
def test_transfer_accept_with_invalid_share_status(self):
share_id = self._create_share(size=5)
transfer = self._create_transfer(share_id)
db.share_update(context.get_admin_context(),
share_id, {'status': 'error'})
body = {"accept": {"auth_key": transfer['auth_key']}}
path = '/v2/fake_project_id/share-transfers/%s/accept' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPBadRequest,
self.v2_controller.accept, req,
transfer['id'], body)
@ddt.data({'overs': {'gigabytes': 'fake'}},
{'overs': {'shares': 'fake'}},
{'overs': {'snapshot_gigabytes': 'fake'}},
{'overs': {'snapshots': 'fake'}})
@ddt.unpack
def test_accept_share_over_quota(self, overs):
share_id = self._create_share()
db_utils.create_snapshot(share_id=share_id, status='available')
transfer = self.share_transfer_api.create(context.get_admin_context(),
share_id, 'test_transfer')
usages = {'gigabytes': {'reserved': 5, 'in_use': 5},
'shares': {'reserved': 10, 'in_use': 10},
'snapshot_gigabytes': {'reserved': 5, 'in_use': 5},
'snapshots': {'reserved': 10, 'in_use': 10}}
quotas = {'gigabytes': 5, 'shares': 10,
'snapshot_gigabytes': 5, 'snapshots': 10}
exc = exception.OverQuota(overs=overs, usages=usages, quotas=quotas)
self.mock_object(quota.QUOTAS, 'reserve', mock.Mock(side_effect=exc))
self.mock_object(quota.QUOTAS, 'commit', mock.Mock())
self.mock_object(share_api.API,
'check_is_share_size_within_per_share_quota_limit',
mock.Mock())
self.mock_object(share_rpcapi.ShareAPI,
'transfer_accept',
mock.Mock())
fake_share_type = {'id': 'fake_id',
'name': 'fake_name',
'is_public': True}
self.mock_object(share_types, 'get_share_type',
mock.Mock(return_value=fake_share_type))
body = {"accept": {"auth_key": transfer['auth_key']}}
path = '/v2/fake_project_id/share-transfers/%s/accept' % transfer['id']
req = fakes.HTTPRequest.blank(path, version=self.microversion)
req.environ['manila.context'] = self.ctxt
req.method = 'POST'
req.headers['Content-Type'] = 'application/json'
req.body = jsonutils.dumps(body).encode("utf-8")
self.assertRaises(webob.exc.HTTPRequestEntityTooLarge,
self.v2_controller.accept, req,
transfer['id'], body)

View File

@ -5083,3 +5083,146 @@ class AsyncOperationDatabaseAPITestCase(test.TestCase):
self.ctxt, test_id)
self.assertEqual({}, actual_result)
class TransfersTestCase(test.TestCase):
"""Test case for transfers."""
def setUp(self):
super(TransfersTestCase, self).setUp()
self.user_id = uuidutils.generate_uuid()
self.project_id = uuidutils.generate_uuid()
self.ctxt = context.RequestContext(user_id=self.user_id,
project_id=self.project_id)
@staticmethod
def _create_transfer(resource_type='share',
resource_id=None, source_project_id=None):
"""Create a transfer object."""
if resource_id and source_project_id:
transfer = db_utils.create_transfer(
resource_type=resource_type,
resource_id=resource_id,
source_project_id=source_project_id)
elif resource_id:
transfer = db_utils.create_transfer(
resource_type=resource_type,
resource_id=resource_id)
elif source_project_id:
transfer = db_utils.create_transfer(
resource_type=resource_type,
source_project_id=source_project_id)
else:
transfer = db_utils.create_transfer(
resource_type=resource_type)
return transfer['id']
def test_transfer_create(self):
# If the resource_id is Null a KeyError exception will be raised.
self.assertRaises(KeyError, self._create_transfer)
share = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)
share_id = share['id']
self._create_transfer(resource_id=share_id)
def test_share_transfer_get(self):
share_id = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)['id']
transfer_id = self._create_transfer(resource_id=share_id)
transfer = db_api.share_transfer_get(self.ctxt, transfer_id)
self.assertEqual(share_id, transfer['resource_id'])
new_ctxt = context.RequestContext(user_id='new_user_id',
project_id='new_project_id')
self.assertRaises(exception.TransferNotFound,
db_api.share_transfer_get, new_ctxt, transfer_id)
transfer = db_api.share_transfer_get(new_ctxt.elevated(), transfer_id)
self.assertEqual(share_id, transfer['resource_id'])
def test_transfer_get_all(self):
share_id1 = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)['id']
share_id2 = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)['id']
self._create_transfer(resource_id=share_id1,
source_project_id=self.project_id)
self._create_transfer(resource_id=share_id2,
source_project_id=self.project_id)
self.assertRaises(exception.NotAuthorized,
db_api.transfer_get_all,
self.ctxt)
transfers = db_api.transfer_get_all(context.get_admin_context())
self.assertEqual(2, len(transfers))
transfers = db_api.transfer_get_all_by_project(self.ctxt,
self.project_id)
self.assertEqual(2, len(transfers))
new_ctxt = context.RequestContext(user_id='new_user_id',
project_id='new_project_id')
transfers = db_api.transfer_get_all_by_project(new_ctxt,
'new_project_id')
self.assertEqual(0, len(transfers))
def test_transfer_destroy(self):
share_id1 = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)['id']
share_id2 = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)['id']
transfer_id1 = self._create_transfer(resource_id=share_id1,
source_project_id=self.project_id)
transfer_id2 = self._create_transfer(resource_id=share_id2,
source_project_id=self.project_id)
transfers = db_api.transfer_get_all(context.get_admin_context())
self.assertEqual(2, len(transfers))
db_api.transfer_destroy(self.ctxt, transfer_id1)
transfers = db_api.transfer_get_all(context.get_admin_context())
self.assertEqual(1, len(transfers))
db_api.transfer_destroy(self.ctxt, transfer_id2)
transfers = db_api.transfer_get_all(context.get_admin_context())
self.assertEqual(0, len(transfers))
def test_transfer_accept_then_rollback(self):
share = db_utils.create_share(size=1, user_id=self.user_id,
project_id=self.project_id)
transfer_id = self._create_transfer(resource_id=share['id'],
source_project_id=self.project_id)
new_ctxt = context.RequestContext(user_id='new_user_id',
project_id='new_project_id')
transfer = db_api.share_transfer_get(new_ctxt.elevated(), transfer_id)
self.assertEqual(share['project_id'], transfer['source_project_id'])
self.assertFalse(transfer['accepted'])
self.assertIsNone(transfer['destination_project_id'])
# accept the transfer
db_api.transfer_accept(new_ctxt.elevated(), transfer_id,
'new_user_id', 'new_project_id')
transfer = db_api.model_query(
new_ctxt.elevated(), models.Transfer,
read_deleted='yes').filter_by(id=transfer_id).first()
share = db_api.share_get(new_ctxt.elevated(), share['id'])
self.assertEqual(share['project_id'], 'new_project_id')
self.assertEqual(share['user_id'], 'new_user_id')
self.assertTrue(transfer['accepted'])
self.assertEqual('new_project_id', transfer['destination_project_id'])
# then test rollback the transfer
db_api.transfer_accept_rollback(new_ctxt.elevated(), transfer_id,
self.user_id, self.project_id)
transfer = db_api.model_query(
new_ctxt.elevated(),
models.Transfer).filter_by(id=transfer_id).first()
share = db_api.share_get(new_ctxt.elevated(), share['id'])
self.assertEqual(share['project_id'], self.project_id)
self.assertEqual(share['user_id'], self.user_id)
self.assertFalse(transfer['accepted'])

View File

@ -299,3 +299,11 @@ def create_message(**kwargs):
'message_level': message_levels.ERROR,
}
return _create_db_row(db.message_create, message_dict, kwargs)
def create_transfer(**kwargs):
transfer = {'display_name': 'display_name',
'salt': 'salt',
'crypt_hash': 'crypt_hash',
'resource_type': constants.SHARE_RESOURCE_TYPE}
return _create_db_row(db.transfer_create, transfer, kwargs)

View File

@ -588,6 +588,26 @@ class CephFSDriverTestCase(test.TestCase):
(self._driver.protocol_helper.get_configured_ip_versions.
assert_called_once_with())
@ddt.data(
([{'id': 'instance_mapping_id1', 'access_id': 'accessid1',
'access_level': 'rw', 'access_type': 'cephx', 'access_to': 'alice'
}], 'fake_project_uuid_1'),
([{'id': 'instance_mapping_id1', 'access_id': 'accessid1',
'access_level': 'rw', 'access_type': 'cephx', 'access_to': 'alice'
}], 'fake_project_uuid_2'),
([], 'fake_project_uuid_1'),
([], 'fake_project_uuid_2'),
)
@ddt.unpack
def test_transfer_accept(self, access_rules, new_project):
fake_share_1 = {"project_id": "fake_project_uuid_1"}
same_project = new_project == 'fake_project_uuid_1'
if access_rules and not same_project:
self.assertRaises(exception.DriverCannotTransferShareWithRules,
self._driver.transfer_accept,
self._context, fake_share_1,
'new_user', new_project, access_rules)
@ddt.ddt
class NativeProtocolHelperTestCase(test.TestCase):

View File

@ -929,8 +929,8 @@ class ShareAPITestCase(test.TestCase):
if replication_type:
# Prevent the raising of an exception, to force the call to the
# function _check_if_replica_quotas_exceeded
self.mock_object(self.api, '_check_if_share_quotas_exceeded')
# function check_if_replica_quotas_exceeded
self.mock_object(self.api, 'check_if_share_quotas_exceeded')
self.assertRaises(
expected_exception,

View File

@ -49,6 +49,7 @@ from manila.tests import fake_notifier
from manila.tests import fake_share as fakes
from manila.tests import fake_utils
from manila.tests import utils as test_utils
from manila.transfer import api as transfer_api
from manila import utils
@ -3994,6 +3995,54 @@ class ShareManagerTestCase(test.TestCase):
api.API.delete.assert_called_once_with(
self.context, share1)
def test_delete_expired_transfers(self):
self.mock_object(db, 'get_all_expired_transfers',
mock.Mock(return_value=[{"id": "transfer1",
"name": "test_tr"}, ]))
self.mock_object(transfer_api.API, 'delete')
self.share_manager.delete_expired_transfers(self.context)
db.get_all_expired_transfers.assert_called_once_with(self.context)
transfer1 = {"id": "transfer1", "name": "test_tr"}
transfer_api.API.delete.assert_called_once_with(
self.context, transfer_id=transfer1["id"])
@ddt.data(True, False)
def test_transfer_accept(self, clear_rules):
share = db_utils.create_share(id="fake")
self.mock_object(db, 'share_get', mock.Mock(return_value=share))
update_access_rules_call = self.mock_object(
self.share_manager.access_helper,
'update_access_rules')
transfer_accept_call = self.mock_object(self.share_manager.driver,
'transfer_accept')
instances, rules = self._setup_init_mocks()
self.mock_object(self.share_manager.db,
'share_access_get_all_for_share',
mock.Mock(return_value=rules))
self.mock_object(self.share_manager.db,
'share_instances_get_all_by_share',
mock.Mock(return_value=instances))
self.mock_object(db, 'share_instance_get',
mock.Mock(return_value=instances[0]))
self.mock_object(self.share_manager,
'_get_share_server',
mock.Mock(return_value=None))
self.share_manager.transfer_accept(self.context, "fake_share_id",
"fake_user_id", "fake_project_id",
clear_rules)
if clear_rules:
update_access_rules_call.assert_called_with(
self.context, instances[0]['id'], delete_all_rules=True)
transfer_accept_call.assert_called_with(
self.context, instances[0], "fake_user_id",
"fake_project_id", access_rules=[],
share_server=None)
else:
transfer_accept_call.assert_called_with(
self.context, instances[0], "fake_user_id",
"fake_project_id", access_rules=rules,
share_server=None)
@mock.patch('manila.tests.fake_notifier.FakeNotifier._notify')
def test_extend_share_invalid(self, mock_notify):
share = db_utils.create_share()

View File

@ -235,6 +235,15 @@ class ShareRpcAPITestCase(test.TestCase):
rpc_method='cast',
share_server=self.fake_share_server)
def test_transfer_accept(self):
self._test_share_api('transfer_accept',
rpc_method='call',
version='1.25',
share=self.fake_share,
new_user='new_user',
new_project='new_project',
clear_rules=False)
def test_extend_share(self):
self._test_share_api('extend_share',
rpc_method='cast',

View File

440
manila/transfer/api.py Normal file
View File

@ -0,0 +1,440 @@
# Copyright (C) 2022 China Telecom Digital Intelligence.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Handles all requests relating to transferring ownership of shares.
"""
import hashlib
import hmac
import os
from oslo_config import cfg
from oslo_log import log as logging
from oslo_utils import excutils
from oslo_utils import strutils
from manila.common import constants
from manila.db import base
from manila import exception
from manila.i18n import _
from manila import policy
from manila import quota
from manila.share import api as share_api
from manila.share import share_types
from manila.share import utils as share_utils
share_transfer_opts = [
cfg.IntOpt('share_transfer_salt_length',
default=8,
help='The number of characters in the salt.',
min=8,
max=255),
cfg.IntOpt('share_transfer_key_length',
default=16,
help='The number of characters in the autogenerated auth key.',
min=16,
max=255),
]
CONF = cfg.CONF
CONF.register_opts(share_transfer_opts)
LOG = logging.getLogger(__name__)
QUOTAS = quota.QUOTAS
class API(base.Base):
"""API for interacting share transfers."""
def __init__(self):
self.share_api = share_api.API()
super().__init__()
def get(self, context, transfer_id):
transfer = self.db.share_transfer_get(context, transfer_id)
return transfer
def delete(self, context, transfer_id):
"""Delete a share transfer."""
transfer = self.db.share_transfer_get(context, transfer_id)
policy.check_policy(context, 'share_transfer', 'delete', target_obj={
'project_id': transfer['source_project_id']})
update_share_status = True
share_ref = None
try:
share_ref = self.db.share_get(context, transfer.resource_id)
except exception.NotFound:
update_share_status = False
if update_share_status:
share_instance = share_ref['instance']
if share_ref['status'] != constants.STATUS_AWAITING_TRANSFER:
msg = (_('Transfer %(transfer_id)s: share id %(share_id)s '
'expected in awaiting_transfer state.'))
msg_payload = {'transfer_id': transfer_id,
'share_id': share_ref['id']}
LOG.error(msg, msg_payload)
raise exception.InvalidShare(reason=msg)
if update_share_status:
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.delete.start")
self.db.transfer_destroy(context, transfer_id,
update_share_status=update_share_status)
if update_share_status:
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.delete.end")
LOG.info('Transfer %s has been deleted successful.', transfer_id)
def get_all(self, context, limit=None, sort_key=None,
sort_dir=None, filters=None, offset=None):
filters = filters or {}
all_tenants = strutils.bool_from_string(filters.pop('all_tenants',
'false'))
query_by_project = False
if all_tenants:
try:
policy.check_policy(context, 'share_transfer',
'get_all_tenant')
except exception.PolicyNotAuthorized:
query_by_project = True
else:
query_by_project = True
if query_by_project:
transfers = self.db.transfer_get_all_by_project(
context, context.project_id,
limit=limit, sort_key=sort_key, sort_dir=sort_dir,
filters=filters, offset=offset)
else:
transfers = self.db.transfer_get_all(context,
limit=limit,
sort_key=sort_key,
sort_dir=sort_dir,
filters=filters,
offset=offset)
return transfers
def _get_random_string(self, length):
"""Get a random hex string of the specified length."""
rndstr = ""
# Note that the string returned by this function must contain only
# characters that the recipient can enter on their keyboard. The
# function sha256().hexdigit() achieves this by generating a hash
# which will only contain hexadecimal digits.
while len(rndstr) < length:
rndstr += hashlib.sha256(os.urandom(255)).hexdigest()
return rndstr[0:length]
def _get_crypt_hash(self, salt, auth_key):
"""Generate a random hash based on the salt and the auth key."""
def _format_str(input_str):
if not isinstance(input_str, (bytes, str)):
input_str = str(input_str)
if isinstance(input_str, str):
input_str = input_str.encode('utf-8')
return input_str
salt = _format_str(salt)
auth_key = _format_str(auth_key)
return hmac.new(salt, auth_key, hashlib.sha256).hexdigest()
def create(self, context, share_id, display_name):
"""Creates an entry in the transfers table."""
LOG.debug("Generating transfer record for share %s", share_id)
try:
share_ref = self.share_api.get(context, share_id)
except exception.NotFound:
msg = _("Share specified was not found.")
raise exception.InvalidShare(reason=msg)
policy.check_policy(context, "share_transfer", "create",
target_obj=share_ref)
share_instance = share_ref['instance']
if share_ref['status'] != "available":
raise exception.InvalidShare(reason=_("Share's status must be "
"available"))
if share_ref['share_network_id']:
raise exception.InvalidShare(reason=_(
"Shares exported over share networks cannot be transferred."))
if share_ref['share_group_id']:
raise exception.InvalidShare(reason=_(
"Shares within share groups cannot be transferred."))
if share_ref.has_replicas:
raise exception.InvalidShare(reason=_(
"Shares with replicas cannot be transferred."))
snapshots = self.db.share_snapshot_get_all_for_share(context, share_id)
for snapshot in snapshots:
if snapshot['status'] != "available":
msg = _("Snapshot: %s status must be "
"available") % snapshot['id']
raise exception.InvalidSnapshot(reason=msg)
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.create.start")
# The salt is just a short random string.
salt = self._get_random_string(CONF.share_transfer_salt_length)
auth_key = self._get_random_string(CONF.share_transfer_key_length)
crypt_hash = self._get_crypt_hash(salt, auth_key)
transfer_rec = {'resource_type': constants.SHARE_RESOURCE_TYPE,
'resource_id': share_id,
'display_name': display_name,
'salt': salt,
'crypt_hash': crypt_hash,
'expires_at': None,
'source_project_id': share_ref['project_id']}
try:
transfer = self.db.transfer_create(context, transfer_rec)
except Exception:
with excutils.save_and_reraise_exception():
LOG.error("Failed to create transfer record for %s", share_id)
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.create.end")
return {'id': transfer['id'],
'resource_type': transfer['resource_type'],
'resource_id': transfer['resource_id'],
'display_name': transfer['display_name'],
'auth_key': auth_key,
'created_at': transfer['created_at'],
'source_project_id': transfer['source_project_id'],
'destination_project_id': transfer['destination_project_id'],
'accepted': transfer['accepted'],
'expires_at': transfer['expires_at']}
def _handle_snapshot_quota(self, context, snapshots, donor_id):
snapshots_num = len(snapshots)
share_snap_sizes = 0
for snapshot in snapshots:
share_snap_sizes += snapshot['size']
try:
reserve_opts = {'snapshots': snapshots_num,
'gigabytes': share_snap_sizes}
reservations = QUOTAS.reserve(context, **reserve_opts)
except exception.OverQuota as e:
reservations = None
overs = e.kwargs['overs']
usages = e.kwargs['usages']
quotas = e.kwargs['quotas']
def _consumed(name):
return (usages[name]['reserved'] + usages[name]['in_use'])
if 'snapshot_gigabytes' in overs:
msg = ("Quota exceeded for %(s_pid)s, tried to accept "
"%(s_size)sG snapshot (%(d_consumed)dG of "
"%(d_quota)dG already consumed).")
LOG.warning(msg, {
's_pid': context.project_id,
's_size': share_snap_sizes,
'd_consumed': _consumed('snapshot_gigabytes'),
'd_quota': quotas['snapshot_gigabytes']})
raise exception.SnapshotSizeExceedsAvailableQuota()
elif 'snapshots' in overs:
msg = ("Quota exceeded for %(s_pid)s, tried to accept "
"%(s_num)s snapshot (%(d_consumed)d of "
"%(d_quota)d already consumed).")
LOG.warning(msg, {'s_pid': context.project_id,
's_num': snapshots_num,
'd_consumed': _consumed('snapshots'),
'd_quota': quotas['snapshots']})
raise exception.SnapshotLimitExceeded(
allowed=quotas['snapshots'])
try:
reserve_opts = {'snapshots': -snapshots_num,
'gigabytes': -share_snap_sizes}
donor_reservations = QUOTAS.reserve(context,
project_id=donor_id,
**reserve_opts)
except exception.OverQuota:
donor_reservations = None
LOG.exception("Failed to update share providing snapshots quota:"
" Over quota.")
return reservations, donor_reservations
@staticmethod
def _check_share_type_access(context, share_type_id, share_id):
share_type = share_types.get_share_type(
context, share_type_id, expected_fields=['projects'])
if not share_type['is_public']:
if context.project_id not in share_type['projects']:
msg = _("Share type of share %(share_id)s is not public, "
"and current project can not access the share "
"type ") % {'share_id': share_id}
LOG.error(msg)
raise exception.InvalidShare(reason=msg)
def _check_transferred_project_quota(self, context, share_ref_size):
try:
reserve_opts = {'shares': 1, 'gigabytes': share_ref_size}
reservations = QUOTAS.reserve(context,
**reserve_opts)
except exception.OverQuota as exc:
reservations = None
self.share_api.check_if_share_quotas_exceeded(context, exc,
share_ref_size)
return reservations
@staticmethod
def _check_donor_project_quota(context, donor_id, share_ref_size,
transfer_id):
try:
reserve_opts = {'shares': -1, 'gigabytes': -share_ref_size}
donor_reservations = QUOTAS.reserve(context.elevated(),
project_id=donor_id,
**reserve_opts)
except Exception:
donor_reservations = None
LOG.exception("Failed to update quota donating share"
" transfer id %s", transfer_id)
return donor_reservations
@staticmethod
def _check_snapshot_status(snapshots, transfer_id):
for snapshot in snapshots:
# Only check snapshot with instances
if snapshot.get('status'):
if snapshot['status'] != 'available':
msg = (_('Transfer %(transfer_id)s: Snapshot '
'%(snapshot_id)s is not in the expected '
'available state.')
% {'transfer_id': transfer_id,
'snapshot_id': snapshot['id']})
LOG.error(msg)
raise exception.InvalidSnapshot(reason=msg)
def accept(self, context, transfer_id, auth_key, clear_rules=False):
"""Accept a share that has been offered for transfer."""
# We must use an elevated context to make sure we can find the
# transfer.
transfer = self.db.share_transfer_get(context.elevated(), transfer_id)
crypt_hash = self._get_crypt_hash(transfer['salt'], auth_key)
if crypt_hash != transfer['crypt_hash']:
msg = (_("Attempt to transfer %s with invalid auth key.") %
transfer_id)
LOG.error(msg)
raise exception.InvalidAuthKey(reason=msg)
share_id = transfer['resource_id']
try:
# We must use an elevated context to see the share that is still
# owned by the donor.
share_ref = self.share_api.get(context.elevated(), share_id)
except exception.NotFound:
msg = _("Share specified was not found.")
raise exception.InvalidShare(reason=msg)
share_instance = share_ref['instance']
if share_ref['status'] != constants.STATUS_AWAITING_TRANSFER:
msg = (_('Transfer %(transfer_id)s: share id %(share_id)s '
'expected in awaiting_transfer state.')
% {'transfer_id': transfer_id, 'share_id': share_id})
LOG.error(msg)
raise exception.InvalidShare(reason=msg)
share_ref_size = share_ref['size']
share_type_id = share_ref.get('share_type_id')
# check share type access
if share_type_id:
self._check_share_type_access(context, share_type_id, share_id)
# check per share quota limit
self.share_api.check_is_share_size_within_per_share_quota_limit(
context, share_ref_size)
# check accept transferred project quotas
reservations = self._check_transferred_project_quota(
context, share_ref_size)
# check donor project quotas
donor_id = share_ref['project_id']
donor_reservations = self._check_donor_project_quota(
context, donor_id, share_ref_size, transfer_id)
snap_res = None
snap_donor_res = None
accept_snapshots = False
snapshots = self.db.share_snapshot_get_all_for_share(
context.elevated(), share_id)
if snapshots:
self._check_snapshot_status(snapshots, transfer_id)
accept_snapshots = True
snap_res, snap_donor_res = self._handle_snapshot_quota(
context, snapshots, share_ref['project_id'])
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.accept.start")
try:
self.share_api.transfer_accept(context,
share_ref,
context.user_id,
context.project_id,
clear_rules=clear_rules)
# Transfer ownership of the share now, must use an elevated
# context.
self.db.transfer_accept(context.elevated(),
transfer_id,
context.user_id,
context.project_id,
accept_snapshots=accept_snapshots)
if reservations:
QUOTAS.commit(context, reservations)
if snap_res:
QUOTAS.commit(context, snap_res)
if donor_reservations:
QUOTAS.commit(context, donor_reservations, project_id=donor_id)
if snap_donor_res:
QUOTAS.commit(context, snap_donor_res, project_id=donor_id)
LOG.info("share %s has been transferred.", share_id)
except Exception:
with excutils.save_and_reraise_exception():
try:
# storage try to rollback
self.share_api.transfer_accept(context,
share_ref,
share_ref['user_id'],
share_ref['project_id'])
# db try to rollback
self.db.transfer_accept_rollback(
context.elevated(), transfer_id,
share_ref['user_id'], share_ref['project_id'],
rollback_snap=accept_snapshots)
finally:
if reservations:
QUOTAS.rollback(context, reservations)
if snap_res:
QUOTAS.rollback(context, snap_res)
if donor_reservations:
QUOTAS.rollback(context, donor_reservations,
project_id=donor_id)
if snap_donor_res:
QUOTAS.rollback(context, snap_donor_res,
project_id=donor_id)
share_utils.notify_about_share_usage(context, share_ref,
share_instance,
"transfer.accept.end")

View File

@ -0,0 +1,4 @@
---
features:
- Share can be transferred between project with API version ``2.77``
and beyond.