Move Metadata Spec from Wallaby to Yoga

Also updates spec details and assignee

Change-Id: I0f6cd9bbc912e8cc139d5fbde9279c6ea96bc2f0
This commit is contained in:
Ashley Rodriguez 2021-10-19 22:20:40 +00:00
parent 816a17b924
commit 833cb6c3e0

View File

@ -29,15 +29,19 @@ Problem description
End users interact with manila API to manage their shared file system storage End users interact with manila API to manage their shared file system storage
resources. These resources include: resources. These resources include:
- Shares - Shares
- Snapshots - Share Export Locations
- Export Locations - Share Access Rules
- Share Replicas - Share Instances
- Share Groups - Share Instance Export Locations
- Share Group Snapshots - Share Replicas
- Security Services - Share Replica Export Locations
- Share Networks - Share Snapshots
- Share Network Subnets - Share Groups
- Share Group Snapshots
- Security Services
- Share Networks
- Share Network Subnets
All of these resources are identifiable by ID. Some of them even allow setting All of these resources are identifiable by ID. Some of them even allow setting
free form text as Name and/or Description. Free form text is usually hard to free form text as Name and/or Description. Free form text is usually hard to
@ -86,10 +90,9 @@ implement API handler code to:
- set a single resource metadata item (update a specific key=value pair) - set a single resource metadata item (update a specific key=value pair)
- unset resource metadata item (delete a specific key=value pair) - unset resource metadata item (delete a specific key=value pair)
To distinguish and protect service owned metadata, all metadata tables will To distinguish and protect service owned or admin only metadata,
include an attribute named ``user_modifiable``. This attribute will there will be an ignore_keys parameter and ignored keys dictionary
determine if users have the ability to manipulate (or delete) to prevent these metadata items from being manipulated by the end user.
the specific metadatum.
Alternatives Alternatives
------------ ------------
@ -125,32 +128,30 @@ necessary. These tables will be created during a database upgrade, and
nullified and destroyed during a database downgrade. nullified and destroyed during a database downgrade.
Existing metadata tables for Shares ("share_metadata"), Export Locations Existing metadata tables for Shares ("share_metadata"), Export Locations
("share_instance_export_locations_metadata") and Share Access ("share_instance_export_locations_metadata") will be modified to include
Rules ("share_access_rules_metadata") will be modified to include a boolean a string attribute called ``deleted``. The default value for this attribute is
attribute called ``user_modifiable``. The default value for this attribute is set to False.
set to True. All existing export location metadata will have this attribute
set to False during the database upgrade step. This is because we expect no
end user created metadata on export locations yet.
Existing metadata tables will also be modified to replace the datatype of Existing metadata tables will also be modified to replace the datatype of
the "id" field to uuids. This will be done to avoid integer overflows and the "id" field to a string of length 36 for UUIDs. This will be done to
provide scalability. avoid integer overflows and provide scalability.
A general ORM schema for a metadata table will be as follows:: A general ORM schema for a metadata table will be as follows
(where 'Resource' is a stand-in for the real resource name)::
class ResourceMetadata(BASE, ManilaBase): class ResourceMetadata(BASE, ManilaBase):
"""Represents a metadata key/value pair for a resource.""" """Represents a metadata key/value pair for a Resource."""
__tablename__ = 'resource_metadata' __tablename__ = 'resource_metadata'
id = Column(String(36), primary_key=True) id = Column(String(36), primary_key=True)
key = Column(String(255), nullable=False) key = Column(String(255), nullable=False)
value = Column(String(1023), nullable=False) value = Column(String(1023), nullable=False)
resource_id = Column(String(36), ForeignKey('resources.id'), nullable=False) resource_id = Column(String(36), ForeignKey('resources.id'), nullable=False)
user_modifiable = Column(Boolean, default=True) deleted = Column(String(36), default='False')
resource = orm.relationship(Resource, backref="resource_metadata", resource = orm.relationship(Resource, backref="resource_metadata",
foreign_keys=resource_id, foreign_keys=resource_id,
primaryjoin='and_(' primaryjoin='and_('
'ResourceMetadata.resource_id == Resource.id,' 'ResourceMetadata.resource_id == Resource.id,'
'ResourceMetadata.deleted == 0)') 'ResourceMetadata.deleted == "False)')
Metadata items are not soft deleted when they are unset by the service or by Metadata items are not soft deleted when they are unset by the service or by
@ -161,12 +162,13 @@ resource has been requested.
REST API impact REST API impact
--------------- ---------------
New API endpoints will be created to get metadata, set metadata, unset a New API endpoints will be created to index metadata, show metadata item,
metadata item, delete all metadata for each resource. The general structure of create metadata, update metadata item, update_all metadata (delete
these APIs is as follows: all existing metadata and update with requested metadata), and delete metadata
item for each resource. The general structure of these APIs is as follows:
Get all Metadata Index Metadata
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
Retrieve all metadata key=value pairs as JSON:: Retrieve all metadata key=value pairs as JSON::
@ -186,8 +188,28 @@ Retrieve all metadata key=value pairs as JSON::
} }
} }
Set/Unset all metadata Show specific metadata item
^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
Retrieve a single metadata key=value pair::
GET /v2/{resource}/metadata/{key}
- *Sample request body*: null
- *Success Codes*: 200
- *Default API policy role*: Project Reader
- *Error Codes*: 401 (Unauthorized), 403 (Policy Not Authorized), 404
(Invalid resource)
- *Sample response body*::
{
"metadata": {
"project": "my_app",
}
}
Update all metadata
^^^^^^^^^^^^^^^^^^^
Replace all metadata with the updated set, can also be used to delete all Replace all metadata with the updated set, can also be used to delete all
metadata:: metadata::
@ -217,10 +239,14 @@ metadata::
} }
} }
Set specific metadata item/s *Note:* Metadata keys that are part of the admin-only dictionary will
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not be deleted and updated if the user requesting this is not an
system or project admin.
Update one or more specific metadata items, leaving the rest unmodified:: Update specific metadata item
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Update a specific metadata item, leaving the rest unmodified::
POST /v2/{resource}/metadata/{key} POST /v2/{resource}/metadata/{key}
@ -251,8 +277,7 @@ Update one or more specific metadata items, leaving the rest unmodified::
Currently, the ``POST /v2/{share}/metadata`` API currently expects a Currently, the ``POST /v2/{share}/metadata`` API currently expects a
``meta`` object. However, the other metadata APIs expect a ``metadata`` ``meta`` object. However, the other metadata APIs expect a ``metadata``
object. For the sake of consistency, this error will be fixed in a new object. For the sake of consistency, this error will be fixed in a new
API microversion. However, use of ``meta`` will be honored for this API even API microversion.
after the new microversion.
Delete specific metadata item Delete specific metadata item
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -302,12 +327,12 @@ Other end user impact
--------------------- ---------------------
Python-manilaclient SDK will include support for the new APIs and we'll Python-manilaclient SDK will include support for the new APIs and we'll
ensure that there are corresponding CLI commands in manilaclient shell as ensure that there are corresponding CLI commands in the new OSC plugin
well as the new OSC plugin shell. Manila UI's support for share, export shell. Manila UI's support for share, export location and access rule
location and access rule metadata is limited. This specification doesn't metadata is limited. This specification doesn't seek to address all the
seek to address all the UI gaps; but all effort will be made to close the UI gaps; but all effort will be made to close the feature parity between
feature parity between the CLI utilities and the UI. Eventually users will the CLI utilities and the UI. Eventually users will be able to perform
be able to perform all metadata interactions via the UI as well. all metadata interactions via the UI as well.
Performance Impact Performance Impact
@ -341,10 +366,10 @@ Assignee(s)
----------- -----------
Primary assignee: Primary assignee:
gouthamr ashrod98 <ashrod98@gmail.com>
Other contributors: Other contributors:
None gouthamr
Work Items Work Items
---------- ----------
@ -352,20 +377,33 @@ Work Items
- Add database migration to convert the ``id`` field of share, export - Add database migration to convert the ``id`` field of share, export
location and access rule metadata to a string from integer and populate location and access rule metadata to a string from integer and populate
the field with UUIDs the field with UUIDs
- Add database migrations to introduce the "user_modifiable" field to share, - Add database migrations to introduce the "deleted" field to share,
export location and access rule metadata tables. export location and access rule metadata tables.
- Add database migrations to create new metadata tables for all other resources - Add database migrations to create new metadata tables for all other resources
- Add ``MetadataControllerMixin``, inherit and extend in all resources and - Add ``MetadataControllerMixin``, inherit and extend in all resources and
bump up the API microversion. bump up the API microversion.
- Add unit and integration tests - Add unit and integration tests
- Add support for metadata APIs in manilaclient SDK, manilaclient shell and OSC - Add support for metadata APIs in manilaclient SDK, and OSC CLI and SDK
- Add support for metadata interactions in the UI - Add support for metadata interactions in the UI
- Add documentation - Add documentation
A further enhancement to this API would be to provide an interface
for administrators to create admin-only metadata values. Currently,
admin-only metadata values are delinated in a dictionary in
manila/manila/common/constants.py. Such a fixated list is sufficient for
the implementation of backend filtering for scheduling according to share
affinities. To implement this customization, we could revisit adjusting
the data-model to include an admin-only boolean value identifying
which keys are adjustable by admins or non-admins. Or we could provide
some configuration option in manila/manila/data/. Such an enhancement
requires further thought, and can be implemented at a later point.
Dependencies Dependencies
============ ============
None Not a direct dependency, but this API change incorporates the metadata
changes necessary to implement this spec: `Affinity and anti-affinity
scheduler filter <https://specs.openstack.org/openstack/manila-specs/specs/xena/affinity-antiaffinity-filter.html>`_
Testing Testing