From e13af0882e3f133c0c5a8737e61980d34e07f0d1 Mon Sep 17 00:00:00 2001 From: Goutham Pacha Ravi Date: Thu, 6 Aug 2020 14:23:56 -0700 Subject: [PATCH] Fix documentation for types and extra specs Some content in the admin docs was incomplete, and irrelevant to administrators. Also add notes on a couple of missing common capabilities: - mount_snapshot_support - availability_zones Partial-Bug: #1816486 Change-Id: Ie55fd5bf80ca2ebacdb652c701aa9c213f758963 Signed-off-by: Goutham Pacha Ravi (cherry picked from commit 36a7ec422bf2ff41aeeeff777c81ab83576ce207) (cherry picked from commit b780ef9ef36e7e4bce3fdba8895e445f8d7eb88b) --- .../admin/capabilities_and_extra_specs.rst | 409 ++++++++---------- .../group_capabilities_and_extra_specs.rst | 58 +-- doc/source/admin/index.rst | 4 +- .../admin/shared-file-systems-share-types.rst | 274 +++++++----- .../contributor/driver_requirements.rst | 125 +++++- 5 files changed, 483 insertions(+), 387 deletions(-) diff --git a/doc/source/admin/capabilities_and_extra_specs.rst b/doc/source/admin/capabilities_and_extra_specs.rst index b17a3c1238..85ecbc15bf 100644 --- a/doc/source/admin/capabilities_and_extra_specs.rst +++ b/doc/source/admin/capabilities_and_extra_specs.rst @@ -2,44 +2,96 @@ Capabilities and Extra-Specs ============================ -Manila Administrators create share types with extra-specs to allow users -to request a type of share to create. The Administrator chooses a name -for the share type and decides how to communicate the significance of -the different share types in terms that the users should understand or -need to know. By design, most of the details of a share type (the extra- -specs) are not exposed to users -- only Administrators. +Cloud Administrators create :ref:`shared_file_systems_share_types` with +extra-specs to: -Share Types ------------ -Refer to the manila client command-line help for information on how to -create a share type and set "extra-spec" key/value pairs for a share type. +- influence the scheduler's decision to place new shares, and +- instruct the Shared File System service or its storage driver/s to perform + certain special actions with respect to the users' shares. -Extra-Specs ------------ -There are 3 types of extra-specs: required, scoped, and un-scoped. +As an administrator, you can choose a descriptive name or provide good +descriptions for your share types to convey the share type capabilities to +end users. End users can view standard ``tenant-visible`` extra-specs that +can let them seek required behavior and automate their applications +accordingly. By design, however, all other extra-specs of a share type are not +exposed to non-privileged users. -Manila *requires* the driver_handles_share_servers extra-spec. +Types of Extra-Specs +-------------------- -*Scoped* extra-specs use a prefix followed by a colon to define a namespace -for scoping the extra-spec. A prefix could be a vendor name or acronym -and is a hint that this extra-spec key/value only applies to that vendor's -driver. Scoped extra-specs are not used by the scheduler to determine -where a share is created (except for the special `capabilities` prefix). -It is up to each driver implementation to determine how to use scoped -extra-specs and to document them. +The Shared File Systems service back-end storage drivers offer a wide range of +capabilities. The variation in these capabilities allows cloud +administrators to provide a storage service catalog to their end users. +Share type extra-specs tie-in with these capabilities. -The prefix "capabilities" is a special prefix to indicate extra-specs that -are treated like un-scoped extra-specs. In the CapabilitiesFilter the -"capabilities:" is stripped from the key and then the extra-spec key and -value are used as an un-scoped extra-spec. +Some back-end capabilities are very specific to a storage system, and are +opaque to the Shared File System service or the end users. These +capabilities are invoked with the help of "scoped" extra-specs. Using scoped +extra-specs is a way to provide programmatic directives to the concerned +storage driver to do something during share creation or share manipulation. +You can learn about the opaque capabilities through driver documentation +and configure these capabilities within share types as scoped +extra-specs (e.g.: hpe3par:nfs_options). The Shared File System service +scheduler ignores scoped extra-specs during its quest to find the right back +end to provision shares. -*Un-scoped* extra-specs have a key that either starts with "capabilities:" or -does not contain a colon. When the CapabilitiesFilter is enabled (it is -enabled by default), the scheduler will only create a share on a backend -that reports capabilities that match the share type's un-scoped extra-spec -keys. +There are some back-end capabilities in manila that do matter to the scheduler. +For our understanding, lets call these non-scoped or non-opaque capabilities. +All non-scoped capabilities can be directly used as share types extra-specs. +They are considered by the scheduler’s capabilities filter (and any custom +filter defined by deployers). -The CapabilitiesFilter uses the following for matching operators: +You can get a list of non-scoped capabilities from the scheduler by using: + +.. code-block:: console + + $ manila pool-list --detail + +The non-scoped capabilities can be of three types: + +- **Capabilities pertaining to a specific back end storage system driver**: For + example, *huawei_smartcache*. + No Shared File System service API relies on non-opaque back end specific + capabilities. +- **Common capabilities that are not visible to end users**: The manila + community has standardized some cross-platform capabilities like + *thin_provisioning*, *dedupe*, *compression*, *qos*, *ipv6_support* and + *ipv4_support*. Values of these options do not matter to any Shared File + System service APIs; however, they can signify something to the manila + services themselves. For example when a back end supports thin_provisioning, + the scheduler service performs over-provisioning, and if a back end does + not report *ipv6_support* as True, the share-manager service drops IPv6 + access rules before invoking the storage driver to update access rules. +- **Common capabilities that are visible to end users**: Some capabilities + affect functionality exposed via the Shared File System service API. For + example, not all back ends support snapshots, and even if they do, they + may not support all of the snapshot operations. For example, cloning + snapshots into new shares, reverting shares in-place to snapshots, etc. + + The support for these capabilities determines whether users would be able + to perform certain control-plane operations with manila. For example, a back + end driver may report *snapshot_support=True* allowing end users to + create share snapshots, however, the driver can report + *create_share_from_snapshot_support=False*. + This reporting allows cloud administrators to create share types that + support snapshots but not creating shares from snapshots. When a user uses + such a share type, they will not be able to clone snapshots into new shares. + Tenant-visible capabilities aid manila in validating requests and failing + fast on requests it cannot accommodate. They also help level set the user + expectations on some failures. For example, if snapshot_support is set to + False on the share type, since users can see this, they will not invoke + the create snapshot API, and even if they do, they will understand the + HTTP 400 (and error message) in better context. + +.. important:: + + All extra-specs are optional, except one: *driver_handles_share_servers*. + +Scheduler's treatment of non-scoped extra specs +----------------------------------------------- + +The CapabilitiesFilter in the Shared File System scheduler uses the following +for matching operators: * No operator This defaults to doing a python ==. Additionally it will match boolean values. @@ -74,63 +126,107 @@ The CapabilitiesFilter uses the following for matching operators: * **s==, s!=, s>=, s>, s<=, s<** - The "s" indicates it is a string comparison. These choose a host that satisfies - the comparison of strings in capability and specification. For example, - if "capabilities:replication_type s== dr", a host that reports - replication_type of "dr" will be chosen. + The "s" indicates it is a string comparison. These choose a host that + satisfies the comparison of strings in capability and specification. For + example, if "capabilities:replication_type s== dr", a host that reports + replication_type of "dr" will be chosen. If "share_backend_name s!= + cephfs" is used, any host not named "cephfs" can be chosen. -For vendor-specific capabilities (which need to be visible to the -CapabilityFilter), it is recommended to use the vendor prefix followed -by an underscore. This is not a strict requirement, but will provide a +For vendor-specific non-scoped capabilities (which need to be visible to the +scheduler), drivers are recommended to use the vendor prefix followed +by an underscore. This is not a strict requirement, but can provide a consistent look along-side the scoped extra-specs and will be a clear indicator of vendor capabilities vs. common capabilities. Common Capabilities ------------------- -For capabilities that apply to multiple backends a common capability can -be created. Like all other backend reported capabilities, these capabilities +Common capabilities apply to multiple backends. +Like all other backend reported capabilities, these capabilities can be used verbatim as extra_specs in share types used to create shares. -* `driver_handles_share_servers` is a special, required, user-visible common - capability. Added in Kilo. +Share type common capability extra-specs that are visible to end users: +----------------------------------------------------------------------- -* `dedupe` - indicates that a backend/pool can provide shares using some +* **driver_handles_share_servers** is a special, required common + capability. When set to True, the scheduler matches requests with back ends + that can isolate user workloads with dedicated share servers exporting + shares on user provided share networks. + +* **snapshot_support** indicates whether snapshots are supported for shares + created on the pool/backend. When administrators do not set this capability + as an extra-spec in a share type, the scheduler can place new shares of that + type in pools without regard for whether snapshots are supported, and those + shares will not support snapshots. + +* **create_share_from_snapshot_support** indicates whether a backend can + create a new share from a snapshot. When administrators do not set this + capability as an extra-spec in a share type, the scheduler can place new + shares of that type in pools without regard for whether creating shares + from snapshots is supported, and those shares will not support creating + shares from snapshots. + +* **revert_to_snapshot_support** indicates that a driver is capable of + reverting a share in place to its most recent snapshot. When administrators + do not set this capability as an extra-spec in a share type, the scheduler + can place new shares of that type in pools without regard for whether + reverting shares to snapshots is supported, and those shares will not support + reverting shares to snapshots. + +* **mount_snapshot_support** indicates that a driver is capable of exporting + share snapshots for mounting. Users can provide and revoke access to + mountable snapshots just like they can with their shares. + +* **replication_type** indicates the style of replication supported for the + backend/pool. This extra_spec will have a string value and could be one + of :term:`writable`, :term:`readable` or :term:`dr`. `writable` replication + type involves synchronously replicated shares where all replicas are + writable. Promotion is not supported and not needed. `readable` and `dr` + replication types involve a single `active` or `primary` replica and one or + more `non-active` or secondary replicas per share. In `readable` type of + replication, `non-active` replicas have one or more export_locations and + can thus be mounted and read while the `active` replica is the only one + that can be written into. In `dr` style of replication, only + the `active` replica can be mounted, read from and written into. + +* **availability_zones** indicates a comma separated list of availability + zones that can be used for provisioning. Users can always provide a specific + availability zone during share creation, and they will receive a + synchronous failure message if they attempt to create a share in an + availability zone that the share type does not permit. If you do not set + this extra-spec, the share type is assumed to be serviceable in all + availability zones known to the Shared File Systems service. + +Share type common capability extra-specs that are not visible to end users: +--------------------------------------------------------------------------- + +* **dedupe** indicates that a backend/pool can provide shares using some deduplication technology. The default value of the dedupe capability (if a - driver doesn't report it) is False. In Liberty, drivers cannot report to the - scheduler that they support both dedupe and non-deduped share. For each pool - it's either always on or always off, even if the drivers can technically - support both dedupe and non-deduped in a pool. Since Mitaka, the logic is - changed to allow a driver to report dedupe=[True, False] if it can support - both dedupe and non-deduped in a pool. Administrators can make a share type - use deduplication by setting this extra-spec to ' True'. Administrators - can prevent a share type from using deduplication by setting this extra-spec - to ' False'. Added in Liberty. + driver doesn't report it) is False. Drivers can support both dedupe and + non-deduped shares in a single storage pool by reporting ``dedupe=[True, + False]``. You can make a share type use deduplication by setting this + extra-spec to ' True', or prevent it by setting this extra-spec + to ' False'. -* `compression` - indicates that a backend/pool can provide shares using some +* **compression** indicates that a backend/pool can provide shares using some compression technology. The default value of the compression capability (if a - driver doesn't report it) is False. In Liberty, drivers cannot report to the - scheduler that they support both compression and non-compression. For each - pool it's either always on or always off, even if the drivers can technically - support both compression and non-compression in a pool. Since Mitaka, the - logic is changed to allow a driver to report compression=[True, False] if it - can support both compression and non-compression in a pool. Administrators - can make a share type use compression by setting this extra-spec to - ' True'. Administrators can prevent a share type from using compression - by setting this extra-spec to ' False'. Added in Liberty. + driver doesn't report it) is False. Drivers can support compressed and + non-compressed shares in a single storage pool by reporting + ``compression=[True, False]``. You can make a share type use compression + by setting this extra-spec to ' True', or prevent it by setting this + extra-spec to ' False'. -* `thin_provisioning` - shares will not be space guaranteed and - overprovisioning will be enabled. This capability defaults to False. - Backends/pools that support thin provisioning must report True for this - capability. Administrators can make a share type use thin provisioned shares - by setting this extra-spec to ' True'. If a driver reports - thin_provisioning=False (the default) then it's assumed that the driver is - doing thick provisioning and overprovisioning is turned off. - This was added in Liberty. In Liberty and Mitaka, the driver was required - to configure one pool for thin and another pool for thick and report - thin_provisioning as either True or False even if an array can technically - support both thin and thick provisioning in a pool. In Newton, the logic is - changed to allow a driver to report thin_provisioning=[True, False] if it - can support both thin and thick provisioning in a pool. To provision a thick +* **thin_provisioning** can be enabled where shares will not be + guaranteed space allocations and overprovisioning will be enabled. This + capability defaults to False. Back ends/pools that support thin + provisioning report True for this capability. Administrators can make a + share type use thin provisioned shares by setting this extra-spec + to ' True'. If a driver reports thin_provisioning=False (the default) + then it's assumed that the driver is doing thick provisioning and + overprovisioning is turned off. A driver can support thin provisioned + and thick provisioned shares in the same pool by reporting + ``thin_provisioning=[True, False]``. + + To provision a thick share on a back end that supports both thin and thick provisioning, set one of the following in extra specs: @@ -141,154 +237,23 @@ can be used verbatim as extra_specs in share types used to create shares. {'capabilities:thin_provisioning': 'False'} {'capabilities:thin_provisioning': ' False'} -* `qos` - indicates that a backend/pool can provide shares using some +* **qos** indicates that a backend/pool can provide shares using some QoS (Quality of Service) specification. The default value of the qos - capability (if a driver doesn't report it) is False. Administrators - can make a share type use QoS by setting this extra-spec to ' True' and - also setting the relevant QoS-related extra specs for the drivers being used. + capability (if a driver doesn't report it) is False. You can make a share + type use QoS by setting this extra-spec to ' True' and also setting + the relevant QoS-related extra specs for the drivers being used. Administrators can prevent a share type from using QoS by setting this - extra-spec to ' False'. Different drivers have different ways of specifying - QoS limits (or guarantees) and this extra spec merely allows the scheduler to - filter by pools that either have or don't have QoS support enabled. Added in - Mitaka. + extra-spec to ' False'. Different drivers have different ways of + specifying QoS limits (or guarantees) and this extra spec merely allows + the scheduler to filter by pools that either have or don't have QoS + support enabled. -* `replication_type` - indicates the style of replication supported for the - backend/pool. This extra_spec will have a string value and could be one - of :term:`writable`, :term:`readable` or :term:`dr`. `writable` replication - type involves synchronously replicated shares where all replicas are - writable. Promotion is not supported and not needed. `readable` and `dr` - replication types involve a single `active` or `primary` replica and one or - more `non-active` or secondary replicas per share. In `readable` type of - replication, `non-active` replicas have one or more export_locations and - can thus be mounted and read while the `active` replica is the only one - that can be written into. In `dr` style of replication, only - the `active` replica can be mounted, read from and written into. Added in - Mitaka. +* **ipv4_support** indicates whether a back end can create a share that + can be accessed via IPv4 protocol. If administrators do not set this + capability as an extra-spec in a share type, the scheduler can place new + shares of that type in pools without regard for whether IPv4 is supported. -* `snapshot_support` - indicates whether snapshots are supported for shares - created on the pool/backend. When administrators do not set this capability - as an extra-spec in a share type, the scheduler can place new shares of that - type in pools without regard for whether snapshots are supported, and those - shares will not support snapshots. - -* `create_share_from_snapshot_support` - indicates whether a backend can create - a new share from a snapshot. When administrators do not set this capability - as an extra-spec in a share type, the scheduler can place new shares of that - type in pools without regard for whether creating shares from snapshots is - supported, and those shares will not support creating shares from snapshots. - -* `revert_to_snapshot_support` - indicates that a driver is capable of - reverting a share in place to its most recent snapshot. When administrators - do not set this capability as an extra-spec in a share type, the scheduler - can place new shares of that type in pools without regard for whether - reverting shares to snapshots is supported, and those shares will not support - reverting shares to snapshots. - -* `ipv4_support` - indicates whether a back end can create a share that can be - accessed via IPv4 protocol. If administrators do not set this capability - as an extra-spec in a share type, the scheduler can place new shares of that - type in pools without regard for whether IPv4 is supported. - -* `ipv6_support` - indicates whether a back end can create a share that can be - accessed via IPv6 protocol. If administrators do not set this capability - as an extra-spec in a share type, the scheduler can place new shares of that - type in pools without regard for whether IPv6 is supported. - -Reporting Capabilities ----------------------- -Drivers report capabilities as part of the updated stats (e.g. capacity) -for their backend/pools. This is how a backend/pool advertizes its ability -to provide a share that matches the capabilities requested in the share -type extra-specs. - -Developer impact ----------------- - -Developers should update their drivers to include all backend and pool -capacities and capabilities in the share stats it reports to scheduler. -Below is an example having multiple pools. "my" is used as an -example vendor prefix: - -:: - - { - 'driver_handles_share_servers': 'False', #\ - 'share_backend_name': 'My Backend', # backend level - 'vendor_name': 'MY', # mandatory/fixed - 'driver_version': '1.0', # stats & capabilities - 'storage_protocol': 'NFS_CIFS', #/ - #\ - 'my_capability_1': 'custom_val', # "my" optional vendor - 'my_capability_2': True, # stats & capabilities - #/ - 'pools': [ - {'pool_name': - 'thin-dedupe-compression pool', #\ - 'total_capacity_gb': 500, # mandatory stats for - 'free_capacity_gb': 230, # pools - 'reserved_percentage': 0, #/ - #\ - 'dedupe': True, # common capabilities - 'compression': True, # - 'snapshot_support': True, # - 'create_share_from_snapshot_support': True, - 'revert_to_snapshot_support': True, - 'qos': True, # this backend supports QoS - 'thin_provisioning': True, # - 'max_over_subscription_ratio': 10, # (mandatory for thin) - 'provisioned_capacity_gb': 270, # (mandatory for thin) - # - # - 'replication_type': 'dr', # this backend supports - # replication_type 'dr' - #/ - 'my_dying_disks': 100, #\ - 'my_super_hero_1': 'Hulk', # "my" optional vendor - 'my_super_hero_2': 'Spider-Man', # stats & capabilities - #/ - #\ - # can replicate to other - 'replication_domain': 'asgard', # backends in - # replication_domain 'asgard' - #/ - 'ipv4_support': True, - 'ipv6_support': True, - - }, - {'pool_name': 'thick pool', - 'total_capacity_gb': 1024, - 'free_capacity_gb': 1024, - 'qos': False, - 'snapshot_support': True, - 'create_share_from_snapshot_support': False, # this pool does not - # allow creating - # shares from - # snapshots - 'revert_to_snapshot_support': True, - 'reserved_percentage': 0, - 'dedupe': False, - 'compression': False, - 'thin_provisioning': False, - 'replication_type': None, - 'my_dying_disks': 200, - 'my_super_hero_1': 'Batman', - 'my_super_hero_2': 'Robin', - 'ipv4_support': True, - 'ipv6_support': True, - }, - ] - } - -Work Flow ---------- - -1) Share Backends report how many pools and what those pools look like and - are capable of to scheduler; - -2) When request comes in, scheduler picks a pool that fits the need best to - serve the request, it passes the request to the backend where the target - pool resides; - -3) Share driver gets the message and lets the target pool serve the request - as scheduler instructed. Share type extra-specs (scoped and un-scoped) - are available for the driver implementation to use as-needed. +* **ipv6_support** - indicates whether a back end can create a share that + can be accessed via IPv6 protocol. If administrators do not set this + capability as an extra-spec in a share type, the scheduler can place new + shares of that type in pools without regard for whether IPv6 is supported. diff --git a/doc/source/admin/group_capabilities_and_extra_specs.rst b/doc/source/admin/group_capabilities_and_extra_specs.rst index 888b98ef4a..cc1b306d59 100644 --- a/doc/source/admin/group_capabilities_and_extra_specs.rst +++ b/doc/source/admin/group_capabilities_and_extra_specs.rst @@ -2,9 +2,8 @@ Group Capabilities and group-specs ================================== -Manila Administrators create share group types with `share types -`_ and group-specs to allow users +Manila Administrators create share group types with +:ref:`shared_file_systems_share_types` and group-specs to allow users to request a group type of share group to create. The Administrator chooses a name for the share group type and decides how to communicate the significance of the different share group types in terms that the users should understand or @@ -40,56 +39,3 @@ create share groups. The default value of the consistent_snapshot_support capability (if a driver doesn't report it) is None. Administrators can make a share group type use consistent snapshot support by setting this group-spec to 'host'. - -Reporting Group Capabilities ----------------------------- -Drivers report group capabilities as part of the updated stats (e.g. capacity) -and filled in 'share_group_stats' node for their back end. This is how a backend -advertizes its ability to provide a share that matches the group capabilities -requested in the share group type group-specs. - -Developer impact ----------------- - -Developers should update their drivers to include all backend and pool -capacities and capabilities in the share stats it reports to scheduler. -Below is an example having multiple pools. "my" is used as an -example vendor prefix: - -:: - - { - 'driver_handles_share_servers': 'False', #\ - 'share_backend_name': 'My Backend', # backend level - 'vendor_name': 'MY', # mandatory/fixed - 'driver_version': '1.0', # stats & capabilities - 'storage_protocol': 'NFS_CIFS', #/ - #\ - 'my_capability_1': 'custom_val', # "my" optional vendor - 'my_capability_2': True, # stats & capabilities - #/ - 'share_group_stats': { - #\ - 'my_group_capability_1': 'custom_val', # "my" optional vendor - 'my_group_capability_2': True, # stats & group capabilities - #/ - 'consistent_snapshot_support': 'host', #\ - # common group capabilities - #/ - }, - ] - } - -Work Flow ---------- - -1) Share Backends report how many pools and what those pools look like and - are capable of to group scheduler; - -2) When request comes in, scheduler picks a backend that fits the need best to - serve the request, it passes the request to the backend where the target - pool resides; - -3) Share driver gets the message and lets the target pool serve the request - as group scheduler instructed. Share group type group-specs (scoped and un-scoped) - are available for the driver implementation to use as-needed. diff --git a/doc/source/admin/index.rst b/doc/source/admin/index.rst index 8d1414059d..3aad82acb9 100644 --- a/doc/source/admin/index.rst +++ b/doc/source/admin/index.rst @@ -45,11 +45,11 @@ deployment. shared-file-systems-key-concepts.rst shared-file-systems-share-management.rst - shared-file-systems-share-server-management.rst - shared-file-systems-share-migration.rst shared-file-systems-share-types.rst shared-file-systems-snapshots.rst + shared-file-systems-share-server-management.rst shared-file-systems-security-services.rst + shared-file-systems-share-migration.rst shared-file-systems-share-replication.rst shared-file-systems-multi-backend.rst shared-file-systems-networking.rst diff --git a/doc/source/admin/shared-file-systems-share-types.rst b/doc/source/admin/shared-file-systems-share-types.rst index 2a55e8d6c3..76d3d20ef1 100644 --- a/doc/source/admin/shared-file-systems-share-types.rst +++ b/doc/source/admin/shared-file-systems-share-types.rst @@ -4,136 +4,218 @@ Share types =========== -A share type enables you to filter or choose back ends before you create a -share and to set data for the share driver. A share type behaves in the same -way as a Block Storage volume type behaves. +The Shared File System service back-end storage drivers offer a wide range +of capabilities. The variation in these capabilities allows cloud +administrators to provide a storage service catalog to their end users. +Share types can be used to create this storage service catalog. +Cloud administrators can influence provisioning of users' shares with the +help of Share types. All shares are associated with a share type. Share +types are akin to ``flavors`` in the OpenStack Compute service (nova), or +``volume types`` in the OpenStack Block Storage service (cinder), or ``storage +classes`` in Kubernetes. You can allow a share type to be accessible to all +users in your cloud if you wish. You can also create private share types that +allow only users belonging to certain OpenStack projects to access them. +You can have an unlimited number of share types in your +cloud, but for practical purposes, you may want to create only a handful of +publicly accessible share types. -In the Shared File Systems configuration file ``manila.conf``, the -administrator can set the share type used by default for the share creation -and then create a default share type. +Each share type is an object that encompasses ``extra-specs`` (extra +specifications). These extra-specs can map to storage back-end capabilities, +or can be directives to the service. -To create a share type, use :command:`manila type-create` command as: +Consider for example, offering three share types in your cloud to map +to "service levels": + ++--------+--------------------------------------------------------------------------------------------------+ +| Type | Capabilities/Instructions | ++========+==================================================================================================+ +| Gold | Allow creating snapshots, reverting to snapshots and share replication, "thick" provision shares | ++--------+--------------------------------------------------------------------------------------------------+ +| Silver | Allow creating snapshots, "thin" provision shares | ++--------+--------------------------------------------------------------------------------------------------+ +| Bronze | Don't allow creating snapshots, "thin" provision shares | ++--------+--------------------------------------------------------------------------------------------------+ + +Capabilities or instructions such as the ones above are coded as extra-specs +that your users and the Shared File System service understand. Users in +OpenStack projects can see all public share types along with private share +types that are made accessible to them. Not all extra-specs that you +configure in a share type are visible to your users. This design helps +preserve the cloud abstraction. Along with the share type names, they can +see the share type descriptions and "tenant-visible" extra-specs. + +For more details on extra-specs, see :ref:`capabilities_and_extra_specs`. + +The Shared File Systems service also allows using quota controls with share +types. Quotas can help you maintain your SLAs by limiting the number of +consumable resources or aid in billing. See :ref:`shared_file_systems_quotas` +for more details. + +Driver Handles Share Servers (DHSS) +----------------------------------- + +To provide secure and hard multi-tenancy on the network data path, the +Shared File Systems service allows users to use their own "share networks". +When shares are created on a share network, users can be sure they have +their own isolated "share servers" that export their shares on the share +network that have the ability plug into user-determined authentication +domains ("security services"). Not all Shared File System service storage +drivers support share networks. Those that do assert the capability +``driver_handles_share_servers=True``. + +When creating a share type, you are *required* to set an extra-spec that +matches this capability. It is visible to end users. + +Default Share Type +------------------ + +When you are operating a cloud where all your tenants are trusted, you may +want to create a "default" share type that applies to all of them. It +simplifies share creation for your end users since they don't need to worry +about share types. + +Use of a default share type is not recommended in a multi-tenant cloud where +you may want to separate your user workloads, or offer different service +capabilities. In such instances, you must always encourage your users to +specify a share type at share creation time, and not rely on the default +share type. + +.. important:: + + If you do not create and configure a default share type, users *must* + specify a valid share type during share creation, or share creation + requests will fail. + +To configure the default share type, edit the ``manila.conf`` file, and set +the configuration option [DEFAULT]/default_share_type. + +You must then create a share type, using :command:`manila type-create`: .. code-block:: console - manila type-create [--snapshot_support ] - [--is_public ] + manila type-create [--is_public ] + [--description ] + [--extra-specs ] -where the ``name`` is the share type name, ``--is_public`` defines the level of -the visibility for the share type, ``snapshot_support`` and -``spec_driver_handles_share_servers`` are the extra specifications used to -filter back ends. Administrators can create share types with these extra -specifications for the back ends filtering: +where: -- ``driver_handles_share_servers``. Required. Defines the driver mode for share - server lifecycle management. Valid values are ``true``/``1`` and - ``false``/``0``. - Set to True when the share driver can manage, or handle, the share server - lifecycle. - Set to False when an administrator, rather than a share driver, manages - the bare metal storage with some net interface instead of the presence - of the share servers. - -- ``snapshot_support``. Filters back ends by whether they do or do not support - share snapshots. Default is ``True``. - Set to True to find back ends that support share snapshots. - Set to False to find back ends that do not support share snapshots. - -.. note:: - - The extra specifications set in the share types are operated in the - :ref:`shared_file_systems_scheduling`. - -Administrators can also set additional extra specifications for a share type -for the following purposes: - -- *Filter back ends*. Unqualified extra specifications written in - this format: ``extra_spec=value``. For example, **netapp_raid_type=raid4**. - -- *Set data for the driver*. Qualified extra specifications always written - with the prefix with a colon, except for the special ``capabilities`` - prefix, in this format: ``vendor:extra_spec=value``. For example, - **netapp:thin_provisioned=true**. - -The scheduler uses the special capabilities prefix for filtering. The scheduler -can only create a share on a back end that reports capabilities matching the -un-scoped extra-spec keys for the share type. For details, see `Capabilities -and Extra-Specs `_. - -Each driver implementation determines which extra specification keys it uses. -For details, see the documentation for the driver. - -An administrator can use the ``policy.json`` file to grant permissions for -share type creation with extra specifications to other roles. - -You set a share type to private or public and -:ref:`manage the access` to the private share types. By -default a share type is created as publicly accessible. Set -``--is_public`` to ``False`` to make the share type private. +- ``name`` is the share type name +- ``is_public`` defines the visibility for the share type (true/false) +- ``description`` is a free form text field to describe the characteristics + of the share type for your users' benefit +- ``extra-specs`` defines a comma separated set of key=value pairs of + optional extra specifications +- ``spec_driver_handles_share_servers`` is the mandatory extra-spec + (true/false) Share type operations --------------------- To create a new share type you need to specify the name of the new share type. You also require an extra spec ``driver_handles_share_servers``. -The new share type can also be public. +The new share type can be public or private. .. code-block:: console - $ manila type-create netapp1 False --is_public True + $ manila manila type-create default-shares False \ + --description "Default share type for the cloud, no fancy capabilities" $ manila type-list - +-----+--------+-----------+-----------+-----------------------------------+-----------------------+ - | ID | Name | Visibility| is_default| required_extra_specs | optional_extra_specs | - +-----+--------+-----------+-----------+-----------------------------------+-----------------------+ - | c0..| netapp1| public | - | driver_handles_share_servers:False| snapshot_support:True | - +-----+--------+-----------+-----------+-----------------------------------+-----------------------+ + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+-------------------------------------------+---------------------------------------------------------+ + | ID | Name | visibility | is_default | required_extra_specs | optional_extra_specs | Description | + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+-------------------------------------------+---------------------------------------------------------+ + | cf1f92ec-4d0a-4b79-8f18-6bb82c22840a | default-shares | public | - | driver_handles_share_servers : False | | Default share type for the cloud, no fancy capabilities | + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+-------------------------------------------+---------------------------------------------------------+ + + $ manila type-show default-shares + +----------------------+---------------------------------------------------------+ + | Property | Value | + +----------------------+---------------------------------------------------------+ + | id | cf1f92ec-4d0a-4b79-8f18-6bb82c22840a | + | name | default-shares | + | visibility | public | + | is_default | NO | + | description | Default share type for the cloud, no fancy capabilities | + | required_extra_specs | driver_handles_share_servers : False | + | optional_extra_specs | | + +----------------------+---------------------------------------------------------+ + +You did not provide optional capabilities, so they are all *assumed to be off +by default*. So, Non-privileged users see some tenant-visible capabilities +explicitly. + +.. code-block:: console + + + $ source demorc + $ manila type-list + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+--------------------------------------------+---------------------------------------------------------+ + | ID | Name | visibility | is_default | required_extra_specs | optional_extra_specs | Description | + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+--------------------------------------------+---------------------------------------------------------+ + | cf1f92ec-4d0a-4b79-8f18-6bb82c22840a | default-shares | public | - | driver_handles_share_servers : False | snapshot_support : False | Default share type for the cloud, no fancy capabilities | + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+--------------------------------------------+---------------------------------------------------------+ + + $ manila type-show default-shares + +----------------------+---------------------------------------------------------+ + | Property | Value | + +----------------------+---------------------------------------------------------+ + | id | cf1f92ec-4d0a-4b79-8f18-6bb82c22840a | + | name | default-shares | + | visibility | public | + | is_default | NO | + | description | Default share type for the cloud, no fancy capabilities | + | required_extra_specs | driver_handles_share_servers : False | + | optional_extra_specs | snapshot_support : False | + | | create_share_from_snapshot_support : False | + | | revert_to_snapshot_support : False | + | | mount_snapshot_support : False | + +----------------------+---------------------------------------------------------+ + You can set or unset extra specifications for a share type -using **manila type-key set ** command. Since it is up -to each driver what extra specification keys it uses, see the documentation -for the specified driver. +using **manila type-key set ** command. .. code-block:: console - $ manila type-key netapp1 set thin_provisioned=True + $ manila type-key default-shares set snapshot_support=True -It is also possible to view a list of current share types and extra -specifications: - -.. code-block:: console - - $ manila extra-specs-list - +-------------+---------+-------------------------------------+ - | ID | Name | all_extra_specs | - +-------------+---------+-------------------------------------+ - | c0086582-...| netapp1 | snapshot_support : True | - | | | thin_provisioned : True | - | | | driver_handles_share_servers : True | - +-------------+---------+-------------------------------------+ + $ manila type-show default-shares + +----------------------+---------------------------------------------------------+ + | Property | Value | + +----------------------+---------------------------------------------------------+ + | id | cf1f92ec-4d0a-4b79-8f18-6bb82c22840a | + | name | default-shares | + | visibility | public | + | is_default | NO | + | description | Default share type for the cloud, no fancy capabilities | + | required_extra_specs | driver_handles_share_servers : False | + | optional_extra_specs | snapshot_support : True | + +----------------------+---------------------------------------------------------+ Use :command:`manila type-key unset ` to unset an extra specification. -The public or private share type can be deleted with the -:command:`manila type-delete ` command. +A share type can be deleted with the :command:`manila type-delete +` command. However, a share type can only be deleted if there +are no shares, share groups or share group types associated with the share +type. .. _share_type_access: -Share type access ------------------ +Share type access control +------------------------- -You can manage access to a private share type for different projects. -Administrators can provide access, remove access, and retrieve -information about access for a specified private share. +You can provide access, revoke access, and retrieve list of allowed projects +for a specified private share. Create a private type: .. code-block:: console - $ manila type-create my_type1 True --is_public False + $ manila type-create my_type1 True \ + --is_public False \ + --extra-specs snapshot_support=True +----------------------+--------------------------------------+ | Property | Value | +----------------------+--------------------------------------+ @@ -148,8 +230,7 @@ Create a private type: .. note:: If you run :command:`manila type-list` only public share types appear. - To see private share types, run :command:`manila type-list` with - ``--all`` optional argument. + To see private share types, run :command:`manila type-list --all``. Grant access to created private type for a demo and alt_demo projects by providing their IDs: @@ -171,9 +252,8 @@ To view information about access for a private share, type ``my_type1``: | e4970f57f1824faab2701db61ee7efdf | +----------------------------------+ -After granting access to the share, the target project -can see the share type in the list, and create private -shares. +After granting access to the share, the users in the allowed projects +can see the share type and use it to create shares. To deny access for a specified project, use :command:`manila type-access-remove ` command. diff --git a/doc/source/contributor/driver_requirements.rst b/doc/source/contributor/driver_requirements.rst index 5afd7777db..c5137fa275 100644 --- a/doc/source/contributor/driver_requirements.rst +++ b/doc/source/contributor/driver_requirements.rst @@ -14,8 +14,8 @@ License for the specific language governing permissions and limitations under the License. -Manila minimum requirements and features since Mitaka -===================================================== +Manila minimum requirements and features +======================================== In order for a driver to be accepted into manila code base, there are certain minimum requirements and features that must be met, in order to ensure @@ -74,8 +74,11 @@ Capabilities ------------ In order for manila to function accordingly to the driver being used, the -driver must provide a set of information to manila, known as capabilities, as -follows: +driver must provide a set of information to manila, known as capabilities. +Share driver can use Share type extra-specs (scoped and un-scoped) to serve +new shares. See :doc:`../admin/capabilities_and_extra_specs` for more +information. At a minimum your driver must report: + - share_backend_name: a name for the backend; - driver_handles_share_servers: driver mode, whether this driver instance @@ -102,7 +105,79 @@ function correctly in manila, such as: - replication_type: string specifying the type of replication supported by the driver. Can be one of ('readable', 'writable' or 'dr'). -.. note:: for more information please see https://docs.openstack.org/manila/latest/admin/capabilities_and_extra_specs.html +Below is an example of drivers with multiple pools. "my" is used as an +example vendor prefix: + +:: + + { + 'driver_handles_share_servers': 'False', #\ + 'share_backend_name': 'My Backend', # backend level + 'vendor_name': 'MY', # mandatory/fixed + 'driver_version': '1.0', # stats & capabilities + 'storage_protocol': 'NFS_CIFS', #/ + #\ + 'my_capability_1': 'custom_val', # "my" optional vendor + 'my_capability_2': True, # stats & capabilities + #/ + 'pools': [ + {'pool_name': + 'thin-dedupe-compression pool', #\ + 'total_capacity_gb': 500, # mandatory stats for + 'free_capacity_gb': 230, # pools + 'reserved_percentage': 0, #/ + #\ + 'dedupe': True, # common capabilities + 'compression': True, # + 'snapshot_support': True, # + 'create_share_from_snapshot_support': True, + 'revert_to_snapshot_support': True, + 'qos': True, # this backend supports QoS + 'thin_provisioning': True, # + 'max_over_subscription_ratio': 10, # (mandatory for thin) + 'provisioned_capacity_gb': 270, # (mandatory for thin) + # + # + 'replication_type': 'dr', # this backend supports + # replication_type 'dr' + #/ + 'my_dying_disks': 100, #\ + 'my_super_hero_1': 'Hulk', # "my" optional vendor + 'my_super_hero_2': 'Spider-Man', # stats & capabilities + #/ + #\ + # can replicate to other + 'replication_domain': 'asgard', # backends in + # replication_domain 'asgard' + #/ + 'ipv4_support': True, + 'ipv6_support': True, + + }, + {'pool_name': 'thick pool', + 'total_capacity_gb': 1024, + 'free_capacity_gb': 1024, + 'qos': False, + 'snapshot_support': True, + 'create_share_from_snapshot_support': False, # this pool does not + # allow creating + # shares from + # snapshots + 'revert_to_snapshot_support': True, + 'reserved_percentage': 0, + 'dedupe': False, + 'compression': False, + 'thin_provisioning': False, + 'replication_type': None, + 'my_dying_disks': 200, + 'my_super_hero_1': 'Batman', + 'my_super_hero_2': 'Robin', + 'ipv4_support': True, + 'ipv6_support': True, + }, + ] + } + Continuous Integration systems ------------------------------ @@ -124,7 +199,7 @@ All drivers submitted must be contemplated with unit tests covering at least framework and be located in-tree using a structure that mirrors the functional code, such as directory names and filenames. See template below: - :: +:: manila/[tests/]path/to/brand/new/[test_]driver.py @@ -206,10 +281,41 @@ additional functionalities such as consistent group snapshot, the driver vendors may report this capability as a group capability, such as: Ordered writes, Consistent snapshots, Group replication. +Drivers need to report group capabilities as part of the updated stats (e.g. +capacity) and filled in 'share_group_stats' node for their back end. Share group +type group-specs (scoped and un-scoped) are available for the driver +implementation to use as-needed. Below is an example of the share stats +payload from the driver having multiple pools and group capabilities. "my" +is used as an example vendor prefix: + +:: + + { + 'driver_handles_share_servers': 'False', #\ + 'share_backend_name': 'My Backend', # backend level + 'vendor_name': 'MY', # mandatory/fixed + 'driver_version': '1.0', # stats & capabilities + 'storage_protocol': 'NFS_CIFS', #/ + #\ + 'my_capability_1': 'custom_val', # "my" optional vendor + 'my_capability_2': True, # stats & capabilities + #/ + 'share_group_stats': { + #\ + 'my_group_capability_1': 'custom_val', # "my" optional vendor + 'my_group_capability_2': True, # stats & group capabilities + #/ + 'consistent_snapshot_support': 'host', #\ + # common group capabilities + #/ + }, + ] + } + + .. note:: - for more information please see - `group capabilities `_ + for more information please see :doc:`../admin/group_capabilities_and_extra_specs` Share Replication ----------------- @@ -219,8 +325,7 @@ recovery) or for load sharing. In order to utilize this feature, drivers must report the ``replication_type`` they support as a capability and implement necessary methods. -More details can be found at: -https://docs.openstack.org/manila/latest/admin/shared-file-systems-share-replication.html +More details can be found at: :doc:`../admin/shared-file-systems-share-replication` Update "used_size" of shares ----------------------------