diff --git a/doc/source/admin/capabilities_and_extra_specs.rst b/doc/source/admin/capabilities_and_extra_specs.rst index b17a3c1238..85ecbc15bf 100644 --- a/doc/source/admin/capabilities_and_extra_specs.rst +++ b/doc/source/admin/capabilities_and_extra_specs.rst @@ -2,44 +2,96 @@ Capabilities and Extra-Specs ============================ -Manila Administrators create share types with extra-specs to allow users -to request a type of share to create. The Administrator chooses a name -for the share type and decides how to communicate the significance of -the different share types in terms that the users should understand or -need to know. By design, most of the details of a share type (the extra- -specs) are not exposed to users -- only Administrators. - -Share Types ------------ -Refer to the manila client command-line help for information on how to -create a share type and set "extra-spec" key/value pairs for a share type. - -Extra-Specs ------------ -There are 3 types of extra-specs: required, scoped, and un-scoped. - -Manila *requires* the driver_handles_share_servers extra-spec. - -*Scoped* extra-specs use a prefix followed by a colon to define a namespace -for scoping the extra-spec. A prefix could be a vendor name or acronym -and is a hint that this extra-spec key/value only applies to that vendor's -driver. Scoped extra-specs are not used by the scheduler to determine -where a share is created (except for the special `capabilities` prefix). -It is up to each driver implementation to determine how to use scoped -extra-specs and to document them. - -The prefix "capabilities" is a special prefix to indicate extra-specs that -are treated like un-scoped extra-specs. In the CapabilitiesFilter the -"capabilities:" is stripped from the key and then the extra-spec key and -value are used as an un-scoped extra-spec. - -*Un-scoped* extra-specs have a key that either starts with "capabilities:" or -does not contain a colon. When the CapabilitiesFilter is enabled (it is -enabled by default), the scheduler will only create a share on a backend -that reports capabilities that match the share type's un-scoped extra-spec -keys. - -The CapabilitiesFilter uses the following for matching operators: +Cloud Administrators create :ref:`shared_file_systems_share_types` with +extra-specs to: + +- influence the scheduler's decision to place new shares, and +- instruct the Shared File System service or its storage driver/s to perform + certain special actions with respect to the users' shares. + +As an administrator, you can choose a descriptive name or provide good +descriptions for your share types to convey the share type capabilities to +end users. End users can view standard ``tenant-visible`` extra-specs that +can let them seek required behavior and automate their applications +accordingly. By design, however, all other extra-specs of a share type are not +exposed to non-privileged users. + +Types of Extra-Specs +-------------------- + +The Shared File Systems service back-end storage drivers offer a wide range of +capabilities. The variation in these capabilities allows cloud +administrators to provide a storage service catalog to their end users. +Share type extra-specs tie-in with these capabilities. + +Some back-end capabilities are very specific to a storage system, and are +opaque to the Shared File System service or the end users. These +capabilities are invoked with the help of "scoped" extra-specs. Using scoped +extra-specs is a way to provide programmatic directives to the concerned +storage driver to do something during share creation or share manipulation. +You can learn about the opaque capabilities through driver documentation +and configure these capabilities within share types as scoped +extra-specs (e.g.: hpe3par:nfs_options). The Shared File System service +scheduler ignores scoped extra-specs during its quest to find the right back +end to provision shares. + +There are some back-end capabilities in manila that do matter to the scheduler. +For our understanding, lets call these non-scoped or non-opaque capabilities. +All non-scoped capabilities can be directly used as share types extra-specs. +They are considered by the scheduler’s capabilities filter (and any custom +filter defined by deployers). + +You can get a list of non-scoped capabilities from the scheduler by using: + +.. code-block:: console + + $ manila pool-list --detail + +The non-scoped capabilities can be of three types: + +- **Capabilities pertaining to a specific back end storage system driver**: For + example, *huawei_smartcache*. + No Shared File System service API relies on non-opaque back end specific + capabilities. +- **Common capabilities that are not visible to end users**: The manila + community has standardized some cross-platform capabilities like + *thin_provisioning*, *dedupe*, *compression*, *qos*, *ipv6_support* and + *ipv4_support*. Values of these options do not matter to any Shared File + System service APIs; however, they can signify something to the manila + services themselves. For example when a back end supports thin_provisioning, + the scheduler service performs over-provisioning, and if a back end does + not report *ipv6_support* as True, the share-manager service drops IPv6 + access rules before invoking the storage driver to update access rules. +- **Common capabilities that are visible to end users**: Some capabilities + affect functionality exposed via the Shared File System service API. For + example, not all back ends support snapshots, and even if they do, they + may not support all of the snapshot operations. For example, cloning + snapshots into new shares, reverting shares in-place to snapshots, etc. + + The support for these capabilities determines whether users would be able + to perform certain control-plane operations with manila. For example, a back + end driver may report *snapshot_support=True* allowing end users to + create share snapshots, however, the driver can report + *create_share_from_snapshot_support=False*. + This reporting allows cloud administrators to create share types that + support snapshots but not creating shares from snapshots. When a user uses + such a share type, they will not be able to clone snapshots into new shares. + Tenant-visible capabilities aid manila in validating requests and failing + fast on requests it cannot accommodate. They also help level set the user + expectations on some failures. For example, if snapshot_support is set to + False on the share type, since users can see this, they will not invoke + the create snapshot API, and even if they do, they will understand the + HTTP 400 (and error message) in better context. + +.. important:: + + All extra-specs are optional, except one: *driver_handles_share_servers*. + +Scheduler's treatment of non-scoped extra specs +----------------------------------------------- + +The CapabilitiesFilter in the Shared File System scheduler uses the following +for matching operators: * No operator This defaults to doing a python ==. Additionally it will match boolean values. @@ -74,221 +126,134 @@ The CapabilitiesFilter uses the following for matching operators: * **s==, s!=, s>=, s>, s<=, s<** - The "s" indicates it is a string comparison. These choose a host that satisfies - the comparison of strings in capability and specification. For example, - if "capabilities:replication_type s== dr", a host that reports - replication_type of "dr" will be chosen. + The "s" indicates it is a string comparison. These choose a host that + satisfies the comparison of strings in capability and specification. For + example, if "capabilities:replication_type s== dr", a host that reports + replication_type of "dr" will be chosen. If "share_backend_name s!= + cephfs" is used, any host not named "cephfs" can be chosen. -For vendor-specific capabilities (which need to be visible to the -CapabilityFilter), it is recommended to use the vendor prefix followed -by an underscore. This is not a strict requirement, but will provide a +For vendor-specific non-scoped capabilities (which need to be visible to the +scheduler), drivers are recommended to use the vendor prefix followed +by an underscore. This is not a strict requirement, but can provide a consistent look along-side the scoped extra-specs and will be a clear indicator of vendor capabilities vs. common capabilities. Common Capabilities ------------------- -For capabilities that apply to multiple backends a common capability can -be created. Like all other backend reported capabilities, these capabilities +Common capabilities apply to multiple backends. +Like all other backend reported capabilities, these capabilities can be used verbatim as extra_specs in share types used to create shares. -* `driver_handles_share_servers` is a special, required, user-visible common - capability. Added in Kilo. +Share type common capability extra-specs that are visible to end users: +----------------------------------------------------------------------- -* `dedupe` - indicates that a backend/pool can provide shares using some - deduplication technology. The default value of the dedupe capability (if a - driver doesn't report it) is False. In Liberty, drivers cannot report to the - scheduler that they support both dedupe and non-deduped share. For each pool - it's either always on or always off, even if the drivers can technically - support both dedupe and non-deduped in a pool. Since Mitaka, the logic is - changed to allow a driver to report dedupe=[True, False] if it can support - both dedupe and non-deduped in a pool. Administrators can make a share type - use deduplication by setting this extra-spec to ' True'. Administrators - can prevent a share type from using deduplication by setting this extra-spec - to ' False'. Added in Liberty. - -* `compression` - indicates that a backend/pool can provide shares using some - compression technology. The default value of the compression capability (if a - driver doesn't report it) is False. In Liberty, drivers cannot report to the - scheduler that they support both compression and non-compression. For each - pool it's either always on or always off, even if the drivers can technically - support both compression and non-compression in a pool. Since Mitaka, the - logic is changed to allow a driver to report compression=[True, False] if it - can support both compression and non-compression in a pool. Administrators - can make a share type use compression by setting this extra-spec to - ' True'. Administrators can prevent a share type from using compression - by setting this extra-spec to ' False'. Added in Liberty. - -* `thin_provisioning` - shares will not be space guaranteed and - overprovisioning will be enabled. This capability defaults to False. - Backends/pools that support thin provisioning must report True for this - capability. Administrators can make a share type use thin provisioned shares - by setting this extra-spec to ' True'. If a driver reports - thin_provisioning=False (the default) then it's assumed that the driver is - doing thick provisioning and overprovisioning is turned off. - This was added in Liberty. In Liberty and Mitaka, the driver was required - to configure one pool for thin and another pool for thick and report - thin_provisioning as either True or False even if an array can technically - support both thin and thick provisioning in a pool. In Newton, the logic is - changed to allow a driver to report thin_provisioning=[True, False] if it - can support both thin and thick provisioning in a pool. To provision a thick - share on a back end that supports both thin and thick provisioning, set one - of the following in extra specs: - -:: - - {'thin_provisioning': 'False'} - {'thin_provisioning': ' False'} - {'capabilities:thin_provisioning': 'False'} - {'capabilities:thin_provisioning': ' False'} - -* `qos` - indicates that a backend/pool can provide shares using some - QoS (Quality of Service) specification. The default value of the qos - capability (if a driver doesn't report it) is False. Administrators - can make a share type use QoS by setting this extra-spec to ' True' and - also setting the relevant QoS-related extra specs for the drivers being used. - Administrators can prevent a share type from using QoS by setting this - extra-spec to ' False'. Different drivers have different ways of specifying - QoS limits (or guarantees) and this extra spec merely allows the scheduler to - filter by pools that either have or don't have QoS support enabled. Added in - Mitaka. - -* `replication_type` - indicates the style of replication supported for the - backend/pool. This extra_spec will have a string value and could be one - of :term:`writable`, :term:`readable` or :term:`dr`. `writable` replication - type involves synchronously replicated shares where all replicas are - writable. Promotion is not supported and not needed. `readable` and `dr` - replication types involve a single `active` or `primary` replica and one or - more `non-active` or secondary replicas per share. In `readable` type of - replication, `non-active` replicas have one or more export_locations and - can thus be mounted and read while the `active` replica is the only one - that can be written into. In `dr` style of replication, only - the `active` replica can be mounted, read from and written into. Added in - Mitaka. +* **driver_handles_share_servers** is a special, required common + capability. When set to True, the scheduler matches requests with back ends + that can isolate user workloads with dedicated share servers exporting + shares on user provided share networks. -* `snapshot_support` - indicates whether snapshots are supported for shares +* **snapshot_support** indicates whether snapshots are supported for shares created on the pool/backend. When administrators do not set this capability as an extra-spec in a share type, the scheduler can place new shares of that type in pools without regard for whether snapshots are supported, and those shares will not support snapshots. -* `create_share_from_snapshot_support` - indicates whether a backend can create - a new share from a snapshot. When administrators do not set this capability - as an extra-spec in a share type, the scheduler can place new shares of that - type in pools without regard for whether creating shares from snapshots is - supported, and those shares will not support creating shares from snapshots. +* **create_share_from_snapshot_support** indicates whether a backend can + create a new share from a snapshot. When administrators do not set this + capability as an extra-spec in a share type, the scheduler can place new + shares of that type in pools without regard for whether creating shares + from snapshots is supported, and those shares will not support creating + shares from snapshots. -* `revert_to_snapshot_support` - indicates that a driver is capable of +* **revert_to_snapshot_support** indicates that a driver is capable of reverting a share in place to its most recent snapshot. When administrators do not set this capability as an extra-spec in a share type, the scheduler can place new shares of that type in pools without regard for whether reverting shares to snapshots is supported, and those shares will not support reverting shares to snapshots. -* `ipv4_support` - indicates whether a back end can create a share that can be - accessed via IPv4 protocol. If administrators do not set this capability - as an extra-spec in a share type, the scheduler can place new shares of that - type in pools without regard for whether IPv4 is supported. +* **mount_snapshot_support** indicates that a driver is capable of exporting + share snapshots for mounting. Users can provide and revoke access to + mountable snapshots just like they can with their shares. -* `ipv6_support` - indicates whether a back end can create a share that can be - accessed via IPv6 protocol. If administrators do not set this capability - as an extra-spec in a share type, the scheduler can place new shares of that - type in pools without regard for whether IPv6 is supported. +* **replication_type** indicates the style of replication supported for the + backend/pool. This extra_spec will have a string value and could be one + of :term:`writable`, :term:`readable` or :term:`dr`. `writable` replication + type involves synchronously replicated shares where all replicas are + writable. Promotion is not supported and not needed. `readable` and `dr` + replication types involve a single `active` or `primary` replica and one or + more `non-active` or secondary replicas per share. In `readable` type of + replication, `non-active` replicas have one or more export_locations and + can thus be mounted and read while the `active` replica is the only one + that can be written into. In `dr` style of replication, only + the `active` replica can be mounted, read from and written into. -Reporting Capabilities ----------------------- -Drivers report capabilities as part of the updated stats (e.g. capacity) -for their backend/pools. This is how a backend/pool advertizes its ability -to provide a share that matches the capabilities requested in the share -type extra-specs. +* **availability_zones** indicates a comma separated list of availability + zones that can be used for provisioning. Users can always provide a specific + availability zone during share creation, and they will receive a + synchronous failure message if they attempt to create a share in an + availability zone that the share type does not permit. If you do not set + this extra-spec, the share type is assumed to be serviceable in all + availability zones known to the Shared File Systems service. -Developer impact ----------------- +Share type common capability extra-specs that are not visible to end users: +--------------------------------------------------------------------------- -Developers should update their drivers to include all backend and pool -capacities and capabilities in the share stats it reports to scheduler. -Below is an example having multiple pools. "my" is used as an -example vendor prefix: +* **dedupe** indicates that a backend/pool can provide shares using some + deduplication technology. The default value of the dedupe capability (if a + driver doesn't report it) is False. Drivers can support both dedupe and + non-deduped shares in a single storage pool by reporting ``dedupe=[True, + False]``. You can make a share type use deduplication by setting this + extra-spec to ' True', or prevent it by setting this extra-spec + to ' False'. + +* **compression** indicates that a backend/pool can provide shares using some + compression technology. The default value of the compression capability (if a + driver doesn't report it) is False. Drivers can support compressed and + non-compressed shares in a single storage pool by reporting + ``compression=[True, False]``. You can make a share type use compression + by setting this extra-spec to ' True', or prevent it by setting this + extra-spec to ' False'. + +* **thin_provisioning** can be enabled where shares will not be + guaranteed space allocations and overprovisioning will be enabled. This + capability defaults to False. Back ends/pools that support thin + provisioning report True for this capability. Administrators can make a + share type use thin provisioned shares by setting this extra-spec + to ' True'. If a driver reports thin_provisioning=False (the default) + then it's assumed that the driver is doing thick provisioning and + overprovisioning is turned off. A driver can support thin provisioned + and thick provisioned shares in the same pool by reporting + ``thin_provisioning=[True, False]``. + + To provision a thick + share on a back end that supports both thin and thick provisioning, set one + of the following in extra specs: :: - { - 'driver_handles_share_servers': 'False', #\ - 'share_backend_name': 'My Backend', # backend level - 'vendor_name': 'MY', # mandatory/fixed - 'driver_version': '1.0', # stats & capabilities - 'storage_protocol': 'NFS_CIFS', #/ - #\ - 'my_capability_1': 'custom_val', # "my" optional vendor - 'my_capability_2': True, # stats & capabilities - #/ - 'pools': [ - {'pool_name': - 'thin-dedupe-compression pool', #\ - 'total_capacity_gb': 500, # mandatory stats for - 'free_capacity_gb': 230, # pools - 'reserved_percentage': 0, #/ - #\ - 'dedupe': True, # common capabilities - 'compression': True, # - 'snapshot_support': True, # - 'create_share_from_snapshot_support': True, - 'revert_to_snapshot_support': True, - 'qos': True, # this backend supports QoS - 'thin_provisioning': True, # - 'max_over_subscription_ratio': 10, # (mandatory for thin) - 'provisioned_capacity_gb': 270, # (mandatory for thin) - # - # - 'replication_type': 'dr', # this backend supports - # replication_type 'dr' - #/ - 'my_dying_disks': 100, #\ - 'my_super_hero_1': 'Hulk', # "my" optional vendor - 'my_super_hero_2': 'Spider-Man', # stats & capabilities - #/ - #\ - # can replicate to other - 'replication_domain': 'asgard', # backends in - # replication_domain 'asgard' - #/ - 'ipv4_support': True, - 'ipv6_support': True, - - }, - {'pool_name': 'thick pool', - 'total_capacity_gb': 1024, - 'free_capacity_gb': 1024, - 'qos': False, - 'snapshot_support': True, - 'create_share_from_snapshot_support': False, # this pool does not - # allow creating - # shares from - # snapshots - 'revert_to_snapshot_support': True, - 'reserved_percentage': 0, - 'dedupe': False, - 'compression': False, - 'thin_provisioning': False, - 'replication_type': None, - 'my_dying_disks': 200, - 'my_super_hero_1': 'Batman', - 'my_super_hero_2': 'Robin', - 'ipv4_support': True, - 'ipv6_support': True, - }, - ] - } - -Work Flow ---------- - -1) Share Backends report how many pools and what those pools look like and - are capable of to scheduler; - -2) When request comes in, scheduler picks a pool that fits the need best to - serve the request, it passes the request to the backend where the target - pool resides; - -3) Share driver gets the message and lets the target pool serve the request - as scheduler instructed. Share type extra-specs (scoped and un-scoped) - are available for the driver implementation to use as-needed. + {'thin_provisioning': 'False'} + {'thin_provisioning': ' False'} + {'capabilities:thin_provisioning': 'False'} + {'capabilities:thin_provisioning': ' False'} + +* **qos** indicates that a backend/pool can provide shares using some + QoS (Quality of Service) specification. The default value of the qos + capability (if a driver doesn't report it) is False. You can make a share + type use QoS by setting this extra-spec to ' True' and also setting + the relevant QoS-related extra specs for the drivers being used. + Administrators can prevent a share type from using QoS by setting this + extra-spec to ' False'. Different drivers have different ways of + specifying QoS limits (or guarantees) and this extra spec merely allows + the scheduler to filter by pools that either have or don't have QoS + support enabled. + +* **ipv4_support** indicates whether a back end can create a share that + can be accessed via IPv4 protocol. If administrators do not set this + capability as an extra-spec in a share type, the scheduler can place new + shares of that type in pools without regard for whether IPv4 is supported. + +* **ipv6_support** - indicates whether a back end can create a share that + can be accessed via IPv6 protocol. If administrators do not set this + capability as an extra-spec in a share type, the scheduler can place new + shares of that type in pools without regard for whether IPv6 is supported. diff --git a/doc/source/admin/group_capabilities_and_extra_specs.rst b/doc/source/admin/group_capabilities_and_extra_specs.rst index 888b98ef4a..cc1b306d59 100644 --- a/doc/source/admin/group_capabilities_and_extra_specs.rst +++ b/doc/source/admin/group_capabilities_and_extra_specs.rst @@ -2,9 +2,8 @@ Group Capabilities and group-specs ================================== -Manila Administrators create share group types with `share types -`_ and group-specs to allow users +Manila Administrators create share group types with +:ref:`shared_file_systems_share_types` and group-specs to allow users to request a group type of share group to create. The Administrator chooses a name for the share group type and decides how to communicate the significance of the different share group types in terms that the users should understand or @@ -40,56 +39,3 @@ create share groups. The default value of the consistent_snapshot_support capability (if a driver doesn't report it) is None. Administrators can make a share group type use consistent snapshot support by setting this group-spec to 'host'. - -Reporting Group Capabilities ----------------------------- -Drivers report group capabilities as part of the updated stats (e.g. capacity) -and filled in 'share_group_stats' node for their back end. This is how a backend -advertizes its ability to provide a share that matches the group capabilities -requested in the share group type group-specs. - -Developer impact ----------------- - -Developers should update their drivers to include all backend and pool -capacities and capabilities in the share stats it reports to scheduler. -Below is an example having multiple pools. "my" is used as an -example vendor prefix: - -:: - - { - 'driver_handles_share_servers': 'False', #\ - 'share_backend_name': 'My Backend', # backend level - 'vendor_name': 'MY', # mandatory/fixed - 'driver_version': '1.0', # stats & capabilities - 'storage_protocol': 'NFS_CIFS', #/ - #\ - 'my_capability_1': 'custom_val', # "my" optional vendor - 'my_capability_2': True, # stats & capabilities - #/ - 'share_group_stats': { - #\ - 'my_group_capability_1': 'custom_val', # "my" optional vendor - 'my_group_capability_2': True, # stats & group capabilities - #/ - 'consistent_snapshot_support': 'host', #\ - # common group capabilities - #/ - }, - ] - } - -Work Flow ---------- - -1) Share Backends report how many pools and what those pools look like and - are capable of to group scheduler; - -2) When request comes in, scheduler picks a backend that fits the need best to - serve the request, it passes the request to the backend where the target - pool resides; - -3) Share driver gets the message and lets the target pool serve the request - as group scheduler instructed. Share group type group-specs (scoped and un-scoped) - are available for the driver implementation to use as-needed. diff --git a/doc/source/admin/index.rst b/doc/source/admin/index.rst index d7ae518926..1d88df1af9 100644 --- a/doc/source/admin/index.rst +++ b/doc/source/admin/index.rst @@ -45,11 +45,11 @@ deployment. shared-file-systems-key-concepts.rst shared-file-systems-share-management.rst - shared-file-systems-share-server-management.rst - shared-file-systems-share-migration.rst shared-file-systems-share-types.rst shared-file-systems-snapshots.rst + shared-file-systems-share-server-management.rst shared-file-systems-security-services.rst + shared-file-systems-share-migration.rst shared-file-systems-share-replication.rst shared-file-systems-multi-backend.rst shared-file-systems-networking.rst diff --git a/doc/source/admin/shared-file-systems-share-types.rst b/doc/source/admin/shared-file-systems-share-types.rst index 2a55e8d6c3..76d3d20ef1 100644 --- a/doc/source/admin/shared-file-systems-share-types.rst +++ b/doc/source/admin/shared-file-systems-share-types.rst @@ -4,136 +4,218 @@ Share types =========== -A share type enables you to filter or choose back ends before you create a -share and to set data for the share driver. A share type behaves in the same -way as a Block Storage volume type behaves. - -In the Shared File Systems configuration file ``manila.conf``, the -administrator can set the share type used by default for the share creation -and then create a default share type. - -To create a share type, use :command:`manila type-create` command as: +The Shared File System service back-end storage drivers offer a wide range +of capabilities. The variation in these capabilities allows cloud +administrators to provide a storage service catalog to their end users. +Share types can be used to create this storage service catalog. +Cloud administrators can influence provisioning of users' shares with the +help of Share types. All shares are associated with a share type. Share +types are akin to ``flavors`` in the OpenStack Compute service (nova), or +``volume types`` in the OpenStack Block Storage service (cinder), or ``storage +classes`` in Kubernetes. You can allow a share type to be accessible to all +users in your cloud if you wish. You can also create private share types that +allow only users belonging to certain OpenStack projects to access them. +You can have an unlimited number of share types in your +cloud, but for practical purposes, you may want to create only a handful of +publicly accessible share types. + +Each share type is an object that encompasses ``extra-specs`` (extra +specifications). These extra-specs can map to storage back-end capabilities, +or can be directives to the service. + +Consider for example, offering three share types in your cloud to map +to "service levels": + ++--------+--------------------------------------------------------------------------------------------------+ +| Type | Capabilities/Instructions | ++========+==================================================================================================+ +| Gold | Allow creating snapshots, reverting to snapshots and share replication, "thick" provision shares | ++--------+--------------------------------------------------------------------------------------------------+ +| Silver | Allow creating snapshots, "thin" provision shares | ++--------+--------------------------------------------------------------------------------------------------+ +| Bronze | Don't allow creating snapshots, "thin" provision shares | ++--------+--------------------------------------------------------------------------------------------------+ + +Capabilities or instructions such as the ones above are coded as extra-specs +that your users and the Shared File System service understand. Users in +OpenStack projects can see all public share types along with private share +types that are made accessible to them. Not all extra-specs that you +configure in a share type are visible to your users. This design helps +preserve the cloud abstraction. Along with the share type names, they can +see the share type descriptions and "tenant-visible" extra-specs. + +For more details on extra-specs, see :ref:`capabilities_and_extra_specs`. + +The Shared File Systems service also allows using quota controls with share +types. Quotas can help you maintain your SLAs by limiting the number of +consumable resources or aid in billing. See :ref:`shared_file_systems_quotas` +for more details. + +Driver Handles Share Servers (DHSS) +----------------------------------- + +To provide secure and hard multi-tenancy on the network data path, the +Shared File Systems service allows users to use their own "share networks". +When shares are created on a share network, users can be sure they have +their own isolated "share servers" that export their shares on the share +network that have the ability plug into user-determined authentication +domains ("security services"). Not all Shared File System service storage +drivers support share networks. Those that do assert the capability +``driver_handles_share_servers=True``. + +When creating a share type, you are *required* to set an extra-spec that +matches this capability. It is visible to end users. + +Default Share Type +------------------ + +When you are operating a cloud where all your tenants are trusted, you may +want to create a "default" share type that applies to all of them. It +simplifies share creation for your end users since they don't need to worry +about share types. + +Use of a default share type is not recommended in a multi-tenant cloud where +you may want to separate your user workloads, or offer different service +capabilities. In such instances, you must always encourage your users to +specify a share type at share creation time, and not rely on the default +share type. + +.. important:: + + If you do not create and configure a default share type, users *must* + specify a valid share type during share creation, or share creation + requests will fail. + +To configure the default share type, edit the ``manila.conf`` file, and set +the configuration option [DEFAULT]/default_share_type. + +You must then create a share type, using :command:`manila type-create`: .. code-block:: console - manila type-create [--snapshot_support ] - [--is_public ] + manila type-create [--is_public ] + [--description ] + [--extra-specs ] -where the ``name`` is the share type name, ``--is_public`` defines the level of -the visibility for the share type, ``snapshot_support`` and -``spec_driver_handles_share_servers`` are the extra specifications used to -filter back ends. Administrators can create share types with these extra -specifications for the back ends filtering: - -- ``driver_handles_share_servers``. Required. Defines the driver mode for share - server lifecycle management. Valid values are ``true``/``1`` and - ``false``/``0``. - Set to True when the share driver can manage, or handle, the share server - lifecycle. - Set to False when an administrator, rather than a share driver, manages - the bare metal storage with some net interface instead of the presence - of the share servers. - -- ``snapshot_support``. Filters back ends by whether they do or do not support - share snapshots. Default is ``True``. - Set to True to find back ends that support share snapshots. - Set to False to find back ends that do not support share snapshots. - -.. note:: - - The extra specifications set in the share types are operated in the - :ref:`shared_file_systems_scheduling`. - -Administrators can also set additional extra specifications for a share type -for the following purposes: - -- *Filter back ends*. Unqualified extra specifications written in - this format: ``extra_spec=value``. For example, **netapp_raid_type=raid4**. - -- *Set data for the driver*. Qualified extra specifications always written - with the prefix with a colon, except for the special ``capabilities`` - prefix, in this format: ``vendor:extra_spec=value``. For example, - **netapp:thin_provisioned=true**. +where: -The scheduler uses the special capabilities prefix for filtering. The scheduler -can only create a share on a back end that reports capabilities matching the -un-scoped extra-spec keys for the share type. For details, see `Capabilities -and Extra-Specs `_. - -Each driver implementation determines which extra specification keys it uses. -For details, see the documentation for the driver. - -An administrator can use the ``policy.json`` file to grant permissions for -share type creation with extra specifications to other roles. - -You set a share type to private or public and -:ref:`manage the access` to the private share types. By -default a share type is created as publicly accessible. Set -``--is_public`` to ``False`` to make the share type private. +- ``name`` is the share type name +- ``is_public`` defines the visibility for the share type (true/false) +- ``description`` is a free form text field to describe the characteristics + of the share type for your users' benefit +- ``extra-specs`` defines a comma separated set of key=value pairs of + optional extra specifications +- ``spec_driver_handles_share_servers`` is the mandatory extra-spec + (true/false) Share type operations --------------------- To create a new share type you need to specify the name of the new share type. You also require an extra spec ``driver_handles_share_servers``. -The new share type can also be public. +The new share type can be public or private. .. code-block:: console - $ manila type-create netapp1 False --is_public True + $ manila manila type-create default-shares False \ + --description "Default share type for the cloud, no fancy capabilities" $ manila type-list - +-----+--------+-----------+-----------+-----------------------------------+-----------------------+ - | ID | Name | Visibility| is_default| required_extra_specs | optional_extra_specs | - +-----+--------+-----------+-----------+-----------------------------------+-----------------------+ - | c0..| netapp1| public | - | driver_handles_share_servers:False| snapshot_support:True | - +-----+--------+-----------+-----------+-----------------------------------+-----------------------+ - -You can set or unset extra specifications for a share type -using **manila type-key set ** command. Since it is up -to each driver what extra specification keys it uses, see the documentation -for the specified driver. + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+-------------------------------------------+---------------------------------------------------------+ + | ID | Name | visibility | is_default | required_extra_specs | optional_extra_specs | Description | + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+-------------------------------------------+---------------------------------------------------------+ + | cf1f92ec-4d0a-4b79-8f18-6bb82c22840a | default-shares | public | - | driver_handles_share_servers : False | | Default share type for the cloud, no fancy capabilities | + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+-------------------------------------------+---------------------------------------------------------+ + + $ manila type-show default-shares + +----------------------+---------------------------------------------------------+ + | Property | Value | + +----------------------+---------------------------------------------------------+ + | id | cf1f92ec-4d0a-4b79-8f18-6bb82c22840a | + | name | default-shares | + | visibility | public | + | is_default | NO | + | description | Default share type for the cloud, no fancy capabilities | + | required_extra_specs | driver_handles_share_servers : False | + | optional_extra_specs | | + +----------------------+---------------------------------------------------------+ + +You did not provide optional capabilities, so they are all *assumed to be off +by default*. So, Non-privileged users see some tenant-visible capabilities +explicitly. .. code-block:: console - $ manila type-key netapp1 set thin_provisioned=True -It is also possible to view a list of current share types and extra -specifications: + $ source demorc + $ manila type-list + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+--------------------------------------------+---------------------------------------------------------+ + | ID | Name | visibility | is_default | required_extra_specs | optional_extra_specs | Description | + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+--------------------------------------------+---------------------------------------------------------+ + | cf1f92ec-4d0a-4b79-8f18-6bb82c22840a | default-shares | public | - | driver_handles_share_servers : False | snapshot_support : False | Default share type for the cloud, no fancy capabilities | + +--------------------------------------+-----------------------------------+------------+------------+--------------------------------------+--------------------------------------------+---------------------------------------------------------+ + + $ manila type-show default-shares + +----------------------+---------------------------------------------------------+ + | Property | Value | + +----------------------+---------------------------------------------------------+ + | id | cf1f92ec-4d0a-4b79-8f18-6bb82c22840a | + | name | default-shares | + | visibility | public | + | is_default | NO | + | description | Default share type for the cloud, no fancy capabilities | + | required_extra_specs | driver_handles_share_servers : False | + | optional_extra_specs | snapshot_support : False | + | | create_share_from_snapshot_support : False | + | | revert_to_snapshot_support : False | + | | mount_snapshot_support : False | + +----------------------+---------------------------------------------------------+ + + +You can set or unset extra specifications for a share type +using **manila type-key set ** command. .. code-block:: console - $ manila extra-specs-list - +-------------+---------+-------------------------------------+ - | ID | Name | all_extra_specs | - +-------------+---------+-------------------------------------+ - | c0086582-...| netapp1 | snapshot_support : True | - | | | thin_provisioned : True | - | | | driver_handles_share_servers : True | - +-------------+---------+-------------------------------------+ + $ manila type-key default-shares set snapshot_support=True + + $ manila type-show default-shares + +----------------------+---------------------------------------------------------+ + | Property | Value | + +----------------------+---------------------------------------------------------+ + | id | cf1f92ec-4d0a-4b79-8f18-6bb82c22840a | + | name | default-shares | + | visibility | public | + | is_default | NO | + | description | Default share type for the cloud, no fancy capabilities | + | required_extra_specs | driver_handles_share_servers : False | + | optional_extra_specs | snapshot_support : True | + +----------------------+---------------------------------------------------------+ Use :command:`manila type-key unset ` to unset an extra specification. -The public or private share type can be deleted with the -:command:`manila type-delete ` command. +A share type can be deleted with the :command:`manila type-delete +` command. However, a share type can only be deleted if there +are no shares, share groups or share group types associated with the share +type. .. _share_type_access: -Share type access ------------------ +Share type access control +------------------------- -You can manage access to a private share type for different projects. -Administrators can provide access, remove access, and retrieve -information about access for a specified private share. +You can provide access, revoke access, and retrieve list of allowed projects +for a specified private share. Create a private type: .. code-block:: console - $ manila type-create my_type1 True --is_public False + $ manila type-create my_type1 True \ + --is_public False \ + --extra-specs snapshot_support=True +----------------------+--------------------------------------+ | Property | Value | +----------------------+--------------------------------------+ @@ -148,8 +230,7 @@ Create a private type: .. note:: If you run :command:`manila type-list` only public share types appear. - To see private share types, run :command:`manila type-list` with - ``--all`` optional argument. + To see private share types, run :command:`manila type-list --all``. Grant access to created private type for a demo and alt_demo projects by providing their IDs: @@ -171,9 +252,8 @@ To view information about access for a private share, type ``my_type1``: | e4970f57f1824faab2701db61ee7efdf | +----------------------------------+ -After granting access to the share, the target project -can see the share type in the list, and create private -shares. +After granting access to the share, the users in the allowed projects +can see the share type and use it to create shares. To deny access for a specified project, use :command:`manila type-access-remove ` command. diff --git a/doc/source/contributor/driver_requirements.rst b/doc/source/contributor/driver_requirements.rst index 5afd7777db..c5137fa275 100644 --- a/doc/source/contributor/driver_requirements.rst +++ b/doc/source/contributor/driver_requirements.rst @@ -14,8 +14,8 @@ License for the specific language governing permissions and limitations under the License. -Manila minimum requirements and features since Mitaka -===================================================== +Manila minimum requirements and features +======================================== In order for a driver to be accepted into manila code base, there are certain minimum requirements and features that must be met, in order to ensure @@ -74,8 +74,11 @@ Capabilities ------------ In order for manila to function accordingly to the driver being used, the -driver must provide a set of information to manila, known as capabilities, as -follows: +driver must provide a set of information to manila, known as capabilities. +Share driver can use Share type extra-specs (scoped and un-scoped) to serve +new shares. See :doc:`../admin/capabilities_and_extra_specs` for more +information. At a minimum your driver must report: + - share_backend_name: a name for the backend; - driver_handles_share_servers: driver mode, whether this driver instance @@ -102,7 +105,79 @@ function correctly in manila, such as: - replication_type: string specifying the type of replication supported by the driver. Can be one of ('readable', 'writable' or 'dr'). -.. note:: for more information please see https://docs.openstack.org/manila/latest/admin/capabilities_and_extra_specs.html +Below is an example of drivers with multiple pools. "my" is used as an +example vendor prefix: + +:: + + { + 'driver_handles_share_servers': 'False', #\ + 'share_backend_name': 'My Backend', # backend level + 'vendor_name': 'MY', # mandatory/fixed + 'driver_version': '1.0', # stats & capabilities + 'storage_protocol': 'NFS_CIFS', #/ + #\ + 'my_capability_1': 'custom_val', # "my" optional vendor + 'my_capability_2': True, # stats & capabilities + #/ + 'pools': [ + {'pool_name': + 'thin-dedupe-compression pool', #\ + 'total_capacity_gb': 500, # mandatory stats for + 'free_capacity_gb': 230, # pools + 'reserved_percentage': 0, #/ + #\ + 'dedupe': True, # common capabilities + 'compression': True, # + 'snapshot_support': True, # + 'create_share_from_snapshot_support': True, + 'revert_to_snapshot_support': True, + 'qos': True, # this backend supports QoS + 'thin_provisioning': True, # + 'max_over_subscription_ratio': 10, # (mandatory for thin) + 'provisioned_capacity_gb': 270, # (mandatory for thin) + # + # + 'replication_type': 'dr', # this backend supports + # replication_type 'dr' + #/ + 'my_dying_disks': 100, #\ + 'my_super_hero_1': 'Hulk', # "my" optional vendor + 'my_super_hero_2': 'Spider-Man', # stats & capabilities + #/ + #\ + # can replicate to other + 'replication_domain': 'asgard', # backends in + # replication_domain 'asgard' + #/ + 'ipv4_support': True, + 'ipv6_support': True, + + }, + {'pool_name': 'thick pool', + 'total_capacity_gb': 1024, + 'free_capacity_gb': 1024, + 'qos': False, + 'snapshot_support': True, + 'create_share_from_snapshot_support': False, # this pool does not + # allow creating + # shares from + # snapshots + 'revert_to_snapshot_support': True, + 'reserved_percentage': 0, + 'dedupe': False, + 'compression': False, + 'thin_provisioning': False, + 'replication_type': None, + 'my_dying_disks': 200, + 'my_super_hero_1': 'Batman', + 'my_super_hero_2': 'Robin', + 'ipv4_support': True, + 'ipv6_support': True, + }, + ] + } + Continuous Integration systems ------------------------------ @@ -124,7 +199,7 @@ All drivers submitted must be contemplated with unit tests covering at least framework and be located in-tree using a structure that mirrors the functional code, such as directory names and filenames. See template below: - :: +:: manila/[tests/]path/to/brand/new/[test_]driver.py @@ -206,10 +281,41 @@ additional functionalities such as consistent group snapshot, the driver vendors may report this capability as a group capability, such as: Ordered writes, Consistent snapshots, Group replication. +Drivers need to report group capabilities as part of the updated stats (e.g. +capacity) and filled in 'share_group_stats' node for their back end. Share group +type group-specs (scoped and un-scoped) are available for the driver +implementation to use as-needed. Below is an example of the share stats +payload from the driver having multiple pools and group capabilities. "my" +is used as an example vendor prefix: + +:: + + { + 'driver_handles_share_servers': 'False', #\ + 'share_backend_name': 'My Backend', # backend level + 'vendor_name': 'MY', # mandatory/fixed + 'driver_version': '1.0', # stats & capabilities + 'storage_protocol': 'NFS_CIFS', #/ + #\ + 'my_capability_1': 'custom_val', # "my" optional vendor + 'my_capability_2': True, # stats & capabilities + #/ + 'share_group_stats': { + #\ + 'my_group_capability_1': 'custom_val', # "my" optional vendor + 'my_group_capability_2': True, # stats & group capabilities + #/ + 'consistent_snapshot_support': 'host', #\ + # common group capabilities + #/ + }, + ] + } + + .. note:: - for more information please see - `group capabilities `_ + for more information please see :doc:`../admin/group_capabilities_and_extra_specs` Share Replication ----------------- @@ -219,8 +325,7 @@ recovery) or for load sharing. In order to utilize this feature, drivers must report the ``replication_type`` they support as a capability and implement necessary methods. -More details can be found at: -https://docs.openstack.org/manila/latest/admin/shared-file-systems-share-replication.html +More details can be found at: :doc:`../admin/shared-file-systems-share-replication` Update "used_size" of shares ----------------------------