==================================== HPE 3PAR Driver for OpenStack Manila ==================================== The HPE 3PAR driver provides NFS and CIFS shared file systems to OpenStack using HPE 3PAR's File Persona capabilities. For information on HPE 3PAR Driver for OpenStack Manila, refer to `content kit page `_. HPE 3PAR File Persona Software Suite concepts and terminology ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The software suite comprises the following managed objects: - File Provisioning Groups (FPGs) - Virtual File Servers (VFSs) - File Stores - File Shares The File Persona Software Suite is built upon the resilient mesh-active architecture of HPE 3PAR StoreServ and benefits from HPE 3PAR storage foundation of wide-striped logical disks and autonomic ``Common Provisioning Groups (CPGs)``. A CPG can be shared between file and block to create the File Shares or the logical unit numbers (LUNs) to provide true convergence. ``A File Provisioning Group (FPG)`` is an instance of the HPE intellectual property Adaptive File System. It controls how files are stored and retrieved. Each FPG is transparently constructed from one or multiple Virtual Volumes (VVs) and is the unit for replication and disaster recovery for File Persona Software Suite. There are up to 16 FPGs supported on a node pair. ``A Virtual File Server (VFS)`` is conceptually like a server. As such, it presents virtual IP addresses to clients, participates in user authentication services, and can have properties for such things as user/group quota management and antivirus policies. Up to 16 VFSs are supported on a node pair, one per FPG. ``File Stores`` are the slice of a VFS and FPG at which snapshots are taken, capacity quota management can be performed, and antivirus scan service policies customized. There are up to 256 File Stores supported on a node pair, 16 File Stores per VFS. ``File Shares`` are what provide data access to clients via SMB, NFS, and the Object Access API, subject to the share permissions applied to them. Multiple File Shares can be created for a File Store and at different directory levels within a File Store. Supported shared filesystems ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The driver supports CIFS and NFS shares. Operations supported ~~~~~~~~~~~~~~~~~~~~ - Create a share. – Share is not accessible until access rules allow access. - Delete a share. - Allow share access. Note the following limitations: – IP access rules are required for NFS share access. – User access rules are not allowed for NFS shares. – User access rules are required for SMB share access. – User access requires a File Persona local user for SMB shares. – Shares are read/write (and subject to ACLs). - Deny share access. - Create a snapshot. - Delete a snapshot. - Create a share from a snapshot. - Extend a share. - Shrink a share. - Share networks. HPE 3PAR File Persona driver can be configured to work with or without share networks. When using share networks, the HPE 3PAR driver allocates an FSIP on the back end FPG (VFS) to match the share network's subnet and segmentation ID. Security groups associated with share networks are ignored. Operations not supported ~~~~~~~~~~~~~~~~~~~~~~~~ - Manage and unmanage - Manila Experimental APIs (consistency groups, replication, and migration) were added in Mitaka but have not yet been implemented by the HPE 3PAR File Persona driver. Requirements ~~~~~~~~~~~~ On the OpenStack host running the Manila share service: - python-3parclient version 4.2.0 or newer from PyPI. On the HPE 3PAR array: - HPE 3PAR Operating System software version 3.2.1 MU3 or higher. - The array class and hardware configuration must support File Persona. Pre-configuration on the HPE 3PAR StoreServ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The following HPE 3PAR CLI commands show how to set up the HPE 3PAR StoreServ to use File Persona with OpenStack Manila. HPE 3PAR File Persona must be initialized, and started on the HPE 3PAR storage. .. code-block:: console cli% startfs 0:2:1 1:2:1 cli% setfs nodeip -ipaddress 10.10.10.11 -subnet 255.255.240.0 0 cli% setfs nodeip -ipaddress 10.10.10.12 -subnet 255.255.240.0 1 cli% setfs dns 192.168.8.80,127.127.5.50 foo.com,bar.com cli% setfs gw 10.10.10.10 - A File Provisioning Group (FPG) must be created for use with the Shared File Systems service. .. code-block:: console cli% createfpg examplecpg examplefpg 18T - A Virtual File Server (VFS) must be created on the FPG. - The VFS must be configured with an appropriate share export IP address. .. code-block:: console cli% createvfs -fpg examplefpg 10.10.10.101 255.255.0.0 examplevfs - A local user in the Administrators group is needed for CIFS (SMB) shares. .. code-block:: console cli% createfsgroup fsusers cli% createfsuser –passwd -enable true -grplist Users,Administrators –primarygroup fsusers fsadmin - The WSAPI with HTTP and/or HTTPS must be enabled and started. .. code-block:: console cli% setwsapi -https enable cli% startwsapi HPE 3PAR shared file system driver configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Install the python-3parclient python package on the OpenStack Block Storage system: .. code-block:: console $ pip install 'python-3parclient>=4.0,<5.0' - Manila configuration file The Manila configuration file (typically ``/etc/manila/manila.conf``) defines and configures the Manila drivers and backends. After updating the configuration file, the Manila share service must be restarted for changes to take effect. - Enable share protocols To enable share protocols, an optional list of supported protocols can be specified using the ``enabled_share_protocols`` setting in the ``DEFAULT`` section of the ``manila.conf`` file. The default is ``NFS, CIFS`` which allows both protocols supported by HPE 3PAR (NFS and SMB). Where Manila uses the term ``CIFS``, HPE 3PAR uses the term ``SMB``. Use the ``enabled_share_protocols`` option if you want to only provide one type of share (for example, only NFS) or if you want to explicitly avoid the introduction of other protocols that can be added for other drivers in the future. - Enable share back ends In the ``[DEFAULT]`` section of the Manila configuration file, use the ``enabled_share_backends`` option to specify the name of one or more back-end configuration sections to be enabled. To enable multiple back ends, use a comma-separated list. .. note:: The name of the backend's configuration section is used (which may be different from the ``share_backend_name`` value) - Configure each back end For each back end, a configuration section defines the driver and back end options. These include common Manila options, as well as driver-specific options. The following ``Driver options`` section describes the parameters that need to be configured in the Manila configuration file for the HPE 3PAR driver. Driver options ~~~~~~~~~~~~~~ The following table contains the configuration options specific to the share driver. .. include:: ../../tables/manila-hpe3par.inc HPE 3PAR Manila driver configuration example ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The following parameters shows a sample subset of the ``manila.conf`` file, which configures two backends and the relevant ``[DEFAULT]`` options. A real configuration would include additional ``[DEFAULT]`` options and additional sections that are not discussed in this document. In this example, the backends are using different FPGs on the same array: .. code-block:: ini [DEFAULT] enabled_share_backends = HPE1,HPE2 enabled_share_protocols = NFS,CIFS default_share_type = default [HPE1] share_backend_name = HPE3PAR1 share_driver = manila.share.drivers.hpe.hpe_3par_driver.HPE3ParShareDriver driver_handles_share_servers = False max_over_subscription_ratio = 1 hpe3par_fpg = examplefpg,10.10.10.101 hpe3par_san_ip = 10.20.30.40 hpe3par_api_url = https://10.20.30.40:8080/api/v1 hpe3par_username = hpe3par_password = hpe3par_san_login = hpe3par_san_password = hpe3par_debug = False hpe3par_cifs_admin_access_username = hpe3par_cifs_admin_access_password = [HPE2] share_backend_name = HPE3PAR2 share_driver = manila.share.drivers.hpe.hpe_3par_driver.HPE3ParShareDriver driver_handles_share_servers = False max_over_subscription_ratio = 1 hpe3par_fpg = examplefpg2,10.10.10.102 hpe3par_san_ip = 10.20.30.40 hpe3par_api_url = https://10.20.30.40:8080/api/v1 hpe3par_username = hpe3par_password = hpe3par_san_login = hpe3par_san_password = hpe3par_debug = False hpe3par_cifs_admin_access_username = hpe3par_cifs_admin_access_password = Network approach ~~~~~~~~~~~~~~~~ Network connectivity between the storage array (SSH/CLI and WSAPI) and the Manila host is required for share management. Network connectivity between the clients and the VFS is required for mounting and using the shares. This includes: - Routing from the client to the external network. - Assigning the client an external IP address, for example a floating IP. - Configuring the Shared File Systems service host networking properly for IP forwarding. - Configuring the VFS networking properly for client subnets. - Configuring network segmentation, if applicable. In the OpenStack Kilo release, the HPE 3PAR driver did not support share networks. Share access from clients to HPE 3PAR shares required external network access (external to OpenStack) and was set up and configured outside of Manila. In the OpenStack Liberty release, the HPE 3PAR driver could run with or without share networks. The configuration option ``driver_handles_share_servers``( ``True`` or ``False`` ) indicated whether share networks could be used. When set to ``False``, the HPE 3PAR driver behaved as described earlier for Kilo. When set to ``True``, the share network's subnet, segmentation ID and IP address range were used to allocate an FSIP on the HPE 3PAR. There is a limit of four FSIPs per VFS. For clients to communicate with shares via this FSIP, the client must have access to the external network using the subnet and segmentation ID of the share network. For example, the client must be routed to the neutron provider network with external access. The Manila host networking configuration and network switches must support the subnet routing. If the VLAN segmentation ID is used, communication with the share will use the FSIP IP address. Neutron networking is required for HPE 3PAR share network support. Flat and VLAN provider networks are supported, but the HPE 3PAR driver does not support share network security groups. Share access ~~~~~~~~~~~~ A share that is mounted before access is allowed can appear to be an empty read-only share. After granting access, the share must be remounted. - IP access rules are required for NFS. - SMB shares require user access rules. With the proper access rules, share access is not limited to the OpenStack environment. Access rules added via Manila or directly in HPE 3PAR CLI can be used to allow access to clients outside of the stack. The HPE 3PAR VFS/FSIP settings determine the subnets visible for HPE 3PAR share access. - IP access rules To allow IP access to a share in the horizon UI, find the share in the Project|Manage Compute|Shares view. Use the ``Manage Rules`` action to add a rule. Select IP as the access type, and enter the external IP address (for example, the floating IP) of the client in the ``Access to`` box. You can also use the command line to allow IP access to a share in the horizon UI with the command: .. code-block:: console $ manila access-allow ip - User access rules To allow user access to a share in the horizon UI, find the share in the Project|Manage Compute|Shares view. Use the ``Manage Rules`` action to add a rule. Select user as the access type and enter user name in the ``Access to`` box. You can also use the command line to allow user access to a share in the horizon UI with the command: .. code-block:: console $ manila access-allow user The user name must be an HPE 3PAR user. Share access is different from file system permissions, for example, ACLs on files and folders. If a user wants to read a file, the user must have at least read permissions on the share and an ACL that grants him read permissions on the file or folder. Even with full control share access, it does not mean every user can do everything due to the additional restrictions of the folder ACLs. To modify the file or folder ACLs, allow access to an HPE 3PAR File Persona local user that is in the administrator's group and connect to the share using that user's credentials. Then, use the appropriate mechanism to modify the ACL or permissions to allow different access than what is provided by default. .. _Share types: Share types ~~~~~~~~~~~ When creating a share, a share type can be specified to determine where and how the share will be created. If a share type is not specified, the ``default_share_type`` set in the Shared File Systems service configuration file is used. Manila share types are a type or label that can be selected at share creation time in OpenStack. These types can be created either in the ``Admin`` horizon UI or using the command line, as follows: .. code-block:: console $ manila --os-username admin --os-tenant-name demo type-create –is_public false false The ```` is the name of the new share type. False at the end specifies ``driver_handles_share_servers=False``. The ``driver_handles_share_servers`` setting in the share type needs to match the setting configured for the back end in the ``manila.conf`` file. ``is_public`` is used to indicate whether this share type is applicable to all tenants or will be assigned to specific tenants. ``--os-username admin --os-tenant-name demo`` are only needed if your environment variables do not specify the desired user and tenant. For share types that are not public, use Manila ``type-access-add`` to assign the share type to a tenant. - Using share types to require share networks The Shared File Systems service requires that the share type include the ``driver_handles_share_servers`` extra-spec. This ensures that the share is created on a back end that supports the requested ``driver_handles_share_servers`` (share networks) capability. From the Liberty release forward, both ``True`` and ``False`` are supported. The ``driver_handles_share_servers`` setting in the share type must match the setting in the back end configuration. - Using share types to select backends by name Administrators can optionally specify that a particular share type be explicitly associated with a single back end (or group of backends) by including the extra spec share_backend_name to match the name specified within the ``share_backend_name`` option in the back end configuration. When a share type is not selected during share creation, the default share type is used. To prevent creating these shares on any back end, the default share type needs to be specific enough to find appropriate default backends (or to find none if the default should not be used). The following example shows how to set share_backend_name for a share type. .. code-block:: console $ manila --os-username admin --os-tenant-name demo type-key set share_backend_name=HPE3PAR2 - Using share types to select backends with capabilities The HPE 3PAR driver automatically reports capabilities based on the FPG used for each back end. An administrator can create share types with extra specs, which controls share types that can use FPGs with or without specific capabilities. With the OpenStack Liberty release or later, below section shows the extra specs used with the capabilities filter and the HPE 3PAR driver: ``hpe3par_flash_cache`` When the value is set to `` True`` (or `` False``), shares of this type are only created on a back end that uses HPE 3PAR Adaptive Flash Cache. For Adaptive Flash Cache, the HPE 3PAR StoreServ Storage array must meet the following requirements: - Adaptive Flash Cache enabled - Available SSDs - Adaptive Flash Cache must be enabled on the HPE 3PAR StoreServ Storage array. This is done with the following CLI command: .. code-block:: console cli% createflashcache ```` must be in 16 GB increments. For example, the below command creates 128 GB of Flash Cache for each node pair in the array. .. code-block:: console cli% createflashcache 128g - Adaptive Flash Cache must be enabled for the VV set used by an FPG. For example, ``setflashcache vvset:``. The VV set name is the same as the FPG name. .. note:: This setting affects all shares in that FPG (on that back end). ``Dedupe`` When the value is set to `` True`` (or `` False``), shares of this type are only created on a back end that uses deduplication. For HPE 3PAR File Persona, the provisioning type is determined when the FPG is created. Using the ``createfpg –tdvv`` option creates an FPG that supports both dedupe and thin provisioning. The thin deduplication must be enabled to use the tdvv option. ``thin_provisioning`` When the value is set to `` True`` (or `` False``), shares of this type are only created on a back end that uses thin (or full) provisioning. For HPE 3PAR File Persona, the provisioning type is determined when the FPG is created. By default, FPGs are created with thin provisioning. The capacity filter uses the total provisioned space and configured ``max_oversubscription_ratio`` when filtering and weighing backends that use thin provisioning. - Using share types to influence share creation options Scoped extra-specs are used to influence vendor-specific implementation details. Scoped extra-specs use a prefix followed by a colon. For HPE 3PAR, these extra specs have a prefix of hpe3par. The following HPE 3PAR extra-specs are used when creating CIFS (SMB) shares: ``hpe3par:smb_access_based_enum`` ``smb_access_based_enum`` (Access Based Enumeration) specifies if users can see only the files and directories to which they have been allowed access on the shares. Valid values are ``True`` or ``False``. The default is ``False``. ``hpe3par:smb_continuous_avail`` ``smb_continuous_avail`` (Continuous Availability) specifies if continuous availability features of SMB3 should be enabled for this share. Valid values are ``True`` or ``False``. The default is ``True``. ``hpe3par:smb_cache`` ``smb_cache`` specifies client-side caching for offline files. The default value is ``manual``. Valid values are: - ``off`` — the client must not cache any files from this share. The share is configured to disallow caching. - ``manual`` — the client must allow only manual caching for the files open from this share. - ``optimized`` — the client may cache every file that it opens from this share. Also, the client may satisfy the file requests from its local cache. The share is configured to allow automatic caching of programs and documents. - ``auto`` — the client may cache every file that it opens from this share. The share is configured to allow automatic caching of documents. When creating NFS shares, the following HPE 3PAR extra-specs are used: ``hpe3par:nfs_options`` Comma separated list of NFS export options. The NFS export options have the following limitations: ``ro`` and ``rw`` are not allowed (will be determined by the driver) ``no_subtree_check`` and ``fsid`` are not allowed per HPE 3PAR CLI support ``(in)secure`` and ``(no_)root_squash`` are not allowed because the HPE 3PAR driver controls those settings All other NFS options are forwarded to the HPE 3PAR as part of share creation. The HPE 3PAR performs additional validation at share creation time. For details, see the HPE 3PAR CLI help. Implementation characteristics ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Shares from snapshots - When a share is created from a snapshot, the share must be deleted before the snapshot can be deleted. This is enforced by the driver. - A snapshot of an empty share will appear to work correctly, but attempting to create a share from an empty share snapshot may fail with an ``NFS Create export`` error. - HPE 3PAR File Persona snapshots are for an entire File Store. In Manila, they appear as snapshots of shares. A share sub-directory is used to give the appearance of a share snapshot when using ``create share from snapshot`` . - Snapshots - For HPE 3PAR File Persona, snapshots are per File Store and not per share. So, the HPE 3PAR limit of 1024 snapshots per File Store results in a Manila limit of 1024 snapshots per tenant on each back end FPG. - Before deleting a share, you must delete its snapshots. This is enforced by Manila. For HPE 3PAR File Persona, this also kicks off a snapshot reclamation. - Size enforcement Manila users create shares with size limits. HPE 3PAR enforces size limits by using File Store quotas. When using ``hpe3par_fstore_per_share``= ``True``(the non-default setting) there is only one share per File Store, so the size enforcement acts as expected. When using ``hpe3par_fstore_per_share`` = ``False`` (the default), the HPE 3PAR Manila driver uses one File Store for multiple shares. In this case, the size of the File Store limit is set to the cumulative limit of its Manila share sizes. This can allow one tenant share to exceed the limit and affect the space available for the same tenant's other shares. One tenant cannot use another tenant's File Store. - File removal When shares are removed and the ``hpe3par_fstore_per_share``=``False`` setting is used (the default), files may be left behind in the File Store. Prior to Mitaka, removal of obsolete share directories and files that have been stranded would require tools outside of OpenStack/Manila. In Mitaka and later, the driver mounts the File Store to remove the deleted share's subdirectory and files. For SMB/CIFS share, it requires the ``hpe3par_cifs_admin_access_username`` and ``hpe3par_cifs_admin_access_password`` configuration. If the mount and delete cannot be performed, an error is logged and the share is deleted in Manila. Due to the potential space held by leftover files, File Store quotas are not reduced when shares are removed. - Multi-tenancy - Network The ``driver_handles_share_servers`` configuration setting determines whether share networks are supported. When ``driver_handles_share_servers`` is set to ``True``, a share network is required to create a share. The administrator creates share networks with the desired network, subnet, IP range, and segmentation ID. The HPE 3PAR is configured with an FSIP using the same subnet and segmentation ID and an IP address allocated from the neutron network. Using share network-specific IP addresses, subnets, and segmentation IDs give the appearance of better tenant isolation. Shares on an FPG, however, are accessible via any of the FSIPs (subject to access rules). Back end filtering should be used for further separation. - Back end filtering A Manila HPE 3PAR back end configuration refers to a specific array and a specific FPG. With multiple backends and multiple tenants, the scheduler determines where shares will be created. In a scenario where an array or back end needs to be restricted to one or more specific tenants, share types can be used to influence the selection of a back end. For more information on using share types, see `Share types`_ . - Tenant limit The HPE 3PAR driver uses one File Store per tenant per protocol in each configured FPG. When only one back end is configured, this results in a limit of eight tenants (16 if only using one protocol). Use multiple back end configurations to introduce additional FPGs on the same array to increase the tenant limit. When using share networks, an FSIP is created for each share network (when its first share is created on the back end). The HPE 3PAR supports 4 FSIPs per FPG (VFS). One of those 4 FSIPs is reserved for the initial VFS IP, so the share network limit is 48 share networks per node pair.