.. Copyright 2015 Red Hat, Inc. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Ganesha Library =============== The Ganesha Library provides base classes that can be used by drivers to provision shares via NFS (NFSv3 and NFSv4), utilizing the NFS-Ganesha NFS server. Supported operations -------------------- - Allow NFS Share access - Only IP access type is supported. - Deny NFS Share access Supported manila drivers ------------------------ - CephFS driver uses ``ganesha.GaneshaNASHelper2`` library class - GlusterFS driver uses ``ganesha.GaneshaNASHelper`` library class Requirements ------------ - Preferred: `NFS-Ganesha `_ v2.4 or later, which allows dynamic update of access rules. Use with manila's ``ganesha.GaneshaNASHelper2`` class as described later in :ref:`using_ganesha_library`. (or) `NFS-Ganesha `_ v2.5.4 or later that allows dynamic update of access rules, and can make use of highly available Ceph RADOS (distributed object storage) as its shared storage for NFS client recovery data, and exports. Use with Ceph v12.2.2 or later, and ``ganesha.GaneshaNASHelper2`` library class in manila Queens release or later. - For use with limitations documented in :ref:`ganesha_known_issues`: `NFS-Ganesha `_ v2.1 to v2.3. Use with manila's ``ganesha.GaneshaNASHelper`` class as described later in :ref:`using_ganesha_library`. NFS-Ganesha configuration ------------------------- The library has just modest requirements against general NFS-Ganesha (in the following: Ganesha) configuration; a best effort was made to remain agnostic towards it as much as possible. This section describes the few requirements. Note that Ganesha's concept of storage backend modules is called FSAL ("File System Abstraction Layer"). The FSAL the driver intends to leverage needs to be enabled in Ganesha config. Beyond that (with default manila config) the following line is needed to be present in the Ganesha config file (that defaults to /etc/ganesha/ganesha.conf): ``%include /etc/ganesha/export.d/INDEX.conf`` The above paths can be customized through manila configuration as follows: - `ganesha_config_dir` = toplevel directory for Ganesha configuration, defaults to /etc/ganesha - `ganesha_config_path` = location of the Ganesha config file, defaults to ganesha.conf in `ganesha_config_dir` - `ganesha_export_dir` = directory where manila generated config bits are stored, defaults to `export.d` in `ganesha_config_dir`. The following line is required to be included (with value expanded) in the Ganesha config file (at `ganesha_config_path`): ``%include /INDEX.conf`` In versions 2.5.4 or later, Ganesha can store NFS client recovery data in Ceph RADOS, and also read exports stored in Ceph RADOS. These features are useful to make Ganesha server that has access to a Ceph (luminous or later) storage backend, highly available. The Ganesha library class `GaneshaNASHelper2` (in manila Queens or later) allows you to store Ganesha exports directly in a shared storage, RADOS objects, by setting the following manila config options in the driver section: - `ganesha_rados_store_enable` = 'True' to persist Ganesha exports and export counter in Ceph RADOS objects - `ganesha_rados_store_pool_name` = name of the Ceph RADOS pool to store Ganesha exports and export counter objects - `ganesha_rados_export_index` = name of the Ceph RADOS object used to store a list of export RADOS object URLs (defaults to 'ganesha-export-index') Check out the `cephfs_driver` documentation for an example driver section that uses these options. To allow Ganesha to read from RADOS objects add the below code block in ganesha's configuration file, substituting values per your setup. .. code-block:: console # To read exports from RADOS objects RADOS_URLS { ceph_conf = "/etc/ceph/ceph.conf"; userid = "admin"; } # Replace with actual pool name, and export index object %url rados:/// # To store client recovery data in the same RADOS pool NFSv4 { RecoveryBackend = "rados_kv"; } RADOS_KV { ceph_conf = "/etc/ceph/ceph.conf"; userid = "admin"; # Replace with actual pool name pool = ; } For a fresh setup, make sure to create the Ganesha export index object as an empty object before starting the Ganesha server. .. code-block:: console echo | sudo rados -p ${GANESHA_RADOS_STORE_POOL_NAME} put ganesha-export-index - Further Ganesha related manila configuration -------------------------------------------- There are further Ganesha related options in manila (which affect the behavior of Ganesha, but do not affect how to set up the Ganesha service itself). These are: - `ganesha_service_name` = name of the system service representing Ganesha, defaults to ganesha.nfsd - `ganesha_db_path` = location of on-disk database storing permanent Ganesha state, e.g. an export ID counter to generate export IDs for shares (or) When `ganesha_rados_store_enabled` is set to True, the ganesha export counter is stored in a Ceph RADOS object instead of in a SQLite database local to the manila driver. The counter can be optionally configured with, `ganesha_rados_export_counter` = name of the Ceph RADOS object used as the Ganesha export counter (defaults to 'ganesha-export-counter') - `ganesha_export_template_dir` = directory from where Ganesha loads export customizations (cf. "Customizing Ganesha exports"). .. _using_ganesha_library: Using Ganesha Library in drivers -------------------------------- A driver that wants to use the Ganesha Library has to inherit from ``driver.GaneshaMixin``. The driver has to contain a subclass of ``ganesha.GaneshaNASHelper2``, instantiate it along with the driver instance and delegate ``update_access`` method to it (when appropriate, i.e., when ``access_proto`` is NFS). .. note:: You can also subclass ``ganesha.GaneshaNASHelper``. It works with NFS-Ganesha v2.1 to v2.3 that doesn't support dynamic update of exports. To update access rules without having to restart NFS-Ganesha server, the class manipulates exports created per share access rule (rather than per share) introducing limitations documented in :ref:`ganesha_known_issues`. In the following we explain what has to be implemented by the ``ganesha.GaneshaNASHelper2`` subclass (to which we refer as "helper class"). Ganesha exports are described by so-called *Ganesha export blocks* (introduced in the 2.* release series), that is, snippets of Ganesha config specifying key-pair values. The Ganesha Library generates sane default export blocks for the exports it manages, with one thing left blank, the so-called *FSAL subblock*. The helper class has to implement the ``_fsal_hook`` method which returns the FSAL subblock (in Python represented as a dict with string keys and values). It has one mandatory key, ``Name``, to which the value should be the name of the FSAL (eg.: ``{"Name": "CEPH"}``). Further content of it is optional and FSAL specific. Customizing Ganesha exports --------------------------- As noted, the Ganesha Library provides sane general defaults. However, the driver is allowed to: - customize defaults - allow users to customize exports The config format for Ganesha Library is called *export block template*. They are syntactically either Ganesha export blocks, (please consult the Ganesha documentation about the format), or isomorphic JSON (as Ganesha export blocks are by-and-large equivalent to arrayless JSON), with two special placeholders for values: ``@config`` and ``@runtime``. ``@config`` means a value that shall be filled from manila config, and ``@runtime`` means a value that's filled at runtime with dynamic data. As an example, we show the library's defaults in JSON format (also valid Python literal): :: { "EXPORT": { "Export_Id": "@runtime", "Path": "@runtime", "FSAL": { "Name": "@config" }, "Pseudo": "@runtime", "SecType": "sys", "Tag": "@runtime", "CLIENT": { "Clients": "@runtime", "Access_Type": "RW" }, "Squash": "None" } } The Ganesha Library takes these values from *manila/share/drivers/ganesha/conf/00-base-export-template.conf* where the same data is stored in Ganesha conf format (also supplied with comments). For customization, the driver has to extend the ``_default_config_hook`` method as follows: - take the result of the super method (a dict representing an export block template) - set up another export block dict that include your custom values, either by - using a predefined export block dict stored in code - loading a predefined export block from the manila source tree - loading an export block from an user exposed location (to allow user configuration) - merge the two export block dict using the ``ganesha_utils.patch`` method - return the result With respect to *loading export blocks*, that can be done through the utility method ``_load_conf_dir``. Known Restrictions ------------------ - The library does not support network segmented multi-tenancy model but instead works over a flat network, where the tenants share a network. .. _ganesha_known_issues: Known Issues ------------ Following issues concern only users of `ganesha.GaneshaNASHelper` class that works with NFS-Ganesha v2.1 to v2.3. - The export location for shares of a driver that uses the Ganesha Library will be of the format ``:/share-``. However, this is incomplete information, because it pertains only to NFSv3 access, which is partially broken. NFSv4 mounts work well but the actual NFSv4 export paths differ from the above. In detail: - The export location is usable only for NFSv3 mounts. - The export location works only for the first access rule that's added for the given share. Tenants that should be allowed to access according to a further access rule will be refused (cf. https://bugs.launchpad.net/manila/+bug/1513061). - The share is, however, exported through NFSv4, just on paths that differ from the one indicated by the export location, namely at: ``:/share---``, where ```` ranges over the ID-s of access rules of the share (and the export with ```` is accessible according to the access rule of that ID). - NFSv4 access also works with pseudofs. That is, the tenant can do a v4 mount of``:/`` and access the shares allowed for her at the respective ``share---`` subdirectories. Deployment considerations ------------------------- When using `NFS-Ganesha`_ v2.4 or later and manila's ``ganesha.GaneshaNASHelper2`` class, dynamic export of access rules is implemented by using the `dbus-send`_ command to signal `NFS-Ganesha`_ to update its exports. The `dbus-send`_ command is executed on the host where `NFS-Ganesha`_ runs. This may be the same host where the :term:`manila-share` service runs, or it may be remote to :term:`manila-share` depending on how the relevant driver has been configured. Either way, the `dbus-send`_ command and `NFS-Ganesha`_ must be able to communicate over an *abstract socket* and *must be in the same namespace*. Consequently, if you deploy `NFS-Ganesha`_ in a container you likely should run the container in the host namespace (e.g. 'docker run --net=host ...') rather than in its own network namespace. For details, see this `article `_. The :mod:`manila.share.drivers.ganesha` Module ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: manila.share.drivers.ganesha :noindex: :members: :undoc-members: :show-inheritance: .. _NFS-Ganesha: https://github.com/nfs-ganesha/nfs-ganesha/wiki .. _dbus-send: https://dbus.freedesktop.org/doc/dbus-send.1.html