The LVM driver was using sudo for one command instead
of proper rootwrap usage. This caused failures when the
user running manila-share didn't have passwordless sudo
access.
Depends-On: Ib1896997f2e7a505b5bf8ec0c9e5ee35942f79a6
Change-Id: I48ea946c2b8de2ebf0e067ef4829b81cabd1464f
Closes-Bug: 1550121
Add admin-only /share-replicas/replica-id/action APIs:
+---------------------+----------------------------------------+
| reset_state | To reset the 'status' attribute of |
| | a replica |
+---------------------+----------------------------------------+
| reset_replica_state | To reset the 'replica_state' attribute |
| | of a replica. |
+---------------------+----------------------------------------+
| force_delete | To delete from the DB and attempt |
| | to remove it from the backend. |
+---------------------+----------------------------------------+
| resync | To attempt update of a replica |
+---------------------+----------------------------------------+
Modify the driver interface for create_replica, delete_replica
and update_replica to hand down more information to the driver
during replica operations.
Implements bp: share-replication-admin-actions
APIImpact
DocImpact
Change-Id: I1352f112f3627af4424eb2e3be428b1d13c1d2d5
This patch provides the scheduler support to filter share
backends matching replication capabilities reported by the
hosts and the replication_type extra_spec provided via the
share_type during share creation.
It also adds wsgi routes, API endpoints and driver entry
routines to support the actions: list, show, create, delete
and promote share replicas. It augments the ShareInstance
DB model with a 'replica_state' attribute and the Share DB
Model with 'replication_type' attribute to support these
workflows.
Replica states are periodically updated from the respective
backends that the replicas are created on.
APIImpact
Impact on existing APIs:
In Microversion 2.11, the /shares APIs return 2 additional
fields during index and show calls for each share: 'has_replicas'
and 'replication_type'. Similarly, the field 'replica_state' is
added to the API response for /share-instances.
Also, deletion of a share that has replicas is forbidden,
returning error code 403.
DocImpact
Co-Authored-By: Alex Meade <mr.alex.meade@gmail.com>
Implements: blueprint manila-share-replication
Change-Id: I10515d55b1291c34777a31d8c6a3a1954f551235
CORS middleware's latent configuration feature, new in 3.0.0,
allows adding headers that apply to all valid origins.
This patch adds headers commonly used in openstack to manila's
paste pipeline, so that operators do not have to be aware of
additional configuration magic to ensure that browsers can talk
to the API.
For more information:
http://docs.openstack.org/developer/oslo.middleware/cors.html#configuration-for-pastedeploy
Change-Id: I7d60cade04f2c9f3ed4517ab26c6151ff25c9531
Reuse code of old LVM share driver.
LVM Driver is 1st party Manila driver with NFS and CIFS support
and no share server support.
LVM Driver doesn't rely on Neutron, Nova, or Cinder. As such, it
is ideal for testing purposes and, after a period of maturation,
it should be useful in production environments.
Move generic driver's helpers to manila/share/drivers/helpers.py
and reuse them in LVM driver.
Implement 'ro' access, user access, extend_share function for
LVM driver.
Implements bp lvm-driver
Change-Id: Ia46c51ed400dbb0f1d87a758fb165ca711ed3d7c
Some upcoming features require more than one export location and
possibility to mark them with specific labels like fast/slow or
rw/ro.
So, implement 'export locations metadata' feature:
- It allows to set any key-value pairs for each export location.
- These key-value pairs can be set only by share manager using
response from various share driver methods.
- Example of update is implemented using Generic driver
"create_instance" method.
- Metadata can be updated for any export location in any place
of share manager where db function "share_export_locations_update"
is called.
- Keys from export location metadata table will be added to 'share' and
'share instances' views as export location attributes.
Also:
- Add new attr 'is_admin_only' for existing export locations model.
If set to True, then only admins will be able to see them. Unless
policy is changed.
- Add APIs for reading export locations by share and share instance IDs.
- Remove 'export_location' and 'export_locations' attrs
from 'share' and 'share instance' views.
- Bump microversion as new APIs implemented.
APIImpact
Implements bp export-location-metadata
Change-Id: I36d1aa8d9302e097ffb08d239cf7a81101d2c1cb
After port of extensions to core API we need to rename their URLs.
So, rename URLs and bump microversion for it.
Make new URLs work only with new microversion(s) 2.7+
and old with old microversions 1.0-2.6.
Add separate API router for v2 API as now we should split v1 and v2
logic.
Also, move updated APIs under v2 directory that will be used by both
API routers - v1 and v2.
List of updated collections is following:
- os-availability-zone -> availability-zones
- os-services -> services
- os-quota-sets -> quota-sets
- os-quota-class-sets -> quota-class-sets
- os-share-manage -> shares/manage
- os-share-unmanage -> shares/%s/action
List of updated member actions is following:
- os-share-unmanage/%(share_id)s/unmanage -> shares/%(share_id)s/action
- types/%(id)s/os-share-type-access -> types/%(id)s/share_type_access
List of updated action names is following:
- os-access_allow -> access_allow
- os-access_deny -> access_deny
- os-access_list -> access_list
- os-reset_status -> reset_status
- os-force_delete -> force_delete
- os-migrate_share -> migrate_share
- os-extend -> extend
- os-shrink -> shrink
List of updated attribute names is following:
- os-share-type-access -> share-type-access
Partially implements bp ext-to-core
Change-Id: I82f00114db985b4b3bf4db0a64191559508ac600
Almost every API method written in Manila performs
a policy check against relevant policies in etc/policy.json.
A method - authorize() was introduced in
manila/api/openstack/wsgi.py
in commit e4428f5e6c324a674c76f81340e1c94a956b4c6c
to perform these policy checks.
This patch refactors the authorize() method as a decorator
for consistency and easy usage in APIs that choose to
implement the policy check prior to any core logic.
Prior unit tests are modified for the APIs impacted through
this change.
The Cgsnapshots controller's policies are corrected for
name consistency without impacting behavior.
Change-Id: Ib34e166ab65513f8f02215c6569175fc9a913244
Closes-Bug: #1515397
Changes:
- Register share type APIs as core API.
- Remove extensions code for share types.
- Leave rename of API 'post' data for future update which
will be done with bump of microversion
after port of all extensions to core API.
Partially implements bp ext-to-core
Change-Id: I334cc0fa8e69c2f9b5226f247149a0b9b748dd60
Changes:
- Register admin actions API as core API.
- Remove extension code for admin actions.
- Leave rename of admin actions API 'post' data for future update which
will be done with bump of microversion
after port of all extensions to core API.
Partially implements bp ext-to-core
Closes-Bug: #1510620
Change-Id: Ic8804cff5fa46003d2991372c6eb2a854b886dec
This adds the CORS support middleware to Manila, allowing a deployer
to optionally configure rules under which a javascript client may
break the single-origin policy and access the API directly.
For Manila, the paste.ini method of deploying the middleware was
chosen, because it needs to be able to annotate error responses
created by other middlewares. If one such middleware throws an
error - such as an auth error from keystone - that error response
must still have CORS headers in order to be readable by the
user agent.
OpenStack CrossProject Spec:
http://specs.openstack.org/openstack/openstack-specs/specs/cors-support.html
Oslo_Middleware Docs:
http://docs.openstack.org/developer/oslo.middleware/cors.html
OpenStack Cloud Admin Guide:
http://docs.openstack.org/admin-guide-cloud/cross_project_cors.html
Change-Id: I9d4b6d0eb380da3eae3867fcebe30fdd5824978d
Changes:
- Register quota and quota class sets API as core API using old links.
- Remove extensions code for quota and quota class sets.
- Leave rename of API urls for future update which will be done with
bump of microversion after port of all extensions to core API.
Partially implements bp ext-to-core
Change-Id: I642e6dc78fd908a4850afc61505dcf34d330acca
Changes:
- Register services API as core API using old link.
- Remove extension code for services.
- Leave rename of API url for future update which will be done with
bump of microversion after port of all extensions to core API.
Partially implements bp ext-to-core
Change-Id: I06005e8339f16c404d3f1ff9917480603801d44f
Changes:
- Register manage/unmanage API extension as core API using old links.
- Remove extensions code.
- Leave rename of API urls for future update which will be done with
bump of microversion after port of all extensions to core API.
Partially implements bp ext-to-core
Change-Id: I773398f13f4b3e4637599cb482a5a2c189958656
Changes:
- Register availability zones API as core API using old link.
- Remove extension code for availability zones.
- Leave rename of API url for future update which will be done with
bump of microversion after port of all extensions to core API.
Partially implements bp ext-to-core
Change-Id: Ifc75ef2d16121634ad12e5e12960c928e4d24b90
Adding GPFS driver back as the third party CI system
for GPFS driver is up and running.
This is a revert of commit 720ec904456594b453cfd220b002bdc8c0d50e08
Change-Id: I86f9b7da21716346f96bae3ce30446589e4c4c37
manila.openstack.common.log.LegacyFormatter no longer exists.
This change removes it from logging_sample.conf.
Change-Id: I67daa0e11458272c0a67bd0754c00902d36e27ef
Closes-Bug: #1499263
This bug, inherited from Cinder, allows a tenant to view share
extra specs using the extra-specs-list CLI command. The Cinder
fix was to check the admin context in the DB layer and filter out
all extra specs for non-admins. This approach doesn't work for
Manila, because some extra specs are required and are effectively
part of the Manila API (DHSS, snapshot_support). So in Manila we
define a set of tenant-visible extra specs and restrict the extra
spec values to that set in the share type view builder. Also, we
add policies for the share type list APIs so that admins can
control access to those if desired.
The separate API to list extra specs already has adequate checking
for non-admin users; the CLI was listing the extra specs returned
by the share type API, which is now filtered as described.
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Change-Id: I9b0a8ddc064c246286f26760b703db6e3e1bcd46
Closes-Bug: #1475285
The Manila community set a deadline of 3 Sep 2015 for all drivers
to have reporting CI systems, and there is no CI system for the
IBM GPFS driver. As agreed, the driver will be removed until such
time that a CI system exists.
Change-Id: I76f238a1a6eb039e4ddc1cb189115ab3e686c56b
To prevent a microversioned client from managing a non-microversioned
Manila server, Manila must update its REST endpoints by adding /v2 for
all microversioned APIs.
This commit does the following:
* Add /v2 to the URL map, connected to all the same /v1 API methods
* Renumber the microversion sequence starting from 2.0
* Update the versions API to reflect v2
* Publish the new endpoint to Keystone in the DevStack plug-in
* Update relevant documentation
* Update Tempest tests for microversions
APIImpact
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Closes-Bug: 1488624
Change-Id: I56a516b5f81914557dd2465746629431cfd6deac
Share Migration allows a share to be migrated from
one host#pool to another host#pool through the
"manila migrate <share> <host#pool>" command. It first
calls the driver to perform it in an optimized way if
possible. If the driver returns that it did not migrate,
it performs a generic migration.
A new field has been added to "shares" table: task_state,
which tracks migration status.
For driver migration, the method migrate_share in driver
base class should be overridden.
For generic migration, drivers may use new config options
to achieve the necessary configuration:
- migration_mounting_backend_ip: If backend has additional
exports IP for admin network, specify it here.
- migration_data_copy_node_ip: IP of entity performing
migration between backends, such as manila node or
data copy service node. This may not apply for
DHSS = true drivers.
- migration_protocol_mount_command: specify mount command
with protocol and additional parameters. Advisable to restrict
protocols per backend. Defaults to "mount -t <share_proto>".
If additional customization is needed, drivers may override
certain methods:
- _mount_share: return the mount command.
- _umount_share: return the umount command.
- _get_access_rule_for_data_copy: return an access rule with
the IP address which will allow the manila node or data copy node
to mount the share after added permission through
allow-access API command.
Change-Id: I8dde892cb7c0180b2b56d8c7d680dfe2320c2ec7
Implements: blueprint share-migration
This patch adds the /consistency-groups and /cgsnapshots
endpoints as well as AdminActions for both.
Partially implements bp manila-consistency-groups
APIImpact
Change-Id: I5fd0d3341766fdba4d92f4a43c3d1186e7a4b38e
Add admin API which allows:
- Get share instances of given share
- Get all share instances
- Get share instance details by id
- Reset share instance state
- Force-delete share instance
Add appropriate unit and tempest tests.
APIImpact
Change-Id: Ie96dfc18b491cb4d9705da4eaca5bc4ce43225ea
Partially-Implements: blueprint share-instances
Rework availability zones support which was inherited from Cinder:
- Add public API extension
- Preserve AZ if creating a share from a snapshot
- Always set AZ in Share API or Share Manager
- Update db schema and create db migration
- Update appropriate unit tests
APIImpact
Partially-Implements: blueprint availability-zones
Change-Id: Iea9fbc3fea5c0128772115c028989121f397e0c5
The OpenStack API working group recommends all projects adopt
Nova-style microversions for versioning their REST APIs. This
commit ports the Nova microversion code and docs to Manila.
With this patch, the API version is bumped to 1.1, and the
versions API (which has always returned horribly outdated
values) is the first API that is versioned. The 1.1 version
of the versions API includes the minimum and current API
version values.
Implements bp: manila-rest-api-microversions
Change-Id: Ifa8e394335a4eb3ad21f53a873530aee241c00e8
The Manila API does not behave properly if it is behind an SSL
termination proxy. If this is the case, the host_url in version
resource are build using http protocol instead of https.
To handle to correct host_url with proxy, first, we should enable
X-Forwarded-Host in proxy so that Manila can get the right client
ip. Second, we should enable X-Forwarded-Proto, with the new SSL
middleware, Manila will get original protocol.
Change-Id: I2dac983481718e6d639453e3f03f41baf8cea56d
Closes-Bug: #1384379
With GlusterFS 3.7.x versions, the delete share operation fails when
deleting the contents of a GlusterFS volume, a share. This is because
two directories are auto-created within a GlusterFS volume when it's
started and GlusterFS refuses to unlink their paths. Fix this issue, by
not trying to remove the two directory paths, but remove their contents
and the rest of the contents of the volume.
Change-Id: I1675bbf593bf578301d6899ee3f9860320080956
Closes-Bug: #1473324
Implement optional API for shrinking shares:
"manila shrink <share-id> <new-size>"
- Implement tenant-facing API for shrinking shares
- Add appropriate unit tests
Partially implements bp share-shrink-api
Change-Id: I48f8e05793a992b11fb180b0b4dbf43bcffb4229
Implement API for extending shares similar to Cinder:
"manila extend <share-id> <new-size>"
- Implement tenant-facing API for extending shares
- Add appropriate unit tests
Partially implements bp share-extend-api
Change-Id: Ic63ecb1c2881ac9c8b59010efe3a37413f18f28d
This patch moves Manila to using olso.middleware, updates it to
use the sizelimit middleware in oslo_middleware namespace.
Change-Id: I10c3cbeb9d43b504f14375df4dac87b323fab5bf
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
New style policy language is much more human readable and the prefered
way to describe policies.
Change-Id: Ib741e2222ad36fa481ca8003387ecc1d9e8231e6
Implements: blueprint switch-to-new-style-policy-lang
The scheduler API 'get_pools' was copied from Cinder and has several
problems. The REST endpoint, /scheduler-stats/get_pools, is not a
REST-appropriate name. The API allows filtering but filters are not
currently supported by the Manila scheduler. The API was added as an
extension, not part of the v1 API as it should be. After restarting
the scheduler service, the API doesn't return any data until a share
has been created (Cinder was used as a model for fixing this one).
This patch fixes all of these issues.
Closes-Bug: #1422042
Change-Id: I7e32efb5390fd4b0a62f644ca5a12dcad14d8a9b
Manage share functionality adds possibility to add existing
shares to manila. On other hand unmanage functionality adds
possibility to remove shares from manila, but without physical
removal from share backend.
Due to high implementation complexity and risks it was
decided to implement manage/unmanage methods only for
driver_manage_share_servers=False driver's and implement
this functionality for other cases in future. Also administrator
can't unmanage shares that were created with a share_server.
- Add manage() and unmanage() methods in admin API
- Add manage_share() and unmanage_share() methods to share rpcapi
and share manager
- Update share rpcapi version to 1.1
- Add manage_existing(), unmanage() methods
to share driver
- Add appropriate unit tests.
Partially implements bp manage-shares
Change-Id: Iedfd85cb6bdfade67c04f62c04756ac192db6b66
Add share_type_access extension which introduces
the ability to manage share type access:
* Share types are public by default
* Private share types can be created by setting
the is_public boolean field to False at creation time.
* Access to a private share type can be controlled
by adding or removing a project from it.
* Private share types without projects are only visible
by users with the admin role/context.
Implementation details and unit tests were mostly adapted
from Cinder and Nova access extensions.
Implements bp private-share-types
Change-Id: I83ee57c6a516b5382d074c0082525ad7feadd59c
An execute call in the ganesha manager module that writes ganesha
export config files is made with 'process_input' argument. This
argument is invalid for processutils's ssh_execute() method which is
used by the execute call to write config files to a remote ganesha
server node. Fix this by not using the 'process_input' argument instead
use other arguments that would still enable the execute call to write
ganesha config files.
Change-Id: I58369468ac7be1de38364a1c1bf6cbf8695b9a1d
Closes-Bug: #1422235
This change introduces pool-aware scheduler to address the need for
supporting multiple pools from one storage controller.
Derived from the Cinder Implementation of Pool-aware Scheduler -
https://review.openstack.org/#/c/98715/ and
https://review.openstack.org/#/c/119938
Implements blueprint: dynamic-storage-pools
Change-Id: I3aee5ed1f96f972f7d40fbd981393559587c1a23
When resetting exports, i.e., remove files present in the ganesha
export directory, during manager service initalization, ignore if the
directory is already empty. This allows ganesha manage service to
initialize even if the ganesha export directory is empty.
Change-Id: Iabe3f36d4f05b0fc451695b261a538b2e87be7a5
Closes-Bug: #1416958
The rootwrap filters for commands 'df' and 'stat', which were part
of the volume.filters before, were recently removed considering those
to be unused. But, GPFS manila driver uses those and fails while
performing certain operations, in their absence.
Hence, adding those back.
Change-Id: Ib736385f15058028b2bc0e38d325d0c256f6dd93
Closes-Bug: #1413544
There is no need to keep rootwrap filters in separate files
with 'volume' and 'share' prefixes.
So, merge it and remove unused filters.
Change-Id: I2fbea254faed5d8fc668cba58211d1df0e0e82f3
Rootwrap supports for a configurable set of trusted
directories to search executables in (exec_dirs),
which defaults to system PATH.
Make the rootwrap filters more portable by removing
the explicit exec_dirs prefix from them. This allows
a binary to be installed in any of the exec_dirs and
still be used by rootwrap, which improves cross-distro
portability.
Change-Id: I9a346c0fcddc7ac8af50f72df6d401b5079e4d58
Closes-Bug: #1405068
Introduce the ganesha share driver helper module
which provides the GaneshaNASHelper class from which
share drivers can derive NFS-Ganesha backed protocol
helpers.
Some utility functions are also added to ease
integration.
Partially implements blueprint gateway-mediated-with-ganesha
Change-Id: I8683ea5eb43d7a8eaf0dfa6af3791782d32b944a
Supporing GPFS exports over kernel NFS and Ganesha NFS
Supporting following Manila functionality:
1. Create, List, Delete Shares
2. Create, List, Delete Share Snapshots
3. Create Share from a Share Snapshot
4. Allow, Deny access to a share based on IP
Supports, local and remote GPFS nodes to the Manila service node,
in a GPFS cluster
Limitation:
1. While using remote GPFS node, with Ganesha NFS, 'gpfs_private_key'
for remote login to the GPFS node must be specified and there must be
a passwordless authentication already setup between the Manila and the
remote GPFS node.
DocImpact
Implements: blueprint gpfs-driver
Change-Id: I6664054ba52d03814cea846cb0d79cd853632814