From 816a17b924bbcacd3bb052f42c80bbfb22ca9a2f Mon Sep 17 00:00:00 2001 From: Maurice Escher Date: Thu, 11 Feb 2021 16:59:37 +0100 Subject: [PATCH] Add Share Affinity/Anti-Affinity Scheduler Filters Change-Id: I66f4dbdc7b6f9d18b8bb4b6fffdea4f5b0849395 --- specs/xena/affinity-antiaffinity-filter.rst | 248 ++++++++++++++++++++ 1 file changed, 248 insertions(+) create mode 100644 specs/xena/affinity-antiaffinity-filter.rst diff --git a/specs/xena/affinity-antiaffinity-filter.rst b/specs/xena/affinity-antiaffinity-filter.rst new file mode 100644 index 0000000..4d2487e --- /dev/null +++ b/specs/xena/affinity-antiaffinity-filter.rst @@ -0,0 +1,248 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +=========================================== +Affinity and anti-affinity scheduler filter +=========================================== + +https://blueprints.launchpad.net/manila/+spec/affinity-antiaffinity-filter + +Copy of cinder feature +https://blueprints.launchpad.net/cinder/+spec/affinity-antiaffinity-filter + +To add scheduler filter to manila that allows scheduler to make placement +decision based on affinity relationship between existing shares and new +share (the one being scheduled). The affinity relationship here means +the location of shares ('host' of share). + +Problem description +=================== + +Manila has done a good job hiding the details of storage back ends from end +users by using share types. However there are use cases where users who +build their application on top of shares would like to be able to 'choose' +where a share be created on. How can manila provide such capability without +hurting the simplicity we have been keeping? Affinity/anti-affinity is one +of the flexibility we can provide without exposing back end details. + +The term affinity/anti-affinity here is to describe the relationship +between two sets of shares in terms of location. To limit the scope, we +describe one share has affinity with the other one only when they reside in +the same share back end; on the contrary, 'anti-affinity' +relation between two sets of shares simply implies they are on different +manila back ends (hosts). + +This affinity/anti-affinity filter filters manila back end based on hint +specified by end user. The hint expresses the affinity or anti-affinity +relation between new shares and existing share(s). This allows end +users to provide hints like 'please put this share to a place that is +different from where share-XYZ resides in'. + +Use Cases +========= + +DB team builds MySQL master onto one share, they'd prefer to put new +shares for slave DBs to different back ends from where the master DB +resides in, for the sake of high availability. + +Proposed change +=============== + +Add two new filters to Manila - AffinityFilter and AntiAffinityFilter. These +two filters will look at the scheduler hint provided by end users and filter +back ends by checking the 'host' of old and new shares see if a back end meets +the requirement (being on the same back end as existing share or not being on +the same back end(s) as existing share(s)). Shares in different pools located +on the same host are still considered as based on the same host. + +The scheduler_hints (same_host, different_host) have to be saved as a share +metadata as soon as new Share record is created in the database. The shares +referenced in those hints will receive the metadata update too, for instance: + +1) There are three shares exist with: UUID1, UUID2, UUID3. +2) User provisions a new share with UUID4 and requests it with a hint to have + different_host than UUID1, UUID2 and UUID3 shares. +3) Share UUID4 gets this hint stored with all other three shares UUIDs. +4) The metadata of each of three shares is also updated to reflect that they + have anti-affinity to a new share with UUID4. + +The respective share metadata is also automatically updated on share deletion +in the similar fashion. + +This extra metadata should be marked as only "admin_modifiable" to prevent +non-admin users from accidential deletion or modification of the values. + +Hint infomation is needed to allow user to understand how were the shares +scheduled and also to be able to respect the hints in case share is migrated +during its lifetime. + +Alternatives +------------ + +There had been one proposal to allow admin user to directly specify the +back end for new shares. It doesn't really provide similar functionality as +affinity filter because it was admin only and it itself has a few drawbacks +(security concern, for example). + +Affinity is also possible with share groups. However, there are some +limitations of share groups to the use case proposed: + +1) Affinity/Anti-affinity is only required at the time of provisioning the +resource, and not at any other point during its lifetime. Shares created +within share groups today are tied to the share group throughout their +lifetime. +2) Current feature limitations with share groups include the inability to +migrate or replicate shares within a share group. +3) While achieving affinity is possible with share groups,anti-affinity +is not. + +There is an option to have a soft OR semantics for both filters instead of +hard AND. That means, the share creation should succeed if at least 1 +condition has been met. The drawback is that there won't be a clear way +to present this information to the user since the share creation is async. +(When a user "fires and forgets" a creation request with hints where not +all conditions are met). + +Alternative to storing scheduler_hints in share metadata can be extension +of Share DB model for saving the hints there. This approach causes more +modifications. + +Data model impact +----------------- + +None. The hints will be saved as additional share metadata. + +REST API impact +--------------- + +Parameter 'scheduler_hints' added to share create, for example:: + + `{ + "share": { + "scheduler_hints": { + "different_host": [ + "share_uuid_1", + "share_uuid_2" + ], + "same_host": ["share_uuid_3"] + } + } + }` + +In this particular example, the share should be scheduled to the same host +where 'share_uuid_3' is located, yet only if both 'share_uuid_1' and +'share_uuid_2' are not located on that host. + +All uuid-s given will be validated if exist and share creation +will fail if at least one share is not existing or uuid malformed. + +The semantics for both filters is hard AND, meaning that the share won't +get created, unless all placement conditions are satisfied. + +If the desired scheduling combination is not possible (e.g. affinity +with two shares located on two different hosts) - the share creation +should fail. The same applies for anti-affinity filter - if there are +only two hosts and each has a share with a uuid that was given in the +'different_host' hint for a new, third share - creation will fail. + +The scheduler hints should also be respected if a share is migrated +between hosts, however there should be an option to override the hints +manually and force migration even if affinity/anti-affinity conditions +won't be met after the migration (e.g add 'force: true' option). + +Microversion of the API is incremented. + +Security impact +--------------- + +Although this change involves using or parsing user-provided - scheduler hints. +This doesn't put Manila in any more danger as it is now. + + +Notifications impact +-------------------- + +None + +Other end user impact +--------------------- + +None + +Performance Impact +------------------ + +New filters would query DB once per request, it only adds slightly latency +to the system and the latency has nothing to do with the size of the system. + +The share metadata update might take extra time, especially when many shares +are specified in the hints. Yet share creation or deletion is an async process +and this impact should not be noticeable by the end user. + +Other deployer impact +--------------------- + +None + +Developer impact +---------------- + +None + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + Dmitry Galkin (galkindmitrii in Gerrit) + Kiran Pawar (kpdev in Gerrit) + +Work Items +---------- + +1. Filter implementation +2. Add scheduler hints parameter +3. Add hints argument for python-manilaclient +4. Add hints support in manila-ui + +Dependencies +============ + +None + +Testing +======= + +Test against AffinityFilter (Same host): + * Create one share A; + * Create another share B with uuid of A and 'same_host' as hint; + * Checks if B is created on same back end as A; + +Test against AntiAffinityFilter (Different host): + * Create one share A; + * Create another share C with uuid of A and 'different_host' as hint; + * Checks if C is created on different back end as A; + +Documentation Impact +==================== + +Need to document the usage of new filters. + + +References +========== + +Nova has been offering similar feature called SameHostFilter and +DifferentHostFilter since *Diablo*. + +https://github.com/openstack/nova/blob/master/nova/scheduler/filters/affinity_filter.py + +Cinder has been offering similar feature called AffinityFilter and +AntiAffinityFilter since *Juno*. + +https://specs.openstack.org/openstack/cinder-specs/specs/juno/affinity-antiaffinity-filter.html