Allow create_volume() to retry when exception happened

Due to the fact that certain volume back-ends cannot easily report
simple total_capacity_gb/free_capacity_gb for their internal
implementation complexity, scheduler is updated to let those back-ends
who report unclear capacity pass capacity filter, thus there is chance
create_volume() request would fail. In a more general case, when a
volume back-end failed to serve create_volume request for whatever reason
it'd be good that we have a mechanism to 'retry' the request.

So the idea is when volume manager catches the exception from
driver.create_volume() call, it checks if the request is allowed to be
rescheduled (requests that are not: clone volume and create volume from
snapshot while 'snapshot_same_host' option is true), it composes a new
request back to scheduler with additional information to mark this specific
back-end has been tried (so that scheduler may choose to skip this back-end
if needed).  Scheduler is (filter scheduler only, simple and chance
scheduler is not supported) is updated as well so that it only retry
scheduler_max_attempts times.  In order to skip/rule out previously tried
back-ends in next schedule task, a new RetryFilter is added.

Changes:
1) volume RPC API create_volume() is updated with new parameters
to save original request information in case rescheduling is needed.
This bumps volume RPC API to 1.4.

2) implementation of create_volume() method in volume API is
refactored in order to distinguish if a request is allowed to
do reschedule (i.e. currently create volume from source volume
bypasses scheduler, not rescheduling is allowed).

3) add reschedule functionality in create_volume() of volume
manager so that it's able to send the request back to scheduler.

4) add schedule_max_attempts config option in scheduler/driver.py

5) add RetryFitler

6) change scheduler_driver default option to FilterScheduler

Change-Id: Ia46b5eb4dc033d73734b6aea82ada34ba5731075
This commit is contained in:
Zhiteng Huang
2013-01-26 02:18:03 +08:00
parent 275ce2d171
commit d17cc23c64
12 changed files with 480 additions and 76 deletions

View File

@@ -242,8 +242,12 @@ class API(base.Base):
self.volume_rpcapi.create_volume(context,
volume_ref,
volume_ref['host'],
snapshot_id,
image_id)
request_spec=request_spec,
filter_properties=
filter_properties,
allow_reschedule=False,
snapshot_id=snapshot_id,
image_id=image_id)
elif source_volid:
source_volume_ref = self.db.volume_get(context,
source_volid)
@@ -255,18 +259,22 @@ class API(base.Base):
self.volume_rpcapi.create_volume(context,
volume_ref,
volume_ref['host'],
snapshot_id,
image_id,
source_volid)
request_spec=request_spec,
filter_properties=
filter_properties,
allow_reschedule=False,
snapshot_id=snapshot_id,
image_id=image_id,
source_volid=source_volid)
else:
self.scheduler_rpcapi.create_volume(
context,
FLAGS.volume_topic,
volume_id,
snapshot_id,
image_id,
request_spec=request_spec,
filter_properties=filter_properties)
self.scheduler_rpcapi.create_volume(context,
FLAGS.volume_topic,
volume_id,
snapshot_id,
image_id,
request_spec=request_spec,
filter_properties=
filter_properties)
@wrap_check_policy
def delete(self, context, volume, force=False):