Allow scheduler to receive volume stats when starting service

Filter scheduler relies on volume services to report their stats in
order to make decisions - this nature makes scheduler service a bit
special when it starts because it can't really do anything without
knowning stats of avilable volume backend.  So the original
implementation of Filter Scheduler added a hook in init_host() to
ask all volume services to send their stats immediately (volume RPC
call 'publish_service_capabitlies()'), this reduced the chance of a
new volume being set to 'error' state because scheduler has no
knowledge of backends to minimum.

             Scheduler       RPC Exchanges    Volume Service(s)
service start | RPC init                          |
              | init_host()                       |
              | --pub_src_caps()-->|  ----------> | (vol fanout)
              |     <-------------  |<-vol stats--| (sch fanout)

However, commit 65fa80c361 moved
init_host() ahead of service RPC initialization because for volume
services, service initialization should be done before the service
starts accepting RPC requests.  This change unfortunately invalidated
scheduler service's init_host() because that needs to be done *AFTER*
RPC is ready (a queue is bound to scheduler fanout exchange).  The
result is scheduler asks vol service to send update, but it misses
the information it asked for because its queue hasn't been created
and bound to the fanout queue yet.  So scheduler always has to wait
for the next update from volume services to be able to work properly.

             Scheduler       RPC Exchanges    Volume Service(s)
service start | init_host()                       |
              | --pub_src_caps()-->|  ----------> | (vol fanout)
              |   (vol stats lost)  |<-vol stats--| (sch fanout)
              | RPC init                          |

This change adds a new hook to Manager class called init_hook_with_rpc()
to allow services like scheduler to do something once RPC is ready.
This should restore scheduler's behavior before commit
65fa80c361.

Change-Id: If6cf9030eb44c39a06ec501ac5c049d460782481
Partial-bug: #1409012

cinder/manager.py

@@ -86,7 +86,19 @@ class Manager(base.Base, periodic_task.PeriodicTasks):
     def init_host(self):
         """Handle initialization if this is a standalone service.
 
-        Child classes should override this method.
+        A hook point for services to execute tasks before the services are made
+        available (i.e. showing up on RPC and starting to accept RPC calls) to
+        other components.  Child classes should override this method.
+
+        """
+        pass
+
+    def init_host_with_rpc(self):
+        """A hook for service to do jobs after RPC is ready.
+
+        Like init_host(), this method is a hook where services get a chance
+        to execute tasks that *need* RPC. Child classes should override
+        this method.
 
         """
         pass

cinder/scheduler/manager.py

@@ -74,7 +74,7 @@ class SchedulerManager(manager.Manager):
         self.driver = importutils.import_object(scheduler_driver)
         super(SchedulerManager, self).__init__(*args, **kwargs)
 
-    def init_host(self):
+    def init_host_with_rpc(self):
         ctxt = context.get_admin_context()
         self.request_service_capabilities(ctxt)
 

cinder/service.py

@@ -159,6 +159,8 @@ class Service(service.Service):
         self.rpcserver = rpc.get_server(target, endpoints, serializer)
         self.rpcserver.start()
 
+        self.manager.init_host_with_rpc()
+
         if self.report_interval:
             pulse = loopingcall.FixedIntervalLoopingCall(
                 self.report_state)