Perform an implicit smart-reconfigure on startup

If the scheduler is stopped, and new projects are added to an existing
tenant's configuration, then the scheduler is started again, the updated
configuration will not take effect because the scheduler considers the
cached configuration valid.

This may be surprising to users, so perform an implicit smart-reconfiguration
after priming the scheduler to pick up any changes.  This should be a
quick no-op if there are no changes.

Note: the zuul-operator essentially does this during its functional test
due to the sequencing involved.

Change-Id: I125e1da0243efff85eea42d7677ecf58b5dafa31

zuul/scheduler.py

@@ -706,13 +706,23 @@ class Scheduler(threading.Thread):
                     else:
                         self.local_layout_state[tenant_name] = layout_state
                     self.connections.reconfigureDrivers(tenant)
-            # TODO(corvus): This isn't quite accurate; we don't really
-            # know when the last reconfiguration took place.  But we
-            # need to set some value here in order for the cleanup
-            # start thread to know that it can proceed.  We should
-            # store the last reconfiguration times in ZK and use them
-            # here.
-            self.last_reconfigured = int(time.time())
+
+        # TODO(corvus): Consider removing this implicit reconfigure
+        # event with v5.  Currently the expectation is that if you
+        # stop a scheduler, change the tenant config, and start it,
+        # the new tenant config should take effect.  If we change that
+        # expectation with multiple schedulers, we can remove this.
+        event = ReconfigureEvent(smart=True)
+        event.zuul_event_ltime = self.zk_client.getCurrentLtime()
+        self._doReconfigureEvent(event)
+
+        # TODO(corvus): This isn't quite accurate; we don't really
+        # know when the last reconfiguration took place.  But we
+        # need to set some value here in order for the cleanup
+        # start thread to know that it can proceed.  We should
+        # store the last reconfiguration times in ZK and use them
+        # here.
+        self.last_reconfigured = int(time.time())
 
         duration = round(time.monotonic() - start, 3)
         self.log.info("Config priming complete (duration: %s seconds)",