Make eventlet hub use a monotonic clock

If system time is adjusted first forward and then backward while a
nova service is running (e.g. nova-compute), then there is a high
probability, that periodic tasks will stop for the duration of time
the system clock was adjusted backward.

This was supposed to be fixed by the following patch to oslo.service
https://review.openstack.org/#/c/286838/ , but the order of imports
in unit tests and production code is different, so nova services
end up starting with the default eventlet hub, that does not use a
monotonic clock and, thus, is affected by changes of system time.

Testing this is problematic, as it's a subject of imports order and
is not reproduced in functional or unit tests (oslo_service is always
imported earlier than eventlet hub is initialized, so it just does
"the right thing"). The alternative is to make an assertion when
services start.

Closes-Bug: #1510234

Change-Id: I110cf31ad2a0c74a0cf30ec08bd94d3a56727b39

nova/__init__.py

@@ -27,4 +27,9 @@ import os
 
 os.environ['EVENTLET_NO_GREENDNS'] = 'yes'
 
+# NOTE(rpodolyaka): import oslo_service first, so that it makes eventlet hub
+# use a monotonic clock to avoid issues with drifts of system time (see
+# LP 1510234 for details)
+import oslo_service  # noqa
+
 import eventlet  # noqa

nova/service.py

@@ -95,6 +95,17 @@ def setup_profiler(binary, host):
         LOG.info(_LI("OSProfiler is enabled."))
 
 
+def assert_eventlet_uses_monotonic_clock():
+    import eventlet.hubs as hubs
+    import monotonic
+
+    hub = hubs.get_hub()
+    if hub.clock is not monotonic.monotonic:
+        raise RuntimeError(
+            'eventlet hub is not using a monotonic clock - '
+            'periodic tasks will be affected by drifts of system time.')
+
+
 class Service(service.Service):
     """Service object for binaries running on hosts.
 
@@ -141,6 +152,8 @@ class Service(service.Service):
         This includes starting an RPC service, initializing
         periodic tasks, etc.
         """
+        assert_eventlet_uses_monotonic_clock()
+
         verstr = version.version_string_with_package()
         LOG.info(_LI('Starting %(topic)s node (version %(version)s)'),
                   {'topic': self.topic, 'version': verstr})