Currently, when we poll hardware related metrics, if Keystone(Apache) or
Nova is down, the central agent will print many exception logs such as:
ConnectionError: ('Connection aborted.', error(111, 'ECONNREFUSED'))
with very long stack trace, but they are all same information.
This is because when we poll metrics, we discover the resources first,
if succeed, then we cache it, if fail, we do not cache it. So in such case,
each metrics will fail at discover steps, then log exception each time.
And actually we have wrapped instance_get_all with a log decorator, which
means each metrics will log twice.
So this patch removes duplicate exception messages by adding a try...except
block in hardware discovery, if there is any exception, we just return an
empty list, so agent can record this result in cache. Finally the log file
will only have one full stack trace exception message for all the hardware
metrics.
The disadvantage of such change is that when Keystone and Nova suddenly become
OK, then we will lose some data can possible collect. But from another
perspective, if in a singe interval, some hardware metrics have data but some
not is not a consistent thing to end user, so I think in such case we drop some
part of data (especially it is in start process or unstable status) is acceptable.
Change-Id: I0045a556cde274be8e4ba6c155d3f59b4e0d5b2c
Closes-Bug: #1493057
9b82f68667