Disable timeouts on the log streaming connection
We set a 5 second timeout on our log callback connection from the executor to the job node. This has created problems when there are quiet periods during logging as there may be no data sent on the stream for more than 5 seconds. The timeout is valuable when setting up the initial connection as it quickly catches underlying network problems, but is less useful when streaming the logs after connecting. Address this by disabling the timeout once we are connected. Change-Id: I66ab17c55d54a3ae0e3f4572bed72826315b4b22
This commit is contained in:
@@ -121,6 +121,11 @@ class CallbackModule(default.CallbackModule):
|
||||
while True:
|
||||
try:
|
||||
s = socket.create_connection((ip, LOG_STREAM_PORT), 5)
|
||||
# Disable the socket timeout after we have successfully
|
||||
# connected to accomodate the fact that jobs may not be writing
|
||||
# logs continously. Without this we can easily trip the 5
|
||||
# second timeout.
|
||||
s.settimeout(None)
|
||||
except socket.timeout:
|
||||
self._log(
|
||||
"Timeout exception waiting for the logger. "
|
||||
|
||||
Reference in New Issue
Block a user