airship/armada
Code Issues Proposed changes

Workaround kubernetes python client deadlock issue

Browse Source 
The kubernetes python client has a bug [1] which results in frequent
deadlocks while being cleaned up, which causes armada to hang at the
end of execution.

This patchset works around that issue by mocking out the associated
thread pools, since they are only needed for async kubernetes api
calls, which armada does not use.

[1]: https://github.com/kubernetes-client/python/issues/411

Change-Id: I71fbfbe355347ae2ddd02ffd26d881368320246b
This commit is contained in:
Sean Eagan 2018-11-26 15:58:39 -06:00
parent a64d435de8
commit 673b1ed4bc
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
armada/handlers/k8s.py
View File

@ -17,7 +17,9 @@ import re
from kubernetes import client
from kubernetes import config
from kubernetes import watch
from kubernetes.client import api_client
from kubernetes.client.rest import ApiException
from unittest.mock import Mock
from oslo_config import cfg
from oslo_log import log as logging
@ -27,6 +29,12 @@ from armada.exceptions import k8s_exceptions as exceptions
CONF = cfg.CONF
LOG = logging.getLogger(__name__)
# TODO: Remove after this bug is fixed and we have uplifted to a fixed version:
# https://github.com/kubernetes-client/python/issues/411
# Avoid creating thread pools in kubernetes api_client.
_dummy_pool = Mock()
api_client.ThreadPool = lambda *args, **kwargs: _dummy_pool
class K8s(object):
'''