There are exceptions in nova pods looking like
ERROR oslo_messaging.rpc.server [-] Exception during message handling: oslo_messaging.rpc.dispatcher.UnsupportedVersion: Endpoint does not support RPC version 1.0. Attempted method: pod_health_probe_method_ignore_errors
ERROR oslo_messaging.rpc.server Traceback (most recent call last):
ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch
ERROR oslo_messaging.rpc.server raise UnsupportedVersion(version, method=method)
ERROR oslo_messaging.rpc.server oslo_messaging.rpc.dispatcher.UnsupportedVersion: Endpoint does not support RPC version 1.0. Attempted method: pod_health_probe_method_ignore_errors
Merged fix from openstack-helm community
Signed-off-by: Martin, Chen <email@example.com>
Each patch included in this commit contains a commit message that
describes the required purpose of the patch.
Signed-off-by: Robert Church <firstname.lastname@example.org>
The keystone-api pod gets stuck in CrashLoopBackOff on
AIO-SX lock/unlock attempt.
When Kubernetes decides to kill the keystone-api pod due to readiness
probe failure or other reasons, it calls the preStop hook immediately
before the container is terminated. This hook starts a graceful
shutdown process which includes removing pid, shared memory segment
and wsgi sock files. If the container is not terminated within the
grace period, a SIGKILL is sent, and the container is forced to shut
When the container was forced to terminate without clean up, the
stale files were left behind. On the restart, the application
detected the file existed, and treated it as configuration failure,
hence the exit. As a result, the pod went into a crash loop.
This update removes any stale files when the pod starts.
Signed-off-by: Tao Liu <email@example.com>
Stale Apache2 pids will prevent Apache2 from starting and will leave
the POD in a crashed state.
Note: the pid file is somewhat confusingly called
/var/run/httpd/httpd.pid and /var/run/apache2 is just a symlink to
This is loosely based off the in-review upstream commit at
Signed-off-by: Chris Friesen <firstname.lastname@example.org>