Add notes on thread dumping and yappi

Copy the bits of the Nodepool Operation and Developer's Guide which
talk about SIGUSR2 usage for producing thread dumps and performing
profiling. While these are technically development-related topics,
they're a useful tool in the admin's troubleshooting belt for filing
informative defect reports.

Change-Id: I07f78be92d1c77ca1e68affd1852de0e9b8630e3
This commit is contained in:
Jeremy Stanley
2020-01-17 19:21:39 +00:00
parent a96a81e0f9
commit 3375209402

View File

@@ -39,3 +39,21 @@ form ``FUNCTION\tTOTAL\tRUNNING\tAVAILABLE_WORKERS``::
zuul:tenant_sql_connection 0 0 1
executor:resume:ze09.openstack.org 0 0 1
...
Thread Dumps and Profiling
==========================
If you send a SIGUSR2 to one of the daemon processes, it will dump a
stack trace for each running thread into its debug log. It is written
under the log bucket ``zuul.stack_dump``. This is useful for tracking
down deadlock or otherwise slow threads::
sudo kill -USR2 `cat /var/run/zuul/executor.pid`
view /var/log/zuul/executor-debug.log +/zuul.stack_dump
When `yappi <https://code.google.com/p/yappi/>`_ (Yet Another Python
Profiler) is available, additional functions' and threads' stats are
emitted as well. The first SIGUSR2 will enable yappi, on the second
SIGUSR2 it dumps the information collected, resets all yappi state and
stops profiling. This is to minimize the impact of yappi on a running
system.