Add notes on thread dumping and yappi
Copy the bits of the Nodepool Operation and Developer's Guide which talk about SIGUSR2 usage for producing thread dumps and performing profiling. While these are technically development-related topics, they're a useful tool in the admin's troubleshooting belt for filing informative defect reports. Change-Id: I07f78be92d1c77ca1e68affd1852de0e9b8630e3
This commit is contained in:
@@ -39,3 +39,21 @@ form ``FUNCTION\tTOTAL\tRUNNING\tAVAILABLE_WORKERS``::
|
||||
zuul:tenant_sql_connection 0 0 1
|
||||
executor:resume:ze09.openstack.org 0 0 1
|
||||
...
|
||||
|
||||
Thread Dumps and Profiling
|
||||
==========================
|
||||
|
||||
If you send a SIGUSR2 to one of the daemon processes, it will dump a
|
||||
stack trace for each running thread into its debug log. It is written
|
||||
under the log bucket ``zuul.stack_dump``. This is useful for tracking
|
||||
down deadlock or otherwise slow threads::
|
||||
|
||||
sudo kill -USR2 `cat /var/run/zuul/executor.pid`
|
||||
view /var/log/zuul/executor-debug.log +/zuul.stack_dump
|
||||
|
||||
When `yappi <https://code.google.com/p/yappi/>`_ (Yet Another Python
|
||||
Profiler) is available, additional functions' and threads' stats are
|
||||
emitted as well. The first SIGUSR2 will enable yappi, on the second
|
||||
SIGUSR2 it dumps the information collected, resets all yappi state and
|
||||
stops profiling. This is to minimize the impact of yappi on a running
|
||||
system.
|
||||
|
||||
Reference in New Issue
Block a user