Manage a pool of nodes for a distributed test infrastructure

Go to file

James E. Blair 6fc61ab201 Format multi-line log entries When tracebacks are sent to a log file or stderr using the default configurations, it can be difficult to work with those logs. Using "grep" to search for messages or time periods will typically only produce a single line of the error. Further, on a busy system the error may be interleaved with other log lines making extraction even harder. To address both of these issues, add a formatter which formats every line of a multi-line log message using the same format specifier. This is quite readable when streaming or viewing a standard log file, and it should cause all log lines to automatically appear in ELK/splunk setups since they will all have the same message format. (Though, those setups are probably better served with a native logging configuration that sends the entire record together.) It may be somewhat difficult if a multi-line log entry is recorded on a busy system with multiple threads. However, some judicious grepping should help with that since all related lineswill have the exact same timestamp (including microseconds). Advanced users/devs could specify a logging format with the thread id if necessary, though I don't think we should do that by default. Since this is implemented as a standard logging formatter, it can be opted-out via logging config files. This enables it by default, which I think is reasonable for new users, but if we decide that we would prefer opt-in, that is a simple change. Sample output (from Zuul): 2021-01-29 10:59:02,004 DEBUG zuul.Scheduler: Run handler awake 2021-01-29 10:59:02,004 ERROR zuul.Scheduler: Exception in run handler: 2021-01-29 10:59:02,004 ERROR zuul.Scheduler: Traceback (most recent call last): 2021-01-29 10:59:02,004 ERROR zuul.Scheduler: File "/home/corvus/git/zuul/zuul/zuul/scheduler.py", line 1254, in run 2021-01-29 10:59:02,004 ERROR zuul.Scheduler: raise Exception("Test") 2021-01-29 10:59:02,004 ERROR zuul.Scheduler: Exception: Test 2021-01-29 10:59:02,005 DEBUG zuul.Scheduler: Run handler sleeping Change-Id: I6d7e7e7a9e19d46a744f9ffac8d532fc6b4bba01		2 years ago
contrib/statsd_exporter	Add statsd_exporter mapping	4 years ago
doc	Merge "Changed informations provided in openshiftpods driver"	2 years ago
etc	Merge "Add systemd drop-in file for CentOS 7"	5 years ago
nodepool	Format multi-line log entries	2 years ago
playbooks	Stop using fedora-30	3 years ago
releasenotes/notes	Format multi-line log entries	2 years ago
roles/nodepool-zuul-functional	nodepool-zuul-functional: switch to editable install	3 years ago
tools	Use rax.dfw mirror for arm64 wheels	3 years ago
.coveragerc	Switch to stestr	5 years ago
.dockerignore	Build images with Dockerfile instead of pbrx	4 years ago
.gitignore	Ignore files produced by tox-cover	5 years ago
.gitreview	OpenDev Migration Patch	4 years ago
.stestr.conf	Switch to stestr	5 years ago
.zuul.yaml	nodepool-functional-k8s: don't run on Xenial	2 years ago
Dockerfile	Dockerfile: add xz-utils	3 years ago
LICENSE	license: remove dos line break	5 years ago
README.rst	Update devstack settings and docs for opendev	4 years ago
TESTING.rst	Add more detail to TESTING.rst	3 years ago
bindep.txt	Stop using fedora-30	3 years ago
requirements.txt	Require diskimage-builder 3.6.0	2 years ago
setup.cfg	Add yappi and objgraph to an extra	3 years ago
setup.py	Bump pbr requirements to >=1.3	8 years ago
test-requirements.txt	Stop using hacking for anything	3 years ago
tox.ini	Merge "Declare support for Python3.8"	3 years ago

README.rst

Nodepool

Nodepool is a system for managing test node resources. It supports launching single-use test nodes from cloud providers as well as managing access to pre-defined pre-existing nodes. Nodepool is part of a suite of tools that form a comprehensive test system, including Zuul.

The latest documentation for Nodepool is published at: https://zuul-ci.org/docs/nodepool/

The latest documentation for Zuul is published at: https://zuul-ci.org/docs/zuul/

Getting Help

There are two Zuul-related mailing lists:

zuul-announce: A low-traffic announcement-only list to which every Zuul operator or power-user should subscribe.
zuul-discuss: General discussion about Zuul, including questions about how to use it, and future development.

You will also find Zuul developers in the #zuul channel on Freenode IRC.

Contributing

To browse the latest code, see: https://opendev.org/zuul/nodepool To clone the latest code, use git clone https://opendev.org/zuul/nodepool

Bugs are handled at: https://storyboard.openstack.org/#!/project/zuul/nodepool

Code reviews are handled by gerrit at https://review.opendev.org

After creating a Gerrit account, use git review to submit patches. Example:

# Do your commits
$ git review
# Enter your username if prompted

Join #zuul on Freenode to discuss development or usage.

License

Nodepool is free software, licensed under the Apache License, version 2.0.

Python Version Support

Nodepool requires Python 3. It does not support Python 2.