openstack/os-loganalyze
Code Issues Proposed changes
RETIRED, log analyzer tools for logs.openstack.org
34 Commits 1 Branch 1 Tag 6.2 MiB
7ffac14837
Go to file
Darragh O'Reilly 7ffac14837 Provide a script that starts a simple WSGI server 
This provides an alternative to using screens in devstack
for reading logs. It works out-of-box, so there is no need
to install and configure something like Apache.

Change-Id: I8811d16cbcb45baaaa4a44b923b813a40714ffa2
2013-12-17 14:15:33 +00:00
apache update apache sample file to have the kind of stanza we want 2013-10-12 07:04:02 -04:00
doc/source initial cookiecutter import 2013-10-11 10:03:03 -04:00
os_loganalyze Provide a script that starts a simple WSGI server 2013-12-17 14:15:33 +00:00
.coveragerc initial cookiecutter import 2013-10-11 10:03:03 -04:00
.gitignore initial cookiecutter import 2013-10-11 10:03:03 -04:00
.gitreview fix pep8 in test 2013-10-12 06:17:54 -04:00
.mailmap initial cookiecutter import 2013-10-11 10:03:03 -04:00
.testr.conf initial cookiecutter import 2013-10-11 10:03:03 -04:00
babel.cfg initial cookiecutter import 2013-10-11 10:03:03 -04:00
CONTRIBUTING.rst updating README to reflect reality 2013-10-12 07:18:30 -04:00
LICENSE initial cookiecutter import 2013-10-11 10:03:03 -04:00
MANIFEST.in initial cookiecutter import 2013-10-11 10:03:03 -04:00
openstack-common.conf initial cookiecutter import 2013-10-11 10:03:03 -04:00
README.rst Provide a script that starts a simple WSGI server 2013-12-17 14:15:33 +00:00
requirements.txt initial cookiecutter import 2013-10-11 10:03:03 -04:00
setup.cfg Provide a script that starts a simple WSGI server 2013-12-17 14:15:33 +00:00
setup.py fix pep8 errors from cookiecutter 2013-10-11 10:04:09 -04:00
test-requirements.txt initial cookiecutter import 2013-10-11 10:03:03 -04:00
tox.ini py27 is good enough for test env for now 2013-10-11 11:06:27 -04:00

README.rst

os_loganalyze

OpenStack tools for gate log analysis

os_loganalyze is designed as a lightweight wsgi filter for openstack logs, making it easier to interact with them on OpenStack's logs.openstack.org repository. This includes colorizing the logs based on log level severity, having bookmarkable links to timestamps in the logs for easy reference, and being able to filter by log level.

This is implemented as a low level wsgi application which returns a generator so that it can act like a pipeline. Some of our logs are 35 MB uncompressed, so if we used a more advanced framework that required we load the entire data stream into memory, the user response would be very poor. As a pipeline and generator the delay added by this script to the user grabbing the logs is largely not noticeable (< 1s).

Features

  • Supports text/html or text/plain dynamically based on content negotiation
  • html highlighting based on severity
  • filtering based on severity using the level=XXXX parameter (works in either text/html or text/plain responses
  • Provides a script named htmlify_server.py that serves htmlified logs over HTTP. To view devstack logs: set SCREEN_LOGDIR=$DEST/logs/screen and LOG_COLOR=false in localrc before running stack.sh, run htmlify_server.py, and point your browser at http://devstack-ip:8000/

Todo

Next steps, roughly in order

  • support swift logs (timestamp linking only, no sevs in swift logs)
  • provide links to logstash for request streams (link well know request ids to logstash queries for them)