Log analyzer tools for logs.openstack.org
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Zuul d80e6a55e0 Merge "Add support for Senlin" 2 weeks ago
apache update apache sample file to have the kind of stanza we want 5 years ago
devstack Set DocumentRoot for devstack plugin 3 years ago
doc/source cleaning up index.rst file 5 years ago
etc/os_loganalyze Remove swift support 2 years ago
os_loganalyze Add support for Senlin 1 month ago
.coveragerc Change ignore-errors to ignore_errors 3 years ago
.gitignore Add ignored Editors to .gitignore 4 years ago
.gitreview OpenDev Migration Patch 1 month ago
.mailmap initial cookiecutter import 5 years ago
.testr.conf initial cookiecutter import 5 years ago
.zuul.yaml Move zuul job definition inside the respository 6 months ago
CONTRIBUTING.rst Workflow documentation is now in infra-manual 4 years ago
LICENSE initial cookiecutter import 5 years ago
MANIFEST.in initial cookiecutter import 5 years ago
README.rst Remove swift support 2 years ago
babel.cfg initial cookiecutter import 5 years ago
openstack-common.conf initial cookiecutter import 5 years ago
requirements.txt Adds libmagic on plaforms missing it 6 months ago
setup.cfg Provide a script that starts a simple WSGI server 5 years ago
setup.py Update requirements 3 years ago
test-requirements.txt Bump hacking to the latest version 2 years ago
tox.ini Bump hacking to 0.9.x series 4 years ago

README.rst

os_loganalyze

OpenStack tools for gate log analysis

os_loganalyze is designed as a lightweight wsgi filter for openstack logs, making it easier to interact with them on OpenStack's logs.openstack.org repository. This includes colorizing the logs based on log level severity, having bookmarkable links to timestamps in the logs for easy reference, and being able to filter by log level.

This is implemented as a low level wsgi application which returns a generator so that it can act like a pipeline. Some of our logs are 35 MB uncompressed, so if we used a more advanced framework that required we load the entire data stream into memory, the user response would be very poor. As a pipeline and generator the delay added by this script to the user grabbing the logs is largely not noticeable (< 1s).

  • Free software: Apache 2.0 license

Features

  • Supports text/html or text/plain dynamically based on content negotiation
  • html highlighting based on severity
  • filtering based on severity using the level=XXXX parameter (works in either text/html or text/plain responses
  • linking and highlighting of lines based on timestamp
  • control of max number of lines that will be returned using the limit=XXXX parameter
  • Provides a script named htmlify_server.py that serves htmlified logs over HTTP. To view devstack logs: set SCREEN_LOGDIR=$DEST/logs/screen and LOG_COLOR=false in localrc before running stack.sh, run htmlify_server.py, and point your browser at http://devstack-ip:8000/

Todo

Next steps, roughly in order

  • provide links to logstash for request streams (link well know request ids to logstash queries for them)

Hacking

If you are working on making changes one of the easiest ways to do this is to run the server stack locally to see how your changes look on same data included for the tests.

This can be done with tox -e run, which will use the script designed for devstack locally pointed at the sample data. A url where you can browse the resultant content will be provided on the command line.