TripleO CI Health logstash and regex queries for logs.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
Wes Hayutin 072f79303e update the README to point to opendev in links 1 week ago
.github/workflows Validate query schema (#6) 4 months ago
build WIP 5 months ago
output Adding tests for sova queries 2 weeks ago
playbooks Adding tests for sova queries 2 weeks ago
samples Adding tests for sova queries 2 weeks ago
src Adding tests for sova queries 2 weeks ago
zuul.d Move zuul jobs to system-config queue 4 weeks ago
.flake8 Add black, isort and flake8 3 weeks ago
.gitignore Adding script to convert queries to er format 3 weeks ago
.gitreview Bootstrap zuul and gerrit config 2 months ago
.pre-commit-config.yaml Add black, isort and flake8 3 weeks ago
.yamllint.yaml Enable pre-commit 4 months ago
LICENSE Initial commit 5 months ago
README.md update the README to point to opendev in links 1 week ago
ansible.cfg Adding script to convert queries to sova format 4 weeks ago
bindep.txt Adding script to convert queries to sova format 4 weeks ago
hosts Adding script to convert queries to sova format 4 weeks ago
requirements.in Adding script to convert queries to er format 3 weeks ago
requirements.txt Adding script to convert queries to er format 3 weeks ago
requirements.yml Adding script to convert queries to sova format 4 weeks ago
tox.ini Adding script to convert queries to er format 3 weeks ago

README.md

queries

Hosts reusable log queries which are built into a single queries.json file.

Query database structure

Queries are defined using the data model from src/model.py which builds a JSON Validation schema, making easy to validate the file.

One example of file can be seen at queries-example.yml

Both elastic-search and artcl can make use of regex searches.

Pattern is supposed to be an exact string match and if multiple are present we could easily convert them into a regex or logstash expression that uses logical AND.

Pattern

On elastic-rechheck queries we have cases with multiple entries used on patterns, like message:foo AND message:bar. This is why we also allow a list of strings.

Categories

A query can have only one category out of a determined list of possible values, currently infra and code are allowed. These can be used to list found matches in section, making them easier to read.

Tags

Tags are also used to build the logstash queries. List of known values already used inside elastic-recheck queries:

tags:
  - console
  - console.html
  - devstack-gate-setup-host.txt
  - grenade.sh.txt
  - job-output.txt
  - screen-c-api.txt
  - screen-c-bak.txt
  - screen-n-cpu.txt
  - screen-n-sch.txt
  - screen-q-agt.txt
  - syslog.txt

When logstash query is build OR is used between multiple tags.

Uncovered cases:

We do not currently support the exclusions like below (2/93 found):

query: >-
  message:"RESULT_TIMED_OUT: [untrusted : git.openstack.org/openstack/tempest/playbooks/devstack-tempest.yaml@master]" AND
  tags:"console" AND NOT
  (build_name:"tempest-all" OR
   build_name:"tempest-slow" OR
   build_name:"tempest-slow-py3")

query2: >-
  (message: "FAILED with status: 137" OR
  message: "FAILED with status: 143" OR
  message: "RUN END RESULT_TIMED_OUT") AND
  NOT message:"POST-RUN END RESULT_TIMED_OUT" AND
  tags: "console"

To allow us to cover for corner cases not covered byt the generic format, we could have an optional logstash key that mentions the query. When this would be present, we woudl avoid building the logstash query ourselves and just use it.

Disable queries per backend

To avoid using a particular query on a particular backend we can make use of skip: ['er', 'artcl'].