Please refer to the recreation instructions

Note: These logs are only available in jobs that use OVB.

• OpenStack Virtual Baremetal Documentation
• baremetal boot log - all the overcloud nodes should have boot logs associated e.g. baremetal_1, baremetal_2 etc.
• baremetal controller log, a.k.a BMC
• logs collected from overcloud-controller-[0-2] node
• logs collected from the overcloud-novacompute-[0-1] node

Check the base directory for a file called *failure_reason* for automatic failure detection. If no known error has been found the file will be named "No_failure_reason_found".

Most of the undercloud and overcloud deployment log files can be found in undercloud/home/zuul tracebacks and other errors are collected in the following log per node:

• undercloud/var/log/extra/errors.txt.txt.gz - the concatenation of all the errors on any node in a single file

Next check the console log and search for PLAY RECAP. There are sometimes multiple ansible runs in a job, usually the last one is the relevant.
If no PLAY RECAP text is found that usually means an infra failure before Quickstart could even start.
If this is a different Ansible error, that could mean either an infra problem (often has UNREACHABLE in the line) or a bug in Quickstart.

Ask on #tripleo to get help or open a bug on Launchpad. Add the "ci" tag if it's a CI issue and "quickstart" if you suspect that the bug is in Quickstart itself.

Finally try rechecking or asking on #tripleo

The logs contain files showing variables used in the job run.

• undercloud/var/log/extra/dump_variables_vars.json.txt.gz - contains the variables used in the running the actual test
• releases.sh - the output of the script setting release-related variables
• playbook_executions.log - prints out the complete commands, with all expanded arguments, to run the playbooks

Upstream OpenStack Health, Elastic Search and Kibana

• http://status.openstack.org/elastic-recheck/ - A tool to track the impact of known bugs in OpenStack CI
• http://logstash.openstack.org - filter in details in the log files from OpenStack CI
• OpenStack Health - upstream job results by project

Tools that will help you spot a trend in TripleO CI

• Job Exploration - check the job history across upstream clouds
• zuul job filter - zuul's job filter per job
• http://cistatus.tripleo.org - Overall check job status
• http://cistatus.tripleo.org/gates/ - Overall gate job status

Tools to compare one job to another.

• jcomparison - A tool to compare results from one job to another
• log reduce - A tool that uses AI features to reduce the noise in logs and present only what is needed for debug

As a debugging step, a job can be run manually with '-dryrun' appended to the job name. When the "playbook dry run" option is invoked, the playbooks will not execute and collect logs will not run but certain log files, including 'toci_env_args_output.log', which contains the environment variables used in the job, and playbook_executions.log will still be produced in the logs directory for inspection. This option serves to assist with debugging and to test the testing scripts themselves.