You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
tripleo-quickstart-extras/roles/collect-logs
Jenkins a4a890ad2e
Merge "Support exclude list for logs collecting"
6 years ago
..
defaults Merge "Support exclude list for logs collecting" 6 years ago
docs updates to the static template for job doc. 6 years ago
meta Integrate undercloud_user var into oooq-extras 6 years ago
scripts Cleanup and move files for collect-logs 6 years ago
tasks Support exclude list for logs collecting 6 years ago
templates collect-logs: add rdoproject env requirements 6 years ago
README.md Support exclude list for logs collecting 6 years ago
doc-requirements.txt Cleanup and move files for collect-logs 6 years ago

README.md

collect-logs

An Ansible role for aggregating logs from TripleO nodes.

Requirements

This role gathers logs and debug information from a target system and collates them in a designated directory, artcl_collect_dir, on the localhost.

Additionally, the role will convert templated bash scripts, created and used by TripleO-Quickstart during deployment, into rST files. These rST files are combined with static rST files and fed into Sphinx to create user friendly post-build-documentation specific to an original deployment.

Finally, the role optionally handles uploading these logs to a rsync server or to an OpenStack Swift object storage. Logs from Swift can be exposed with os-loganalyze.

Role Variables

  • artcl_collect_list -- A list of files and directories to gather from the target. Directories are collected recursively. Can include joker characters that bash understands. Should be specified as a YaML list, e.g.:
artcl_collect_list:
    - /etc/nova
    - /home/stack/*.log
    - /var/log
  • artcl_exclude_list -- A list of files and directories to remove from the collected logs. First of all we collect everything and then remove all we have in artcl_exclude_list (/etc/puppet/modules by default).

  • artcl_collect_dir -- A local directory where the logs should be gathered, without a trailing slash.

  • artcl_gzip_only: false/true -- When true, gathered files are gzipped one by one in artcl_collect_dir, when false, a tar.gz file will contain all the logs.

  • artcl_gen_docs: false/true -- If true, the role will use build artifacts and Sphinx and produce user friendly documentation (default: false)
  • artcl_docs_source_dir -- a local directory that serves as the Sphinx source directory.
  • artcl_docs_build_dir -- A local directory that serves as the Sphinx build output directory.
  • artcl_create_docs_payload -- Dictionary of lists that direct what and how to construct documentation.
    • included_deployment_scripts -- List of templated bash scripts to be converted to rST files.
    • included_deployment_scripts -- List of static rST files that will be included in the output documentation.
    • table_of_contents -- List that defines the order in which rST files will be laid out in the output documentation.
  • artcl_verify_sphinx_build -- false/true -- If true, verify items defined in artcl_create_docs_payload.table_of_contents exist in sphinx generated index.html (default: false)
artcl_create_docs_payload:
  included_deployment_scripts:
    - undercloud-install
    - undercloud-post-install
  included_static_docs:
    - env-setup-virt
  table_of_contents:
    - env-setup-virt
    - undercloud-install
    - undercloud-post-install
  • artcl_publish: true/false -- If true, the role will attempt to rsync logs to the target specified by artcl_rsync_url. Uses BUILD_URL, BUILD_TAG vars from the environment (set during a Jenkins job run) and requires the next to variables to be set.
  • artcl_txt_rename: false/true -- rename text based file to end in .txt.gz to make upstream log servers display them in the browser instead of offering them to download
  • artcl_publish_timeout: the maximum seconds the role can spend uploading the logs, the default is 1800 (30 minutes)
  • artcl_use_rsync: false/true -- use rsync to upload the logs
  • artcl_rsync_use_daemon: false/true -- use rsync daemon instead of ssh to connect
  • artcl_rsync_url -- rsync target for uploading the logs. The localhost needs to have passwordless authentication to the target or the PROVISIONER_KEY Var specificed in the environment.
  • artcl_use_swift: false/true -- use swift object storage to publish the logs
  • artcl_swift_auth_url -- the OpenStack auth URL for Swift
  • artcl_swift_username -- OpenStack username for Swift
  • artcl_swift_password -- password for the Swift user
  • artcl_swift_tenant_name -- OpenStack tenant name for Swift
  • artcl_swift_container -- the name of the Swift container to use, default is logs
  • artcl_swift_delete_after -- The number of seconds after which Swift will remove the uploaded objects, the default is 2678400 seconds = 31 days.
  • artcl_artifact_url -- a HTTP URL at which the uploaded logs will be accessible after upload.
  • artcl_collect_sosreport -- true/false -- If true, create and collect a sosreport for each host.

Example Playbook

---
- name: Gather logs
  hosts: all:!localhost
  roles:
    - collect-logs

Templated Bash to rST Conversion Notes

Templated bash scripts used during deployment are converted to rST files during the create-docs portion of the role's call. Shell scripts are fed into an awk script and output as restructured text. The awk script has several simple rules:

  1. Only lines between ### ---start_docs and ### ---stop_docs will be parsed.
  2. Lines containing # nodoc will be excluded.
  3. Lines containing ## :: indicate subsequent lines should be formatted as code blocks
  4. Other lines beginning with ## <anything else> will have the prepended ## removed. This is how and where general rST formatting is added.
  5. All other lines, including shell comments, will be indented by four spaces.

Enabling sosreport Collection

sosreport is a unified tool for collecting system logs and other debug information. To enable creation of sosreport(s) with this role, create a custom config (you can use centosci-logs.yml as a template) and ensure that artcl_collect_sosreport: true is set.

License

Apache 2.0

Author Information

RDO-CI Team