Timeout cleanup playbook tasks

This adds timeouts to cleanup playbook tasks so that if they have
problems running the entire job does not stall. We noticed that df in
particular can hang on broken nfs mounts and if that happens the job's
cleanup playbook does not end (even after a couple days).

This change adds the timeouts to the testing playbook. If testing shows
this works well we will add it to the production playbooks too.

Change-Id: Ibc875bf99e6da29e2fffbddae8590ecde06b5c3b
This commit is contained in:
Clark Boylan 2019-11-12 12:27:42 -08:00
parent 932cabb772
commit 16af26d558
1 changed files with 4 additions and 4 deletions

View File

@ -4,13 +4,13 @@
when: not (zuul_success | bool)
block:
- name: get df disk usage
raw: df
raw: timeout -k 5 90 df
- name: get df inode usage
raw: df -i
raw: timeout -k 5 90 df -i
- name: get l2 networking
raw: ip link
raw: timeout -k 5 90 ip link
- name: get l3 networking
raw: ip addr
raw: timeout -k 5 90 ip addr