Browse Source

CI: Collect a snapshot of network connections

A recent magnum bug (https://storyboard.openstack.org/#!/story/2008494)
when running with uwsgi has yielded an interesting question if Ironic
is orphaning rpc clients or not. Ironic's code is slightly different
but also very similar, and similar bugs have been observed in the past
where the python garbage collection never cleans up the old connection
objects and effectively orphans the connection.

So we should likely at least try to collect some of the information
so we can determine if this is the case in our CI jobs. Hence this
change and attempt to collect that data after running CI.

Change-Id: I4e80c56449c0b3d07b160ae6c933a7a16c63c5c5
changes/81/770481/1
Julia Kreger 9 months ago
parent
commit
d44b3b6fec
  1. 8
      playbooks/ci-workarounds/get_extra_logging.yaml
  2. 5
      playbooks/ci-workarounds/get_tftpd.yaml
  3. 2
      zuul.d/ironic-jobs.yaml

8
playbooks/ci-workarounds/get_extra_logging.yaml

@ -0,0 +1,8 @@
- hosts: all
tasks:
- name: Get tftpd info from journald
shell: "journalctl -t in.tftpd > {{ zuul_output_dir }}/logs/tftpd-journal.txt"
become: yes
- name: Get network connection information for ironic process
shell: "netstat -apn |grep ironic > {{ zuul_output_dir }}/logs/post-job-network-connections.txt"
become: yes

5
playbooks/ci-workarounds/get_tftpd.yaml

@ -1,5 +0,0 @@
- hosts: all
tasks:
- name: Get tftpd info from journald
shell: "journalctl -t in.tftpd > {{ zuul_output_dir }}/logs/tftpd-journal.txt"
become: yes

2
zuul.d/ironic-jobs.yaml

@ -4,7 +4,7 @@
description: Base job for devstack/tempest based ironic jobs.
parent: devstack-tempest
nodeset: openstack-single-node-focal
post-run: playbooks/ci-workarounds/get_tftpd.yaml
post-run: playbooks/ci-workarounds/get_extra_logging.yaml
timeout: 10800
required-projects:
- openstack/ironic

Loading…
Cancel
Save