StarlingX miscellaneous tools and utilities
Go to file
Eric MacDonald b6c944292e Enhance collect to support collecting from subclouds
This update introduces a new collect command line
option --subcloud (or -sc) to support the collect
of subclouds from the system controller.

This update also

- defaults to a 1 month dated collect
- defaults to parallel collect ; instead of one by one
- introduces an --inline or -in option to collect hosts
  or subclouds one by one ; i.e. the legacy collect mode
  before the parallel collect was introduced.
- adds a check_host_reachable access test to each host
  or subcloud to verify access before trying to collect.
- adds collect --continue support for collecting a large
  number of subclouds when there is not enough scratch
  space to hold them all in one go.
- scale subcloud collect timeout with number of subclouds
- show early initial progress for subcloud collect
- improved speed of subcloud name verification for large
  number of listed subclouds
- improved scratch space management on final tar create

Test Plan:

Orchestrated subcloud(s) collect ; parallel and inline:

PASS: Verify single subcloud collect
PASS: Verify listed subcloud collect ; one and several
PASS: Verify named subcloud collect (-sc -a -p -n <name>)
PASS: Verify all subcloud collect in parallel
PASS: Verify subcloud collect continue option handling

Active controller host(s) collect ; parallel and inline:

PASS: Verify single host collect
PASS: Verify listed host collect  ; one and several
PASS: Verify named collect
PASS: Verify all hosts collect

Misc New Features:

PASS: Verify new defaulted 1 month dated collect
PASS: Verify new --file option for subcloud collect
PASS: Verify collect --clean for local and remote hosts
             and subclouds
PASS: Verify collect tar cleanup on hosts and subclouds
             following collect
PASS: Verify parallel collect early progress with .'s
PASS: Verify subcloud collect continue warning message

Failure Cases:

PASS: Verify subcloud collect with failing dcmanager process
PASS: Verify subcloud collect with no provisioned subclouds
PASS: Verify fault handling surrounding use of new --file option
PASS: Verify partial collect after one or more subcloud collect
             errors or timeouts
PASS: Verify subcloud collect is only accepted on a system controller
PASS: Verify handling of unreachable host or subcloud
PASS: Verify handling of host or subcloud that reboots during collect
PASS: Verify collect of subcloud with a lot of /var/log
PASS: Verify collect handling when remote host or subcloud
             runs out of space
PASS: Verify subcloud collect handling when system controller
             runs out of space
PASS: Verify host collect handling when active controller
             runs out of space
PASS: Verify all report_error case handling for collect subcloud
PASS: Verify subcloud collect timeout on remote subcloud is
             reported as a subcloud timeout
PASS: Verify host or subcloud collect with no valid hosts or
             subclouds found or specified
PASS: Verify collect continue option failure handling

Regression:

PASS: Verify host and subcloud specification options (-a -l … , … )
PASS: Verify --all option overrides --list option
PASS: Verify collect drops duplicate or unknown host/subclouds
PASS: Verify host or subcloud collect clean option behavior
PASS: Verify host or subcloud collect reject with -lt 25% free
             scratch space
PASS: Verify permission and incorrect password error handling
PASS: Verify collect handling for unresponsive host or subcloud
PASS: Verify subcloud collect clean of unresponsive host or subcloud

PASS: Verify handling of 'control c' during collect
PASS: Verify collect logging on all hosts and subclouds
PASS: Verify shellcheck static analysis
PASS: Verify bashate static analysis

Change-Id: Ie76bfc86b1ee5eab83f42b65b643ccdf13ad7580
Story: 2009055
Task: 42836
Signed-off-by: Eric MacDonald <eric.macdonald@windriver.com>
2021-08-18 08:58:25 -04:00
ceph Enable bandit for utilities 2021-06-17 01:45:52 +00:00
security Add auto-version for remaining stx/utilities packages 2020-12-17 13:20:49 -05:00
tools Enhance collect to support collecting from subclouds 2021-08-18 08:58:25 -04:00
utilities Merge "Remove InfluxDB" 2021-07-16 16:47:10 +00:00
.gitignore Collect filesystem initial commit 2020-02-21 14:57:37 -06:00
.gitreview Add a .gitreview file to the new repo 2019-09-09 09:48:42 -05:00
.zuul.yaml Enable bandit for utilities 2021-06-17 01:45:52 +00:00
bindep.txt Turn off legacy resolver workaround in pip 2020-12-17 17:04:41 -06:00
centos_build_layer.cfg Build layering, add layer build config file and tarball lst file 2019-10-15 15:17:50 +08:00
centos_dev_wheels.inc Add python-cephclient stable/dev wheel file to wheel tarball 2019-11-14 11:16:17 +08:00
centos_guest_image_rt.inc Config file changes to add 'ceph/ceph-manager ceph/python-cephclient utilities/nfscheck utilities/logmgmt security/tpm2-openssl-engine security/stx-ssl tools/collector tools/engtools/hostdata-collectors utilities/build-info utilities/namespace-utils utilities/pci-irq-affinity-agent utilities/platform-util utilities/stx-extensions utilities/update-motd ' after relocation from 'stx-integ' 2019-09-04 11:08:27 -04:00
centos_guest_image.inc Config file changes to add 'ceph/ceph-manager ceph/python-cephclient utilities/nfscheck utilities/logmgmt security/tpm2-openssl-engine security/stx-ssl tools/collector tools/engtools/hostdata-collectors utilities/build-info utilities/namespace-utils utilities/pci-irq-affinity-agent utilities/platform-util utilities/stx-extensions utilities/update-motd ' after relocation from 'stx-integ' 2019-09-04 11:08:27 -04:00
centos_iso_image.inc Merge "Remove pm-qos-mgr daemon" 2019-10-25 21:33:23 +00:00
centos_pkg_dirs Remove pm-qos-mgr daemon 2019-10-24 13:23:53 -04:00
centos_stable_docker_images.inc Build image for ceph-manager 2020-12-01 14:15:04 +08:00
centos_stable_wheels.inc Add python-cephclient stable/dev wheel file to wheel tarball 2019-11-14 11:16:17 +08:00
centos_tarball-dl.lst Build layering, add layer build config file and tarball lst file 2019-10-15 15:17:50 +08:00
CONTRIBUTING.rst Adding zuul jobs for new repo 2019-09-09 13:43:49 -05:00
HACKING.rst Adding zuul jobs for new repo 2019-09-09 13:43:49 -05:00
pylint.rc Enable a check in pylint for import-error 2019-09-19 13:49:57 -05:00
requirements.txt Turn off legacy resolver workaround in pip 2020-12-17 17:04:41 -06:00
test-requirements.txt Turn off legacy resolver workaround in pip 2020-12-17 17:04:41 -06:00
tox.ini Enable bandit for utilities 2021-06-17 01:45:52 +00:00