utilities/tools
Eric MacDonald b6c944292e Enhance collect to support collecting from subclouds
This update introduces a new collect command line
option --subcloud (or -sc) to support the collect
of subclouds from the system controller.

This update also

- defaults to a 1 month dated collect
- defaults to parallel collect ; instead of one by one
- introduces an --inline or -in option to collect hosts
  or subclouds one by one ; i.e. the legacy collect mode
  before the parallel collect was introduced.
- adds a check_host_reachable access test to each host
  or subcloud to verify access before trying to collect.
- adds collect --continue support for collecting a large
  number of subclouds when there is not enough scratch
  space to hold them all in one go.
- scale subcloud collect timeout with number of subclouds
- show early initial progress for subcloud collect
- improved speed of subcloud name verification for large
  number of listed subclouds
- improved scratch space management on final tar create

Test Plan:

Orchestrated subcloud(s) collect ; parallel and inline:

PASS: Verify single subcloud collect
PASS: Verify listed subcloud collect ; one and several
PASS: Verify named subcloud collect (-sc -a -p -n <name>)
PASS: Verify all subcloud collect in parallel
PASS: Verify subcloud collect continue option handling

Active controller host(s) collect ; parallel and inline:

PASS: Verify single host collect
PASS: Verify listed host collect  ; one and several
PASS: Verify named collect
PASS: Verify all hosts collect

Misc New Features:

PASS: Verify new defaulted 1 month dated collect
PASS: Verify new --file option for subcloud collect
PASS: Verify collect --clean for local and remote hosts
             and subclouds
PASS: Verify collect tar cleanup on hosts and subclouds
             following collect
PASS: Verify parallel collect early progress with .'s
PASS: Verify subcloud collect continue warning message

Failure Cases:

PASS: Verify subcloud collect with failing dcmanager process
PASS: Verify subcloud collect with no provisioned subclouds
PASS: Verify fault handling surrounding use of new --file option
PASS: Verify partial collect after one or more subcloud collect
             errors or timeouts
PASS: Verify subcloud collect is only accepted on a system controller
PASS: Verify handling of unreachable host or subcloud
PASS: Verify handling of host or subcloud that reboots during collect
PASS: Verify collect of subcloud with a lot of /var/log
PASS: Verify collect handling when remote host or subcloud
             runs out of space
PASS: Verify subcloud collect handling when system controller
             runs out of space
PASS: Verify host collect handling when active controller
             runs out of space
PASS: Verify all report_error case handling for collect subcloud
PASS: Verify subcloud collect timeout on remote subcloud is
             reported as a subcloud timeout
PASS: Verify host or subcloud collect with no valid hosts or
             subclouds found or specified
PASS: Verify collect continue option failure handling

Regression:

PASS: Verify host and subcloud specification options (-a -l … , … )
PASS: Verify --all option overrides --list option
PASS: Verify collect drops duplicate or unknown host/subclouds
PASS: Verify host or subcloud collect clean option behavior
PASS: Verify host or subcloud collect reject with -lt 25% free
             scratch space
PASS: Verify permission and incorrect password error handling
PASS: Verify collect handling for unresponsive host or subcloud
PASS: Verify subcloud collect clean of unresponsive host or subcloud

PASS: Verify handling of 'control c' during collect
PASS: Verify collect logging on all hosts and subclouds
PASS: Verify shellcheck static analysis
PASS: Verify bashate static analysis

Change-Id: Ie76bfc86b1ee5eab83f42b65b643ccdf13ad7580
Story: 2009055
Task: 42836
Signed-off-by: Eric MacDonald <eric.macdonald@windriver.com>
2021-08-18 08:58:25 -04:00
..
collect_filesystem Collect filesystem initial commit 2020-02-21 14:57:37 -06:00
collector Enhance collect to support collecting from subclouds 2021-08-18 08:58:25 -04:00
engtools/hostdata-collectors Add auto-version for remaining stx/utilities packages 2020-12-17 13:20:49 -05:00