Add USM prestaging support. This commit introduces support for the
--for-sw-deploy and --for-install prestaging options. The --for-install
option is the default, and is the equivalent of the previous release
prestaging behaviour - the subcloud ostree repo is prestaged in the
platform-backup partition. The --for-sw-deploy option is the new
prestaging behaviour for USM major and minor releases. This commit
focuses on prestaging minor release updates.
The --for-sw-deploy option requires synchronization of ostree commits
and /opt/software/metadata contents from the system controller to the
subcloud.
See the header file of
playbookconfig/src/playbooks/roles/prestage/
sync-software-metadata/tasks/main.yml
for a detailed overview of the algorithm to synchronize the ostree
commits and metadata from the system controller onto the subcloud.
Notes on --release YY.MM and --release YY.MM.nn handling:
- The --release value can either be a major release, in format YY.MM, or
a specific minor release, in format YY.MM.nn.
- If format is YY.MM, the behaviour is to prestage ALL minor releases
present on the system controller to the subcloud.
- If format is YY.MM.nn, the behaviour is to prestage ONLY the given
minor release
- Note: there is no sanity checking for dependent minor updates.
If update YY.MM.nn is given but YY.MM.nn-1 is not present, the
operation is currently allowed. This will be addressed in a future
commit (see TODO below)
- TODO: Future commit will expand on this behaviour for YY.MM.nn to
prestage all missing commits up to and including the YY.MM.nn commit
Unit tests for ostree-metadata-sync.sh:
A local BATS unit test script is added to unit test the
ostree-metadata-sync.sh script. The unit tests must be run manually -
there is no zuul support for BATS testing. See the README.md file in the
test directory for information on running the tests for this file.
This commit also handles the metadata synchronization when a subcloud is
installed. In this case, the /opt/software/metadata is synchronized
during the subcloud install, so that when the subcloud pulls from the
system controller feed ostree repo, the subcloud is bootstrapped with
the correct /opt/software/metadata hierarchy.
NOTE: this commit also turns off line-length checking for bashate. We do
not enforce line-length for bash scripts in metal, since it arguably
makes it more difficult to write bash scripts (due to challenges with
quoting, etc.)
Test Cases
PASS
- Verify when --release YY.MM is given, all minor releases are included
for synchronizaton to the subcloud
- Verify when there is only one release on system controller, the
synchronization is skipped (there are no updates to synchronize)
- For prestaging a minor release, verify the minor release appears in
the correct location/state onthe subcloud:
| System Controller state | Subcloud state | Subcloud result |
| ----------------------- | -------------- | --------------- |
| available | - | available |
| deploying | - | available |
| deployed | - | available |
| deployed | deployed | deployed |
| | | |
- Verify prestaging --for-install
- verify that prestaging works as in previous release:
- ostree_repo staged in /opt/platform-backup
Story: 2010676
Task: 50325
Signed-off-by: Kyle MacLeod <kyle.macleod@windriver.com>
Change-Id: I4c67f03d0cfcf60e1bf78fc4c80ec18271fc49c1
This change will allow this repo to pass zuul now
that this has merged:
https://review.opendev.org/c/zuul/zuul-jobs/+/866943
Tox 4 deprecated whitelist_externals.
Replace whitelist_externals with allowlist_externals
Partial-Bug: #2000399
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I879b2d49fca6437dbe33802c85d20394d6bdc60e
This commit aims to refactor the way we select folders for backup in a
single place, giving the option to remove folders/subfolders from the
backup archive using paths/globs. This will give the opportunity to
provide a more detailed input of what we want add to and exclude from
the backup and reduce the backup archive size to allow for a quicker
backup & restore process.
TEST PLAN
-- CentOS --
PASS backup and restore on SX
PASS add new deployments before taking backup, check if they are present
and working well after the restore
PASS backup system with stx-openstack installed, restored and checked
if stx-openstack was working
-- Debian --
PASS backup and restore on SX (local and remote play)
PASS backup and restore on DX
PASS backup system with storage backend configured as Ceph, restore
without issues
PASS backup system with no storage backend configured, restore with no
issues
PASS add new deployments before taking backup, check if they are present
and working well after the restore
Story: 2010117
Task: 45980
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I750be41aaaf8e136ecfaa5442a6d0d304ac9490f
This change fixes the cases where ansible is looking for vars file on
the AnsibleController instead of the remote StarlingX. Also, this
file adds requirements.txt and uses it for the tox "venv" setup in
order to provide an wasy way for a user to create an environment to
remotely run ansible with the same toolset we have on a StarlingX ISO.
TEST PLAN
PASS remote play for backup
PASS local play for backup
PASS remote play for restore
PASS local play for restore
Closes-Bug: #1986691
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I27ea61428afafbd12434fca6f7bef4526ee01559
Running tox results in ansible-lint creating a $HOME/.ansible/tmp
directory. This update sets the ANSIBLE_LOCAL_TEMP environment
variable to direct ansible to create this directory under the .tox
work directory, rather than the default.
Change-Id: I26055e071bfa2ecbd2b44ab9d1bda3273d8c67ed
Closes-Bug: 1886865
Signed-off-by: Don Penney <don.penney@windriver.com>
The list of all ansible-lint error codes is documented here:
https://docs.ansible.com/ansible-lint/rules/default_rules.html
The failing ansible-lint checks are suppressed by the skip_list
entry in .ansible-lint file. Those should be fixed in followup
commits by an ansible expert.
The tox target can be invoked locally by: tox -e ansible-lint
The ansible-lint target is automatically included in the linters
tox targer, which is already inoked by zuul.
Story: 2004515
Task: 37144
Change-Id: I9a3e90aebdf6915f3a55b894bda2cd0919b073fd
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
Enabling yamllint for the ansible playbook .yml files. Modified
the default yamllint configuration to ignore some minor issues
so yamllint will pass. Made some very basic formatting changes
to the playbook files to address basic warnings/errors and
get the maximum line length to 125 characters.
Change-Id: I3b6adf77025dceeb168222ff154a98178d2bb7aa
Story: 2004695
Task: 34286
Signed-off-by: Bart Wensley <barton.wensley@windriver.com>
This picks up a number of changes made to the original ansible-playbooks
repo between copying from starlingx/config and the import review
https://review.opendev.org/#/c/665437/.
Signed-off-by: Dean Troyer <dtroyer@gmail.com>