10 Commits

Author SHA1 Message Date
Kyle MacLeod
311386a353 Prestaging support for --for-sw-deploy/--for-install
Add USM prestaging support. This commit introduces support for the
--for-sw-deploy and --for-install prestaging options. The --for-install
option is the default, and is the equivalent of the previous release
prestaging behaviour - the subcloud ostree repo is prestaged in the
platform-backup partition. The --for-sw-deploy option is the new
prestaging behaviour for USM major and minor releases. This commit
focuses on prestaging minor release updates.

The --for-sw-deploy option requires synchronization of ostree commits
and /opt/software/metadata contents from the system controller to the
subcloud.

See the header file of
playbookconfig/src/playbooks/roles/prestage/
    sync-software-metadata/tasks/main.yml
for a detailed overview of the algorithm to synchronize the ostree
commits and metadata from the system controller onto the subcloud.

Notes on --release YY.MM and --release YY.MM.nn handling:
- The --release value can either be a major release, in format YY.MM, or
  a specific minor release, in format YY.MM.nn.
- If format is YY.MM, the behaviour is to prestage ALL minor releases
  present on the system controller to the subcloud.
- If format is YY.MM.nn, the behaviour is to prestage ONLY the given
  minor release
    - Note: there is no sanity checking for dependent minor updates.
      If update YY.MM.nn is given but YY.MM.nn-1 is not present, the
      operation is currently allowed. This will be addressed in a future
      commit (see TODO below)
- TODO: Future commit will expand on this behaviour for YY.MM.nn to
  prestage all missing commits up to and including the YY.MM.nn commit

Unit tests for ostree-metadata-sync.sh:
A local BATS unit test script is added to unit test the
ostree-metadata-sync.sh script. The unit tests must be run manually -
there is no zuul support for BATS testing. See the README.md file in the
test directory for information on running the tests for this file.

This commit also handles the metadata synchronization when a subcloud is
installed. In this case, the /opt/software/metadata is synchronized
during the subcloud install, so that when the subcloud pulls from the
system controller feed ostree repo, the subcloud is bootstrapped with
the correct /opt/software/metadata hierarchy.

NOTE: this commit also turns off line-length checking for bashate. We do
not enforce line-length for bash scripts in metal, since it arguably
makes it more difficult to write bash scripts (due to challenges with
quoting, etc.)

Test Cases
PASS
- Verify when --release YY.MM is given, all minor releases are included
  for synchronizaton to the subcloud
- Verify when there is only one release on system controller, the
  synchronization is skipped (there are no updates to synchronize)
- For prestaging a minor release, verify the minor release appears in
  the correct location/state onthe subcloud:

  | System Controller state | Subcloud state | Subcloud result |
  | ----------------------- | -------------- | --------------- |
  | available               | -              | available       |
  | deploying               | -              | available       |
  | deployed                | -              | available       |
  | deployed                | deployed       | deployed        |
  |                         |                |                 |

- Verify prestaging --for-install
    - verify that prestaging works as in previous release:
        - ostree_repo staged in /opt/platform-backup

Story: 2010676
Task: 50325

Signed-off-by: Kyle MacLeod <kyle.macleod@windriver.com>
Change-Id: I4c67f03d0cfcf60e1bf78fc4c80ec18271fc49c1
2024-06-19 00:38:00 +00:00
Al Bailey
6d68a2e26d Update tox.ini to work with tox 4
This change will allow this repo to pass zuul now
that this has merged:
https://review.opendev.org/c/zuul/zuul-jobs/+/866943

Tox 4 deprecated whitelist_externals.
Replace whitelist_externals with allowlist_externals

Partial-Bug: #2000399

Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I879b2d49fca6437dbe33802c85d20394d6bdc60e
2022-12-26 21:52:05 +00:00
Thiago Brito
c53535bffc Refactor backup content
This commit aims to refactor the way we select folders for backup in a
single place, giving the option to remove folders/subfolders from the
backup archive using paths/globs. This will give the opportunity to
provide a more detailed input of what we want add to and exclude from
the backup and reduce the backup archive size to allow for a quicker
backup & restore process.

TEST PLAN
-- CentOS --
PASS backup and restore on SX
PASS add new deployments before taking backup, check if they are present
     and working well after the restore
PASS backup system with stx-openstack installed, restored and checked
     if stx-openstack was working
-- Debian --
PASS backup and restore on SX (local and remote play)
PASS backup and restore on DX
PASS backup system with storage backend configured as Ceph, restore
     without issues
PASS backup system with no storage backend configured, restore with no
     issues
PASS add new deployments before taking backup, check if they are present
     and working well after the restore

Story: 2010117
Task: 45980
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I750be41aaaf8e136ecfaa5442a6d0d304ac9490f
2022-08-19 17:08:16 -03:00
Thiago Brito
a80d03e5af Fix remote play for backup and restore
This change fixes the cases where ansible is looking for vars file on
the AnsibleController instead of the remote StarlingX. Also, this
file adds requirements.txt and uses it for the tox "venv" setup in
order to provide an wasy way for a user to create an environment to
remotely run ansible with the same toolset we have on a StarlingX ISO.

TEST PLAN
PASS remote play for backup
PASS local play for backup
PASS remote play for restore
PASS local play for restore

Closes-Bug: #1986691
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I27ea61428afafbd12434fca6f7bef4526ee01559
2022-08-17 14:51:35 -03:00
Chen, Haochuan Z
4a840d7f64 Enable bandit for ansible-playbooks
Story: 2008929
Task: 42531

Change-Id: Ibc7ba6ec327f87775a61f96c709654b75ce78489
Signed-off-by: Chen, Haochuan Z <haochuan.z.chen@intel.com>
2021-06-09 14:11:38 +08:00
Don Penney
9e3a52f571 Update tox to set ANSIBLE_LOCAL_TEMP
Running tox results in ansible-lint creating a $HOME/.ansible/tmp
directory. This update sets the ANSIBLE_LOCAL_TEMP environment
variable to direct ansible to create this directory under the .tox
work directory, rather than the default.

Change-Id: I26055e071bfa2ecbd2b44ab9d1bda3273d8c67ed
Closes-Bug: 1886865
Signed-off-by: Don Penney <don.penney@windriver.com>
2020-07-08 15:45:35 -04:00
Al Bailey
2850e2267a Add in the ability to call ansible-lint from tox.
The list of all ansible-lint error codes is documented here:
https://docs.ansible.com/ansible-lint/rules/default_rules.html

The failing ansible-lint checks are suppressed by the skip_list
entry in .ansible-lint file.  Those should be fixed in followup
commits by an ansible expert.

The tox target can be invoked locally by: tox -e ansible-lint

The ansible-lint target is automatically included in the linters
tox targer, which is already inoked by zuul.

Story: 2004515
Task: 37144
Change-Id: I9a3e90aebdf6915f3a55b894bda2cd0919b073fd
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2019-10-16 08:31:18 -05:00
Bart Wensley
46be1e020a Enable yamllint for ansible playbook .yml files
Enabling yamllint for the ansible playbook .yml files. Modified
the default yamllint configuration to ignore some minor issues
so yamllint will pass. Made some very basic formatting changes
to the playbook files to address basic warnings/errors and
get the maximum line length to 125 characters.

Change-Id: I3b6adf77025dceeb168222ff154a98178d2bb7aa
Story: 2004695
Task: 34286
Signed-off-by: Bart Wensley <barton.wensley@windriver.com>
2019-06-24 13:27:26 -05:00
Tee Ngo
50f61acab2 Clean up repo import
This picks up a number of changes made to the original ansible-playbooks
repo between copying from starlingx/config and the import review
https://review.opendev.org/#/c/665437/.

Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2019-06-15 14:21:19 -05:00
Scott Little
3376dbf852 Initial zuul / TOX setup
Change-Id: Ie2afd3904203aa5a8feaad650854cc3ff78428b2
Story: 2004695
Task: 33662
Signed-off-by: Scott Little <scott.little@windriver.com>
2019-06-15 14:03:17 -05:00