115 Commits

Author SHA1 Message Date
Sreejith Punnapuzha
78a9f2312a change libvirt-bin package
libvirt-bin package have been split into 2 packages,
libvirt-daemon-system and libvirt-clients. this commit changes the
libvirt-bin package to the new packages

Signed-off-by: Sreejith Punnapuzha <Sreejith.Punnapuzha@outlook.com>
Change-Id: I24c2d32d2b10ae86684c4a4ef82c2d994064c24e
2021-01-13 08:47:36 -06:00
Sirajudeen
5ad2a3fd06 Make resources configurable for log collection
* Updated the clustered objects and namespaced objects to
  use a custom list when defined, otherwise fallback to
  the existing default list.
* This will allow other providers to trim down or change the
   resource names for log collection

Change-Id: I4e19e558470186943f66b6e1f69221b85bab739d
2020-12-02 15:47:06 +00:00
Dinesh Garg
396a160271 Allow proxy and container repository customization
Modify following roles to accept the proxy and env
variables from zuul jobs and project manifest so they
can be overwritten to run them on downstream zuul

- airshipctl-build-images
- airshipctl-publish-images

Change-Id: Ied06e60507f125c60ca7b9d47331e967fd69bf14
2020-11-18 15:29:41 +00:00
Roman Gorshunov
5fcc994f51 Add local containers cache support to CI
OpenDev Zuul CI hosts cache of docker containers on
{{ zuul_site_mirror_fqdn }} host. Set up docker on main VM to use it.

This change reduces time needed to run majority of our jobs by 1-3 min.

Change-Id: I6f7c88ff0e45fb48a0fa1369c6dc593f85cb7c3e
2020-11-11 17:18:51 +00:00
Andrii Ostapenko
691fac6115
Gather helmrelease description in describe-kubernetes-objects
Change-Id: Iac3b65e4c98575124df67d0d83bbf7b24bb210c6
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-26 11:26:03 -05:00
Andrii Ostapenko
cd1ca0b787
Fix describe-kubernetes-objects role
According to [0] kubectl get all -A does not include all types (e.g.
configmaps) and is going to be deprecated. Instead of getting all
objects with one kubectl request, doing the same for each object type
separately.

[0] https://github.com/kubernetes/kubectl/issues/151

Change-Id: I624e785b95e0c5697a567251b42855545d771179
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-24 13:15:06 -05:00
Andrii Ostapenko
81efbceea6
Collect description of k8s object to job logs
Change-Id: I15226b9700acf169c100431d5dc7fafe41fbcb87
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-23 11:58:29 -05:00
Dmitry Ukov
d783d71f05 Introduce Replacement Transformer plugin container
Relates-To: #341
Change-Id: I65a5b8cf3bcf94321fc02f240b1d8075bca45c0c
2020-10-16 16:00:56 +04:00
Vladimir Kozhukalov
0dc4ab7491 Replace PrimaryRepositoryName with PhaseRepositoryName
Change-Id: I2276a3b3ea50b8e81dd240815cd00a0a2a2af96e
Relates-To: #356
2020-10-12 15:28:27 +03:00
Ruslan Aliev
d8a6139e81 Add document pull step to gate script runner
Currently we don't test document pull step by rewriting
manifest_directory variable as current source path, which
makes a mess in target_path, manifest_path and related paths
(concatenation of target_path+manifest_path won't return
an existing and proper location). This patch organizes
mentioned variables and enables document pull step which
starts working properly. The latest repo state used as
git source for this command.

Change-Id: I5abce73877441c7529f2f77add79cf410e2226d8
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-25 00:39:28 -05:00
Zuul
06029bc1bc Merge "Allow do gather pod logs for different clusters" 2020-09-14 22:34:36 +00:00
Kostiantyn Kalynovskyi
a5c6b12e7e Allow do gather pod logs for different clusters
Relates-To: #346
Closes: #346

Change-Id: I80d16fdafde3d39cdac65942f49e892297cbfeda
2020-09-10 17:54:16 -05:00
Stanislav Egorov
393b75bc08 Worker node deployment
Change-Id: I1a1c06ee9fa16e2c56374e98f8aaea5da80898cd
2020-09-10 12:11:10 -07:00
Sreejith Punnapuzha
f95da5a56c Enhance target node deployment on gates
* Add script to install clusterctl
  * Add controlplane yamls for target node
  * Add script to perform cluster move
  * Add script to zuul jobs

Closes: #288
Change-Id: Ia6891df9c9b1da333396e76f11332deeb17ab807
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-08-27 11:12:05 -05:00
Zuul
d6c0e4303c Merge "Add multi-site support to gate runner" 2020-08-21 15:45:56 +00:00
Stanislav Egorov
2605cd9159 Gate scripts refactoring
Added flexible boot mode configuration for VMs based on BareMetalHost definition.

Change-Id: I1332ff99e7ec539ac280efda515f736b7c18f459
2020-08-19 14:49:03 -07:00
Drew Walters
0706575ef4 Add multi-site support to gate runner
This change adds multi-site support to the gate runner by leveraging
environment variables used by the deployment scripts. Zuul jobs can
change the site by defining the site variable in the job definition. If
no site name is specified, test-site is used as a default.

Example:

  - job:
      name: gate-script-runner-site-b
      vars:
        site: site-b

This change will allow the treasuremap repository to test multiple sites
using the same job definitions.

Change-Id: I5ed02d2563db2d2b3b1587fcac94ad2bce805ced
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-08-19 16:38:42 +00:00
Alexander Noskov
1fcb6df061 Enable UEFI in addition to legacy bootmode
In order to enable UEFI support we need to configure guests VM to use OVMF
(Open Virtual Machine Firmware)

Closes: #319

Change-Id: I4b1c3b9fa8f1f0fe42ced7ec3e57b9d95dd4a3f5
2020-08-13 13:14:37 -05:00
Nikolay Fedorov
bd6f2f8e32 Add usage MAC for Ephemeral VM deploy
We should use interface naming independent on OS, type, speed, slot etc.
This CR takes an approach that is already used for the target node.
To define VMs, their script uses MAC addresses from manifests.

Relates-To: #285

Change-Id: Iafffd23bc584d2daf3fed5ee301491f447498193
2020-08-05 12:29:39 +04:00
Stanislav Egorov
e20c8ae1fa Preparation for workers deployment
Added air-worker VM
Import kubeconfig and check cluster
Initinfra and capi for target cluster

Change-Id: Ib05f805b428a1eab20d62df67ac97670714eebd0
2020-07-28 18:35:19 +00:00
Ruslan Aliev
d21c02c87b Fix airshipctl-roles-test gate job
Due to incorrect processing IP:port pairs using 'regex_replace' filter,
roles-test gate job fails. This patches fixes all the related errors and
CI gate check itself.

Change-Id: Ifd7f890166195df0af9b34e1ccff59b7abd2d9ab
Relates-To: #305
Closes: #305
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-07-21 20:51:53 -05:00
Stanislav Egorov
f1725f3123 Cleaning for gate scripts
Removed orphaned local scripts because some roles were removed.
Fixed env vars for proper configuration.
Updated documentation accordingly.

Change-Id: I37f0c8d038fb51ddaa57664a65d347056df6f007
2020-07-09 13:16:05 -07:00
Roman Gorshunov
eddd09f0a0 CI: remove redundant Ansiblle-based CI jobs
CI jobs have been rewritten in Bash scripts and seem to be working fine for the
past few weeks. Removing now redundant Ansiblle-based CI jobs.

This would reduce workload we impose in CI infrastructure.

Change-Id: I690405f5401da0beeb4b176ab22cffdfda24a09f
2020-07-08 16:19:56 +02:00
Ruslan Aliev
876c6043ee Replace apt/yum ansible modules, use package instead
apt/yum use is not convenient, package module automatically uses
the underlying OS package manager. Also, some ansible roles currently
use only apt module without yum, therefore patch fixes this bug too.

Change-Id: I5dd49d513d1a791ab51ca6ce6eb1c079542c5624
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-06-29 20:28:49 +00:00
Sreejith Punnapuzha
c64ad2db4f Update kubectl version
This commit updates kubectl version from 1.16.2 to 1.17.4 as part of
gate setup

Change-Id: I88728be65474ee7ce57f243e1356b84c7609a45f
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 14:31:31 -05:00
Sreejith Punnapuzha
c3002cabda Add bash scripts to run airshipctl
* Add playbook to run the scripts
 * Add script to install kubectl
 * Add script to create airshipctl executable
 * Add script to generate test configuration
 * Add script to pull site documents
 * Add script to build ephemeral iso
 * Add script to deploy ephemeral node
 * Add script to deploy control plane

Closes: #223
Closes: #224
Closes: #225
Closes: #226
Closes: #227

Change-Id: Ied63e2a733c3abcc6b1a822624c5ffce92e05a44
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 15:44:06 +00:00
Sreejith Punnapuzha
1108d4d2bf Add python3-setuptools to needed packages
This commit adds python3-setuptools to needed packages so that
sushy-tool installation will not fail

Change-Id: I7b6de0ebf09b7c187061030463327831c7d856f3
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 08:25:13 -05:00
Zuul
953e55f04b Merge "Remove deprecated fields of remoteDirect" 2020-06-12 19:59:49 +00:00
Yasin, Siraj (SY495P)
54e13ceaad Added new command for Image Build.
* command: airshipctl image build
* For now it is just a wrapper for baremetal isogen.
* Removed isogen subcommand from baremetal

Change-Id: I5c1260c4b167e5768eba27a7cd83c9b60ab29184
Relates-To:#251
Closes:#251
2020-06-12 03:01:26 +00:00
Dmitry Ukov
df2fff0acf Add control plane deployment
K8s control plane is deployed by KubeadmControlPlane controller. This
controller creates CAPI machines and infrastructure objects
(Metal3Machine). Metal3Machine objects are created based on a template
which contins host selector label. Control plane label is assigned to a
particular BareMetalHots object defined inside of the shared
kustomization.

Relates-To: #149
Closes: #221
Change-Id: I3be1750aacf9736ece2944045c036f405e404561
2020-06-11 13:44:42 +00:00
Ian Howell
5a187e1810 Remove deprecated fields of remoteDirect
This change removes the deprecated "remoteType", "useProxy", and
"insecure" fields from the remoteDirect field of bootstrapInfo

Change-Id: I64d28b715b81464c871f99ad0348fadc2e9fc5f6
Relates-To: #242
2020-06-10 10:23:41 -05:00
Stanislav Egorov
37bba4e2fc Locked version of sushy-tools
0.9.1 is not working with current implementation of airshipctl
Error below:
  Setting boot device to 'CD'.
  BMC responded '500 INTERNAL SERVER ERROR'. Attempting to unmarshal the raw BMC error response.
  Unable to decode BMC response. "Unable to decode Redfish response.
  Key 'error.@Message.ExtendedInfo.Message' is missing or has unknown format."
  redfish client encountered an error: BMC responded '500 INTERNAL SERVER ERROR'.

0.9.0 works fine

Change-Id: I34907cc42bbe78624e2741a581bc95c2e2f68bea
2020-05-28 15:24:36 -07:00
Dmitry Ukov
a301de202b Add CAPI initialization steps to gating
This change enables execute 'airshipctl cluster init' command
as a part of zuul pipeline

Co-Authored-By: Kostyantyn Kalynovskyi <kkalynovskyi@mirantis.com>
Change-Id: I9d499ae66f8d847759c2e61e5d4415f2c9f3c860
2020-05-20 15:17:20 +04:00
Zuul
66980ed432 Merge "Replace become: true notation" 2020-05-20 04:27:57 +00:00
Ruslan Aliev
76d8716968 Replace become: true notation
Ansible documentation recommends to use yes/no as boolean value,
the last mention of possible using true/yes related to version 2.4 [1].
Starting from version 2.5 only become: yes specified in docs [2][3].
Since we have both variations, this patch aligns become statement to
use only yes value.

[1] https://docs.ansible.com/ansible/2.4/become.html
[2] https://docs.ansible.com/ansible/2.5/user_guide/become.html
[3] https://docs.ansible.com/ansible/latest/user_guide/become.html

Change-Id: I706e6e1b9d706d6bfcc38c1a3d0d7304575946ce
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-05-16 01:48:42 -05:00
Ruslan Aliev
917d6896b6 Remove "cluster initinfra" command
Command "cluster initinfra" should be removed since
"phase apply" was introduced instead.

Change-Id: I263f8988a4456fc03314093a55ee1c1da8048404
Relates-To: #162
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-05-16 01:45:34 -05:00
Alexander Noskov
84f2bc60c9 Fix default Airshipctl config parameters
Problems description:
Once we execute airshipctl config init, the default config in
~/.airship/config doesn't contain default values for:
currentContext: ~
clusters:
  default:
    clusterType:
      target:
        bootstrapInfo: ~
        managementConfiguration: ~
---
This PS set default values for Airshipctl config to use it
out of the box.

Closes: #234

Change-Id: I57449e678f19ab1c0e2212c38b01ea195f62aa44
2020-05-12 16:02:05 -05:00
Zuul
60ece21b96 Merge "Fix system power regressions" 2020-05-12 16:42:31 +00:00
Zuul
528407edc7 Merge "Add pod log collecting role" 2020-05-12 16:10:18 +00:00
Dmitry Ukov
ea79cd6a71 Add pod log collecting role
Closes: #220
Change-Id: I5a4b682f8f4b94fa6827e6d304d07d825811916b
2020-05-12 18:08:03 +04:00
Drew Walters
4017e9c221 Fix system power regressions
A change [0] recently merged that makes additional remote parameters
configurable; however, it changed the default wait value to two seconds
and made the wait value a duration in time between two nanoseconds.

This change restores the default wait value of thirty seconds and makes
the wait value representative of seconds again.

[0] https://review.opendev.org/717812

Change-Id: I9fad30d479419efe95fca7dc78154bfc3ca66697
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-11 20:33:50 +00:00
Zuul
936b5639f5 Merge "Fix image push zuul job" 2020-05-11 16:14:11 +00:00
Zuul
3fb7c774e7 Merge "Reusing apache-server instead of httpd" 2020-05-11 15:39:30 +00:00
Sreejith Punnapuzha
ad71f4704b Fix image push zuul job
This commit changes airshipctl-publish-images role to upload images
to quay.io

Closes: #217
Change-Id: I67b3e07ddc170da6df7c34dce929356449863c10
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-05-08 18:54:39 -05:00
Zuul
f334b45638 Merge "Make remote parameters configurable" 2020-05-08 22:01:18 +00:00
James Gu
301de51467 Fixed document pull issue in gate script 21
Changed systemwide-executable to run in local_src_dir because the script needs
to build airshipctl binary first, before doc pull command can be invoked and populate
the remote_work_dir. Changed the order of the roles to allow systemwide-executable
to run before test-config and pull-docs.

Change-Id: Ia47423d91cdfb2dbbf0259344daab1c7b8cd47f5
Signed-off-by: James Gu <james.gu@att.com>
2020-05-08 18:30:51 +00:00
Drew Walters
f729fdd1a1 Replace venv with tox for documentation builds
OpenStack infra has created Zuul project templates that will enable us
to push our documentation to Read the Docs and the Airship website.
While adding a tox file to this project is not ideal, it appears to be
the only way to leverage these jobs at the moment.

This change adds a tox.ini file to airshipctl to build our documentation
and removes the venv solution. The Airship Working Committee will work
with the OpenStack infra team, if possible, to make the publish jobs
more flexible in the future and remove the tox.ini file.

Change-Id: I274cf69a7c79e0aad1d4c9b1decfaf5630f0b18f
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-08 15:47:41 +00:00
Vladislav Kuzmin
9c1724f6d8 Make remote parameters configurable
Remove constants and define these parameteres
in config file for remote configuration.

Closes: #138

Change-Id: Ib9250b8d39c01ef43a1262b62e4d37215481bc43
2020-05-08 17:23:59 +04:00
Zuul
a9211bf787 Merge "[#112] Add doc pull in deployment scripts" 2020-05-05 14:58:01 +00:00
sai battina
f844596a05 Wait for control pods to be ready before moving to initinfra
Change-Id: I091fe3f712c4bc29d9e0c5f03e234c75523dfa90
2020-05-02 19:21:44 +00:00