124 Commits

Author SHA1 Message Date
Shturm, Svetlana
dd9a904fcd Use Zuul project variable for build images
Change-Id: Ie3b5ca9f17d3f908e7771dd5a6a6b09b9ee409c8
2021-04-15 21:26:41 +00:00
Roman Gorshunov
7998615a7b Add base images override support
Go builder and base images override support for downstream
deployment customizations for both airshipctl and plugins.
If required, adjust the following Zuul job variables:
 - docker.base_go_image
 - docker.base_release_image
 - docker.base_plugins_build_image
 - docker.base_plugins_release_image

Closes: #515
Change-Id: Iaf4b54a353207b06c9ed6bdcae876537e73f6e44
2021-04-12 19:47:31 +02:00
Andrii Ostapenko
64e96269ea Add ability to override airship images prefix
Change-Id: I13ccca42773246bf0ef6803d3bd1bd9cb44b88e0
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2021-04-05 21:26:41 +00:00
James Gu
1342ab708f Enhanced deployment script to support baremetal multi-node
Additionally aligned between Treasuremap and Airchipctl to use the same
environemnt variable "SITE" for a site name.

Change-Id: I50494b50691a40efe68996cd11ccede6517b23d6
Signed-off-by: James Gu <james.gu@att.com>
2021-03-23 06:21:31 -07:00
Zuul
63a5d1146c Merge "Copy only current logs from apache2" 2021-03-04 12:29:16 +00:00
siraj.yasin
bf588f993a Copy only current logs from apache2
* `cp -r /var/log/apache2` copies all the archive(*.gz) logs also.
   So just copy only the current logs
    access.log, error.log, *vhost.log

Change-Id: I2ec55b3da61eb258337f0c0023eb7ebcd9ce1913
2021-03-03 08:54:08 +00:00
Andrii Ostapenko
e53f52549a Optimize pod logs and cluster object description gathering
Don't gather logs for unavailable cluster that is always the case for
ephemeral cluster in successful deployments. Cheap ~5 mins on each
deployment.

Change-Id: I32f84e9dadbad11fe92337cd0dc04870d1a0ff3e
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2021-02-25 14:46:27 +00:00
Matt McEuen
0398c974ed Update sushy-tools to latest
This migrates away from a custom fork of sushy-tools, which worked
around the booting issue described here:
https://storyboard.openstack.org/#!/story/2007689

This issue has been fixed, according to the following issue, so
this patchset migrates us back to upstream sushy-tools.
https://review.opendev.org/c/openstack/sushy-tools/+/733152

Change-Id: I421c9d267640720571ac6fa2341af5102afbafb2
Closes: #467
2021-02-11 17:49:42 -06:00
Sirajudeen
e732a58f03 Added tags to control execution of tasks
* Added tag to zuul specific tasks, so that it can be skipped
  when running the playbooks in local setup or in a 3rd party gate

Change-Id: If512eac4e0c5889a48e4aa98e357ca2ed1feefd8
Relates-To: #449
2021-01-22 19:42:39 +00:00
Sreejith Punnapuzha
78a9f2312a change libvirt-bin package
libvirt-bin package have been split into 2 packages,
libvirt-daemon-system and libvirt-clients. this commit changes the
libvirt-bin package to the new packages

Signed-off-by: Sreejith Punnapuzha <Sreejith.Punnapuzha@outlook.com>
Change-Id: I24c2d32d2b10ae86684c4a4ef82c2d994064c24e
2021-01-13 08:47:36 -06:00
Sirajudeen
5ad2a3fd06 Make resources configurable for log collection
* Updated the clustered objects and namespaced objects to
  use a custom list when defined, otherwise fallback to
  the existing default list.
* This will allow other providers to trim down or change the
   resource names for log collection

Change-Id: I4e19e558470186943f66b6e1f69221b85bab739d
2020-12-02 15:47:06 +00:00
Dinesh Garg
396a160271 Allow proxy and container repository customization
Modify following roles to accept the proxy and env
variables from zuul jobs and project manifest so they
can be overwritten to run them on downstream zuul

- airshipctl-build-images
- airshipctl-publish-images

Change-Id: Ied06e60507f125c60ca7b9d47331e967fd69bf14
2020-11-18 15:29:41 +00:00
Roman Gorshunov
5fcc994f51 Add local containers cache support to CI
OpenDev Zuul CI hosts cache of docker containers on
{{ zuul_site_mirror_fqdn }} host. Set up docker on main VM to use it.

This change reduces time needed to run majority of our jobs by 1-3 min.

Change-Id: I6f7c88ff0e45fb48a0fa1369c6dc593f85cb7c3e
2020-11-11 17:18:51 +00:00
Andrii Ostapenko
691fac6115
Gather helmrelease description in describe-kubernetes-objects
Change-Id: Iac3b65e4c98575124df67d0d83bbf7b24bb210c6
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-26 11:26:03 -05:00
Andrii Ostapenko
cd1ca0b787
Fix describe-kubernetes-objects role
According to [0] kubectl get all -A does not include all types (e.g.
configmaps) and is going to be deprecated. Instead of getting all
objects with one kubectl request, doing the same for each object type
separately.

[0] https://github.com/kubernetes/kubectl/issues/151

Change-Id: I624e785b95e0c5697a567251b42855545d771179
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-24 13:15:06 -05:00
Andrii Ostapenko
81efbceea6
Collect description of k8s object to job logs
Change-Id: I15226b9700acf169c100431d5dc7fafe41fbcb87
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-23 11:58:29 -05:00
Dmitry Ukov
d783d71f05 Introduce Replacement Transformer plugin container
Relates-To: #341
Change-Id: I65a5b8cf3bcf94321fc02f240b1d8075bca45c0c
2020-10-16 16:00:56 +04:00
Vladimir Kozhukalov
0dc4ab7491 Replace PrimaryRepositoryName with PhaseRepositoryName
Change-Id: I2276a3b3ea50b8e81dd240815cd00a0a2a2af96e
Relates-To: #356
2020-10-12 15:28:27 +03:00
Ruslan Aliev
d8a6139e81 Add document pull step to gate script runner
Currently we don't test document pull step by rewriting
manifest_directory variable as current source path, which
makes a mess in target_path, manifest_path and related paths
(concatenation of target_path+manifest_path won't return
an existing and proper location). This patch organizes
mentioned variables and enables document pull step which
starts working properly. The latest repo state used as
git source for this command.

Change-Id: I5abce73877441c7529f2f77add79cf410e2226d8
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-25 00:39:28 -05:00
Zuul
06029bc1bc Merge "Allow do gather pod logs for different clusters" 2020-09-14 22:34:36 +00:00
Kostiantyn Kalynovskyi
a5c6b12e7e Allow do gather pod logs for different clusters
Relates-To: #346
Closes: #346

Change-Id: I80d16fdafde3d39cdac65942f49e892297cbfeda
2020-09-10 17:54:16 -05:00
Stanislav Egorov
393b75bc08 Worker node deployment
Change-Id: I1a1c06ee9fa16e2c56374e98f8aaea5da80898cd
2020-09-10 12:11:10 -07:00
Sreejith Punnapuzha
f95da5a56c Enhance target node deployment on gates
* Add script to install clusterctl
  * Add controlplane yamls for target node
  * Add script to perform cluster move
  * Add script to zuul jobs

Closes: #288
Change-Id: Ia6891df9c9b1da333396e76f11332deeb17ab807
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-08-27 11:12:05 -05:00
Zuul
d6c0e4303c Merge "Add multi-site support to gate runner" 2020-08-21 15:45:56 +00:00
Stanislav Egorov
2605cd9159 Gate scripts refactoring
Added flexible boot mode configuration for VMs based on BareMetalHost definition.

Change-Id: I1332ff99e7ec539ac280efda515f736b7c18f459
2020-08-19 14:49:03 -07:00
Drew Walters
0706575ef4 Add multi-site support to gate runner
This change adds multi-site support to the gate runner by leveraging
environment variables used by the deployment scripts. Zuul jobs can
change the site by defining the site variable in the job definition. If
no site name is specified, test-site is used as a default.

Example:

  - job:
      name: gate-script-runner-site-b
      vars:
        site: site-b

This change will allow the treasuremap repository to test multiple sites
using the same job definitions.

Change-Id: I5ed02d2563db2d2b3b1587fcac94ad2bce805ced
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-08-19 16:38:42 +00:00
Alexander Noskov
1fcb6df061 Enable UEFI in addition to legacy bootmode
In order to enable UEFI support we need to configure guests VM to use OVMF
(Open Virtual Machine Firmware)

Closes: #319

Change-Id: I4b1c3b9fa8f1f0fe42ced7ec3e57b9d95dd4a3f5
2020-08-13 13:14:37 -05:00
Nikolay Fedorov
bd6f2f8e32 Add usage MAC for Ephemeral VM deploy
We should use interface naming independent on OS, type, speed, slot etc.
This CR takes an approach that is already used for the target node.
To define VMs, their script uses MAC addresses from manifests.

Relates-To: #285

Change-Id: Iafffd23bc584d2daf3fed5ee301491f447498193
2020-08-05 12:29:39 +04:00
Stanislav Egorov
e20c8ae1fa Preparation for workers deployment
Added air-worker VM
Import kubeconfig and check cluster
Initinfra and capi for target cluster

Change-Id: Ib05f805b428a1eab20d62df67ac97670714eebd0
2020-07-28 18:35:19 +00:00
Ruslan Aliev
d21c02c87b Fix airshipctl-roles-test gate job
Due to incorrect processing IP:port pairs using 'regex_replace' filter,
roles-test gate job fails. This patches fixes all the related errors and
CI gate check itself.

Change-Id: Ifd7f890166195df0af9b34e1ccff59b7abd2d9ab
Relates-To: #305
Closes: #305
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-07-21 20:51:53 -05:00
Stanislav Egorov
f1725f3123 Cleaning for gate scripts
Removed orphaned local scripts because some roles were removed.
Fixed env vars for proper configuration.
Updated documentation accordingly.

Change-Id: I37f0c8d038fb51ddaa57664a65d347056df6f007
2020-07-09 13:16:05 -07:00
Roman Gorshunov
eddd09f0a0 CI: remove redundant Ansiblle-based CI jobs
CI jobs have been rewritten in Bash scripts and seem to be working fine for the
past few weeks. Removing now redundant Ansiblle-based CI jobs.

This would reduce workload we impose in CI infrastructure.

Change-Id: I690405f5401da0beeb4b176ab22cffdfda24a09f
2020-07-08 16:19:56 +02:00
Ruslan Aliev
876c6043ee Replace apt/yum ansible modules, use package instead
apt/yum use is not convenient, package module automatically uses
the underlying OS package manager. Also, some ansible roles currently
use only apt module without yum, therefore patch fixes this bug too.

Change-Id: I5dd49d513d1a791ab51ca6ce6eb1c079542c5624
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-06-29 20:28:49 +00:00
Sreejith Punnapuzha
c64ad2db4f Update kubectl version
This commit updates kubectl version from 1.16.2 to 1.17.4 as part of
gate setup

Change-Id: I88728be65474ee7ce57f243e1356b84c7609a45f
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 14:31:31 -05:00
Sreejith Punnapuzha
c3002cabda Add bash scripts to run airshipctl
* Add playbook to run the scripts
 * Add script to install kubectl
 * Add script to create airshipctl executable
 * Add script to generate test configuration
 * Add script to pull site documents
 * Add script to build ephemeral iso
 * Add script to deploy ephemeral node
 * Add script to deploy control plane

Closes: #223
Closes: #224
Closes: #225
Closes: #226
Closes: #227

Change-Id: Ied63e2a733c3abcc6b1a822624c5ffce92e05a44
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 15:44:06 +00:00
Sreejith Punnapuzha
1108d4d2bf Add python3-setuptools to needed packages
This commit adds python3-setuptools to needed packages so that
sushy-tool installation will not fail

Change-Id: I7b6de0ebf09b7c187061030463327831c7d856f3
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 08:25:13 -05:00
Zuul
953e55f04b Merge "Remove deprecated fields of remoteDirect" 2020-06-12 19:59:49 +00:00
Yasin, Siraj (SY495P)
54e13ceaad Added new command for Image Build.
* command: airshipctl image build
* For now it is just a wrapper for baremetal isogen.
* Removed isogen subcommand from baremetal

Change-Id: I5c1260c4b167e5768eba27a7cd83c9b60ab29184
Relates-To:#251
Closes:#251
2020-06-12 03:01:26 +00:00
Dmitry Ukov
df2fff0acf Add control plane deployment
K8s control plane is deployed by KubeadmControlPlane controller. This
controller creates CAPI machines and infrastructure objects
(Metal3Machine). Metal3Machine objects are created based on a template
which contins host selector label. Control plane label is assigned to a
particular BareMetalHots object defined inside of the shared
kustomization.

Relates-To: #149
Closes: #221
Change-Id: I3be1750aacf9736ece2944045c036f405e404561
2020-06-11 13:44:42 +00:00
Ian Howell
5a187e1810 Remove deprecated fields of remoteDirect
This change removes the deprecated "remoteType", "useProxy", and
"insecure" fields from the remoteDirect field of bootstrapInfo

Change-Id: I64d28b715b81464c871f99ad0348fadc2e9fc5f6
Relates-To: #242
2020-06-10 10:23:41 -05:00
Stanislav Egorov
37bba4e2fc Locked version of sushy-tools
0.9.1 is not working with current implementation of airshipctl
Error below:
  Setting boot device to 'CD'.
  BMC responded '500 INTERNAL SERVER ERROR'. Attempting to unmarshal the raw BMC error response.
  Unable to decode BMC response. "Unable to decode Redfish response.
  Key 'error.@Message.ExtendedInfo.Message' is missing or has unknown format."
  redfish client encountered an error: BMC responded '500 INTERNAL SERVER ERROR'.

0.9.0 works fine

Change-Id: I34907cc42bbe78624e2741a581bc95c2e2f68bea
2020-05-28 15:24:36 -07:00
Dmitry Ukov
a301de202b Add CAPI initialization steps to gating
This change enables execute 'airshipctl cluster init' command
as a part of zuul pipeline

Co-Authored-By: Kostyantyn Kalynovskyi <kkalynovskyi@mirantis.com>
Change-Id: I9d499ae66f8d847759c2e61e5d4415f2c9f3c860
2020-05-20 15:17:20 +04:00
Zuul
66980ed432 Merge "Replace become: true notation" 2020-05-20 04:27:57 +00:00
Ruslan Aliev
76d8716968 Replace become: true notation
Ansible documentation recommends to use yes/no as boolean value,
the last mention of possible using true/yes related to version 2.4 [1].
Starting from version 2.5 only become: yes specified in docs [2][3].
Since we have both variations, this patch aligns become statement to
use only yes value.

[1] https://docs.ansible.com/ansible/2.4/become.html
[2] https://docs.ansible.com/ansible/2.5/user_guide/become.html
[3] https://docs.ansible.com/ansible/latest/user_guide/become.html

Change-Id: I706e6e1b9d706d6bfcc38c1a3d0d7304575946ce
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-05-16 01:48:42 -05:00
Ruslan Aliev
917d6896b6 Remove "cluster initinfra" command
Command "cluster initinfra" should be removed since
"phase apply" was introduced instead.

Change-Id: I263f8988a4456fc03314093a55ee1c1da8048404
Relates-To: #162
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-05-16 01:45:34 -05:00
Alexander Noskov
84f2bc60c9 Fix default Airshipctl config parameters
Problems description:
Once we execute airshipctl config init, the default config in
~/.airship/config doesn't contain default values for:
currentContext: ~
clusters:
  default:
    clusterType:
      target:
        bootstrapInfo: ~
        managementConfiguration: ~
---
This PS set default values for Airshipctl config to use it
out of the box.

Closes: #234

Change-Id: I57449e678f19ab1c0e2212c38b01ea195f62aa44
2020-05-12 16:02:05 -05:00
Zuul
60ece21b96 Merge "Fix system power regressions" 2020-05-12 16:42:31 +00:00
Zuul
528407edc7 Merge "Add pod log collecting role" 2020-05-12 16:10:18 +00:00
Dmitry Ukov
ea79cd6a71 Add pod log collecting role
Closes: #220
Change-Id: I5a4b682f8f4b94fa6827e6d304d07d825811916b
2020-05-12 18:08:03 +04:00
Drew Walters
4017e9c221 Fix system power regressions
A change [0] recently merged that makes additional remote parameters
configurable; however, it changed the default wait value to two seconds
and made the wait value a duration in time between two nanoseconds.

This change restores the default wait value of thirty seconds and makes
the wait value representative of seconds again.

[0] https://review.opendev.org/717812

Change-Id: I9fad30d479419efe95fca7dc78154bfc3ca66697
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-11 20:33:50 +00:00