87 Commits

Author SHA1 Message Date
Zuul
df3764b7ac Merge "Remove document plugin subcommand" 2020-11-12 03:03:33 +00:00
Dmitry Ukov
9c69138990 Remove document plugin subcommand
This change finalises shift to new version of kustomize plugins

Change-Id: Ie47f42a9e2c4dc03fc4dca437020ad71a564d894
2020-11-11 14:59:14 +04:00
Rajat Sharma
854e25355b Incorporate HWCC in deployment gate.
Added sample HCC profiles and deployment script to apply profiles.

Change-Id: If8152a0afda142c40a53ddf9a4bd6866b435b0e5
Co-Authored-By: Rajat Sharma <rajatcu143@gmail.com>
Relates-To-Issue: #326
Depends-On: #738410
2020-11-10 16:12:28 +05:30
Sirajudeen
a360ebf9da Check if commit is linked to github issue
* Non-Voting gate to check if commit is linked to github issue

Change-Id: I8a4ea350382f8f2ed58ce49d52f1386d9c0d9c92
Closes: #387
2020-11-06 18:45:34 +00:00
Sirajudeen
c0f116a68c Separate lint job from the test
* Split lint and test jobs
* Add lint job to be run against all files

Change-Id: I74da7999324283984d61f38ae9fb394ee5b37ce6
Closes: #386
2020-11-05 22:35:32 +00:00
Zuul
340bed7ac5 Merge "Use token for github Authentication" 2020-10-26 18:32:56 +00:00
Sirajudeen
c33f5f5104 Use token for github Authentication
* Move from user/passwd to token based auth for
  github api access
* Github password based authentication will be deprecated soon
  https://developer.github.com/changes/2020-02-14-deprecating-password-auth/

Change-Id: Ic5af5bcf6ceb9828f2df7cca97e73c713dc79c1e
Closes: #373
2020-10-26 13:44:27 +00:00
Andrii Ostapenko
81efbceea6
Collect description of k8s object to job logs
Change-Id: I15226b9700acf169c100431d5dc7fafe41fbcb87
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-23 11:58:29 -05:00
Dmitry Ukov
d783d71f05 Introduce Replacement Transformer plugin container
Relates-To: #341
Change-Id: I65a5b8cf3bcf94321fc02f240b1d8075bca45c0c
2020-10-16 16:00:56 +04:00
Zuul
f92e8bd042 Merge "Replace PrimaryRepositoryName with PhaseRepositoryName" 2020-10-13 20:50:53 +00:00
Sirajudeen
8307148482 Remove clusterctl install from gate script
* clusterctl commands are driven through phase run.
  So removing install clusterctl CLI utility from gate
  script

Change-Id: I1c57aa07a9e19495c94c3080d0ce40dfc0e5dd47
2020-10-13 15:11:59 +00:00
Vladimir Kozhukalov
0dc4ab7491 Replace PrimaryRepositoryName with PhaseRepositoryName
Change-Id: I2276a3b3ea50b8e81dd240815cd00a0a2a2af96e
Relates-To: #356
2020-10-12 15:28:27 +03:00
Sidney Shiba
d0683139fe Integration of Azure provider to Airship 2.0
This commit integrates the Azure provider to the Airship 2.0 project.

It adds the following folders:
- manifest/function/capz: This folder contains all manifests required
for the integration of Azure provider.
- manifest/function/k8scontrol-capz: This folder contains the base
manifest for the Azure Workload cluster.
- manifest/site/az-test-site: This folder contains the manifests used
for initializing the CAPI and CAPZ components on the management cluster
invoking "airshipctl cluster init" and manifests used for deploying a
Workload cluster on the Azure Cloud by invoking the command
"airshipctl phase apply azure".
- tools/deployment/azure: provides the script shell that are used in the
zuul gates and local test.

Updated files:
- zuul.d/project.yaml and zuul.d/jobs.yaml have been updated to include
gates for validating the Azure provider integration.

Change-Id: Icbdc7f6f42c159f48dd11e35626da3bc016f5487
2020-10-05 19:27:31 +00:00
Sean Eagan
eb40a5700f Add gating for helm release management
This adds gating for helm release management by including a minimal
example workload phase which consists of a HelmRelease for the nginx
ingress controller and a corresponding deployment script including
supporting validation logic.

Change-Id: Ia21a799030289c7e40a0e61292578987ea0f6c63
Relates-To: #351
2020-09-23 14:59:24 +00:00
Sreejith Punnapuzha
a6528dc564 Enhance log collection zuul jobs
* Change name for gate script runner job
  * Make log collection job parameterize

Change-Id: I9f7c711556996af2da350d0160e572e1a26efd0f
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-09-11 14:46:50 -05:00
Stanislav Egorov
393b75bc08 Worker node deployment
Change-Id: I1a1c06ee9fa16e2c56374e98f8aaea5da80898cd
2020-09-10 12:11:10 -07:00
Sreejith Punnapuzha
f95da5a56c Enhance target node deployment on gates
* Add script to install clusterctl
  * Add controlplane yamls for target node
  * Add script to perform cluster move
  * Add script to zuul jobs

Closes: #288
Change-Id: Ia6891df9c9b1da333396e76f11332deeb17ab807
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-08-27 11:12:05 -05:00
Stanislav Egorov
2605cd9159 Gate scripts refactoring
Added flexible boot mode configuration for VMs based on BareMetalHost definition.

Change-Id: I1332ff99e7ec539ac280efda515f736b7c18f459
2020-08-19 14:49:03 -07:00
Alexander Noskov
582894d389 Switch to Ubuntu focal (20.04) based image
Since we have switched from debian to ubuntu based image and changed the
name (in https://review.opendev.org/733078) we need to align it with
airshipctl repository.
Renaming the following names:
1) debian-custom.iso -> ubuntu-focal.iso
2) quay.io/airshipit/isogen:latest-debian_stable -> quay.io/airshipit/isogen:latest-ubuntu_focal

Updating the following packages and resources:
1) k8s 1.17.3 -> 1.18.6
2) docker 19.03.9 -> 19.03.12
3) Calico 3.9 -> 3.15

Change-Id: I7eaf382acb79016a511db6e0955fa932c02963c4
2020-08-13 13:11:25 -05:00
Nikolay Fedorov
bd6f2f8e32 Add usage MAC for Ephemeral VM deploy
We should use interface naming independent on OS, type, speed, slot etc.
This CR takes an approach that is already used for the target node.
To define VMs, their script uses MAC addresses from manifests.

Relates-To: #285

Change-Id: Iafffd23bc584d2daf3fed5ee301491f447498193
2020-08-05 12:29:39 +04:00
Stanislav Egorov
e20c8ae1fa Preparation for workers deployment
Added air-worker VM
Import kubeconfig and check cluster
Initinfra and capi for target cluster

Change-Id: Ib05f805b428a1eab20d62df67ac97670714eebd0
2020-07-28 18:35:19 +00:00
Matt McEuen
0324993d60 Add an m3 host generator function
This PS has a function which constructs a collection of Metal3 BareMetalHost
resources, along with associated configuration Secrets.
It solves for a couple of things:

1. pulling the nitty gritty details for generating BMH into one reusable place,
2. allowing the site-specific details to be filled in via catalogues of values

This function leverages a couple of different plugins in sequence:
The airshipctl Replacement plugin, which pulls the site-specific data from
the catalogue documents into a Templater plugin configuration; and then
the airshipctl Templater plugin, which generates a variable number of
BMHs in a data-driven fashion.

More details can be found in the README.md in this patchset.

Closes: #245

Change-Id: I3ddbd36dc53ea6afbd633098c985f4b28bcbb793
2020-07-17 15:30:26 -05:00
Stanislav Egorov
f1725f3123 Cleaning for gate scripts
Removed orphaned local scripts because some roles were removed.
Fixed env vars for proper configuration.
Updated documentation accordingly.

Change-Id: I37f0c8d038fb51ddaa57664a65d347056df6f007
2020-07-09 13:16:05 -07:00
Stanislav Egorov
7e5ba1a195 Reverting back vars for build gate
Change-Id: Ia880686a9398ce0ce8b4d3b062d94601a53ca66f
2020-07-08 22:58:14 -07:00
Roman Gorshunov
eddd09f0a0 CI: remove redundant Ansiblle-based CI jobs
CI jobs have been rewritten in Bash scripts and seem to be working fine for the
past few weeks. Removing now redundant Ansiblle-based CI jobs.

This would reduce workload we impose in CI infrastructure.

Change-Id: I690405f5401da0beeb4b176ab22cffdfda24a09f
2020-07-08 16:19:56 +02:00
Ruslan Aliev
876c6043ee Replace apt/yum ansible modules, use package instead
apt/yum use is not convenient, package module automatically uses
the underlying OS package manager. Also, some ansible roles currently
use only apt module without yum, therefore patch fixes this bug too.

Change-Id: I5dd49d513d1a791ab51ca6ce6eb1c079542c5624
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-06-29 20:28:49 +00:00
Sirajudeen
1fd55a7abd install python3-pip explicitly.
* zuul gate failing due to pip3 missing, so had to install it explicitly

Change-Id: I8d24cc506d58312f338e3632e97a567b23dc71a1
2020-06-24 16:54:34 -05:00
Sirajudeen
6aaceb1e7f Use sudo to install python3-setuptools
Change-Id: I4119486e8388ca69fce8a135d785f66cc9c4e880
2020-06-23 17:47:38 -05:00
Sirajudeen
08c5d6acbb Add python3-setuptools to required packages
* This commit adds python3-setuptools to required packages, so that
  the job airship-airshipctl-update-github-issues will not fail

Change-Id: I7b5a93e83175bb19dd6145f1af6bf7f6fd7a4d61
2020-06-23 09:30:26 -05:00
Sreejith Punnapuzha
c3002cabda Add bash scripts to run airshipctl
* Add playbook to run the scripts
 * Add script to install kubectl
 * Add script to create airshipctl executable
 * Add script to generate test configuration
 * Add script to pull site documents
 * Add script to build ephemeral iso
 * Add script to deploy ephemeral node
 * Add script to deploy control plane

Closes: #223
Closes: #224
Closes: #225
Closes: #226
Closes: #227

Change-Id: Ied63e2a733c3abcc6b1a822624c5ffce92e05a44
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 15:44:06 +00:00
Ruslan Aliev
19c37fef7f Fix contol plane deployment ansible task
There is need to ensure that appropriate directory does not exist
before clonining source code into it, otherwise git throws an error.
The destination folder name for cloning was fixed. Variable name which
uses for task fail verification was corrected.

Relates-To: #271
Closes: #271
Change-Id: I9c2bd03f68e6c34fc28b900f11dbaadd190087e8
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-06-15 02:17:22 -05:00
Yasin, Siraj (SY495P)
458fb2579b Fix gate scripts for local setup
* when remote_work_dir is set, document pull happens from the
  remote_work_dir (/tmp/airship) with a folder created within
  remote_work_dir with repo name (/tmp/airship/airshipctl).
  However the manifest yamls of kind "Clusterctl"
  refers to a relative path from work_dir and it fails
  finding expected objects.
   example: manifests/function/capm3/v0.3.1
* So trying to avoid document pull, and use current dir for workspace.
  similar to Zuul gate implementation

Change-Id: I63fd5476247f957745e15cbdfceb5fb483758e83
2020-06-12 03:00:32 +00:00
Dmitry Ukov
df2fff0acf Add control plane deployment
K8s control plane is deployed by KubeadmControlPlane controller. This
controller creates CAPI machines and infrastructure objects
(Metal3Machine). Metal3Machine objects are created based on a template
which contins host selector label. Control plane label is assigned to a
particular BareMetalHots object defined inside of the shared
kustomization.

Relates-To: #149
Closes: #221
Change-Id: I3be1750aacf9736ece2944045c036f405e404561
2020-06-11 13:44:42 +00:00
Matt McEuen
958d783919 Add site doc validation gate
This adds a gate which loops over all phases in all sites,
and performs an airshipctl apply --dry-run on them to ensure YAML
validity and schema adherence.  Aside from installation tasks,
the gate is run via a makefile entrypoint so that it can be
easily consumed by developers or by non-zuul CICD platforms.

Change-Id: Ie4ab246848a580ab20c3153af1e3749a27e3f770
2020-06-04 18:04:47 -05:00
Ian Howell
7243176e44 Add golint as a non-voting gate
Change-Id: I9850528a720ff517a3f400553fa8ab66bf4b1b3b
Closes: #253
2020-05-29 11:40:45 -05:00
Yasin, Siraj (SY495P)
9cde7fc8f5 Fixed the script path issue
* created a files dir within playbooks and moved
  update_github_issues.py

Change-Id: Icb621ec00bab0b431c09405370968c0b0ac975b7
Relates-To: #196
2020-05-27 14:34:03 +00:00
Zuul
57b47c9640 Merge "Ansible syntax fix and added message decrypt." 2020-05-22 15:55:10 +00:00
Yasin, Siraj (SY495P)
b248f6e4a4 Ansible syntax fix and added message decrypt.
* Decrypt message which is base64 encoded in secrets
* read change_url from zuul dictionary instead of its
  subelement items

Change-Id: I2137be52d4ad8c0a158b6192b6e0a08527c60328
Relates-To: #196
2020-05-21 12:25:34 -05:00
Dmitry Ukov
a301de202b Add CAPI initialization steps to gating
This change enables execute 'airshipctl cluster init' command
as a part of zuul pipeline

Co-Authored-By: Kostyantyn Kalynovskyi <kkalynovskyi@mirantis.com>
Change-Id: I9d499ae66f8d847759c2e61e5d4415f2c9f3c860
2020-05-20 15:17:20 +04:00
Zuul
66980ed432 Merge "Replace become: true notation" 2020-05-20 04:27:57 +00:00
Ruslan Aliev
76d8716968 Replace become: true notation
Ansible documentation recommends to use yes/no as boolean value,
the last mention of possible using true/yes related to version 2.4 [1].
Starting from version 2.5 only become: yes specified in docs [2][3].
Since we have both variations, this patch aligns become statement to
use only yes value.

[1] https://docs.ansible.com/ansible/2.4/become.html
[2] https://docs.ansible.com/ansible/2.5/user_guide/become.html
[3] https://docs.ansible.com/ansible/latest/user_guide/become.html

Change-Id: I706e6e1b9d706d6bfcc38c1a3d0d7304575946ce
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-05-16 01:48:42 -05:00
Ruslan Aliev
917d6896b6 Remove "cluster initinfra" command
Command "cluster initinfra" should be removed since
"phase apply" was introduced instead.

Change-Id: I263f8988a4456fc03314093a55ee1c1da8048404
Relates-To: #162
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-05-16 01:45:34 -05:00
Zuul
528407edc7 Merge "Add pod log collecting role" 2020-05-12 16:10:18 +00:00
Dmitry Ukov
ea79cd6a71 Add pod log collecting role
Closes: #220
Change-Id: I5a4b682f8f4b94fa6827e6d304d07d825811916b
2020-05-12 18:08:03 +04:00
Zuul
3fb7c774e7 Merge "Reusing apache-server instead of httpd" 2020-05-11 15:39:30 +00:00
Zuul
4e0af74f46 Merge "Fixed document pull issue in gate script 21" 2020-05-08 20:47:48 +00:00
James Gu
301de51467 Fixed document pull issue in gate script 21
Changed systemwide-executable to run in local_src_dir because the script needs
to build airshipctl binary first, before doc pull command can be invoked and populate
the remote_work_dir. Changed the order of the roles to allow systemwide-executable
to run before test-config and pull-docs.

Change-Id: Ia47423d91cdfb2dbbf0259344daab1c7b8cd47f5
Signed-off-by: James Gu <james.gu@att.com>
2020-05-08 18:30:51 +00:00
Albin Vass
ee2b91efed Use in repo docker-install instead of install-docker from zuul-jobs
Change-Id: Id94ef7f5a0327aff5c184a85d11f457e0436630f
2020-05-08 18:59:00 +02:00
Drew Walters
f729fdd1a1 Replace venv with tox for documentation builds
OpenStack infra has created Zuul project templates that will enable us
to push our documentation to Read the Docs and the Airship website.
While adding a tox file to this project is not ideal, it appears to be
the only way to leverage these jobs at the moment.

This change adds a tox.ini file to airshipctl to build our documentation
and removes the venv solution. The Airship Working Committee will work
with the OpenStack infra team, if possible, to make the publish jobs
more flexible in the future and remove the tox.ini file.

Change-Id: I274cf69a7c79e0aad1d4c9b1decfaf5630f0b18f
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-08 15:47:41 +00:00
Stanislav Egorov
b599664bb7 Fixed local gate runner
Role airshipctl-test-configs should be run after building airshipctl
because it's running binary that is not built yet

Change-Id: I7d352bfbd80f9db50913fae3620170bea9460325
2020-05-06 11:14:51 -07:00