129 Commits

Author SHA1 Message Date
Anderson, Craig (ca846m)
2daacf5f2a image-builder integration for ISO builds
This introduces airshipctl integration with image-builder [0], which
replaces the existing isogen tool for ephemeral ISO generation.

The airshipctl isogen executor has been updated for building ephemeral
ISOs using the image-builder container. The ability for user-declared
filenames for cloud-init user data and network data was removed, since
the user's only interest is in supplying the relevant overrides, not in
transparent naming coordination with the image-builder container. A new
object is added to the document package to identify the document kind,
label, and key to retrieve data from since this is pattern we will
reuse elsewhere.

Progress flag removed as requsted. Progress is reported directly by the
image-builder container.

Isogen debug flag removed in favor of using log.DebugEnabled()

[0] https://review.opendev.org/#/c/730777/

Depends-On: https://review.opendev.org/c/airship/images/+/730777/
Change-Id: I545004feaf2116f8ffb29faf6f7f7f5fcfe24fff
2020-12-17 14:47:53 -08:00
Zuul
4c213b5fe4 Merge "Add toleration for taint" 2020-12-15 18:28:57 +00:00
Sreejith Punnapuzha
02da44720c Add toleration for taint
This commit adds toleration for taint master NoSchedule to different
components. This will help us in managing components that gets deployed
in kubernetes master nodes

Relates-To: #406

Change-Id: I9f3a30be9c4eed65dcdd1c41514abbfd9c384541
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-12-14 14:56:51 -06:00
Rishabh Kumar Jain
ae6ce6931b Integrate common infra provider zuul scripts
* add common scripts for providers capd/capg/capz/capo
    to do the following:
    - install kind (reuses  tools/document/get_kind.sh)
    - improve tools/document/start_kind.sh to use
      custom kind config file
    - created a script similar to 26_deploy_metal3_capi_ephemeral_node.sh,
      called tools/deployment/26_deploy_capi_ephemeral_node.sh and
      made it flexible for all providers (metal3/capd/capz/capg/capo)
    - deploy ephemeral control plane
    - initialize target node
    - move crds from ephemeral to target
    - deploy target workers

Closes: #413
Change-Id: If3520d000a03e5ed26d54e1ceedb2febfceccb3d
2020-12-11 16:45:07 -05:00
Ruslan Aliev
4258d1904e Add missing metadata.yaml definitions
Some of sites defined in document model have no metadata.yaml,
which is essential to work properly with that sites.
 * az-test-site and gcp-test-site metadata.yaml added
 * metadata.yaml for test-site moved to appropriate directory
   for consistency

Change-Id: Ibec109e41a5e3fd36794dc6a879888cfc610c9cc
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #19
2020-12-03 19:34:09 -06:00
Dmitry Ukov
3acd9ab8f5 Add DeepCopy generation target to Makefile
Change-Id: I974f8472f2ad8231a6612849f1bc73895c96ef81
Closes: #384
2020-11-18 18:20:25 +04:00
Zuul
df3764b7ac Merge "Remove document plugin subcommand" 2020-11-12 03:03:33 +00:00
Dmitry Ukov
9c69138990 Remove document plugin subcommand
This change finalises shift to new version of kustomize plugins

Change-Id: Ie47f42a9e2c4dc03fc4dca437020ad71a564d894
2020-11-11 14:59:14 +04:00
Rajat Sharma
854e25355b Incorporate HWCC in deployment gate.
Added sample HCC profiles and deployment script to apply profiles.

Change-Id: If8152a0afda142c40a53ddf9a4bd6866b435b0e5
Co-Authored-By: Rajat Sharma <rajatcu143@gmail.com>
Relates-To-Issue: #326
Depends-On: #738410
2020-11-10 16:12:28 +05:30
Sean Eagan
2ba6561854 Add helm-controller and source-controller
This adds functions for the helm-controller and source-controller from
Flux v2 [0]. The helm-controller depends on the source-controller and
so this includes a composite to wrap both. These components replace the
Helm Operator, which is now removed from the gating setup. The Helm Operator
function itself will be removed after a suitable migration period.

This also adds a format for Helm chart and Helm repository sourcing
to the versions catalog.

[0]: https://fluxcd.github.io/flux2

Closes: #372

Change-Id: I7287f1bb5b1ebd1dba9920cc0bed8c867720b4c1
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
2020-11-06 09:52:16 +00:00
Michael Beaver
193dc935f1 Refactor subPath out of config manifests
This change removes the SubPath entry portion of the config manifests, and
refactors several areas that used it. This also updates unit tests to
reflect the new changes.

Closes: #255

Change-Id: Ibbb519a3f1fb28a1594e971274c5de54b99c867e
2020-11-04 09:30:03 -06:00
Ahmad Mahmoudi
ff6b5ecbff (fix) Add -E for sudo to pick user env
In 10_buid_scrip.sh script sudo call needs user's environment
variables for proxy configuration, in case it is run behind a
proxy.

Change-Id: Ia084cba548e19340d25ef941e96336d9e420bec0
2020-10-27 14:27:27 +00:00
Zuul
d812b6c165 Merge "Introduce Replacement Transformer plugin container" 2020-10-19 18:51:49 +00:00
Ruslan Aliev
0c870e5244 Remove clustertype-related functions
We don't use clustertype anymore, so all the related functions must
be deleted or adjusted as well as appropriate config fields.

Change-Id: I3931fdc71d4318e916f8bbc2d94e062c9df5f641
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #349
2020-10-16 11:31:58 -05:00
Zuul
23224e57c5 Merge "Phase docs are targetPath + phaseRepo + phase.DocumentEntrypoint" 2020-10-16 15:02:04 +00:00
Dmitry Ukov
d783d71f05 Introduce Replacement Transformer plugin container
Relates-To: #341
Change-Id: I65a5b8cf3bcf94321fc02f240b1d8075bca45c0c
2020-10-16 16:00:56 +04:00
Vladimir Kozhukalov
dc68640389 Phase docs are targetPath + phaseRepo + phase.DocumentEntrypoint
Change-Id: I1d5ea75d1c19eb4ebaf37b28918ccde771bcef85
Relates-To: #356
2020-10-15 13:55:59 +03:00
Zuul
f92e8bd042 Merge "Replace PrimaryRepositoryName with PhaseRepositoryName" 2020-10-13 20:50:53 +00:00
Sirajudeen
8307148482 Remove clusterctl install from gate script
* clusterctl commands are driven through phase run.
  So removing install clusterctl CLI utility from gate
  script

Change-Id: I1c57aa07a9e19495c94c3080d0ce40dfc0e5dd47
2020-10-13 15:11:59 +00:00
Vladimir Kozhukalov
0dc4ab7491 Replace PrimaryRepositoryName with PhaseRepositoryName
Change-Id: I2276a3b3ea50b8e81dd240815cd00a0a2a2af96e
Relates-To: #356
2020-10-12 15:28:27 +03:00
Roman Gorshunov
585a75acf3 Fix shell syntax causing gates to silently fail
Fix shell syntax causing gates to silently fail with error
  ./tools/deployment/35_deploy_workload.sh:
  line 33: 404: command not found

Change-Id: Id49dde7e7c50f6596359e5ec7b6643401797ac9a
2020-10-08 14:18:09 +00:00
Ruslan Aliev
fc33287204 Remove kubeconfig from config struct
Change-Id: I5ab8722dd151d9e652e0b20e1d82988cd98505af
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-10-07 14:35:07 -05:00
Zuul
10620da2cc Merge "Changes gating scripts to use the airshipctl move command." 2020-10-07 18:30:41 +00:00
Zuul
4b8209f100 Merge "Add CI metadata path override" 2020-10-06 16:27:39 +00:00
Arvinderpal Wander
e3065784ee Changes gating scripts to use the airshipctl move command.
Change-Id: I383343f4cde3aa3d431844f5792ee75d305d5210
2020-10-06 06:57:54 -07:00
Sidney Shiba
d0683139fe Integration of Azure provider to Airship 2.0
This commit integrates the Azure provider to the Airship 2.0 project.

It adds the following folders:
- manifest/function/capz: This folder contains all manifests required
for the integration of Azure provider.
- manifest/function/k8scontrol-capz: This folder contains the base
manifest for the Azure Workload cluster.
- manifest/site/az-test-site: This folder contains the manifests used
for initializing the CAPI and CAPZ components on the management cluster
invoking "airshipctl cluster init" and manifests used for deploying a
Workload cluster on the Azure Cloud by invoking the command
"airshipctl phase apply azure".
- tools/deployment/azure: provides the script shell that are used in the
zuul gates and local test.

Updated files:
- zuul.d/project.yaml and zuul.d/jobs.yaml have been updated to include
gates for validating the Azure provider integration.

Change-Id: Icbdc7f6f42c159f48dd11e35626da3bc016f5487
2020-10-05 19:27:31 +00:00
Ruslan Aliev
9751968fa8 Enhance cleanup script
There is need to remove airshipctl artifacts in /tmp/airship,
docker containers/images and left processes which use port 8000 or 8099.

Change-Id: I682cf5a6f41875d5be77aa552e4743c9b8cf17df
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-29 15:38:33 +00:00
Drew Walters
43d4ebe164 Add CI metadata path override
This change adds an environment variable override to change the metadata
path in the Airship config file when using the CI-driven deployment
scripts. This will enable the Treasuremap repository to store
metadata.yaml at the site level while using the same deployment scripts
from airshipctl.

Change-Id: I0862a87417ad7a4d8666df7b34e25327bb40e251
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-09-29 14:43:05 +00:00
Zuul
918291f932 Merge "git diff check updated to watch for md changes" 2020-09-28 22:48:57 +00:00
Sirajudeen
b373fff4bc git diff check updated to watch for md changes
* Whenever the CLI usage is updated, it is required to run
  make cli-docs. So added that check in git-diff to capture in gates
  if missed by user

Change-Id: I47be48e921505593e745c11ce139f8079abc686a
2020-09-28 21:16:08 +00:00
Ruslan Aliev
d8a6139e81 Add document pull step to gate script runner
Currently we don't test document pull step by rewriting
manifest_directory variable as current source path, which
makes a mess in target_path, manifest_path and related paths
(concatenation of target_path+manifest_path won't return
an existing and proper location). This patch organizes
mentioned variables and enables document pull step which
starts working properly. The latest repo state used as
git source for this command.

Change-Id: I5abce73877441c7529f2f77add79cf410e2226d8
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-25 00:39:28 -05:00
Sean Eagan
eb40a5700f Add gating for helm release management
This adds gating for helm release management by including a minimal
example workload phase which consists of a HelmRelease for the nginx
ingress controller and a corresponding deployment script including
supporting validation logic.

Change-Id: Ia21a799030289c7e40a0e61292578987ea0f6c63
Relates-To: #351
2020-09-23 14:59:24 +00:00
Ruslan Aliev
2f5402ae80 Remove phase apply command
Command phase run will be used instead, so phase apply became no
longer needed.

Change-Id: I8bdbd6bcad7efbb03632978d8952700fc39d55ba
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-23 01:42:16 -05:00
Ruslan Aliev
0c736af2e8 Implement render methods
Change-Id: I03b7aad33576eaa9ef7dfaabfb579ac90cc26a12
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-23 03:06:51 +00:00
Kostiantyn Kalynovskyi
1eba48d8cd Adding bootstrap phase
Change-Id: I76a5fd0d724ddb7f2e3467f0807d6c312b1c836b
2020-09-22 00:18:13 +00:00
Ruslan Aliev
5d52339bd6 Remove clusters, users, reconcilation and config import
All of these entities are no longer required and should be deleted.

Change-Id: Id4776efe668265df6961a38ce984b8ac37b27097
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #327
2020-09-16 18:35:12 -05:00
Kostiantyn Kalynovskyi
f4baf5a971 Switch to using phase documents
This commit allows to stop using phases with hardcoded paths to
kustomize entrypoints. Instead we use phase documents defined
in bundles. Phase document has a reference to executor such as
kubernetes apply and clusterctl init. More executors are being
developed

Relates-To: #342

Change-Id: I8babd9d0379763773c443e263373e9816f9dbd39
2020-09-16 11:17:41 -05:00
Zuul
c9be1643b5 Merge "Prepare gate to remove airshipctl import" 2020-09-15 22:48:07 +00:00
Zuul
fd4bd1beb9 Merge "Enhance cleanup script" 2020-09-15 21:36:49 +00:00
Sirajudeen
196150649a Prepare gate to remove airshipctl import
* Inject target-cluster-ca as sceret
* Create kubeconfig using gate scripts with both ephemeral and
  target clusters.
* With both contexts predefined in kubeconfig,
  just switch/use context in gate scripts as required.
* Removed the use of airshipctl import from gate script

Change-Id: I5a9a56db71f0a80d9b35144eacbdd980c13c9ffb
Relates-To: #329
Closes: #329
2020-09-15 15:22:27 -05:00
Zuul
2266d0e0b3 Merge "Add DEBIAN_FRONTEND=noninteractive to setup script" 2020-09-15 18:45:37 +00:00
Sreejith Punnapuzha
87d9738bb7 Enhance cleanup script
* add steps to clean iso and network for gates

Change-Id: I7ffc62cac9690e524c559af554ff6d1a70683368
Signed-off-by: Sreejith Punnapuzha <Sreejith.Punnapuzha@outlook.com>
2020-09-15 15:14:42 +00:00
Zuul
53c96ea413 Merge "Make clean script executable" 2020-09-15 12:29:37 +00:00
Stanislav Egorov
78ec01a1f2 Make clean script executable
Change-Id: Iafbdc12b29139ec73509bdd658ad57f8e77e260b
2020-09-14 14:56:27 -07:00
Sreejith Punnapuzha
f48456a69f Uplift provider components in test site
* uplift capi, cabpk, cacpk and capm3 in test site defination
  * uplift images in version catalogue
  * add api timeout in controlplane
  * add timeout for phases

Change-Id: I9de15c4e1979c12eeedb55fbe721b5e7adfe496b
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-09-14 20:00:11 +00:00
Zuul
4f6b5252c2 Merge "Add node availability check for deploy initinfra script" 2020-09-14 18:00:45 +00:00
Zuul
3401586acc Merge "Worker node deployment" 2020-09-11 17:18:36 +00:00
Zuul
2f355d884b Merge "Added cleanup script" 2020-09-11 17:18:34 +00:00
Ruslan Aliev
165ccb2d7a Add node availability check for deploy initinfra script
Deploy initinfra script can fail due to "node01 not found error".
We need to wait until node01 become ready and available before
performing manipulations. Around 10% of gate script runner test
fails due to this error. Example of error can be found here[1],
example of handling - here[2].

[1] https://zuul.opendev.org/t/openstack/build/5f789f5b6f7c42af903a1892cc69ebf8
[2] https://zuul.opendev.org/t/openstack/build/be22de82d32d4c79a9f993d73fcd6d58/console

Change-Id: If525042ec1260afae278542b033abb14b97347f6
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-11 11:58:33 -05:00
Ruslan Aliev
2d014c1c93 Remove left over artifacts from set/get-cluster/credentials cmds
Change-Id: Ic44083049ae82e1ff65669af76af08553b500fda
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-10 15:02:51 -05:00