209 Commits

Author SHA1 Message Date
Ratnopam Chakrabarti
27031d6f70 Update CAPG provider phase
Update phase for calico deployment in the
target-cluster

Relates-To: #446
Change-Id: Icead39c6bd95c903bc7a912b7a2d161bf09a0401
2021-01-27 15:56:10 +00:00
Battina, Sai (sb464f)
0f0e01ea0e Update PROXY, TARGET_NODE parameters
This PS adds PROXY parameter system executable script.
make uses PROXY parameter to pass proxy while building image

This change also adds TARGET_NODE,CLUSTER_NAMESPACE parameter to pass
node details and cluster details for further operations

Change-Id: I9ff8e12ff679526b728c55ffd23c3ed513db4589
2021-01-22 17:54:09 +00:00
Sirajudeen
9f794360be Image override for CAPO components
Change-Id: I1cdc3f17bf10cd4831670c5aabc509e639c9ebc3
Relates-To: #431
2021-01-18 21:04:46 +00:00
Vladislav Kuzmin
8dba799c18 Add secrets generator phase
This phase builded on top of generic executor container.
It uses kustomize generator to generate secrets
and SOPS function to encrypt secrets.

Usage:
    1. `curl -fsSL -o key.asc https://raw.githubusercontent.com/mozilla/sops/master/pgp/sops_functional_tests_key.asc`
       Copy existing key from sops project
    2. `export SOPS_IMPORT_PGP="$(cat key.asc)" && export SOPS_PGP_FP="FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4"`
    3. `airshipctl phase run secret-generate`
        It will generate and encrypt secret in
        manifests/site/test-site/target/generator/results/generated/
    4. `KUSTOMIZE_PLUGIN_HOME=$(pwd)/manifests SOPS_IMPORT_PGP=$(cat key.asc) kustomize build --enable_alpha_plugins
        manifests/site/test-site/target/catalogues/ > output.txt`
	It will decrypt encrypted secret

Co-authored-by: Alexey Odinokov <aodinokov@mirantis.com>
Change-Id: I1682d71b7805eb36c407e712dcb747de799bc8bb
Relates-To: #379
2021-01-14 18:57:15 +00:00
Zuul
2e66fc59f6 Merge "Implement kustomize sink in generic container executor" 2021-01-14 18:40:14 +00:00
Vladislav Kuzmin
3de8b5b2b2 Implement kustomize sink in generic container executor
Sink function allows to write configurations to an external system.

Change-Id: If9c6904239a542ea4c2bef2920965b6d87feb1e6
Relates-To: #202
Relates-To: #369
2021-01-13 17:56:04 +04:00
Sirajudeen
95af33581d Image override for CAPG components
Change-Id: Ibc2c3174897de4417e274f2bb887ce2f8649148d
Relates-To: #431
2021-01-13 03:09:44 +00:00
Sirajudeen
de625b10d8 Image override for CAPZ components
* This is follow-up commit for image override (CAPZ)

  CAPM3, CAPI, CACPK, CABPK =>
		  https://review.opendev.org/c/airship/airshipctl/+/766228
  CAPD => https://review.opendev.org/c/airship/airshipctl/+/767235

* Also added the replacements to consume values from versions-airshipctl

Change-Id: I9c908c071a76b6fca0d0e03ea27b36517bf7052a
Relates-To: #431
2021-01-12 19:47:10 +00:00
Zuul
2e5818a58c Merge "Image override for CAPD components" 2021-01-12 19:40:28 +00:00
bijayasharma
2615afa87b Add phase tree command in cmd module
*This is the smaller patchset from earlier work:
 https://review.opendev.org/#/c/750449/

* This is the second part of huge PS
* First: https://review.opendev.org/#/c/762386/

Change-Id: I8e7a13445f73b46a8b83f79711551a6fd5f5ce1f
Relates-To: #296
2021-01-08 13:26:08 +00:00
Zuul
4c0cbecf6d Merge "Update Flux manifests" 2021-01-08 05:02:48 +00:00
Sean Eagan
d64404864b Update Flux manifests
This updates the Flux manifests to the latest versions, which includes
a fix [0] to Helm chart rendering which was impacting the OSH charts.

[0]: https://github.com/fluxcd/helm-controller/pull/172

This also adds Kptfiles to each of the functions to make it trivial
to update them.

Relates-To: #430
Change-Id: Ic12bc6a8460542fd943ed3539cf1be19b6525dbc
2021-01-05 13:58:03 -06:00
guhaneswaran20
bfe88fda56 Introducing secret generation template
This patchset contains the function-manifests containing the template
to generate secrets. The secrets include both certificates and
passphrases.

Change-Id: Ie26fac9fe7f3918c8ebb746259d1d9bc0b423489
2021-01-05 19:50:42 +05:30
Sirajudeen
f212cfd55b Deploy Calico using Tigera Operator
* Migrate the Calico function in airshipctl to deploy the
  Tigera Operator, instead of raw manifests
* Added a new executor `kubernetes-apply-nowait` to run phase with nowait
* nowait had to be done for phase `initinfra-networking-<ephemeral/target>`
  because of an issue with phase status-check for a specific resource
  `installation` which is part of tigera operator CR

Change-Id: I748813667cdc5d05c9f0758d9c1e28082d79bdbe
Closes: #368
2021-01-01 04:32:22 +00:00
Sirajudeen
30e69fedf5 Client Certificate and key re-generated
* Current kubernetes client certificate expired on Dec 25th 2020
* Re-generated client certificate and key with same CA

Change-Id: Ifb7eacbf5264bed9373c0b9d9287ee087ebd36cb
Relates-To: #441
2020-12-31 19:50:08 +00:00
Zuul
86f961ac33 Merge "Bootstrap Ephemeral - Phase and Executor Manifests" 2020-12-22 22:44:08 +00:00
Zuul
e075b78d85 Merge "image-builder integration for ISO builds" 2020-12-18 02:04:36 +00:00
Anderson, Craig (ca846m)
2daacf5f2a image-builder integration for ISO builds
This introduces airshipctl integration with image-builder [0], which
replaces the existing isogen tool for ephemeral ISO generation.

The airshipctl isogen executor has been updated for building ephemeral
ISOs using the image-builder container. The ability for user-declared
filenames for cloud-init user data and network data was removed, since
the user's only interest is in supplying the relevant overrides, not in
transparent naming coordination with the image-builder container. A new
object is added to the document package to identify the document kind,
label, and key to retrieve data from since this is pattern we will
reuse elsewhere.

Progress flag removed as requsted. Progress is reported directly by the
image-builder container.

Isogen debug flag removed in favor of using log.DebugEnabled()

[0] https://review.opendev.org/#/c/730777/

Depends-On: https://review.opendev.org/c/airship/images/+/730777/
Change-Id: I545004feaf2116f8ffb29faf6f7f7f5fcfe24fff
2020-12-17 14:47:53 -08:00
Ratnopam Chakrabarti
93b7de1ba6 Update CAPG provider config
Added CAPG specific envs to initialize ephemeral cluster
with CAPG provider components
Added support to replace env vars

Relates-To: #425

Change-Id: I5d38a3f703683b68b18f4ccbaa52331de8484d6e
2020-12-17 12:01:50 -05:00
Zuul
dbb006c02d Merge "Update CAPG test site to support phase run" 2020-12-16 23:52:55 +00:00
Sidney Shiba
446a66454b Bootstrap Ephemeral - Phase and Executor Manifests
This patchset includes the Phase and Executor manifests that handles
the creation of a K8S cluster on private/public cloud provider.

There are three sets of Phases and Executors for each provider:
- ephemeral-<provider>-genesis: deploys a K8S cluster
- ephemeral-<provider>-cleanup: deletes a K8S cluster
- ephemeral-<provider>-help: returns the help text for the container

where <provider> can be:
- az: for Azure cloud platform
- gcp: for Google cloud platform
- os: for Openstack cloud platform

Change-Id: Ia5d9aeca89c507c2e98a1beb1fb3ff906f9540f6
2020-12-16 10:02:10 -06:00
Sirajudeen
fba17f625e Image override for CAPD components
* Follow-up commit of https://review.opendev.org/c/airship/airshipctl/+/766228
  for CAPD

Change-Id: I9aa2e012ff27bce90ad8fb9630a697d6123b971f
Relates-To: #431
2020-12-16 02:28:10 +00:00
Sreejith Punnapuzha
02da44720c Add toleration for taint
This commit adds toleration for taint master NoSchedule to different
components. This will help us in managing components that gets deployed
in kubernetes master nodes

Relates-To: #406

Change-Id: I9f3a30be9c4eed65dcdd1c41514abbfd9c384541
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-12-14 14:56:51 -06:00
Zuul
a980243565 Merge "Image override for cluster components" 2020-12-14 20:42:19 +00:00
Sreejith Punnapuzha
6f57582508 Add Cert-manager definition
* Add cert-manager to airshipctl
  * include cert-manager to infra composite

Relates-To: #408

Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
Change-Id: I3977b1fee7c44724c8a4ec47bb3e8b5be413386d
2020-12-14 10:59:06 -06:00
Sirajudeen
00dcad041e Image override for cluster components
* Used images tag to inject image override to clustetctl object,
  the same way the image override is done for cert-manager.
* Using this way, the upstream manifests for cluster components does
  not need any customization (IMAGE URL as variable) and can be consumed as is.
* If this approach is approved, then will update the same for other proviers
  ( CAPO, CAPD, CAPZ, cAPG ) and will get tested.

Change-Id: If920f544d111d94e5b7075d5406ed2f87a5e6929
Closes: #431
2020-12-14 15:27:35 +00:00
Zuul
d4711d36e8 Merge "Change target cluster name for CAPG provider" 2020-12-11 21:27:08 +00:00
Zuul
9774bb9728 Merge "Update IPA packages in ironic" 2020-12-10 22:15:58 +00:00
Zuul
1f666fc64b Merge "Revert "Use global certmanager for clusterctl"" 2020-12-10 21:34:30 +00:00
Sirajudeen
27dee33bfd Revert "Use global certmanager for clusterctl"
This reverts commit 813bdd3614d13cc6bacbf0846c139d8f6da027af.

Reason for revert: This commit was done to avoid duplication of certmanager across all clusterctl components. However with the current notion of maintaining the upstream version as is, this change should be reverted. Also there is WIP to implement cert-manager as a standalone component even before clusterctl tries to deploy its embedded certmanager. So this commit will avoid manipulating the override in the standalone certmanager component

Change-Id: I16b0b002da990eb3f070f69d025cb904f7ef9e27
2020-12-10 15:40:03 +00:00
Battina, Sai (sb464f)
f49b509534 Update IPA packages in ironic
This PS updates ironic deployment with IPA init container
which contains python agent agent packages which are ubuntu based
packages. The docker image for this can found here [0]

[0]
https://opendev.org/airship/images/src/branch/master/ipa-downloader-image

Change-Id: Idaaafb2ddb562cf22a62df36100dd1e6c76211fc
2020-12-10 15:07:04 +00:00
Ratnopam Chakrabarti
630464bb8d Change target cluster name for CAPG provider
Change target cluster name from gtc to target-cluster for
CAPG provider

Relates-To: #426

Change-Id: I0e694cfae1957a84b668cd36f344dd7c2d00db3d
2020-12-07 18:06:00 +00:00
Ratnopam Chakrabarti
e9f9d27732 Update CAPG test site to support phase run
Added support for following phases in CAPG provider
 clusterctl-init-ephemeral
 controlplane-ephemeral
 initinfra-target
 clusterctl-init-target
 clusterctl-move
 workers-target

Added support for deploying calico CNI for CAPG
through initinfra-target phase

Relates-To: #425
Change-Id: I91b29e561444d82d33802157eba5a1d94b25ba04
2020-12-07 11:45:47 -05:00
Ruslan Aliev
4258d1904e Add missing metadata.yaml definitions
Some of sites defined in document model have no metadata.yaml,
which is essential to work properly with that sites.
 * az-test-site and gcp-test-site metadata.yaml added
 * metadata.yaml for test-site moved to appropriate directory
   for consistency

Change-Id: Ibec109e41a5e3fd36794dc6a879888cfc610c9cc
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #19
2020-12-03 19:34:09 -06:00
Zuul
4be3ae03eb Merge "Azure provider integration - CAPZ Calico" 2020-12-02 18:56:37 +00:00
Sidney Shiba
3e3dadb8b7 Azure provider integration - CAPZ Calico
Azure does not currently suport Calico networking.
As a workaround, this patch set includes CAPZ Calico manifests that uses
VXLAN, instead.

The CAPZ Calico manifest are located under
manifest/function/cni/calico-capz.

Change-Id: Iadb2d5e10131e6a2df8cef49e2ec189ab948eeb9
2020-11-30 17:29:26 -06:00
Vladislav Kuzmin
258b81122e Add generic container executor
This executor based on kyaml.RunFns[1].
It allows to execute arbitrary container as RunFns function
and handle any input data as ExecutorBundle.
Be aware that runtimeutils.FunctionSpec used for version 0.4.1[2]
and it can be changed in the future. It means that yaml scheme
of executor can be changed during kyaml uplift.
This executor transforms executor yaml to RunFns function acceptable format.

[1] https://github.com/kubernetes-sigs/kustomize/blob/master/kyaml/runfn/runfn.go
[2] https://github.com/kubernetes-sigs/kustomize/blob/kyaml/v0.4.1/kyaml/fn/runtime/runtimeutil/functiontypes.go#L22

Change-Id: I9eb648783f1ed462d2b8359d9dc86320c4db2a83
Closes: #202
Closes: #369
2020-11-26 17:11:56 +04:00
Zuul
c18db07043 Merge "Azure provider integration - K8S-Control & Workers" 2020-11-23 18:41:40 +00:00
Zuul
7d52fa392f Merge "Azure provider integration - Phase Run support" 2020-11-23 18:33:02 +00:00
Zuul
d7a4c11a30 Merge "Azure provider integration - CAPZ v0.4.9" 2020-11-23 18:23:05 +00:00
Zuul
26fcff2509 Merge "Fix ephemeral disk mounting" 2020-11-23 17:01:20 +00:00
Sidney Shiba
b0eef10ca9 Azure provider integration - Phase Run support
This patch set includes the site specifig for Azure (az-test-site) that
contains the manifests and kustomization files needed to deploy a
Workload cluster on the Azure cloud platform.

This patch set uses the Azure CAPZ v0.4.9 for the Workload cluster
deployment.

Change-Id: Ie71630bf55edadfcc11527c04aea41aa2161bdbd
2020-11-19 13:42:04 -06:00
Sidney Shiba
14e01c9b70 Azure provider integration - K8S-Control & Workers
This patch set includes the manifest templates for the Workload cluster
deployment for CAPZ v0.4.9.

Also, added the new folder manifest/function/workers-capz and moved the
manifests for v0.4.8 under it, as well as for v0.4.9

Change-Id: Ie08b6e7092f90ac8ee533c574a26eaf9b6600c5c
2020-11-18 11:53:47 -06:00
Sidney Shiba
abb3a10032 Azure provider integration - CAPZ v0.4.9
This patchset includes the manifests for CAPZ version v0.4.9, which has
been added under airshipctl/manifests/function/capz/v0.4.9

Also added an entry in the clusterctl/clusterctl.yaml for CAPZ

Change-Id: I3a52257320af1f68ec2d8f4302e7176b73f227dd
2020-11-17 18:47:14 -06:00
Ratnopam Chakrabarti
58f9f99d83 openstack-test-site for capo
adjust openstack-test-site to execute the below phases
for openstack provider(capo)

  clusterctl-init-ephemeral
  controlplane-ephemeral
  clusterctl-init-target
  clusterctl-move
  workers-target

A detailed test run of the phases can be found in
https://hackmd.io/OheCGmq8RX64SPw2sF3qGQ

Change-Id: Id982273d28515eb3a44c39d869eaeb229923339c
2020-11-13 15:45:08 +00:00
Zuul
411bbf899a Merge "Align CAPO to use catalogue-driven versions" 2020-11-13 02:59:09 +00:00
Sirajudeen
b266e18fd4 Image override for cert-manager components
* Images added to clusterctl object like Providers
  So that repository and tag for one or all of the cert-manager
  components can be overriden using patch/replacement

clusterctl Documentation for Image Override:
  https://cluster-api.sigs.k8s.io/clusterctl/configuration.html#image-overrides

Change-Id: Id9de8d1967e49aeb3293f6802e51d66d598333ae
Closes: #350
2020-11-12 17:12:23 +00:00
Zuul
df3764b7ac Merge "Remove document plugin subcommand" 2020-11-12 03:03:33 +00:00
Ratnopam Chakrabarti
379c42a7b2 Align CAPO to use catalogue-driven versions
Update CAPO provider manifest to inherit clusterctl from
  functions and patch the infrastructure provider details
  Add capo versions in base-catalog

Change-Id: I8f1f59857dcbf163af01a5a766451e6579ded148
2020-11-11 15:18:40 +00:00
Zuul
7a404c6fa6 Merge "Incorporate HWCC in deployment gate." 2020-11-11 12:08:22 +00:00