Integration of Azure provider to Airship 2.0

This commit integrates the Azure provider to the Airship 2.0 project. It adds the following folders: - manifest/function/capz: This folder contains all manifests required for the integration of Azure provider. - manifest/function/k8scontrol-capz: This folder contains the base manifest for the Azure Workload cluster. - manifest/site/az-test-site: This folder contains the manifests used for initializing the CAPI and CAPZ components on the management cluster invoking "airshipctl cluster init" and manifests used for deploying a Workload cluster on the Azure Cloud by invoking the command "airshipctl phase apply azure". - tools/deployment/azure: provides the script shell that are used in the zuul gates and local test. Updated files: - zuul.d/project.yaml and zuul.d/jobs.yaml have been updated to include gates for validating the Azure provider integration. Change-Id: Icbdc7f6f42c159f48dd11e35626da3bc016f5487
2020-09-09 12:41:04 -05:00
parent 8c180daf4e
commit d0683139fe
77 changed files with 4671 additions and 0 deletions
--- a/tools/deployment/azure/100_clean_up_resources.sh
+++ b/tools/deployment/azure/100_clean_up_resources.sh
@@ -0,0 +1,23 @@
+#!/bin/bash
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+set -xe
+
+# Deleting Target Cluster resources from Azure Cloud
+echo "Deleting Target Cluster on Azure Cloud ..."
+airshipctl phase render azure -k Cluster | kubectl delete --kubeconfig ~/.airship/kubeconfig -f -
+
+# Deleting the Management cluster
+kind delete cluster --name capi-azure
--- a/tools/deployment/azure/11_build_kind_cluster.sh
+++ b/tools/deployment/azure/11_build_kind_cluster.sh
@@ -0,0 +1,50 @@
+#!/bin/bash
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+set -xe
+
+#Default wait timeout is 3600 seconds
+export TIMEOUT=${TIMEOUT:-3600}
+export KUBECONFIG=${KUBECONFIG:-"$HOME/.kube/config"}
+
+REMOTE_WORK_DIR=/tmp
+
+# get kind
+echo "Fetching kind from ${KIND_URL}..."
+TMP=$(KIND_URL=${KIND_URL} ./tools/document/get_kind.sh)
+export KIND=${TMP}/kind
+
+$KIND delete cluster --name capi-azure
+$KIND create cluster --name capi-azure
+
+#Wait till CAPI/CAPZ Control Plane Node is ready
+end=$(($(date +%s) + $TIMEOUT))
+echo "Waiting $TIMEOUT seconds for Capi Docker Control Plane node to be ready."
+while true; do
+    if (kubectl --request-timeout 20s --kubeconfig $KUBECONFIG get nodes capi-azure-control-plane -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}' | grep -q True) ; then
+        echo -e "\nCAPI Docker Control Plane Node is ready."
+        kubectl --request-timeout 20s --kubeconfig $KUBECONFIG get nodes
+        break
+    else
+        now=$(date +%s)
+        if [ $now -gt $end ]; then
+            echo -e "\nCAPI Azure Control Plane Node was not ready before TIMEOUT."
+            exit 1
+        fi
+        echo -n .
+        sleep 15
+    fi
+done
+
--- a/tools/deployment/azure/200_configure_test_vm.sh
+++ b/tools/deployment/azure/200_configure_test_vm.sh
@@ -0,0 +1,58 @@
+#!/bin/bash
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script transfers the local Airship project to the remote machine,
+# copies the test/validation script to the remote machine and then executes it.
+#
+# This script is a local test simulating Zuul gates used to test and validate
+# the Azure clout integration.
+
+if [ $# -eq 3 ]; then
+    echo "--- Remote VM Username@IP = $1"
+    echo "--- Local Airship Dir = $2"
+    echo "--- Credentials Script = $3"
+
+    export REMOTE_VM=$1
+    export LOCAL_AIRSHIP_DIR=$2
+    export CREDENTIALS=$3
+else
+    echo "Syntax: 200_configure-remote-vm.sh <Remote VM Username> <Remote VM IP> <Local Airship Dir>"
+    echo "    <Remote VM Username>: Username@VM_IP to login to the Remote VM"
+    echo "    <Local Airship Dir>: Directory containing the Airship project, e.g., /home/esidshi/projects/airshipctl/"
+    echo "    <Credentials script>: script to be used by remote VM for setting the credentials for public Clouds"
+    exit 1
+fi
+
+echo "Remote Username@VM = $REMOTE_VM"
+echo "Local Airship Dir" = $LOCAL_AIRSHIP_DIR
+echo "Credentials Script = $CREDENTIALS"
+
+# Pushing local SSH Public Key to Remote VM
+echo "Adding local VM public in the Remote VM ..."
+ssh-copy-id -o StrictHostKeyChecking=no -i ~/.ssh/id_rsa.pub $REMOTE_VM
+
+# Installing Docker in the remote VM
+echo "Installing Docker ..."
+# ssh $REMOTE_VM 'sudo apt update -y && sudo apt install docker.io && sudo usermod -aG docker $USER'
+ssh $REMOTE_VM 'sudo apt-get remove docker docker-engine docker.io containerd runc && sudo apt-get update'
+ssh $REMOTE_VM 'sudo apt-get install -y apt-transport-https ca-certificates curl gnupg-agent software-properties-common'
+ssh $REMOTE_VM 'curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - && sudo apt-key fingerprint 0EBFCD88'
+ssh $REMOTE_VM 'sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"'
+ssh $REMOTE_VM 'sudo apt-get update && sudo apt-get install -y docker-ce docker-ce-cli containerd.io && sudo usermod -aG docker $USER'
+
+# Installing Make in the remote VM
+ssh $REMOTE_VM 'sudo apt-get update -y && sudo apt install make'
+
+# Transfer the manifests to the remote VM and start the local test
+$LOCAL_AIRSHIP_DIR/tools/deployment/azure/200_transfer_airshipctl.sh $REMOTE_VM $LOCAL_AIRSHIP_DIR $CREDENTIALS
--- a/tools/deployment/azure/200_transfer_airshipctl.sh
+++ b/tools/deployment/azure/200_transfer_airshipctl.sh
@@ -0,0 +1,78 @@
+#!/bin/bash
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script transfers the local Airship project to the remote machine,
+# copies the test/validation script to the remote machine and then executes it.
+#
+# This script is a local test simulating Zuul gates used to test and validate
+# the Azure clout integration.
+
+if [ -n "$3" ]; then
+    echo "--- Remote username@VM = $1"
+    echo "--- Local Airship Dir = $2"
+    echo "--- Credentials script = $3"
+
+    export REMOTE_VM=$1
+    export LOCAL_AIRSHIP_DIR=$2
+    export CREDENTIALS=$3
+else
+    echo "Syntax: 200_transfer_airshipctl.sh <Remote VM Username> <Remote VM IP> <Local Airship Dir>"
+    echo "    <Remote VM>: Username@VM to login to the Remote VM"
+    echo "    <Local Airship Dir>: Directory containing the Airship project, e.g., /home/esidshi/projects/airshipctl/"
+    echo "    <Credentials script>: used by remote VM to set public Cloud credentials"
+    exit 1
+fi
+
+export REMOTE_USERNAME=$(echo $REMOTE_VM | cut -d'@' -f 1)
+echo "Remote Username = $REMOTE_USERNAME"
+echo "Remote VM = $REMOTE_VM"
+echo "Local Airshipt Dir = $LOCAL_AIRSHIP_DIR"
+echo "Credentials Script = $CREDENTIALS"
+
+# Preparing the Remote VM to the "Zero Touch" Validation
+cd $LOCAL_AIRSHIP_DIR
+echo "sudo mkdir /home/zuul"
+ssh -o StrictHostKeyChecking=no $REMOTE_VM 'sudo mkdir /home/zuul'
+
+echo "sudo chown ${REMOTE_USERNAME} /home/zuul"
+ssh  $REMOTE_VM "sudo chown ${REMOTE_USERNAME} /home/zuul"
+
+echo "sudo chgrp ${REMOTE_USERNAME} /home/zuul"
+ssh  $REMOTE_VM "sudo chgrp ${REMOTE_USERNAME} /home/zuul"
+
+echo "mkdir /home/zuul/src"
+ssh  $REMOTE_VM 'mkdir /home/zuul/src'
+
+echo "mkdir /home/zuul/src/opendev.org"
+ssh  $REMOTE_VM 'mkdir /home/zuul/src/opendev.org'
+
+echo "mkdir /home/zuul/src/opendev.org/airship"
+ssh  $REMOTE_VM 'mkdir /home/zuul/src/opendev.org/airship'
+
+echo "scp -r $LOCAL_AIRSHIP_DIR/  $REMOTE_VM:/home/zuul/src/opendev.org/airship/airshipctl"
+scp -r $LOCAL_AIRSHIP_DIR/  $REMOTE_VM:/home/zuul/src/opendev.org/airship/airshipctl
+
+echo "scp $LOCAL_AIRSHIP_DIR/tools/deployment/azure/201_zuul_local_test.sh  $REMOTE_VM:~"
+scp $LOCAL_AIRSHIP_DIR/tools/deployment/azure/201_zuul_local_test.sh  $REMOTE_VM:~
+scp $CREDENTIALS  $REMOTE_VM:~
+
+# echo "ssh  $REMOTE_VM 'bash ~/201_zuul_local_test.sh'"
+export CREDENTIALS_FILENAME=$(echo ${CREDENTIALS##*/}) # extract the script filename only
+echo "CREDENTIALS_FILENAME = $CREDENTIALS_FILENAME"
+
+# Setting Public Cloud credentials as environment variables in the remote VM
+ssh $REMOTE_VM "cat ${CREDENTIALS_FILENAME} >> ~/.profile"
+
+# Executing the local test
+ssh $REMOTE_VM '/bin/bash ~/201_zuul_local_test.sh'
--- a/tools/deployment/azure/201_zuul_local_test.sh
+++ b/tools/deployment/azure/201_zuul_local_test.sh
@@ -0,0 +1,100 @@
+#!/bin/bash
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script simulates the Zuul gates for validation of Azure cloud integration.
+# It goes through all gates for locally in the remote VM.
+
+echo "----- Using default airship directory"
+export AIRSHIPDIR="/home/zuul/src/opendev.org/airship/airshipctl"
+export AIRSHIPCTL_WS="/home/zuul/src/opendev.org/airship/airshipctl"
+export PATH=$PATH:/usr/local/go/bin/
+
+# Setting Public Cloud Credentials as Environment Variables
+echo "source ~/.profile"
+source ~/.profile
+
+echo "Azure Subscription ID = $AZURE_SUBSCRIPTION_ID"
+echo "Azure Tenant ID = $AZURE_TENANT_ID"
+echo "Google Cloud Project ID = $GCP_PROJECT"
+echo "Google Cloud Account ID = $GCP_ACCOUNT"
+
+cd $AIRSHIPCTL_WS
+echo "----- Airship Directory = $AIRSHIPCTL_WS"
+
+# Installation of Kubectl
+echo "************************************************************************"
+echo "***** Installation of Kubectl ..."
+./tools/deployment/01_install_kubectl.sh
+if [ $? -ne 0 ]; then
+    echo ">>>>> Failed to Install Kubectl CLI"
+    exit 1
+fi
+
+# Build Kind Cluster
+echo "************************************************************************"
+echo "***** Building Kind Cluster ..."
+./tools/deployment/azure/11_build_kind_cluster.sh
+if [ $? -ne 0 ]; then
+    echo ">>>>> Failed to build Kind cluster"
+    exit 1
+fi
+
+# Building airshipctl command
+echo "************************************************************************"
+echo "***** Building airshipctl command ..."
+./tools/deployment/21_systemwide_executable.sh
+if [ $? -ne 0 ]; then
+    echo ">>>>> Failed to build airshipctl CLI"
+    exit 1
+fi
+
+# Creating Airship config file
+echo "************************************************************************"
+echo "***** Creating Airship config file ..."
+./tools/deployment/azure/31_create_configs.sh
+if [ $? -ne 0 ]; then
+    echo ">>>>> Failed to create airshipctl config file"
+    exit 1
+fi
+
+# Initializing CAPI and CAPZ components for the Managemeng cluster
+echo "************************************************************************"
+echo "***** Initializing CAPI and CAPZ components for the Managemeng cluster ..."
+./tools/deployment/azure/41_initialize_management_cluster.sh
+if [ $? -ne 0 ]; then
+    echo ">>>>> Failed to initialize the Ephemeral cluster with CAPI/CAPZ components"
+    exit 1
+fi
+
+# Deploying the Target Cluster in Azure cloud
+echo "************************************************************************"
+echo "***** Deploying the Target Cluster in azure cloud ..."
+./tools/deployment/azure/51_deploy_workload_cluster.sh
+if [ $? -ne 0 ]; then
+    echo ">>>>> Failed to deploy Target/Workload cluster on Azure Cloud"
+    exit 1
+fi
+
+# Sleep for 15 min before start cleaning up everything.
+echo "Waiting for 15 min..."
+sleep 15m
+
+# Cleaning up Resources
+echo "************************************************************************"
+echo "***** Cleaning up resources ..."
+./tools/deployment/azure/100_clean_up_resources.sh
+if [ $? -ne 0 ]; then
+    echo ">>>>> Failed to clean up all public cloud resources created to this test"
+    exit 1
+fi
--- a/tools/deployment/azure/31_create_configs.sh
+++ b/tools/deployment/azure/31_create_configs.sh
@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -xe
+
+export ISO_DIR=${ISO_DIR:-"/srv/iso"}
+export SERVE_PORT=${SERVE_PORT:-"8099"}
+export AIRSHIPCTL_WS=${AIRSHIPCTL_WS:-$PWD}
+export USER_NAME=${USER:-"ubuntu"}
+export USE_PROXY=${USE_PROXY:-"false"}
+export HTTPS_PROXY=${HTTPS_PROXY:-${https_proxy}}
+export HTTPS_PROXY=${HTTP_PROXY:-${http_proxy}}
+export NO_PROXY=${NO_PROXY:-${no_proxy}}
+export REMOTE_WORK_DIR=${remote_work_dir:-"/tmp/airship"}
+export AIRSHIP_CONFIG_ISO_GEN_TARGET_PATH=${ISO_DIR}
+export AIRSHIP_CONFIG_ISO_BUILDER_DOCKER_IMAGE=${BUILDER_IMAGE:-"quay.io/airshipit/isogen:latest-debian_stable"}
+export REMOTE_TYPE=redfish
+export REMOTE_INSECURE=true
+export REMOTE_PROXY=false
+export AIRSHIP_CONFIG_ISO_SERVE_HOST=${HOST:-"localhost"}
+export AIRSHIP_CONFIG_ISO_PORT=${SERVE_PORT}
+export AIRSHIP_CONFIG_ISO_NAME=${ISO_NAME:-"debian-custom.iso"}
+export SYSTEM_ACTION_RETRIES=30
+export SYSTEM_REBOOT_DELAY=30
+export AIRSHIP_CONFIG_PRIMARY_REPO_BRANCH=${BRANCH:-"master"}
+# the git repo url or local file system path to a cloned repo, e.g., /home/stack/airshipctl
+export AIRSHIP_CONFIG_PRIMARY_REPO_URL=${REPO:-"https://review.opendev.org/airship/airshipctl"}
+export AIRSHIP_SITE_NAME="airshipctl/manifests/site/az-test-site"
+export AIRSHIP_CONFIG_MANIFEST_DIRECTORY=${remote_work_dir}
+export AIRSHIP_CONFIG_CA_DATA=$(cat tools/deployment/certificates/airship_config_ca_data| base64 -w0)
+export AIRSHIP_CONFIG_EPHEMERAL_IP=${IP_Ephemeral:-"10.23.25.101"}
+export AIRSHIP_CONFIG_CLIENT_CERT_DATA=$(cat tools/deployment/certificates/airship_config_client_cert_data| base64 -w0)
+export AIRSHIP_CONFIG_CLIENT_KEY_DATA=$(cat tools/deployment/certificates/airship_config_client_key_data| base64 -w0)
+
+#Remove and Create .airship folder
+rm -rf $HOME/.airship
+mkdir -p $HOME/.airship
+
+cp ~/.kube/config ~/.airship/kubeconfig
+
+echo "Generate ~/.airship/config and ~/.airship/kubeconfig"
+envsubst <"${AIRSHIPCTL_WS}/tools/deployment/templates/azure_airshipconfig_template" > ~/.airship/config
--- a/tools/deployment/azure/41_initialize_management_cluster.sh
+++ b/tools/deployment/azure/41_initialize_management_cluster.sh
@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -xe
+
+export AIRSHIPCTL_WS=${AIRSHIPCTL_WS:-$PWD}
+export KUBECONFIG=${KUBECONFIG:-"$HOME/.airship/kubeconfig"}
+
+echo "Initialize Management Cluster with CAPI and CAPZ Components"
+airshipctl cluster init --debug
+
+echo "Waiting for all pods to come up"
+kubectl --kubeconfig $KUBECONFIG wait --for=condition=ready pods --all --timeout=1000s -A
+kubectl --kubeconfig $KUBECONFIG get pods -A
+
--- a/tools/deployment/azure/51_deploy_workload_cluster.sh
+++ b/tools/deployment/azure/51_deploy_workload_cluster.sh
@@ -0,0 +1,77 @@
+#!/usr/bin/env bash
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+echo "Deploy Target Workload Cluster"
+airshipctl phase apply azure
+
+echo "Get kubeconfig from secret"
+KUBECONFIG=""
+N=0
+MAX_RETRY=30
+DELAY=60
+until [ "$N" -ge ${MAX_RETRY} ]
+do
+  KUBECONFIG=$(kubectl --kubeconfig ~/.airship/kubeconfig --namespace=default get secret/az-workload-cluster-kubeconfig -o jsonpath={.data.value}  || true)
+
+  if [[ ! -z "$KUBECONFIG" ]]; then
+      break
+  fi
+
+  N=$((N+1))
+  echo "$N: Retry to get target cluster kubeconfig from secret."
+  sleep ${DELAY}
+done
+
+if [[ -z "$KUBECONFIG" ]]; then
+  echo "Could not get target cluster kubeconfig from sceret."
+  exit 1
+fi
+
+echo "Create kubeconfig"
+echo ${KUBECONFIG} | base64 -d > /tmp/target.kubeconfig
+
+echo "Get Machine State"
+kubectl get machines
+
+echo "Check kubectl version"
+VERSION=""
+N=0
+MAX_RETRY=30
+DELAY=60
+until [ "$N" -ge ${MAX_RETRY} ]
+do
+  VERSION=$(timeout 20 kubectl --kubeconfig /tmp/target.kubeconfig version | grep 'Server Version' || true)
+
+  if [[ ! -z "$VERSION" ]]; then
+      break
+  fi
+
+  N=$((N+1))
+  echo "$N: Retry to get kubectl version."
+  sleep ${DELAY}
+done
+
+if [[ -z "$VERSION" ]]; then
+  echo "Could not get kubectl version."
+  exit 1
+fi
+
+echo "Check nodes status"
+
+kubectl --kubeconfig /tmp/target.kubeconfig wait --for=condition=Ready node --all --timeout 900s
+kubectl get nodes --kubeconfig /tmp/target.kubeconfig
+
+echo "Get cluster state"
+kubectl --kubeconfig ${HOME}/.airship/kubeconfig get cluster
+
--- a/tools/deployment/azure/README.md
+++ b/tools/deployment/azure/README.md
@@ -0,0 +1,60 @@
+# Zuul Gate Scripts for Bootstrap Container/Ephemeral Cluster/Azure Target Cluster
+This folder container the Zuul Gate Scripts for configuring the tools necessary to
+automatically validate the building of Bootstrap Container (Go app + Docker Image),
+deployment of Ephemeral Cluster on Azure Cloud and Google Cloud, then using the
+Azure Ephemeral Cluster to deploy the Target Cluster on Azure Cloud.
+
+## Installing and Configuring Tools
+The following scripts are used to install and configure tools such as "kubectl", "Go", "Kustomize" and "airshipctl":
+- 01_install_kubectl.sh - installs "kubectl" CLI
+- 02_install_go.sh - installs the "Go" language
+- 03_install_kustomize_docker.sh - install "kustomize" CLI
+- 21_systemwide_executable.sh - build the "airshipctl" CLI
+
+## Bootstrap Container and Ephemeral Cluster
+The following scrips are used to deploy the Ephemeral cluster on Azure and Google Cloud.
+- 41_deploy_azure_ephemeral_cluster.sh - creates the Azure Bootstrap container that deploys the Azure (AKS) Ephemeral cluster
+- 41_initialize_management_cluster.sh - creates the GCP Bootstrap container that deploys the GCP (GKE) Ephemeral cluster
+
+> NOTE: the Bootstrap container images shall be built and pushed to **quay.io** registry prior to executing these scripts.
+## Initializing the Ephemeral cluster and Deploying the Target Cluster
+The following scripts initialize the Ephemeral cluster with CAPI and CAPZ components
+and deploy the Target/Workload cluster on the Azure Cloud platform.
+- 41_initialize_management_cluster.sh - initializes the Azure Ephemeral cluster with CAPI and CAPZ components
+- 51_deploy_workload_cluster.sh - deploys a Target/Workload cluster on the Azure Cloud platform
+
+And last but not least, the following scripts is a clean up script, deleting all resources created
+the public clouds, including the ephemeral clusters.
+- 100_clean_up_resources.sh
+
+## Supporting Local Test Scripts
+The scripts in this section are used for testing the end-to-end testing pipeline outside the Zuul
+environment. It simulates the Zuul pipeline on a clean remote VM, e.g., VM created on Azure Cloud.
+- 201_zuul_local_test.sh - simulates the sequence of scripts to run on a Zuul environment.
+- 200_transfer_airshipctl.sh - this script transfers the airshipctl local repository to the test VM then executes 201_zuul_local_test.sh
+- 200_configure_test_vm.sh - Prepares the test VM with basic tools such as "make" and "docker", then executes 200_transfer_airshipctl.sh.
+
+By executing *200_configure_test_vm.sh* on a development server will trigger the entire test pipeline, i.e., "Zero Touch" local test.
+
+Pre-requisite: the *200_configure_test_vm.sh* requires a special script file that exports environment variables specific for the
+Azure and GCP Cloud account credentials. See template for this script below:
+
+```bash
+# Azure cloud authentication credentials.
+export AZURE_SUBSCRIPTION_ID="<Your Azure Subscription ID>"
+export AZURE_TENANT_ID="<Your Tenant ID>"
+export AZURE_CLIENT_ID="<Your Service Principal ID>"
+export AZURE_CLIENT_SECRET="<Your Service Principal Secret>"
+
+# To use the default public cloud, otherwise set to AzureChinaCloud|AzureGermanCloud|AzureUSGovernmentCloud
+export AZURE_ENVIRONMENT="AzurePublicCloud"
+
+export AZURE_SUBSCRIPTION_ID_B64="$(echo -n "$AZURE_SUBSCRIPTION_ID" | base64 | tr -d '\n')"
+export AZURE_TENANT_ID_B64="$(echo -n "$AZURE_TENANT_ID" | base64 | tr -d '\n')"
+export AZURE_CLIENT_ID_B64="$(echo -n "$AZURE_CLIENT_ID" | base64 | tr -d '\n')"
+export AZURE_CLIENT_SECRET_B64="$(echo -n "$AZURE_CLIENT_SECRET" | base64 | tr -d '\n')"
+
+# GCP Environment Variables
+export GCP_PROJECT=<Your Google Cloud Project ID>
+export GCP_ACCOUNT=<Your Google Cloud Account ID>
+```