airshipctl/docs/source/phases.rst

Phase execution engine

General phase architecture

Airshipctl performs operations such as kubectl apply, clusterctl init etc... using phases defined in a phase bundle. You can think of a phase as a step that has to be performed in order to achieve a desired state of the managed site.

A phase is usually run against an executor bundle, which is a document set containing kubernetes resources.

Executor bundles and phase bundles are separate bundles and it is important to understand the difference between them:

• A phase bundle can be thought of as an airshipctl configuration bundle, it defines what will be executed and against which executor bundles. Important: path to phase bundle is constructed using the airship config current manifest TargetPath, phaseRepositoryName directory and phase.path field taken from metadata file.
• An executor bundle defines a document set that the executor will run against, for example kubernetes manifests, that will be applied to the kubernetes cluster by the KubernetesApply executor. The path to an executor bundle is constructed using targetPath, phaseRepoDir, docEntryPointPrefix(optional), documentEntryPoint.

architecture diagram

Steps performed during phase execution based on diagram above:

1. Load metadata file based on airshipctl config from manifest
2. Build phase bundle (please see phase bundle section for the details on how the path to the phase bundle is constructed using manifest and metadata.yaml file)

3. Prepare executor config

   • Find requested phase in phase document bundle
   • Load kubeconfig and cluster map from phase document bundle
   • Build executor bundle based on phase document

4. Find executor document in phase Bundle and load executor providing config from step 3
5. Run executor, (validate/render available as well)

Phase

Phase is a basic step that is executed by airshipctl to get a site to a desired state. Phases are defined in phase document bundles as a phase document containing:

• type meta

  apiVersion: airshipit.org/v1alpha1
  kind: Phase

• object meta

  Note that the clusterName that the phase belongs to comes from objects metadata and not from phase config. ClusterName is used to identify kubeconfig context and if needed cluster name of a parent cluster to be used during phase execution, also this clusterName must be present in the cluster map.

  metadata:
    name: initinfra-ephemeral
    clusterName: ephemeral-cluster

• config

  Can include documentEntryPoint relative to TargetPath + phaseRepoDir + docEntryPointPrefix, some executors don't need a bundle, for example clusterctl move, and in such cases documentEntryPoint field can be omitted. Also it includes an executorRef, which is a k8s ObjectReference, to an executor document, which defines an executor that will run the phase. The executor document referenced must exist in phase bundle

  config:
    executorRef:
      apiVersion: airshipit.org/v1alpha1
      kind: KubernetesApply
      name: kubernetes-apply
    documentEntryPoint: ephemeral/initinfra

Complete phase example:

apiVersion: airshipit.org/v1alpha1
kind: Phase
metadata:
  name: initinfra-ephemeral
  clusterName: ephemeral-cluster
config:
  executorRef:
    apiVersion: airshipit.org/v1alpha1
    kind: KubernetesApply
    name: kubernetes-apply
  documentEntryPoint: ephemeral/initinfra

Phase Bundle

A phase bundle controls how airshipctl will run, it contains but not limited to objects depicted in the following table:

Name	Description	API object source
Cluster map	Defines relationship between clusters	Cluster map API object source code
Phase	Basic step that is executed by airshipctl	phase document
Kubeconfig	Credentials for kubernetes clusters	kubeconfig api object
Phase plan	Sequence of phase execution	plan api object
Executor document	Define what is going to be used by phases	see executor section

How is the path to a phase bundle constructed? - A phase bundle is constructed using kustomize root that consists of TargetPath defined in current context manifest, and phase path taken from metadata file that is located inside the manifest: TargetPath + Metadata.Phase.Path

Executor Bundle

• Executor bundle may contain any documents, they are available to executor during execution. For example these documents can be k8s resources, such as Deployments, Secrets, Services etc, and KubernetesApply executor will apply those resources to k8s cluster that phase belongs to.
• Some executors may require an executor bundle, some may not. Clusterctl move is an example of an executor that doesn't need a bundle
• Each executor may perform different actions based on the document set
• The path to the executor bundle is currently constructed using TargetPath + phaseRepositoryName + metadata.Phase.DocEntryPointPrefix + DocumentEntrypoint

Executor

Executor is what actually performs execution of the phase. If a phase is considered as an abstract step of execution process, then Executor is an implementation of this step. Executors are defined as executor documents inside phase bundle, these documents contain configuration for the executor, Kind and apiVersion that is mapped by airshipctl internally to execution module that is invoked during airshipctl phase commands.

Executor has access to kubeconfig, cluster map and its cluster name, which allows the executor to find its context in kubeconfig, and if needed parent cluster name and context as well.

Currently there is a limited, but growing number of executors:

• KubernetesApply: applies resources to kubernetes.
• Clusterctl: performs clusterctl commands based on its config.

Note: for more information about each executor please refer to the code base, in the future more documentation will be developed for each executor.

Clusterctl executor document example

• Executor source code
• Executor API object source code

apiVersion: airshipit.org/v1alpha1
kind: Clusterctl
metadata:
  labels:
    airshipit.org/deploy-k8s: "false"
  name: clusterctl_init
init-options:
  core-provider: "cluster-api:v0.3.7"
  bootstrap-providers:
    - "kubeadm:v0.3.7"
  infrastructure-providers:
    - "metal3:v0.3.2"
  control-plane-providers:
    - "kubeadm:v0.3.7"
action: init
providers:
  - name: "metal3"
    type: "InfrastructureProvider"
    variable-substitution: true
    versions:
      v0.3.2: airshipctl/manifests/function/capm3/v0.3.2
  - name: "kubeadm"
    type: "BootstrapProvider"
    variable-substitution: true
    versions:
      v0.3.7: airshipctl/manifests/function/cabpk/v0.3.7
  - name: "cluster-api"
    type: "CoreProvider"
    variable-substitution: true
    versions:
      v0.3.7: airshipctl/manifests/function/capi/v0.3.7
  - name: "kubeadm"
    type: "ControlPlaneProvider"
    variable-substitution: true
    versions:
      v0.3.7: airshipctl/manifests/function/cacpk/v0.3.7
additional-vars:
  CONTAINER_CAPM3_MANAGER: quay.io/metal3-io/cluster-api-provider-metal3:v0.3.2

KubernetesApply executor document example

• Executor source code
• Executor API object source code

apiVersion: airshipit.org/v1alpha1
kind: KubernetesApply
metadata:
  labels:
    airshipit.org/deploy-k8s: "false"
  name: kubernetes-apply
config:
  waitOptions:
    timeout: 2000
  pruneOptions:
    prune: false

Kubeconfig

Executors have access to site's kubeconfig. In kubeconfig, context names must correspond to cluster names, for example if cluster name is workload01, then kubeconfig should have context named workload01 to be able to access this kubernetes cluster. (TODO: update this later when https://github.com/airshipit/airshipctl/issues/380 is closed)

Kubeconfig can come from various sources:

• Phase bundle, standard way of supplying kubeconfig is putting it into phase bundle as kubeconfig api object, Kubeconfig api object example. This is a good option when you are supplying predefined certificates for some of the clusters (ephemeral and target clusters) while still leaving possibility to use dynamic kubeconfig from a parent cluster for worker clusters.
• User provided , if a user wants to override kubeconfig, and use their own, they can do so using --kubeconfig flag in airshipctl, but they MUST be aware that kubeconfig context names must correspond to cluster names. (TODO: update this later when https://github.com/airshipit/airshipctl/issues/380 is closed)
• Parent cluster (not implemented), airshipctl uses clusterctl to deploy k8s clusters. And can use parent cluster to get kubeconfig secret from it, in such case dynamic kubeconfig should be allowed to be used in cluster map, this is done by setting dynamicKubeConf: true in cluster map for specific cluster, default is false.

Kubeconfig api object example

apiVersion: airshipit.org/v1alpha1
kind: KubeConfig
metadata:
  name: default
  labels:
    airshipit.org/deploy-k8s: "false"
config:
  apiVersion: v1
  clusters:
  - cluster:
      certificate-authority-data: <ca-data>
      server: https://10.23.25.102:6443
    name: target-cluster
  - cluster:
      certificate-authority-data: <ca-data>
      server: https://10.23.25.101:6443
    name: ephemeral-cluster
  contexts:
  - context:
      cluster: target-cluster
      user: target-cluster-admin
    name: target-cluster
  - context:
      cluster: ephemeral-cluster
      user: ephemeral-cluster-admin
    name: ephemeral-cluster
  current-context: ""
  kind: Config
  preferences: {}
  users:
  - name: ephemeral-cluster-admin
    user:
      client-certificate-data: <client-cert-data>
      client-key-data: <client-key-data>
  - name: target-cluster-admin
    user:
      client-certificate-data: <client-cert-data>
      client-key-data: <client-key-data>

Cluster Map

Cluster map defines parent-child relationship between clusters, allows dynamic kubeconfig for clusters. When kubeconfig must be sourced from a parent cluster, cluster map will be used to find parent cluster and request kubeconfig from it.

Also, ClusterAPIRef is a part of Cluster Map and will be used to find cluster object in kubernetes parent cluster. It also maps a clusterapi name and namespaces for a given cluster. In the below example object, the cluster api ref describes the reference to the cluster api object. The cluster workload01 in the cluster map has an clusterAPIRef to corresponding cluster-api cluster object (kind: cluster) with name workload01 and inside namespace tenant01-namespace.

Cluster map is defined in Phase bundle as a document.

• Cluster map API object source code
• Cluster map interface source code

Example of cluster map

apiVersion: airshipit.org/v1alpha1
kind: ClusterMap
metadata:
  labels:
    airshipit.org/deploy-k8s: "false"
  name: main-map
map:
  target-cluster:
    parent: ephemeral-cluster
    # dynamicKubeConf: false # default value
  ephemeral-cluster: {}
  workload01:
    parent: target-cluster
    dynamicKubeConf: true
    clusterAPIRef:
      name: workload01
      namespace: tenant01-namespace

Metadata file

Phase plan

TODO expand this part of documentation when we utilize phase plan