Support for specifying site manifest location in AIAP

- Addded new container artifact-setup in lieu of
  airshipctl-builder
- Added/Renamed env MANIFEST_REPO_NAME, MANIFEST_REPO_URL, MANIFEST_REPO_REF
- Added examples for airshipctl/treasuremap as kustomize files.

Relates-To: #477

Closes: #477

Depends-On: I98b605d16940fab2a728e4646e52c7d1ad992c2d

Change-Id: I6759c765c1aa9eaef8a989fcefe300679513a1be
This commit is contained in:
ma257n 2021-03-27 12:25:34 +00:00 committed by Manoj Alva(ma257n)
parent 8d882fcc85
commit 418c64ba44
11 changed files with 384 additions and 57 deletions

View File

@ -1,5 +1,5 @@
IMAGE_REGISTRY ?= quay.io/airshipit
IMAGES := infra-builder airshipctl-builder runner
IMAGES := infra-builder artifact-setup runner
IMAGE_TAG ?= latest
PUSH_IMAGES ?= false
@ -34,6 +34,12 @@ ifeq (true, $(PUSH_IMAGES))
docker push $(IMAGE_REGISTRY)/aiap-$@:$(IMAGE_TAG)
endif
test: build ## Test airship-in-a-pod
kubectl delete -f airship-in-a-pod.yaml || true
kubectl create -f airship-in-a-pod.yaml
test: test-airshipctl test-treasuremap
test-airshipctl: build
kubectl delete pod airship-in-a-pod || true
kustomize build ./examples/airshipctl | kubectl apply -f -
test-treasuremap: build
kubectl delete pod airship-in-a-pod || true
kustomize build ./examples/airshipctl | kubectl apply -f -

View File

@ -4,8 +4,10 @@ Airship in a pod is a Kubernetes pod definition which describes all of the
components required to deploy a fully functioning Airship 2 deployment. The pod
consists of the following "Task" containers:
* `airshipctl-builder`: This container builds the airshipctl binary and makes it
available to the other containers
* `artifact-setup`: This container builds the airshipctl binary and makes it
available to the other containers. Also, based on the configuration provided
in the airship-in-a-pod manifest, airshipctl/treasuremap(based on the usecase) git repositories
will be downloaded and the required tag or commitId will be checked out.
* `infra-builder`: This container creates the various virtual networks and
machines required for an Airship deployment
* `runner`: The runner container is the "meat" of the pod, and executes the
@ -21,32 +23,115 @@ The pod also contains the following "Support" containers:
## Prerequisites
In order to deploy Airship in a Pod for development, you must first have a
working Kubernetes cluster. This guide assumes that a developer will deploy
using [minikube](https://minikube.sigs.k8s.io/docs/start/):
### Nested Virtualisation
If deployment is done on a VM, ensure that nested virtualization is enabled.
### Setup shared directory
Create the following directory with appropriate r+w permissions.
```
sudo mkdir /opt/.airship
```
### Environment variable setup
If you are within a proxy environment, ensure that the following environment
variables are defined, and NO_PROXY has the IP address which minikube uses.
For retrieving minikube ip refer: [minikube-ip](https://minikube.sigs.k8s.io/docs/commands/ip/)
```
export HTTP_PROXY=http://username:password@host:port
export HTTPS_PROXY=http://username:password@host:port
export NO_PROXY="localhost,127.0.0.1,10.23.0.0/16,10.96.0.0/12,10.1.1.44"
export PROXY=http://username:password@host:port
export USE_PROXY=true
export http_proxy=http://username:password@host:port
export https_proxy=http://username:password@host:port
export no_proxy="localhost,127.0.0.1,10.23.0.0/16,10.96.0.0/12,10.1.1.44"
export proxy=http://username:password@host:port
```
### To start minikube
Within the environment, with appropriate env variables set, run the following command.
```
sudo -E minikube start --driver=none
```
Refer [minikube](https://minikube.sigs.k8s.io/docs/start/)for more details.
## Usage
Since Airship in a Pod is just a pod definition, deploying and using it is as
simple as deploying and using any Kubernetes pod.
simple as deploying and using any Kubernetes pod with kustomize tool.
### Pod configuration
The below section provides steps to configure site with [airshipctl](https://github.com/airshipit/airshipctl)/[treasuremap](https://github.com/airshipit/treasuremap) manifests.
#### For airshipctl
Within the examples/airshipctl directory, update the existing patchset.yaml
file to reflect the airshipctl branch reference as required.
filepath : airshipctl/tools/airship-in-a-pod/examples/airshipctl/patchset.yaml
#### Deploy the Pod
```
kubectl apply -f airship-in-a-pod.yaml
- op: replace
path: "/spec/containers/4/env/4/value"
value: <branch reference>
```
#### View Pod Logs
#### For treasuremap
For treasuremap related manifests, use the patchset.yaml from
examples/treasuremap and update the following to reflect
the treasuremap branch reference and the pinned airshipctl reference
as required. The pinned airshipctl reference is the tag/commitId with
which treasuremap is tested and found working satisfactorily. This
could be found listed as 'AIRSHIPCTL_REF' attribute under the zuul.d
directory of treasuremap repository.
filepath : airshipctl/tools/airship-in-a-pod/examples/treasuremap/patchset.yaml
```
- op: replace
path: "/spec/containers/4/env/4/value"
value: <branch reference>
- op: replace
path: "/spec/containers/4/env/6/value"
value: <airshipctl_ref>
```
For more details, please consult the examples directory.
### Deploy the Pod
Once patchset.yaml for either airshipctl/treasuremap is ready, run the following
command against the running minikube cluster as shown below.
For example to run AIAP with treasuremap manifests, run the following commands.
```
cd tools/airship-in-a-pod/examples/{either airshipctl or treasuremap}
kustomize build . | kubectl apply -f -
```
### View Pod Logs
```
kubectl logs airship-in-a-pod -c $CONTAINER
```
#### Interact with the Pod
### Interact with the Pod
```
kubectl exec -it airship-in-a-pod -c $CONTAINER -- bash
@ -54,12 +139,38 @@ kubectl exec -it airship-in-a-pod -c $CONTAINER -- bash
where `$CONTAINER` is one of the containers listed above.
### Inspect Cluster
Once AIAP is fully installed with a target cluster (air-target-1 and air-worker-1 nodes)
installed and running, the cluster could be monitored using the following steps.
#### Log into the runner container
```
kubectl exec -it airship-in-a-pod -c runner -- bash
```
Run the .profile file using the following command to run kubectl/airshipctl commands
as below.
```
source ~/.profile
```
To run kubectl commands on Target cluster, use --kubeconfig and --context params
within kubectl as below.
```
kubectl --kubeconfig /root/.airship/kubeconfig --context target-cluster get pods -A'
```
### Output
Airship-in-a-pod produces the following outputs:
* The airshipctl repo and associated binary used with the deployment
* The airshipctl repo, manifest repo, and airshipctl binary used with the deployment.
* A tarball containing the generated ephemeral ISO, as well as the
configuration used during generation.
@ -76,7 +187,7 @@ caching:
* If using a cached `airshipctl`, the `airshipctl` binary must be stored in the
`$CACHE_DIR/airshipctl/bin/` directory, and the developer must have set
`USE_CACHED_AIRSHIPCTL` to `true`.
`USE_CACHED_ARTIFACTS` to `true`.
* If using a cached ephemeral iso, the iso must first be contained in a tarball named `iso.tar.gz`, must be stored in the
`$CACHE_DIR/` directory, and the developer must have set
`USE_CACHED_ISO` to `true`.

View File

@ -3,15 +3,21 @@ FROM ianhowell/base:latest
SHELL ["bash", "-exc"]
ENV DEBIAN_FRONTEND noninteractive
ARG USE_CACHED_AIRSHIPCTL="false"
ENV USE_CACHED_AIRSHIPCTL="false"
ARG USE_CACHED_ARTIFACTS="false"
ENV USE_CACHED_ARTIFACTS="false"
ARG AIRSHIPCTL_REPO=https://review.opendev.org/airship/airshipctl
ARG MANIFEST_REPO_URL=https://review.opendev.org/airship/airshipctl
ENV MANIFEST_REPO_URL=$MANIFEST_REPO_URL
ARG MANIFEST_REPO_REF=master
ENV MANIFEST_REPO_REF=$MANIFEST_REPO_REF
ARG MANIFEST_REPO_NAME=airshipctl
ENV MANIFEST_REPO_NAME=$MANIFEST_REPO_NAME
ARG AIRSHIPCTL_REF=
ENV AIRSHIPCTL_REF=$AIRSHIPCTL_REF
ARG AIRSHIPCTL_REF=master
ENV AIRSHIPCTL_REPO=$AIRSHIPCTL_REPO
# Update distro and install ansible
RUN apt-get update ;\
apt-get dist-upgrade -y ;\

View File

@ -14,25 +14,46 @@
set -ex
if [[ "$USE_CACHED_AIRSHIPCTL" = "true" ]]
function cloneRepo(){
repo_name=$1
repo_url=$2
repo_ref=$3
repo_dir="$ARTIFACTS_DIR/$repo_name"
mkdir -p "$repo_dir"
cd "$repo_dir"
git init
git fetch "$repo_url" "$repo_ref"
git checkout FETCH_HEAD
}
if [[ "$USE_CACHED_ARTIFACTS" = "true" ]]
then
printf "Using cached airshipctl\n"
cp -r "$CACHE_DIR/airshipctl" "$ARTIFACTS_DIR/airshipctl"
cp -r "$CACHE_DIR/*" "$ARTIFACTS_DIR"
else
printf "Waiting 30 seconds for the libvirt, sushy, and docker services to be ready\n"
sleep 30
airshipctl_dir="$ARTIFACTS_DIR/airshipctl"
mkdir -p "$airshipctl_dir"
cd "$airshipctl_dir"
cloneRepo $MANIFEST_REPO_NAME $MANIFEST_REPO_URL $MANIFEST_REPO_REF
git init
git fetch "$AIRSHIPCTL_REPO" "$AIRSHIPCTL_REF"
git checkout FETCH_HEAD
if [[ "$MANIFEST_REPO_NAME" != "airshipctl" ]]
then
cloneRepo airshipctl https://github.com/airshipit/airshipctl $AIRSHIPCTL_REF
fi
cd $ARTIFACTS_DIR/$MANIFEST_REPO_NAME
if [[ "$MANIFEST_REPO_NAME" == "airshipctl" ]]
then
./tools/deployment/21_systemwide_executable.sh
else
./tools/deployment/airship-core/21_systemwide_executable.sh
fi
mkdir -p bin
cp "$(which airshipctl)" bin
fi
/signal_complete airshipctl-builder
/signal_complete artifact-setup

View File

@ -0,0 +1,23 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../base
patchesJson6902:
- target:
version: v1 # apiVersion
kind: Pod
name: airship-in-a-pod
path: patchset.yaml

View File

@ -0,0 +1,37 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- op: replace
path: "/spec/containers/4/env/3/value"
value: https://review.opendev.org/airship/airshipctl
- op: replace
path: "/spec/containers/4/env/4/value"
value: master
- op: replace
path: "/spec/containers/4/env/5/value"
value: airshipctl
# runner container
- op: replace
path: "/spec/containers/6/env/3/value"
value: airshipctl
# for local testing
#- op: add
# path: "/spec/containers/4/imagePullPolicy"
# value: Never
#
#- op: add
# path: "/spec/containers/6/imagePullPolicy"
# value: Never

View File

@ -20,9 +20,11 @@ spec:
containers:
- name: libvirt
image: ianhowell/libvirt:latest
image: quay.io/airshipit/libvirt:aiap-v1
# Set to true for debugging
tty: false
# 18June2021 : reverting #578 fix due to erratic behaviour
# for “setctty: operation not permitted" error, set tty:false
tty: true
securityContext:
privileged: true
#SYS_ADMIN required for systemd, need to work out reqs for libvirt
@ -180,9 +182,13 @@ spec:
mountPath: /var/run/
- name: srv
mountPath: /srv/
- name: airship-config
mountPath: /root/.airship
- name: var-run-libvirt
mountPath: /var/run/libvirt
- name: airshipctl-builder
image: quay.io/airshipit/aiap-airshipctl-builder:latest
- name: artifact-setup
image: quay.io/airshipit/aiap-artifact-setup:latest
command:
- bash
- -cex
@ -194,18 +200,22 @@ spec:
command:
- test
- -e
- /tmp/completed/airshipctl-builder
- /tmp/completed/artifact-setup
env:
- name: CACHE_DIR
value: /opt/aiap-cache
- name: USE_CACHED_AIRSHIPCTL
- name: USE_CACHED_ARTIFACTS
value: "false"
- name: ARTIFACTS_DIR
value: /opt/aiap-artifacts
- name: AIRSHIPCTL_REPO
- name: MANIFEST_REPO_URL
value: https://review.opendev.org/airship/airshipctl
- name: AIRSHIPCTL_REF
- name: MANIFEST_REPO_REF
value: master
- name: MANIFEST_REPO_NAME
value: airshipctl
- name: AIRSHIPCTL_REF
value:
volumeMounts:
- name: tmp
mountPath: /tmp
@ -217,6 +227,8 @@ spec:
mountPath: /tmp/completed
- name: var-run-docker
mountPath: /var/run
- name: var-run-libvirt
mountPath: /var/run/libvirt
- name: infra-builder
image: quay.io/airshipit/aiap-infra-builder:latest
@ -284,6 +296,8 @@ spec:
value: /opt/aiap-artifacts
- name: USE_CACHED_ISO
value: "false"
- name: MANIFEST_REPO_NAME
value: airshipctl
volumeMounts:
- name: cache
mountPath: /opt/aiap-cache
@ -305,6 +319,8 @@ spec:
mountPath: /var/log/
- name: var-run-docker
mountPath: /var/run
- name: airship-config
mountPath: /root/.airship
volumes:
- name: cache
@ -321,6 +337,8 @@ spec:
- name: tmp
emptyDir:
medium: "Memory"
- name: airship-config
emptyDir: {}
- name: run
emptyDir:
medium: "Memory"

View File

@ -0,0 +1,14 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
resources:
- airship-in-a-pod.yaml

View File

@ -0,0 +1,23 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../base
patchesJson6902:
- target:
version: v1 # apiVersion
kind: Pod
name: airship-in-a-pod
path: patchset.yaml

View File

@ -0,0 +1,41 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- op: replace
path: "/spec/containers/4/env/3/value"
value: https://review.opendev.org/airship/treasuremap
- op: replace
path: "/spec/containers/4/env/4/value"
value: master
- op: replace
path: "/spec/containers/4/env/5/value"
value: treasuremap
- op: replace
path: "/spec/containers/4/env/6/value"
value: 63c1faf718fd3341fc5bd975e575e3cf41647206
# runner container
- op: replace
path: "/spec/containers/6/env/3/value"
value: treasuremap
# for local testing
#- op: add
# path: "/spec/containers/4/imagePullPolicy"
# value: Never
#
#- op: add
# path: "/spec/containers/6/imagePullPolicy"
# value: Never

View File

@ -14,8 +14,8 @@
set -ex
# Wait until airshipctl and libvirt infrastructure has been built
/wait_for airshipctl-builder
# Wait until artifact-setup and libvirt infrastructure has been built
/wait_for artifact-setup
/wait_for infra-builder
export USER=root
@ -28,35 +28,62 @@ kustomize_download_url="https://github.com/kubernetes-sigs/kustomize/releases/do
curl -sSL "$kustomize_download_url" | tar -C /tmp -xzf -
install /tmp/kustomize /usr/local/bin
cp "$ARTIFACTS_DIR/airshipctl/bin/airshipctl" /usr/local/bin/airshipctl
cp -r "$ARTIFACTS_DIR/airshipctl/" /opt/airshipctl
cd /opt/airshipctl
cp "$ARTIFACTS_DIR/$MANIFEST_REPO_NAME/bin/airshipctl" /usr/local/bin/airshipctl
if [ $MANIFEST_REPO_NAME != "airshipctl" ]
then
export AIRSHIP_CONFIG_PHASE_REPO_URL="https://opendev.org/airship/treasuremap"
cp -r $ARTIFACTS_DIR/airshipctl/ /opt/airshipctl
fi
cp -r $ARTIFACTS_DIR/$MANIFEST_REPO_NAME/ /opt/$MANIFEST_REPO_NAME
cd /opt/$MANIFEST_REPO_NAME
curl -fsSL -o key.asc https://raw.githubusercontent.com/mozilla/sops/master/pgp/sops_functional_tests_key.asc
SOPS_IMPORT_PGP="$(cat key.asc)"
SOPS_PGP_FP="FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4"
export SOPS_IMPORT_PGP SOPS_PGP_FP
curl -fsSL -o /tmp/key.asc https://raw.githubusercontent.com/mozilla/sops/master/pgp/sops_functional_tests_key.asc
echo 'export SOPS_IMPORT_PGP="$(cat /tmp/key.asc)"' >> ~/.profile
echo "export SOPS_PGP_FP=${SOPS_PGP_FP}" >> ~/.profile
source ~/.profile
export AIRSHIP_CONFIG_MANIFEST_DIRECTORY="/tmp/airship"
# By default, don't build airshipctl - use the binary from the shared volume instead
# ./tools/deployment/21_systemwide_executable.sh
./tools/deployment/22_test_configs.sh
./tools/deployment/23_pull_documents.sh
./tools/deployment/23_generate_secrets.sh
if [ "$MANIFEST_REPO_NAME" == "airshipctl" ]
then
./tools/deployment/22_test_configs.sh
./tools/deployment/23_pull_documents.sh
./tools/deployment/23_generate_secrets.sh
else
./tools/deployment/airship-core/22_test_configs.sh
./tools/deployment/airship-core/23_pull_documents.sh
./tools/deployment/airship-core/23_generate_secrets.sh
sed -i -e 's#bmcAddress: redfish+http://\([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+\):8000#bmcAddress: redfish+https://10.23.25.1:8443#' /tmp/airship/airshipctl/manifests/site/test-site/target/catalogues/hosts.yaml
sed -i -e 's#root#username#' /tmp/airship/airshipctl/manifests/site/test-site/target/catalogues/hosts.yaml
sed -i -e 's#r00tme#password#' /tmp/airship/airshipctl/manifests/site/test-site/target/catalogues/hosts.yaml
sed -i -e 's#disableCertificateVerification: false#disableCertificateVerification: true#' /tmp/airship/airshipctl/manifests/site/test-site/target/catalogues/hosts.yaml
fi
sed -i -e 's#bmcAddress: redfish+http://\([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+\):8000#bmcAddress: redfish+https://10.23.25.1:8443#' "/tmp/airship/$MANIFEST_REPO_NAME/manifests/site/test-site/target/catalogues/hosts.yaml"
sed -i -e 's#root#username#' "/tmp/airship/$MANIFEST_REPO_NAME/manifests/site/test-site/target/catalogues/hosts.yaml"
sed -i -e 's#r00tme#password#' "/tmp/airship/$MANIFEST_REPO_NAME/manifests/site/test-site/target/catalogues/hosts.yaml"
sed -i -e 's#disableCertificateVerification: false#disableCertificateVerification: true#' "/tmp/airship/$MANIFEST_REPO_NAME/manifests/site/test-site/target/catalogues/hosts.yaml"
if [[ "$USE_CACHED_ISO" = "true" ]]; then
mkdir -p /srv/images
tar -xzf "$CACHE_DIR/iso.tar.gz" --directory /srv/images
else
if [ "$MANIFEST_REPO_NAME" == "airshipctl" ]
then
./tools/deployment/24_build_images.sh
else
./tools/deployment/airship-core/24_build_images.sh
fi
tar -czf "$ARTIFACTS_DIR/iso.tar.gz" --directory=/srv/images .
fi
./tools/deployment/25_deploy_gating.sh
if [ "$MANIFEST_REPO_NAME" == "airshipctl" ]
then
./tools/deployment/25_deploy_gating.sh
else
./tools/deployment/airship-core/25_deploy_gating.sh
fi
/signal_complete runner