Browse Source

Use overrides (values) to describe VMs

Previously the VM configuration was embedded in the charts templates;
this change moves things into values as we iterate over things created
resources as required.

Change-Id: I75e60bd31ee5debda90a703775215e92480d572c
Chris Wedgwood 1 year ago
parent
commit
92a678bb5c

+ 11
- 0
Dockerfile View File

@@ -0,0 +1,11 @@
1
+FROM ubuntu:16.04
2
+
3
+RUN apt-get update && apt-get install -y qemu-kvm dnsmasq bridge-utils mkisofs curl jq wget iptables
4
+RUN apt-get clean
5
+RUN rm -f /var/lib/apt/lists/* || true
6
+
7
+ENTRYPOINT ["/bin/sleep", "infinity"]
8
+
9
+VOLUME "/image"
10
+EXPOSE 22
11
+CMD []

+ 8
- 7
Makefile View File

@@ -9,22 +9,23 @@ test: install
9 9
 
10 10
 install: build
11 11
 	@echo
12
-	-helm delete --purge berth >>helm.log 2>&1
12
+	-helm delete --purge berth
13 13
 	@echo
14
-	@[ -f override.yaml ] || touch override.yaml
15
-	helm install ./berth-0.1.0.tgz --values=override.yaml --name=berth >>helm.log 2>&1
16
-	@sleep 5.0 # give k8s a chance to see the IP
14
+	helm install --name=berth --debug ./berth
15
+	helm upgrade --debug berth ./berth \
16
+			--values examples/cirros-test.yaml \
17
+			--values examples/demo-ub14-apache.yaml \
18
+			--values examples/ub16-smp-test.yaml
19
+	@sleep 5 # give k8s a chance to see the IP
17 20
 	@echo
18 21
 	kubectl get pods -o wide
19 22
 
20 23
 build:
21 24
 	@echo
22 25
 	helm lint berth
23
-	@echo
24
-	helm package berth
25 26
 
26 27
 clean:
27
-	rm -f berth-0.1.0.tgz helm.log
28
+	rm -f *~ */*~ */*/*~ berth-0.1.0.tgz
28 29
 
29 30
 .PHONY:
30 31
 	all default build clean

+ 105
- 0
README.md View File

@@ -0,0 +1,105 @@
1
+Berth is a deliberately minimalist VM runner for Kubernetes.
2
+
3
+## TL;DR Installation Guide
4
+
5
+Install the bare Berth chart:
6
+```
7
+# Have (recent) Helm and Kubernetes (2.5.x and 1.6.x or later)
8
+#
9
+helm install --name=berth ./berth
10
+kubectl get pods -o wide
11
+```
12
+
13
+This should happen error free and be fairly quick.  At this stage
14
+install one of the examples:
15
+```
16
+# helm upgrade berth ./berth --values=examples/cirros-test.yaml
17
+```
18
+
19
+You should be able to SSH to your VM at the Kubernetes IP for the
20
+container which you can retrieve with `kubectl get all -o wide`.  VNC
21
+access is available on port 5900.
22
+
23
+Additional examples may be added with additional `--values` arguments.
24
+For example:
25
+```
26
+# helm upgrade berth ./berth --values=examples/cirros-test.yaml --values=examples/demo-ub14-apache.yaml
27
+```
28
+
29
+You can use a similar method to purge all VMs:
30
+```
31
+# helm upgrade berth ./berth --set 'vmlist={}'
32
+
33
+```
34
+
35
+
36
+<!-- https://mostsecure.pw/  -->
37
+
38
+### Example
39
+
40
+[Quick installation / sample](https://asciinema.org/a/4VazbwsokL3zpnGPf27eyFIfe)
41
+
42
+### Why this?
43
+
44
+The requirements are very narrow right now and the existing
45
+alternatives don't align well at present.  This will likely change in
46
+time at which point we can realign the internal implementation.
47
+
48
+#### Minimalist requirements
49
+* Run VMs from inside of Kubernetes
50
+* Work with Calico
51
+* Have VM life-cycle match that of pods
52
+* Have VMs benefit from Kubernetes resiliency
53
+* Allow for persistent storage
54
+* Allow for state injection/access from a ConfigMaps
55
+
56
+## Requirements:
57
+* Helm 2.5.x
58
+* Kubernetes 1.6.x
59
+
60
+This does not need to be installed as part of the OpenStack chart
61
+collection.
62
+
63
+## How it works:
64
+
65
+At a high level, it works like this:
66
+* Create a SNAT/DNAT enabled linux bridge.
67
+* Assign the bridge a private IP address from a small /30 subnet
68
+  (controlled with `VM_IP` and `VM_GW`)
69
+* Plug the VM network interface into the bridge.
70
+* Run a dnsmasq process to allocate the VM the right name-servers, and
71
+  DNS search strings extracted from the parent container.  Assign the
72
+  private IP address to the VM and have it use the bridge's IP as its
73
+  default gateway.
74
+* Setup SNAT/DNAT on the parent container to do 1:1 mapping of all
75
+  ports, all protocols to the VM, except for TCP:5900 to allow for VNC
76
+  access (can be controlled with NO_VNC environment variable).
77
+* At this point, VM essentially assumes Pod Assigned IP.
78
+* Feed any meta-data or user-data down into the VM by leveraging these
79
+  ConfigMap mounts with the same name and turning them into an ISO
80
+  presented to the guest.
81
+
82
+The startvm entry-point supports several environment variables:
83
+
84
+* `IMG_SOURCE` which is an http or https URL that contains a qcow2
85
+  image.  It can also be a full path to a local file baked into the
86
+  container image, e.g. "/image.qcow"
87
+* `IMG_TARGET` the name to save the image above as in the shared
88
+  volume.
89
+
90
+It also supports two files, which should be mounted as ConfigMaps if
91
+using Kubernetes at `/userdata` and `/metadata` as YAML files
92
+containing, obviously meta-data and user-data as YAML that will be fed
93
+to the VM as a config-drive iso.
94
+
95
+The "pet" version of the image, which is created using qemu-img -b to
96
+base it on the source, is stored in a separate volume dedicated to the
97
+VM itself, and named after the container hostname.
98
+
99
+There are a few other parameters you can control as an operator:
100
+
101
+* `VM_IP` is the IP address the VM should be allocated by DHCP.  The
102
+  container will 1:1 NAT except for port 5900 for VNC access (defaults
103
+  to 192.168.254.2)
104
+* `VM_GW` is the gateway IP address the VM should use for its default
105
+  route (defaults to 192.168.254.1)

+ 0
- 23
TODO View File

@@ -1,23 +0,0 @@
1
-[Put these in Jira]
2
-
3
-*Require* ssh key, throw an error if missing.
4
-
5
-Chart definition vs values.yaml:
6
-
7
-    Move image source/target details from chart to values
8
-    (IMG_SOURCE/IMG_TARGET)
9
-
10
-    Move the VM definitions entirely into the values.yaml file; have
11
-    helm deployment process iterate over that creating multiple pods,
12
-    one for each VM.
13
-
14
-    Ideally this will allow incremental updates of pods without having
15
-    to know about the chart internals.
16
-
17
-    XXX Get examples of where we've done this before from Alan XXX
18
-
19
-Add Dockerfile; update the container image we use to Ubuntu 16.04 with
20
-a suitable KVM/Qemu.
21
-
22
-Consider moving to stateful sets for cleaner PVC associations.  See
23
-OSH MariaDB chart for an example of this.

+ 70
- 41
berth/templates/bin/_startvm.txt View File

@@ -1,16 +1,17 @@
1 1
 #!/bin/bash
2 2
 
3
-set -x
3
+set -ex
4 4
 
5
-echo "VER-0.1.0-1.2"
5
+# FIXME; right now this doens't work, need to work out why
6
+#set -o pipefail
6 7
 
7 8
 # Returns the integer representation of an IP arg, passed in ascii
8 9
 # dotted-decimal notation (x.x.x.x)
9 10
 atoi() {
10 11
     IP=$1; IPNUM=0
11 12
     for (( i=0 ; i<4 ; ++i )); do
12
-	((IPNUM+=${IP%%.*}*$((256**$((3-${i}))))))
13
-	IP=${IP#*.}
13
+        ((IPNUM+=${IP%%.*}*$((256**$((3-${i}))))))
14
+        IP=${IP#*.}
14 15
     done
15 16
     echo $IPNUM
16 17
 }
@@ -27,23 +28,23 @@ itoa() {
27 28
 generate_cloud_drive() {
28 29
     metadata=/metadata
29 30
     if [ ! -f $metadata ]; then
30
-	metadata=""
31
+        metadata=""
31 32
     fi
32 33
 
33 34
     userdata=/userdata
34 35
     if [ ! -f $userdata ]; then
35
-	userdata=""
36
+        userdata=""
36 37
     fi
37 38
 
38 39
     if [ "$metadata" == "" -a "$userdata" == "" ]; then
39
-	return
40
+        return
40 41
     fi
41 42
 
42 43
     TMPDIR=`mktemp -d -t aicvm.XXXXXX`
43 44
 
44 45
     if [ $? -ne 0 ]; then
45
-	echo "Fail to create temporaily directory"
46
-	exit 1
46
+        echo "Fail to create temporaily directory"
47
+        exit 1
47 48
     fi
48 49
 
49 50
     # create form of config drive
@@ -54,17 +55,17 @@ generate_cloud_drive() {
54 55
     cd $OLD_PWD
55 56
 
56 57
     if [ -f $metadata ]; then
57
-	cp $metadata ${TMPDIR}/openstack/2012-08-10/meta_data.json
58
+        cp $metadata ${TMPDIR}/openstack/2012-08-10/meta_data.json
58 59
     fi
59 60
     if [ -f $userdata ]; then
60
-	cp $userdata ${TMPDIR}/openstack/2012-08-10/user_data
61
+        cp $userdata ${TMPDIR}/openstack/2012-08-10/user_data
61 62
     fi
62 63
 
63 64
     iso="cloud-drive.iso"
64 65
     mkisofs -R -V config-2 -o $iso ${TMPDIR}
65 66
     if [ $? -ne 0 ]; then
66
-	echo Fail to create cloud-drive ISO image for cloud-init
67
-	exit 1
67
+        echo Fail to create cloud-drive ISO image for cloud-init
68
+        exit 1
68 69
     fi
69 70
     echo $iso
70 71
 }
@@ -110,38 +111,38 @@ if [ -e /dev/vm/root ]; then
110 111
 else
111 112
 
112 113
     if [ -e "${IMG_TARGET}" ]; then
113
-	BASE=${IMG_TARGET}
114
+        BASE=${IMG_TARGET}
114 115
     else
115 116
 
116
-	if [ ! -d "/image" ]; then
117
-	    echo "/image directory does not exist, failed to mount volume?"
118
-	    exit 2
119
-	fi
117
+        if [ ! -d "/image" ]; then
118
+            echo "/image directory does not exist, failed to mount volume?"
119
+            exit 2
120
+        fi
120 121
 
121
-	if [ ! -e "/image/${IMG_TARGET}" ]; then
122
-	    echo "Fetching missing image target"
123
-	    curl ${IMG_SOURCE} > /image/${IMG_TARGET}
124
-	fi
122
+        if [ ! -e "/image/${IMG_TARGET}" ]; then
123
+            echo "Fetching missing image target"
124
+            curl ${IMG_SOURCE} > /image/${IMG_TARGET}
125
+        fi
125 126
 
126
-	BASE=/image/${IMG_TARGET}
127
+        BASE=/image/${IMG_TARGET}
127 128
     fi
128 129
 
129 130
     if [ ! -d "/image" ]; then
130
-	echo "/image directory does not exist, failed to mount volume /image?"
131
-	exit 2
131
+        echo "/image directory does not exist, failed to mount volume /image?"
132
+        exit 2
132 133
     fi
133 134
 
134 135
     if [ -z "${HOSTNAME}" ]; then
135
-    	echo "Could not find HOSTNAME var.  Did you specify a HOSTNAME environment variable?"
136
+        echo "Could not find HOSTNAME var.  Did you specify a HOSTNAME environment variable?"
136 137
     fi
137 138
 
138 139
     KVM_IMAGE=/image/${HOSTNAME}.qcow2
139 140
 
140 141
     if [ -e "${KVM_IMAGE}" ]; then
141
-    	echo "Image ${KVM_IMAGE} already exists.  Not recreating"
142
+        echo "Image ${KVM_IMAGE} already exists.  Not recreating"
142 143
     else
143 144
         qemu-img create -f qcow2 -b ${BASE} \
144
-		 $KVM_IMAGE > /dev/null
145
+                 $KVM_IMAGE > /dev/null
145 146
         if [[ $? -ne 0 ]]; then
146 147
             echo "Failed to create qcow2 image"
147 148
             exit 3
@@ -173,14 +174,14 @@ cidr2mask() {
173 174
     local partial_octet=$(($1%8))
174 175
 
175 176
     for ((i=0;i<4;i+=1)); do
176
-	if [ $i -lt $full_octets ]; then
177
-	    mask+=255
178
-	elif [ $i -eq $full_octets ]; then
179
-	    mask+=$((256 - 2**(8-$partial_octet)))
180
-	else
181
-	    mask+=0
182
-	fi
183
-	test $i -lt 3 && mask+=.
177
+        if [ $i -lt $full_octets ]; then
178
+            mask+=255
179
+        elif [ $i -eq $full_octets ]; then
180
+            mask+=$((256 - 2**(8-$partial_octet)))
181
+        else
182
+            mask+=0
183
+        fi
184
+        test $i -lt 3 && mask+=.
184 185
     done
185 186
 
186 187
     echo $mask
@@ -196,6 +197,24 @@ setup_bridge_networking() {
196 197
     NAMESERVER=( `grep nameserver /etc/resolv.conf | grep -v "#" | cut -f2 -d ' '` )
197 198
     NAMESERVERS=`echo ${NAMESERVER[*]} | sed "s/ /,/"`
198 199
     SEARCH=( `grep -E ^search /etc/resolv.conf | grep -v "#" | cut -f2- -d ' ' | tr ' ' ','` )
200
+    # MAC=$(ip addr show $IFACE | grep ether | sed -e 's/^[[:space:]]*//g' -e 's/[[:space:]]*\$//g' | cut -f2 -d ' ')
201
+    # HOST_IP=$(ip addr show dev $IFACE | grep "inet $IP" | awk '{print $2}' | cut -f1 -d/)
202
+    # HOST_CIDR=$(ip addr show dev $IFACE | grep "inet $IP" | awk '{print $2}' | cut -f2 -d/)
203
+    # HOST_NETMASK=$(cidr2mask $HOST_CIDR)
204
+    # HOST_GATEWAY=$(ip route get 8.8.8.8 | grep via | cut -f3 -d ' ')
205
+    # NAMESERVER=$(grep nameserver /etc/resolv.conf | grep -v "#" | cut -f2 -d ' ') )
206
+    # NAMESERVERS=$(echo ${NAMESERVER[*]} | sed "s/ /,/")
207
+    # SEARCH=$(grep -E ^search /etc/resolv.conf | grep -v "#" | cut -f2- -d ' ' | tr ' ' ',')
208
+
209
+    # fail if any of the above aren't suitable    # here
210
+    [ -n "$MAC" ]
211
+    [ -n "$HOST_IP" ]
212
+    [ -n "$HOST_CIDR" ]
213
+    [ -n "$HOST_NETMASK" ]
214
+    [ -n "$HOST_GATEWAY" ]
215
+    [ -n "$NAMESERVER" ]
216
+    [ -n "$NAMESERVERS" ]
217
+    [ -n "$SEARCH" ]
199 218
 
200 219
     # we must enable forwarding inside the container
201 220
     echo 1 > /proc/sys/net/ipv4/ip_forward
@@ -205,11 +224,11 @@ setup_bridge_networking() {
205 224
     # specify NO_VNC as an environment variable to disable this
206 225
     # functionality
207 226
     if [ -z $NO_VNC ]; then
208
-   	iptables -t nat -A PREROUTING -p tcp \! --dport 5900 -d $HOST_IP -j DNAT --to-destination $VM_IP
209
-   	iptables -t nat -A POSTROUTING -s $VM_IP -j SNAT --to-source $HOST_IP
227
+        iptables -t nat -A PREROUTING -p tcp \! --dport 5900 -d $HOST_IP -j DNAT --to-destination $VM_IP
228
+        iptables -t nat -A POSTROUTING -s $VM_IP -j SNAT --to-source $HOST_IP
210 229
     else
211
-	iptables -t nat -A PREROUTING -d $HOST_IP -j DNAT --to-destination $VM_IP
212
-	iptables -t nat -A POSTROUTING -s $VM_IP -j SNAT --to-source $HOST_IP
230
+        iptables -t nat -A PREROUTING -d $HOST_IP -j DNAT --to-destination $VM_IP
231
+        iptables -t nat -A POSTROUTING -s $VM_IP -j SNAT --to-source $HOST_IP
213 232
     fi
214 233
 
215 234
     # generate VM specifics
@@ -258,4 +277,14 @@ setup_bridge_networking
258 277
 HOST_IP=`ip addr show dev $IFACE | grep "inet $IP" | awk '{print $2}' | cut -f1 -d/`
259 278
 VNC="-vnc $HOST_IP:0"
260 279
 
261
-exec $LAUNCHER qemu-system-x86_64 -enable-kvm  $VNC `eval echo $KVM_BLK_OPTS` `eval echo $KVM_NET_OPTS` -usbdevice tablet -nographic $KVM_ARGS
280
+exec $LAUNCHER qemu-system-x86_64 \
281
+     -smp "$IMG_VCPU" \
282
+     -m "$IMG_RAM_MB" \
283
+     -machine q35 \
284
+     -cpu host,+x2apic \
285
+     -vga vmware \
286
+     -enable-kvm \
287
+     $VNC \
288
+     `eval echo $KVM_BLK_OPTS` \
289
+     `eval echo $KVM_NET_OPTS` \
290
+     -usbdevice tablet -nographic $KVM_ARGS

+ 0
- 132
berth/templates/deployment.yaml View File

@@ -1,132 +0,0 @@
1
-# FIXME(cw); refactor into multiple per-function files
2
-
3
-# FIXME(cw) consider using OSH helm-toolkit.utils.template
4
-{{- define "template" -}}
5
-{{- $name := index . 0 -}}
6
-{{- $context := index . 1 -}}
7
-{{- $last := base $context.Template.Name }}
8
-{{- $wtf := $context.Template.Name | replace $last $name -}}
9
-{{ include $wtf $context }}
10
-{{- end -}}
11
-
12
----
13
-apiVersion: v1
14
-kind: ConfigMap
15
-metadata:
16
-  name: cloudinit
17
-data:
18
-  metadata: |
19
-    { "uuid": "example-01-vm.mydomain.com" }
20
-  userdata: |
21
-    #cloud-config
22
-    fqdn: example-01-vm.mydomain.com
23
-    users:
24
-      - name: root
25
-        ssh-authorized-keys:
26
-          - {{ .Values.auth.ssh_key }}
27
-    ssh_pwauth: True
28
-    runcmd:
29
-     - [ apt-get, update ]
30
-     - [ apt-get, install, -y, --force-yes, apache2 ]
31
----
32
-apiVersion: v1
33
-kind: ConfigMap
34
-metadata:
35
-  name: configmap-startvm
36
-data:
37
-  startvm: |
38
-    #!/bin/bash
39
-    #
40
-    # start of startvm
41
-{{ tuple "bin/_startvm.txt" . | include "template" | indent 4 }}
42
-    # end of startvm
43
----
44
-kind: PersistentVolumeClaim
45
-apiVersion: v1
46
-metadata:
47
-  name: example-01-vm
48
-spec:
49
-  accessModes: [ "ReadWriteOnce" ]
50
-  resources:
51
-    requests:
52
-      storage: {{ .Values.volume.size }}
53
-{{ if not .Values.volume.class_name }}
54
-  storageClassName: {{ .Values.volume.class_name }}
55
-{{ end }}
56
----
57
-apiVersion: extensions/v1beta1
58
-kind: Deployment
59
-metadata:
60
-  name: berth
61
-spec:
62
-  replicas: 1
63
-  template:
64
-    metadata:
65
-      labels:
66
-        app: berth
67
-      annotations:
68
-        pod.beta.kubernetes.io/hostname: example-01-vm
69
-    spec:
70
-      nodeSelector:
71
-        {{ .Values.labels.node_selector_key }}: {{ .Values.labels.node_selector_value }}
72
-      hostNetwork: false
73
-      hostPID: false
74
-      securityContext:
75
-        runAsUser: 0
76
-      containers:
77
-        - name: example-01-vm
78
-          imagePullPolicy: IfNotPresent
79
-          image: {{ .Values.images.entrypoint }}
80
-          env:
81
-            - name: IMG_SOURCE
82
-              value: http://stupidest.org/vm/ubuntu-14.04-amd64.img
83
-            - name: IMG_TARGET
84
-              value: ubuntu-14.04-amd64.img
85
-          securityContext:
86
-            privileged: true
87
-          command:
88
-            - /usr/local/bin/startvm
89
-          ports:
90
-            - containerPort: {{ .Values.network.port }}
91
-            - containerPort: {{ .Values.network.vnc }}
92
-          readinessProbe:
93
-            tcpSocket:
94
-              port: {{ .Values.network.vnc }}
95
-          volumeMounts:
96
-            - name: volume-startvm
97
-              mountPath: /usr/local/bin/startvm
98
-              subPath: startvm
99
-            - name: image
100
-              mountPath: /image
101
-            - name: dev
102
-              mountPath: /dev
103
-            - name: sys
104
-              mountPath: /sys
105
-            - name: cloudinit
106
-              mountPath: /userdata
107
-              subPath: userdata
108
-            - name: cloudinit
109
-              mountPath: /metadata
110
-              subPath: metadata
111
-      volumes:
112
-        - name: volume-startvm
113
-          configMap:
114
-            name: configmap-startvm
115
-            defaultMode: 0755
116
-        - name: image
117
-          persistentVolumeClaim:
118
-            claimName: example-01-vm
119
-        - name: dev
120
-          hostPath:
121
-              path: /dev
122
-        - name: sys
123
-          hostPath:
124
-              path: /sys
125
-        - name: cloudinit
126
-          configMap:
127
-            name: cloudinit
128
-            items:
129
-            - key: userdata
130
-              path: userdata
131
-            - key: metadata
132
-              path: metadata

+ 33
- 0
berth/templates/iter-configmap.yaml View File

@@ -0,0 +1,33 @@
1
+
2
+{{- define "template" -}}
3
+{{- $name := index . 0 -}}
4
+{{- $context := index . 1 -}}
5
+{{- $last := base $context.Template.Name }}
6
+{{- $wtf := $context.Template.Name | replace $last $name -}}
7
+{{ include $wtf $context }}
8
+{{- end -}}
9
+
10
+---
11
+
12
+apiVersion: v1
13
+kind: ConfigMap
14
+metadata:
15
+  name: configmap-generic-startvm
16
+data:
17
+  startvm: |
18
+{{ tuple "bin/_startvm.txt" . | include "template" | indent 4 }}
19
+
20
+---
21
+
22
+{{- range $name, $vm := .Values.vmlist }}
23
+{{- if $vm.enabled }}
24
+apiVersion: v1
25
+kind: ConfigMap
26
+metadata:
27
+  name: configmap-cloudconfig-{{ $name }}
28
+data:
29
+  metadata: {{ toYaml $vm.cloudconfig.metadata | indent 4 }}
30
+  userdata: {{ toYaml $vm.cloudconfig.userdata | indent 4 }}
31
+---
32
+{{- end }}
33
+{{- end }}

+ 98
- 0
berth/templates/iter-deployment.yaml View File

@@ -0,0 +1,98 @@
1
+
2
+{{- $envAll := . }}
3
+
4
+{{ range $name, $vm := .Values.vmlist }}
5
+
6
+# id: {{- $name }}
7
+{{- if $vm.enabled }}
8
+# vm enabled
9
+apiVersion: extensions/v1beta1
10
+kind: Deployment
11
+metadata:
12
+  name: berth-{{ $name }}
13
+spec:
14
+  replicas: 1
15
+  template:
16
+    metadata:
17
+      labels:
18
+        app: berth
19
+      annotations:
20
+        pod.beta.kubernetes.io/hostname: {{ $name }}
21
+    spec:
22
+      nodeSelector:
23
+        {{ $envAll.Values.labels.node_selector_key }}: {{ $envAll.Values.labels.node_selector_value }}
24
+      hostNetwork: false
25
+      hostPID: false
26
+      securityContext:
27
+        runAsUser: 0
28
+      containers:
29
+        - name: {{ $name }}
30
+          imagePullPolicy: IfNotPresent
31
+          image: {{ $envAll.Values.images.vmrunner }}
32
+          env:
33
+            - name: IMG_SOURCE
34
+              value: {{ $vm.vmconfig.rootfs.sourceurl }}
35
+            - name: IMG_TARGET
36
+              value: {{ $vm.vmconfig.rootfs.localtarget }}
37
+            - name: IMG_VCPU
38
+              value: "{{ $vm.vmconfig.cpu.vcpu }}"
39
+            - name: IMG_RAM_MB
40
+              value: "{{ $vm.vmconfig.cpu.ram_mb }}"
41
+          securityContext:
42
+            privileged: true
43
+          command:
44
+            - /usr/local/bin/startvm
45
+{{- if $vm.netconfig.ports }}
46
+          ports:
47
+{{- range $for, $port := $vm.netconfig.ports }}
48
+            - containerPort: {{ $port }}
49
+{{- end }}
50
+{{- end }}
51
+{{- if $vm.netconfig.readinessTcpProbe }}
52
+          readinessProbe:
53
+            tcpSocket:
54
+              port: {{ $vm.netconfig.readinessTcpProbe }}
55
+{{- end }}
56
+          volumeMounts:
57
+            - name: volume-startvm
58
+              mountPath: /usr/local/bin/startvm
59
+              subPath: startvm
60
+            - name: image
61
+              mountPath: /image
62
+            - name: dev
63
+              mountPath: /dev
64
+            - name: sys
65
+              mountPath: /sys
66
+            - name: volume-cloudinit
67
+              mountPath: /userdata
68
+              subPath: userdata
69
+            - name: volume-cloudinit
70
+              mountPath: /metadata
71
+              subPath: metadata
72
+      volumes:
73
+        - name: volume-startvm
74
+          configMap:
75
+            name: configmap-generic-startvm
76
+            defaultMode: 0755
77
+        - name: image
78
+          persistentVolumeClaim:
79
+            claimName: {{ $name }}
80
+        - name: dev
81
+          hostPath:
82
+              path: /dev
83
+        - name: sys
84
+          hostPath:
85
+              path: /sys
86
+        - name: volume-cloudinit
87
+          configMap:
88
+            name: configmap-cloudconfig-{{ $name }}
89
+            items:
90
+            - key: userdata
91
+              path: userdata
92
+            - key: metadata
93
+              path: metadata
94
+{{- else }}
95
+# {{ $name }} not enabled!
96
+{{- end }}
97
+---
98
+{{- end }}

+ 23
- 0
berth/templates/iter-pvc.yaml View File

@@ -0,0 +1,23 @@
1
+
2
+{{- range $name, $vm := .Values.vmlist }}
3
+# id: {{- $name }}
4
+{{- if $vm.enabled }}
5
+# vm enabled
6
+# {{ $vm.vmconfig.rootfs.pvc_size }} {{ $vm.vmconfig.rootfs.pvc_class }}
7
+kind: PersistentVolumeClaim
8
+apiVersion: v1
9
+metadata:
10
+  name: {{ $name }}
11
+spec:
12
+  accessModes: [ "ReadWriteOnce" ]
13
+  resources:
14
+    requests:
15
+      storage: {{ $vm.vmconfig.rootfs.pvc_size }}
16
+{{- if $vm.vmconfig.rootfs.pvc_class }}
17
+  storageClassName: {{ $vm.vmconfig.rootfs.pvc_class}}
18
+{{- end }}
19
+{{- else }}
20
+# vm disabled - skipping
21
+{{- end }}
22
+---
23
+{{- end }}

+ 4
- 10
berth/values.yaml View File

@@ -1,18 +1,12 @@
1 1
 
2
-auth:
3
-  ssh_key:
4
-
5 2
 images:
6
-  entrypoint: quay.io/attcomdev/kvm-manager:latest
3
+  vmrunner: quay.io/attcomdev/kvm-manager:latest
7 4
 
8 5
 labels:
9 6
   node_selector_key: openstack-control-plane
10 7
   node_selector_value: enabled
11 8
 
12
-volume:
13
-  class_name:
14
-  size: 25Gi
9
+# by default in the chart's values.yaml this is empty; use and
10
+# override file and helm --values=... for your specific values
11
+vmlist: { }
15 12
 
16
-network:
17
-  port: 22
18
-  vnc: 5900

+ 10
- 0
examples/bogus-vm.yaml View File

@@ -0,0 +1,10 @@
1
+# this vm isn't fully defined, not enabling it is enough to have the
2
+# templates skip over the details and not barf
3
+vmlist:
4
+  - bogus-vm:
5
+    enabled: false
6
+    extra: "thang"
7
+    vmconfig: "this is wrong"
8
+    cloudconfig:
9
+      metadata: "{ "
10
+      userdata: false

+ 22
- 0
examples/cirros-test.yaml View File

@@ -0,0 +1,22 @@
1
+vmlist:
2
+  cirros-test:
3
+    enabled: true
4
+    vmconfig:
5
+      cpu:
6
+        vcpu: 1
7
+        ram_mb: 256
8
+      rootfs:
9
+        sourceurl: http://stupidest.org/vm/cirros-0.3.5-x86_64-disk.img
10
+        localtarget: cirros-vm.qcow2
11
+        pvc_size: 128Mi
12
+    netconfig:
13
+      ports:
14
+        ssh: 22
15
+        vnc: 5900
16
+    cloudconfig:
17
+      metadata: |
18
+        { "uuid": "093772fe-d6a3-4eea-84bc-5966661a0c3e" }
19
+      userdata: |
20
+        #cloud-config
21
+        fqdn: cirros.example.com
22
+      bogus_not_used: "this won't hurt anthing but shouldn't be here"

+ 33
- 0
examples/demo-ub14-apache.yaml View File

@@ -0,0 +1,33 @@
1
+vmlist:
2
+  demo-ub14-apache:
3
+    enabled: true
4
+    vmconfig:
5
+      cpu:
6
+        vcpu: 1
7
+        ram_mb: 1024
8
+      rootfs:
9
+        sourceurl: http://stupidest.org/vm/ubuntu-14.04-amd64.img
10
+        localtarget: ubuntu-14.04-amd64.qcow2
11
+        pvc_size: 5Gi
12
+        pvc_class: nfs
13
+    netconfig:
14
+      ports:
15
+        ssh: 22
16
+        vnc: 5900
17
+      readinessTcpProbe: 22
18
+    cloudconfig:
19
+      metadata: |
20
+        { "uuid": "apache-demo.example.com" }
21
+      userdata: |
22
+        #cloud-config
23
+        fqdn: apache-demo.example.com
24
+        users:
25
+          - name: root
26
+            ssh-authorized-keys:
27
+              - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII+k1AYpUX7Y6+pVzkw3JPbRPNpoh7m1rZBP4Qa37Wz2 user@host"
28
+              - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPE/zGmNx3W52ztb/2vvTcgUN7RGbq172QXGcXKAagU1 user@host"
29
+              - "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKVeMUpC6Ba0UwyLeCCWexr/sc4kYwKW6mTSTpcQHDm+F5UqoW3pnbGPn3WSJB2AbCMP9oG0qQrLP2zHmE7fyog= otheruser@otherhost"
30
+        ssh_pwauth: True
31
+        runcmd:
32
+          - [ apt-get, update ]
33
+          - [ apt-get, install, -y, --force-yes, apache2 ]

+ 36
- 0
examples/ub16-smp-test.yaml View File

@@ -0,0 +1,36 @@
1
+vmlist:
2
+  ub16-smp-test:
3
+    enabled: true
4
+    vmconfig:
5
+      cpu:
6
+        vcpu: 2
7
+        ram_mb: 2048
8
+      rootfs:
9
+        sourceurl: http://stupidest.org/vm/cw-ub16-test-3a.qcow2
10
+        localtarget: cw-ub16-test.qcow2
11
+        pvc_size: 7Gi
12
+    netconfig:
13
+      ports:
14
+        ssh: 22
15
+        vnc: 5900
16
+    cloudconfig:
17
+      metadata: |
18
+        {
19
+            "uuid": "093772fe-d6a3-4eea-84bc-5966661a0c3e",
20
+            "name": "my-name",
21
+            "instance-id": "my-instance-id",
22
+            "availability-zone": "my-az",
23
+            "hostname": "my-hostname",
24
+            "local-hostname": "my-local-hostname",
25
+            "launch-index": "123"
26
+        }
27
+      userdata: |
28
+        #cloud-config
29
+        fqdn: ub16-smp-test.example.com
30
+        users:
31
+          - name: root
32
+            ssh-authorized-keys:
33
+              - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII+k1AYpUX7Y6+pVzkw3JPbRPNpoh7m1rZBP4Qa37Wz2 user@host"
34
+              - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPE/zGmNx3W52ztb/2vvTcgUN7RGbq172QXGcXKAagU1 user@host"
35
+              - "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKVeMUpC6Ba0UwyLeCCWexr/sc4kYwKW6mTSTpcQHDm+F5UqoW3pnbGPn3WSJB2AbCMP9oG0qQrLP2zHmE7fyog= otheruser@otherhost"
36
+        ssh_pwauth: True

+ 13
- 0
tools/gate/setup.sh View File

@@ -0,0 +1,13 @@
1
+#!/bin/sh
2
+
3
+set -ex
4
+
5
+. /etc/os-release
6
+type=${ID_LIKE:=ID}
7
+
8
+if [ "$type" == "debian" ] ; then
9
+    apt-get update
10
+    apt-get install netcat-openbsd jq
11
+else
12
+    yum install netcat jq
13
+fi

+ 41
- 0
tools/gate/test.sh View File

@@ -0,0 +1,41 @@
1
+#!/bin/bash
2
+
3
+set -ex
4
+
5
+NS=berth
6
+
7
+helm install --name=berth --debug ./berth --values=examples/cirros-test.yaml --namespace="${NS}"
8
+
9
+# wait until we get a PODIP
10
+while : ; do
11
+    PODIP=$(kubectl -n "${NS}" get pods -o wide -o json | jq -r '.items[].status.podIP')
12
+    if [ -n "$PODIP" -a "null" != "$PODIP" ] ; then
13
+	break
14
+    fi
15
+    echo "waiting for PODIP"
16
+    # XXX
17
+    kubectl get pods --all-namespaces
18
+    sleep 2
19
+done
20
+
21
+kubectl -n "${NS}" get pods
22
+
23
+# wait for pod to come up say something on ssh
24
+timeout=60
25
+t=0
26
+while : ; do
27
+    if echo "bye" | nc "${PODIP}" 22 | grep --quiet ^SSH ; then
28
+        echo "VM up"
29
+	break
30
+    fi
31
+    if [ $t -gt $timeout ] ; then
32
+        exit 2
33
+    fi
34
+    t=$(($t + 5))
35
+    sleep 2
36
+done
37
+
38
+# verify we can cleanup
39
+helm upgrade berth ./berth --values=berth/values.yaml
40
+
41
+helm delete --purge berth

Loading…
Cancel
Save