Merge larsks/heat-kubernetes to Magnum

In 20150106 IRC meeting, we decide to add two templates to magnum
for creating bays. One for virt and the other is for ironic.

larsks/heat-kubernetes is for virt and we can merge it first.

Change-Id: I9db19c006db9c9b725a562f532448d447761542f

etc/magnum/templates/heat-kubernetes/COPYING

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

etc/magnum/templates/heat-kubernetes/README.md

@@ -0,0 +1,116 @@
+A Kubernetes cluster with Heat
+==============================
+
+These [Heat][] templates will deploy an *N*-node [Kubernetes][] cluster,
+where *N* is the value of the `number_of_minions` parameter you
+specify when creating the stack.
+
+[heat]: https://wiki.openstack.org/wiki/Heat
+[kubernetes]: https://github.com/GoogleCloudPlatform/kubernetes
+
+The cluster uses [Flannel][] to provide an overlay network connecting
+pods deployed on different minions.
+
+[flannel]: https://github.com/coreos/flannel
+
+## Requirements
+
+### OpenStack
+
+These templates will work with the Juno version of Heat.
+
+### Guest image
+
+These templates will work with either CentOS Atomic Host or Fedora 21
+Atomic.  You will need an image dated later than 2015-01-01, or you
+will need to create an image that includes Flannel by booting an
+existing image, running `atomic ugprade`, and saving the new image.
+
+You can enable the VXLAN backend for flannel by setting the
+"flannel_use_vxlan" parameter to "true", but I have run into kernel
+crashes using that backend with CentOS 7.  It seems to work fine with
+Fedora 21.
+
+## Creating the stack
+
+Creating an environment file `local.yaml` with parameters specific to
+your environment:
+
+    parameters:
+      ssh_key_name: lars
+      external_network_id: 028d70dd-67b8-4901-8bdd-0c62b06cce2d
+      dns_nameserver: 192.168.200.1
+      server_image: centos-7-atomic-20150101
+
+And then create the stack, referencing that environment file:
+
+    heat stack-create -f kubecluster.yaml -e local.yaml my-kube-cluster
+
+You must provide values for:
+
+- `ssh_key_name`
+- `external_network_id`
+- `server_image`
+
+## Interacting with Kubernetes
+
+You can get the ip address of the Kubernetes master using the `heat
+output-show` command:
+
+    $ heat output-show my-kube-cluster kube_master
+    "192.168.200.86"
+
+You can ssh into that server as the `minion` user:
+
+    $ ssh minion@192.168.200.86
+
+And once logged in you can run `kubectl`, etc:
+
+    $ kubectl get minions
+    NAME                LABELS
+    10.0.0.4            <none>
+
+You can log into your minions using the `minion` user as well.  You
+can get a list of minion addresses by running:
+
+    $ heat output-show my-kube-cluster kube_minions_external
+    [
+      "192.168.200.182"
+    ]
+
+## Testing
+
+The templates install an example Pod and Service description into
+`/etc/kubernetes/examples`.  You can deploy this with the following
+commands:
+
+    $ kubectl create -f /etc/kubernetes/examples/web.service
+    $ kubectl create -f /etc/kubernetes/examples/web.pod
+
+This will deploy a minimal webserver and a service.  You can use
+`kubectl get pods` and `kubectl get services` to see the results of
+these commands.
+
+## License
+
+Copyright 2014 Lars Kellogg-Stedman <lars@redhat.com>
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use these files except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+## Contact
+
+Please report bugs using the [GitHub issue tracker][] at
+https://github.com/larsks/heat-kubernetes/issues.
+
+[github issue tracker]: https://github.com/larsks/heat-kubernetes/issues
+

etc/magnum/templates/heat-kubernetes/beaker.yaml

@@ -0,0 +1,7 @@
+parameters:
+  ssh_key_name: lars_redhat
+  server_image: fedora-21
+  server_flavor: m1.medium
+  external_network_id: 59bcbd61-f5ed-4c77-8b60-b7a004ed40b3
+  dns_nameserver: 10.16.36.29
+  fixed_network_cidr: 192.168.113.0/24

etc/magnum/templates/heat-kubernetes/fragments/add-to-docker-group.sh

@@ -0,0 +1,14 @@
+#!/bin/sh
+
+# Under atomic, we need to make sure the 'docker' group exists in 
+# /etc/group (because /lib/group cannot be modified by usermod).
+echo "making 'docker' group editable"
+if ! grep -q docker /etc/group; then
+	grep docker /lib/group >> /etc/group
+fi
+
+# make 'minion' user a member of the docker group
+# (so you can run docker commands as the 'minion' user)
+echo "adding 'minion' user to 'docker' group"
+usermod -G docker minion
+

etc/magnum/templates/heat-kubernetes/fragments/cfn-signal.sh

@@ -0,0 +1,11 @@
+#!/bin/sh
+
+. /etc/sysconfig/heat-params
+
+echo "notifying heat"
+curl -sf -X PUT -H 'Content-Type: application/json' \
+	--data-binary '{"Status": "SUCCESS",
+	"Reason": "Setup complete",
+	"Data": "OK", "UniqueId": "00000"}' \
+	"$WAIT_HANDLE"
+

etc/magnum/templates/heat-kubernetes/fragments/configure-flannel.sh

@@ -0,0 +1,33 @@
+#!/bin/sh
+
+. /etc/sysconfig/heat-params
+. /etc/sysconfig/flanneld
+
+if [ "$FLANNEL_USE_VXLAN" == "true" ]; then
+	use_vxlan=1
+fi
+
+# Generate a flannel configuration that we will
+# store into etcd using curl.
+cat > /etc/sysconfig/flannel-network.json <<EOF
+{
+  "Network": "$FLANNEL_NETWORK_CIDR",
+  "Subnetlen": $FLANNEL_NETWORK_SUBNETLEN${use_vxlan:+",
+  "Backend": {
+    "Type": "vxlan"
+  }"}
+}
+EOF
+
+# wait for etcd to become active (we will need it to push the flanneld config)
+while ! curl -sf -o /dev/null $FLANNEL_ETCD/v2/keys/; do
+  echo "waiting for etcd"
+  sleep 1
+done
+
+# put the flannel config in etcd
+echo "creating flanneld config in etcd"
+curl -sf -L $FLANNEL_ETCD/v2/keys/coreos.com/network/config \
+  -X PUT \
+  --data-urlencode value@/etc/sysconfig/flannel-network.json
+

etc/magnum/templates/heat-kubernetes/fragments/configure-kubernetes-master.sh

@@ -0,0 +1,19 @@
+#!/bin/sh
+
+. /etc/sysconfig/heat-params
+
+echo "configuring kubernetes (master)"
+
+sed -i '
+  /^KUBE_ALLOW_PRIV=/ s/=.*/="--allow_privileged='"$KUBE_ALLOW_PRIV"'"/
+' /etc/kubernetes/config
+
+sed -i '
+  /^KUBE_API_ADDRESS=/ s/=.*/="--address=0.0.0.0"/
+  /^KUBE_SERVICE_ADDRESSES=/ s|=.*|="--portal_net='"$PORTAL_NETWORK_CIDR"'"|
+' /etc/kubernetes/apiserver
+
+sed -i '
+  /^KUBELET_ADDRESSES=/ s/=.*/="--machines='"$MINION_ADDRESSES"'"/
+' /etc/kubernetes/controller-manager
+

etc/magnum/templates/heat-kubernetes/fragments/configure-kubernetes-minion.sh

@@ -0,0 +1,31 @@
+#!/bin/sh
+
+. /etc/sysconfig/heat-params
+
+echo "configuring kubernetes (minion)"
+
+myip=$(ip addr show eth0 |
+awk '$1 == "inet" {print $2}' | cut -f1 -d/)
+myip_last_octet=${myip##*.}
+
+sed -i '
+/^KUBE_ALLOW_PRIV=/ s/=.*/="--allow_privileged='"$KUBE_ALLOW_PRIV"'"/
+/^KUBE_ETCD_SERVERS=/ s|=.*|="--etcd_servers=http://'"$KUBE_MASTER_IP"':4001"|
+' /etc/kubernetes/config
+
+sed -i '
+/^KUBELET_ADDRESS=/ s/=.*/="--address=0.0.0.0"/
+/^KUBELET_HOSTNAME=/ s/=.*/="--hostname_override='"$myip"'"/
+' /etc/kubernetes/kubelet
+
+sed -i '
+/^KUBE_MASTER=/ s/=.*/="--master='"$KUBE_MASTER_IP"':8080"/
+' /etc/kubernetes/apiserver
+
+sed -i '
+/^FLANNEL_ETCD=/ s|=.*|="http://'"$KUBE_MASTER_IP"':4001"|
+' /etc/sysconfig/flanneld
+
+cat >> /etc/environment <<EOF
+KUBERNETES_MASTER=http://$KUBE_MASTER_IP:8080
+EOF

etc/magnum/templates/heat-kubernetes/fragments/disable-selinux.sh

@@ -0,0 +1,8 @@
+#cloud-boothook
+#!/bin/sh
+
+setenforce 0
+
+sed -i '
+  /^SELINUX=/ s/=.*/=permissive/
+' /etc/selinux/config

etc/magnum/templates/heat-kubernetes/fragments/docker.service.yaml

@@ -0,0 +1,65 @@
+#cloud-config
+merge_how: dict(recurse_array)+list(append)
+write_files:
+  - path: /etc/systemd/system/docker.service
+    owner: "root:root"
+    permissions: "0644"
+    content: |
+      [Unit]
+      Description=Docker Application Container Engine
+      Documentation=http://docs.docker.com
+      After=network.target docker.socket wait-for-flanneld.service
+      Requires=docker.socket wait-for-flanneld.service
+
+      [Service]
+      Type=notify
+      EnvironmentFile=-/etc/sysconfig/docker
+      EnvironmentFile=-/etc/sysconfig/docker-storage
+      
+      # This EnvironmentFile is required.
+      EnvironmentFile=/run/flannel/subnet.env
+
+      # This works around https://github.com/docker/docker/issues/9468
+      Environment=DOCKER_CERT_PATH=/etc/docker
+
+      ExecStart=/usr/bin/docker -d -H fd:// --bip $FLANNEL_SUBNET --mtu $FLANNEL_MTU $OPTIONS $DOCKER_STORAGE_OPTIONS
+      Restart=on-failure
+      LimitNOFILE=1048576
+      LimitNPROC=1048576
+
+      [Install]
+      WantedBy=multi-user.target
+  - path: /usr/local/bin/wait-for-flanneld
+    owner: "root:root"
+    permissions: "0755"
+    content: |
+      #!/bin/sh
+      
+      # This script waits for flannel to provide the /run/flannel/subnet.env
+      # file. This can be used to synchronize docker startup with the
+      # availability of this file.
+
+      while ! [ -f /run/flannel/subnet.env ]; do
+        echo "waiting for flanneld"
+        sleep 1
+      done
+
+      echo "flanneld is active"
+
+      exit 0
+  - path: /etc/systemd/system/wait-for-flanneld.service
+    owner: "root:root"
+    permissions: "0644"
+    content: |
+      [Unit]
+      Description=Wait for flanneld to provide subnet/mtu information
+      After=network.target flanneld.service
+      Requires=flanneld.service
+
+      [Service]
+      Type=oneshot
+      ExecStart=/usr/local/bin/wait-for-flanneld
+
+      [Install]
+      WantedBy=multi-user.target
+

etc/magnum/templates/heat-kubernetes/fragments/enable-services-master.sh

@@ -0,0 +1,10 @@
+#!/bin/sh
+
+echo starting services
+for service in etcd kube-apiserver kube-scheduler kube-controller-manager; do
+	echo "activating service $service"
+	systemctl enable $service
+	systemctl --no-block start $service
+done
+
+

etc/magnum/templates/heat-kubernetes/fragments/enable-services-minion.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+
+# docker is already enabled and possibly running on centos atomic host
+# so we need to stop it first and delete the docker0 bridge (which will
+# be re-created using the flannel-provided subnet).
+echo "stopping docker"
+systemctl stop docker
+ip link del docker0
+
+for service in wait-for-flanneld flanneld docker.socket docker kubelet kube-proxy; do
+	echo "activating service $service"
+	systemctl enable $service
+	systemctl --no-block start $service
+done
+

etc/magnum/templates/heat-kubernetes/fragments/kube-examples.yaml

@@ -0,0 +1,32 @@
+#cloud-config
+merge_how: dict(recurse_array)+list(append)
+write_files:
+  - path: /etc/kubernetes/examples/web.pod
+    owner: "root:root"
+    permissions: "0644"
+    content: |
+      kind: Pod
+      apiVersion: v1beta1
+      labels:
+        name: web
+      desiredState:
+        manifest:
+          version: v1beta1
+          id: web
+          containers:
+            - name: web
+              image: larsks/onehttpd
+              command: [/onehttpd, -p, 80, /data]
+              ports:
+                - containerPort: 80
+  - path: /etc/kubernetes/examples/web.service
+    owner: "root:root"
+    permissions: "0644"
+    content: |
+      kind: Service
+      apiVersion: v1beta1
+      id: web
+      port: 8000
+      selector:
+        name: web
+      containerPort: 80

etc/magnum/templates/heat-kubernetes/fragments/kube-user.yaml

@@ -0,0 +1,10 @@
+#cloud-config
+system_info:
+  default_user:
+    name: minion
+    lock_passwd: true
+    gecos: Kubernetes Interactive User
+    groups: [wheel, adm, systemd-journal]
+    sudo: ["ALL=(ALL) NOPASSWD:ALL"]
+    shell: /bin/bash
+

etc/magnum/templates/heat-kubernetes/fragments/write-heat-params-master.yaml

@@ -0,0 +1,14 @@
+#cloud-config
+merge_how: dict(recurse_array)+list(append)
+write_files:
+  - path: /etc/sysconfig/heat-params
+    owner: "root:root"
+    permissions: "0644"
+    content: |
+      MINION_ADDRESSES="$MINION_ADDRESSES"
+      KUBE_ALLOW_PRIV="$KUBE_ALLOW_PRIV"
+      WAIT_HANDLE="$WAIT_HANDLE"
+      FLANNEL_NETWORK_CIDR="$FLANNEL_NETWORK_CIDR"
+      FLANNEL_NETWORK_SUBNETLEN="$FLANNEL_NETWORK_SUBNETLEN"
+      FLANNEL_USE_VXLAN="$FLANNEL_USE_VXLAN"
+      PORTAL_NETWORK_CIDR="$PORTAL_NETWORK_CIDR"

etc/magnum/templates/heat-kubernetes/fragments/write-heat-params.yaml

@@ -0,0 +1,11 @@
+#cloud-config
+merge_how: dict(recurse_array)+list(append)
+write_files:
+  - path: /etc/sysconfig/heat-params
+    owner: "root:root"
+    permissions: "0644"
+    content: |
+      KUBE_ALLOW_PRIV="$KUBE_ALLOW_PRIV"
+      KUBE_MASTER_IP="$KUBE_MASTER_IP"
+      WAIT_HANDLE="$WAIT_HANDLE"
+

etc/magnum/templates/heat-kubernetes/kubecluster.yaml

@@ -0,0 +1,318 @@
+heat_template_version: 2013-05-23
+
+description: >
+  This template will boot a Kubernetes cluster with one or more
+  minions (as specified by the number_of_minions parameter, which
+  defaults to "2").
+
+parameters:
+
+  #
+  # REQUIRED PARAMETERS
+  #
+  ssh_key_name:
+    type: string
+    description: name of ssh key to be provisioned on our server
+
+  external_network_id:
+    type: string
+    description: uuid of a network to use for floating ip addresses
+
+  #
+  # OPTIONAL PARAMETERS
+  #
+  server_image:
+    type: string
+    default: centos-atomic
+    description: glance image used to boot the server
+
+  server_flavor:
+    type: string
+    default: m1.small
+    description: flavor to use when booting the server
+
+  dns_nameserver:
+    type: string
+    description: address of a dns nameserver reachable in your environment
+    default: 8.8.8.8
+
+  number_of_minions:
+    type: string
+    description: how many kubernetes minions to spawn
+    default: 1
+  
+  fixed_network_cidr:
+    type: string
+    description: network range for fixed ip network
+    default: "10.0.0.0/24"
+
+  portal_network_cidr:
+    type: string
+    description: >
+      address range used by kubernetes for service portals
+    default: "10.254.0.0/16"
+
+  flannel_network_cidr:
+    type: string
+    description: network range for flannel overlay network
+    default: "10.100.0.0/16"
+
+  flannel_network_subnetlen:
+    type: string
+    description: size of subnet assigned to each minion
+    default: 24
+
+  flannel_use_vxlan:
+    type: string
+    description: >
+      if true use the vxlan backend, otherwise use the default
+      udp backend
+    default: "false"
+    constraints:
+      - allowed_values: ["true", "false"]
+
+  kube_allow_priv:
+    type: string
+    description: >
+      whether or not kubernetes should permit privileged containers.
+    default: "true"
+    constraints:
+      - allowed_values: ["true", "false"]
+
+resources:
+
+  master_wait_handle:
+    type: "AWS::CloudFormation::WaitConditionHandle"
+
+  master_wait_condition:
+    type: "AWS::CloudFormation::WaitCondition"
+    depends_on:
+      - kube_master
+    properties:
+      Handle:
+        get_resource: master_wait_handle
+      Timeout: "6000"
+
+  ######################################################################
+  #
+  # network resources.  allocate a network and router for our server.
+  # it would also be possible to take advantage of existing network
+  # resources (and have the deployer provide network and subnet ids,
+  # etc, as parameters), but I wanted to minmize the amount of
+  # configuration necessary to make this go.
+
+  fixed_network:
+    type: "OS::Neutron::Net"
+
+  # This is the subnet on which we will deploy our server.
+  fixed_subnet:
+    type: "OS::Neutron::Subnet"
+    properties:
+      cidr: {get_param: fixed_network_cidr}
+      network_id:
+        get_resource: fixed_network
+      dns_nameservers:
+        - get_param: dns_nameserver
+
+  # create a router attached to the external network provided as a
+  # parameter to this stack.
+  extrouter:
+    type: "OS::Neutron::Router"
+    properties:
+      external_gateway_info:
+        network:
+          get_param: external_network_id
+
+  # attached fixed_subnet to our extrouter router.
+  extrouter_inside:
+    type: "OS::Neutron::RouterInterface"
+    properties:
+      router_id:
+        get_resource: extrouter
+      subnet_id:
+        get_resource:
+          fixed_subnet
+
+  ######################################################################
+  #
+  # security groups.  we need to permit network traffic of various
+  # sorts.
+  #
+
+  # permit ssh access
+  secgroup_base:
+    type: "OS::Neutron::SecurityGroup"
+    properties:
+      rules:
+        - protocol: icmp
+        - protocol: tcp
+          port_range_min: 22
+          port_range_max: 22
+
+  # open ports for kubernetes and etcd
+  secgroup_kubernetes:
+    type: "OS::Neutron::SecurityGroup"
+    properties:
+      rules:
+        - protocol: tcp
+          port_range_min: 7080
+          port_range_max: 7080
+        - protocol: tcp
+          port_range_min: 8080
+          port_range_max: 8080
+        - protocol: tcp
+          port_range_min: 4001
+          port_range_max: 4001
+        - protocol: tcp
+          port_range_min: 7001
+          port_range_max: 7001
+
+  ######################################################################
+  #
+  # software configs.  these are components that are combined into
+  # a multipart MIME user-data archive.
+  #
+
+  write_heat_params:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config:
+        str_replace:
+          template: {get_file: fragments/write-heat-params-master.yaml}
+          params:
+            "$MINION_ADDRESSES": {"Fn::Join": [",", {get_attr: [kube_minions, kube_node_ip]}]}
+            "$KUBE_ALLOW_PRIV": {get_param: kube_allow_priv}
+            "$WAIT_HANDLE": {get_resource: master_wait_handle}
+            "$FLANNEL_NETWORK_CIDR": {get_param: flannel_network_cidr}
+            "$FLANNEL_NETWORK_SUBNETLEN": {get_param: flannel_network_subnetlen}
+            "$FLANNEL_USE_VXLAN": {get_param: flannel_use_vxlan}
+            "$PORTAL_NETWORK_CIDR": {get_param: portal_network_cidr}
+
+  configure_kubernetes:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/configure-kubernetes-master.sh}
+
+  configure_flannel:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/configure-flannel.sh}
+
+  enable_services:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/enable-services-master.sh}
+
+  kube_user:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/kube-user.yaml}
+
+  kube_examples:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/kube-examples.yaml}
+
+  cfn_signal:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/cfn-signal.sh}
+
+  disable_selinux:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/disable-selinux.sh}
+
+  kube_master_init:
+    type: "OS::Heat::MultipartMime"
+    properties:
+      parts:
+        - config: {get_resource: disable_selinux}
+        - config: {get_resource: write_heat_params}
+        - config: {get_resource: kube_user}
+        - config: {get_resource: configure_kubernetes}
+        - config: {get_resource: enable_services}
+        - config: {get_resource: configure_flannel}
+        - config: {get_resource: kube_examples}
+        - config: {get_resource: cfn_signal}
+
+  ######################################################################
+  #
+  # databases server.  this sets up a Kubernetes server
+  #
+  kube_master:
+    type: "OS::Nova::Server"
+    depends_on:
+      - extrouter_inside
+    properties:
+      image:
+        get_param: server_image
+      flavor:
+        get_param: server_flavor
+      key_name:
+        get_param: ssh_key_name
+      user_data_format: RAW
+      user_data: {get_resource: kube_master_init}
+      networks:
+        - port:
+            get_resource: kube_master_eth0
+
+  kube_master_eth0:
+    type: "OS::Neutron::Port"
+    properties:
+      network_id:
+        get_resource: fixed_network
+      security_groups:
+        - get_resource: secgroup_base
+        - get_resource: secgroup_kubernetes
+      fixed_ips:
+        - subnet_id:
+            get_resource: fixed_subnet
+
+  kube_master_floating:
+    type: "OS::Neutron::FloatingIP"
+    depends_on:
+      - extrouter_inside
+    properties:
+      floating_network_id:
+        get_param: external_network_id
+      port_id:
+        get_resource: kube_master_eth0
+
+  kube_minions:
+    type: "OS::Heat::ResourceGroup"
+    depends_on:
+      - extrouter_inside
+    properties:
+      count: {get_param: number_of_minions}
+      resource_def:
+        type: kubenode.yaml
+        properties:
+          ssh_key_name: {get_param: ssh_key_name}
+          server_image: {get_param: server_image}
+          server_flavor: {get_param: server_flavor}
+          fixed_network_id: {get_resource: fixed_network}
+          fixed_subnet_id: {get_resource: fixed_subnet}
+          kube_master_ip: {get_attr: [kube_master_eth0, fixed_ips, 0, ip_address]}
+          external_network_id: {get_param: external_network_id}
+          kube_allow_priv: {get_param: kube_allow_priv}
+
+outputs:
+
+  kube_master:
+    value: {get_attr: [kube_master_floating, floating_ip_address]}
+
+  kube_minions:
+    value: {get_attr: [kube_minions, kube_node_ip]}
+
+  kube_minions_external:
+    value: {get_attr: [kube_minions, kube_node_external_ip]}
+

etc/magnum/templates/heat-kubernetes/kubenode.yaml

@@ -0,0 +1,197 @@
+heat_template_version: 2013-05-23
+
+description: >
+  This is a nested stack that defines a single Kubernetes minion,
+  based on a vanilla Fedora 20 cloud image.  This stack is included by
+  a ResourceGroup resource in the parent template (kubecluster.yaml).
+
+parameters:
+
+  server_image:
+    type: string
+    default: fedora-20-x86_64-updated
+    description: glance image used to boot the server
+
+  server_flavor:
+    type: string
+    default: m1.small
+    description: flavor to use when booting the server
+
+  ssh_key_name:
+    type: string
+    description: name of ssh key to be provisioned on our server
+    default: lars
+
+  external_network_id:
+    type: string
+    description: uuid of a network to use for floating ip addresses
+
+  kube_allow_priv:
+    type: string
+    description: >
+      whether or not kubernetes should permit privileged containers.
+    default: "false"
+    constraints:
+      - allowed_values: ["true", "false"]
+
+  # The following are all generated in the parent template.
+  kube_master_ip:
+    type: string
+    description: IP address of the Kubernetes master server.
+  fixed_network_id:
+    type: string
+    description: Network from which to allocate fixed addresses.
+  fixed_subnet_id:
+    type: string
+    description: Subnet from which to allocate fixed addresses.
+
+resources:
+
+  node_wait_handle:
+    type: "AWS::CloudFormation::WaitConditionHandle"
+
+  node_wait_condition:
+    type: "AWS::CloudFormation::WaitCondition"
+    depends_on:
+      - kube_node
+    properties:
+      Handle:
+        get_resource: node_wait_handle
+      Timeout: "6000"
+
+  ######################################################################
+  #
+  # security groups.  we need to permit network traffic of various
+  # sorts.
+  #
+
+  secgroup_all_open:
+    type: "OS::Neutron::SecurityGroup"
+    properties:
+      rules:
+        - protocol: icmp
+        - protocol: tcp
+        - protocol: udp
+
+  ######################################################################
+  #
+  # software configs.  these are components that are combined into
+  # a multipart MIME user-data archive.
+  #
+
+  write_heat_params:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config:
+        str_replace:
+          template: {get_file: fragments/write-heat-params.yaml}
+          params:
+            "$KUBE_ALLOW_PRIV": {get_param: kube_allow_priv}
+            "$KUBE_MASTER_IP": {get_param: kube_master_ip}
+            "$WAIT_HANDLE": {get_resource: node_wait_handle}
+
+  add_to_docker_group:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/add-to-docker-group.sh}
+
+  configure_kubernetes_minion:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/configure-kubernetes-minion.sh}
+
+  kube_user:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/kube-user.yaml}
+
+  kube_examples:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/kube-examples.yaml}
+
+  docker_service:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/docker.service.yaml}
+
+  enable_services:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/enable-services-minion.sh}
+
+  cfn_signal:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/cfn-signal.sh}
+
+  disable_selinux:
+    type: "OS::Heat::SoftwareConfig"
+    properties:
+      group: ungrouped
+      config: {get_file: fragments/disable-selinux.sh}
+
+  kube_node_init:
+    type: "OS::Heat::MultipartMime"
+    properties:
+      parts:
+        - config: {get_resource: disable_selinux}
+        - config: {get_resource: write_heat_params}
+        - config: {get_resource: kube_user}
+        - config: {get_resource: kube_examples}
+        - config: {get_resource: add_to_docker_group}
+        - config: {get_resource: configure_kubernetes_minion}
+        - config: {get_resource: docker_service}
+        - config: {get_resource: enable_services}
+        - config: {get_resource: cfn_signal}
+
+  kube_node:
+    type: "OS::Nova::Server"
+    properties:
+      image:
+        get_param: server_image
+      flavor:
+        get_param: server_flavor
+      key_name:
+        get_param: ssh_key_name
+      user_data_format: RAW
+      user_data: {get_resource: kube_node_init}
+      networks:
+        - port:
+            get_resource: kube_node_eth0
+
+  kube_node_eth0:
+    type: "OS::Neutron::Port"
+    properties:
+      network_id:
+        get_param: fixed_network_id
+      security_groups:
+        - get_resource: secgroup_all_open
+      fixed_ips:
+        - subnet_id:
+            get_param: fixed_subnet_id
+
+  kube_node_floating:
+    type: "OS::Neutron::FloatingIP"
+    properties:
+      floating_network_id:
+        get_param: external_network_id
+      port_id:
+        get_resource: kube_node_eth0
+
+outputs:
+
+  kube_node_ip:
+    value: {get_attr: [kube_node_eth0, fixed_ips, 0, ip_address]}
+  
+  kube_node_external_ip:
+    value: {get_attr: [kube_node_floating, floating_ip_address]}
+