openstack/magnum
Code Issues Proposed changes

DIB elements to support dcos for magnum

Browse Source 
This is a patch to build a centos image to support installing DC/OS as
a magnum coe.

Two elements are included: dcos and docker, only used to build a centos
image.

1. Element docker will install and configure docker in centos.
2. Element dcos will download dcos_generate_config.sh and do some
configurations for DC/OS.

Design spec in contrib/drivers/dcos_centos_v1/image/README.md.

Partially-Implements: blueprint mesos-dcos

Change-Id: I30fa4c102205aa2475b9491398c06da0d1e86f84
This commit is contained in:
fengbeihong 2016-09-02 14:52:41 +08:00
parent 42abb07835
commit 10e9132bb8
15 changed files with 281 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

0
contrib/drivers/dcos_centos_v1/__init__.py Normal file
View File

86
contrib/drivers/dcos_centos_v1/image/README.md Normal file
View File

@ -0,0 +1,86 @@
=============
centos-dcos
=============
This directory contains `[diskimage-builder](https://github.com/openstack/diskimage-builder)`
elements to build an centos image which contains dcos.
Pre-requisites to run diskimage-builder
---------------------------------------
For diskimage-builder to work, following packages need to be
present:
* kpartx
* qemu-utils
* curl
* xfsprogs
* yum
* yum-utils
* git
For Debian/Ubuntu systems, use::
apt-get install kpartx qemu-utils curl xfsprogs yum yum-utils git
For CentOS and Fedora < 22, use::
yum install kpartx qemu-utils curl xfsprogs yum yum-utils git
For Fedora >= 22, use::
dnf install kpartx @virtualization curl xfsprogs yum yum-utils git
How to generate Centos image with DC/OS 1.8.x
---------------------------------------------
1. Download and export element path
git clone https://git.openstack.org/openstack/magnum
git clone https://git.openstack.org/openstack/diskimage-builder.git
git clone https://git.openstack.org/openstack/dib-utils.git
git clone https://git.openstack.org/openstack/tripleo-image-elements.git
git clone https://git.openstack.org/openstack/heat-templates.git
export PATH="${PWD}/diskimage-builder/bin:$PATH"
export PATH="${PWD}/dib-utils/bin:$PATH"
export ELEMENTS_PATH=magnum/contrib/drivers/dcos_centos_v1/image
export ELEMENTS_PATH=${ELEMENTS_PATH}:diskimage-builder/elements
export ELEMENTS_PATH=${ELEMENTS_PATH}:tripleo-image-elements/elements:heat-templates/hot/software-config/elements
2. Export environment path of the url to download dcos_generate_config.sh
This default download url is for DC/OS 1.8.4
export DCOS_GENERATE_CONFIG_SRC=https://downloads.dcos.io/dcos/stable/commit/e64024af95b62c632c90b9063ed06296fcf38ea5/dcos_generate_config.sh
Or specify local file path
export DCOS_GENERATE_CONFIG_SRC=`pwd`/dcos_generate_config.sh
3. Set file system type to `xfs`
Only XFS is currently supported for overlay.
See https://dcos.io/docs/1.8/administration/installing/custom/system-requirements/install-docker-centos/#recommendations
export FS_TYPE=xfs
4. Create image
disk-image-create \
centos7 vm docker dcos selinux-permissive \
os-collect-config os-refresh-config os-apply-config \
heat-config heat-config-script \
-o centos-7-dcos.qcow2
5. (Optional) Create user image for bare metal node
Create with elements dhcp-all-interfaces and devuser
export DIB_DEV_USER_USERNAME=centos
export DIB_DEV_USER_PWDLESS_SUDO=YES
disk-image-create \
centos7 vm docker dcos selinux-permissive dhcp-all-interfaces devuser \
os-collect-config os-refresh-config os-apply-config \
heat-config heat-config-script \
-o centos-7-dcos-bm.qcow2

2
contrib/drivers/dcos_centos_v1/image/dcos/elements-deps Normal file
View File

@ -0,0 +1,2 @@
package-installs
docker

5
contrib/drivers/dcos_centos_v1/image/dcos/environment.d/10-dcos-install-url Executable file
View File

@ -0,0 +1,5 @@
# Specify download url, default DC/OS version 1.8.4
export DCOS_GENERATE_CONFIG_SRC=${DCOS_GENERATE_CONFIG_SRC:-https://downloads.dcos.io/dcos/stable/commit/e64024af95b62c632c90b9063ed06296fcf38ea5/dcos_generate_config.sh}
# or local file path
# export DCOS_GENERATE_CONFIG_SRC=${DCOS_GENERATE_CONFIG_SRC:-${PWD}/dcos_generate_config.sh}

23
contrib/drivers/dcos_centos_v1/image/dcos/extra-data.d/99-download-generate-config Executable file
View File

@ -0,0 +1,23 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
# This scrpit file is used to dowload dcos_generate_config.sh outside chroot.
# Ihis file is essential that the size of dcos_generate_config.sh is more than
# 700M, we should download it into the image in advance.
sudo mkdir -p $TMP_MOUNT_PATH/opt/dcos
if [ -f $DCOS_GENERATE_CONFIG_SRC ]; then
# If $DCOS_GENERATE_CONFIG_SRC is a file path, copy the file
sudo cp $DCOS_GENERATE_CONFIG_SRC $TMP_MOUNT_PATH/opt/dcos
else
# If $DCOS_GENERATE_CONFIG_SRC is a url, download it
# Please make sure curl is installed on your host environment
cd $TMP_MOUNT_PATH/opt/dcos
sudo -E curl -O $DCOS_GENERATE_CONFIG_SRC
fi

6
contrib/drivers/dcos_centos_v1/image/dcos/package-installs.yaml Normal file
View File

@ -0,0 +1,6 @@
tar:
xz:
unzip:
curl:
ipset:
ntp:

10
contrib/drivers/dcos_centos_v1/image/dcos/post-install.d/99-add-norgoup Executable file
View File

@ -0,0 +1,10 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
# nogroup will be used on Mesos masters and agents.
sudo groupadd nogroup

12
contrib/drivers/dcos_centos_v1/image/dcos/post-install.d/99-disable-firewall Executable file
View File

@ -0,0 +1,12 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
# CentOS-7 introduced firewalld, which is a wrapper around iptables
# and can conflict with Docker.
# See https://docs.docker.com/v1.6/installation/centos/#firewalld
sudo systemctl disable firewalld

9
contrib/drivers/dcos_centos_v1/image/dcos/post-install.d/99-enable-ntp Normal file
View File

@ -0,0 +1,9 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
sudo systemctl enable ntpd

1
contrib/drivers/dcos_centos_v1/image/docker/elements-deps Normal file
View File

@ -0,0 +1 @@
package-installs

24
contrib/drivers/dcos_centos_v1/image/docker/install.d/50-install-docker Executable file
View File

@ -0,0 +1,24 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
# Install the Docker engine, daemon, and service.
#
# The supported versions of Docker are:
# 1.7.x
# 1.8.x
# 1.9.x
# 1.10.x
# 1.11.x
# Docker 1.12.x is NOT supported.
# Docker 1.9.x - 1.11.x is recommended for stability reasons.
# https://github.com/docker/docker/issues/9718
#
# See DC/OS installtion guide for details
# https://dcos.io/docs/1.8/administration/installing/custom/system-requirements/install-docker-centos/
#
sudo -E yum install -y docker-engine-1.11.2

9
contrib/drivers/dcos_centos_v1/image/docker/post-install.d/60-enable-docker-service Executable file
View File

@ -0,0 +1,9 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
sudo systemctl enable docker

26
contrib/drivers/dcos_centos_v1/image/docker/pre-install.d/10-enable-overlay Executable file
View File

@ -0,0 +1,26 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
# Upgrade CentOS to 7.2
sudo -E yum upgrade --assumeyes --tolerant
sudo -E yum update --assumeyes
# Verify that the kernel is at least 3.10
function version_gt() { test "$(echo "$@" | tr " " "\n" | sort -V | head -n 1)" != "$1"; }
kernel_version=`uname -r | cut --bytes=1-4`
expect_version=3.10
if version_gt $expect_version $kernel_version; then
echo "Error: kernel version at least $expect_version, current version $kernel_version"
exit 1
fi
# Enable OverlayFS
sudo tee /etc/modules-load.d/overlay.conf <<-'EOF'
overlay
EOF

33
contrib/drivers/dcos_centos_v1/image/docker/pre-install.d/20-configure-docker-service Executable file
View File

@ -0,0 +1,33 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
# Configure yum to use the Docker yum repo
sudo tee /etc/yum.repos.d/docker.repo <<-'EOF'
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/7/
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
EOF
# Configure systemd to run the Docker Daemon with OverlayFS
# Manage Docker on CentOS with systemd.
# systemd handles starting Docker on boot and restarting it when it crashes.
#
# Docker 1.11.x will be installed, so issue for Docker 1.12.x on Centos7
# won't happen.
# https://github.com/docker/docker/issues/22847
# https://github.com/docker/docker/issues/25098
#
sudo mkdir -p /etc/systemd/system/docker.service.d
sudo tee /etc/systemd/system/docker.service.d/override.conf <<- 'EOF'
[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon --storage-driver=overlay -H fd://
EOF

35
contrib/drivers/dcos_centos_v1/image/validate_dcos_image.sh Executable file
View File

@ -0,0 +1,35 @@
#!/bin/bash
#
# Copyright (c) 2016 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -e
# check that image is valid
qemu-img check -q $1
# validate estimated size
FILESIZE=$(stat -c%s "$1")
MIN_SIZE=1231028224 # 1.15GB
MAX_SIZE=1335885824 # 1.25GB
if [ $FILESIZE -lt $MIN_SIZE ] ; then
echo "Error: generated image size is lower than expected."
exit 1
fi
if [ $FILESIZE -gt $MAX_SIZE ] ; then
echo "Error: generated image size is higher than expected."
exit 1
fi