Split the debootstrap functions into an element

debootstrap is not debian or ubuntu specific. We can make a debootstrap
element that knows how to do all of the things, and then a
debian-minimal and ubuntu-minimal image that use it. Finally, make
the debian element simply be a collection of the extra things we do to
make it look like a cloud-init based cloud image.

Change-Id: Iaf46c8e61bf1cac9a096cbfd75d6d6a9111b701e
This commit is contained in:
Monty Taylor 2015-03-22 16:11:55 -04:00 committed by Gregory Haynes
parent c1d7cb4d6c
commit 23aa76dff0
21 changed files with 276 additions and 155 deletions

View File

@ -0,0 +1,40 @@
==============
debian-minimal
==============
Create a minimal image based on Debian. We default to unstable but DIB_RELEASE
is mapped to any series of Debian.
Note that the default Debian series is `unstable`, and the default
mirrors for Debian can be problematic for `unstable`. Because apt does
not handle changing Packages files well across multiple out of sync
mirrors, it is recommended that you choose a single mirror of debian,
and pass it in via `DIB_DISTRIBUTION_MIRROR`.
If necessary, a custom apt keyring and debootstrap script can be
supplied to the `debootstrap` command via `DIB_APT_KEYRING` and
`DIB_DEBIAN_DEBOOTSTRAP_SCRIPT` respectively. Both options require the
use of absolute rather than relative paths.
Use of this element will also require the tool 'debootstrap' to be
available on your system. It should be available on Ubuntu, Debian,
and Fedora. It is also recommended that the 'debian-keyring' package
be installed.
The `DIB_OFFLINE` or more specific `DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE`
variables can be set to prefer the use of a pre-cached root filesystem
tarball.
The `DIB_DEBOOTSTRAP_EXTRA_ARGS` environment variable may be used to
pass extra arguments to the debootstrap command used to create the
base filesystem image. If --keyring is is used in `DIB_DEBOOTSTRAP_EXTRA_ARGS`,
it will override `DIB_APT_KEYRING` if that is used as well.
-------------------
Note on ARM systems
-------------------
Because there is not a one-to-one mapping of `ARCH` to a kernel package, if
you are building an image for ARM on debian, you need to specify which kernel
you want in the environment variable `DIB_ARM_KERNEL`. For instance, if you want
the `linux-image-mx5` package installed, set `DIB_ARM_KERNEL` to `mx5`.

View File

@ -0,0 +1 @@
debootstrap

View File

@ -0,0 +1 @@
operating-system

View File

@ -0,0 +1,3 @@
export DISTRO_NAME=debian
export DIB_RELEASE=${DIB_RELEASE:-unstable}
export DIB_DISTRIBUTION_MIRROR=${DIB_DISTRIBUTION_MIRROR:-http://http.debian.net/debian}

View File

@ -0,0 +1,39 @@
#!/bin/bash
# Copyright (c) 2014 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
#
# See the License for the specific language governing permissions and
# limitations under the License.
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
# We should manage this in a betterer way
cat << EOF >/etc/apt/sources.list
deb $DIB_DISTRIBUTION_MIRROR $DIB_RELEASE main
EOF
if [ $DIB_RELEASE != unstable ] ; then
cat << EOF >>/etc/apt/sources.list
deb $DIB_DISTRIBUTION_MIRROR $DIB_RELEASE-backports main restricted universe
EOF
fi
# Need to update to retrieve the signed Release file
apt-get update
apt-get clean
apt-get dist-upgrade -y

View File

@ -11,7 +11,7 @@ mirrors, it is recommended that you choose a single mirror of debian,
and pass it in via `DIB_DISTRIBUTION_MIRROR`.
If necessary, a custom apt keyring and debootstrap script can be
supplied to the `debootstrap` command via `DIB_DEBIAN_KEYRING` and
supplied to the `debootstrap` command via `DIB_APT_KEYRING` and
`DIB_DEBIAN_DEBOOTSTRAP_SCRIPT` respectively. Both options require the
use of absolute rather than relative paths.
@ -26,4 +26,13 @@ tarball.
The `DIB_DEBOOTSTRAP_EXTRA_ARGS` environment variable may be used to
pass extra arguments to the debootstrap command used to create the
base filesystem image. If --keyring is is used in `DIB_DEBOOTSTRAP_EXTRA_ARGS`,
it will override `DIB_DEBIAN_KEYRING` if that is used as well.
it will override `DIB_APT_KEYRING` if that is used as well.
-------------------
Note on ARM systems
-------------------
Because there is not a one-to-one mapping of `ARCH` to a kernel package, if
you are building an image for ARM on debian, you need to specify which kernel
you want in the environment variable `DIB_ARM_KERNEL`. For instance, if you want
the `linux-image-mx5` package installed, set `DIB_ARM_KERNEL` to `mx5`.

View File

@ -1,2 +1 @@
dib-run-parts
dpkg
debian-minimal

View File

@ -1 +0,0 @@
export DISTRO_NAME=debian

View File

@ -0,0 +1,41 @@
#!/bin/bash
#
# Copyright 2014 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
DIB_DEBIAN_ALT_INIT_PACKAGE=${DIB_DEBIAN_ALT_INIT_PACKAGE:-sysvinit}
adduser --gecos Debian-cloud-init-user --disabled-password --quiet debian
install -d -m 0755 -o root -g root /etc/sudoers.d
echo 'debian ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/debian-cloud-init
chmod 0440 /etc/sudoers.d/debian-cloud-init
if [ "$DIB_DEBIAN_ALT_INIT_PACKAGE" != "sysvinit" ]; then
# To avoid a conflict against an essential package, we need to remove sysvinit first
dpkg --purge --force remove-essential sysvinit
apt-get install -y $apt_get_bp_extra_opts $DIB_DEBIAN_ALT_INIT_PACKAGE
printf \"Package: sysvinit\nPin: origin ""\nPin-Priority: -1\n\" > \
/etc/apt/preferences.d/sysvinit > /etc/apt/preferences.d/sysvinit
fi
# specify a hostname so that cloud-init does not default to (None)
echo 'hostname: debian' > /etc/cloud/cloud.cfg.d/01_hostname.cfg

View File

@ -0,0 +1,10 @@
openssh-server:
file:
less:
kbd:
curl:
rsync:
bash-completion:
cloud-init:
cloud-utils:
cloud-initramfs-growroot:

View File

@ -1,116 +0,0 @@
#!/bin/bash
#
# Copyright 2014 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
if [ -f ${TARGET_ROOT}/.extra_settings ] ; then
. ${TARGET_ROOT}/.extra_settings
fi
DISTRO_NAME=${DISTRO_NAME:-debian}
DIB_RELEASE=${DIB_RELEASE:-unstable}
DIB_DEBIAN_ALT_INIT_PACKAGE=${DIB_DEBIAN_ALT_INIT_PACKAGE:-sysvinit}
DIB_DEBOOTSTRAP_EXTRA_ARGS=${DIB_DEBOOTSTRAP_EXTRA_ARGS:-}
DEBOOTSTRAP_TARBALL=$DIB_IMAGE_CACHE/debootstrap-${DISTRO_NAME}-${DIB_DEBIAN_ALT_INIT_PACKAGE}-${DIB_RELEASE}-${ARCH}.tar.gz
if [ -n "${DIB_DEBIAN_MIRROR:-}" ]; then
echo "Use of DIB_DEBIAN_MIRROR environment variable to configure mirror is deprecated."
echo "Please use DIB_DISTRIBUTION_MIRROR instead."
DIB_DISTRIBUTION_MIRROR=$DIB_DEBIAN_MIRROR
fi
DIB_DISTRIBUTION_MIRROR=${DIB_DISTRIBUTION_MIRROR:-http://http.debian.net/debian}
http_proxy=${http_proxy:-}
if [ -n "$DIB_OFFLINE" -o -n "${DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE:-}" ] && [ -f $DEBOOTSTRAP_TARBALL ] ; then
echo $DEBOOTSTRAP_TARBALL found in cache. Using.
sudo tar -C $TARGET_ROOT --numeric-owner -xzf $DEBOOTSTRAP_TARBALL
else
echo Building new tarball for Debian $DIB_RELEASE ARCH=$ARCH
ADD_PACKAGES=sudo,adduser,locales,openssh-server,file,less,kbd,curl,rsync,bash-completion,linux-image-amd64
KEYRING_OPT=
if [ -n "${DIB_DEBIAN_KEYRING:-}" ] ; then
KEYRING_OPT="--keyring=${DIB_DEBIAN_KEYRING}"
fi
sudo sh -c "http_proxy=$http_proxy debootstrap --verbose \
--arch=${ARCH} \
--include=${ADD_PACKAGES} \
$KEYRING_OPT \
$DIB_DEBOOTSTRAP_EXTRA_ARGS \
$DIB_RELEASE \
$TARGET_ROOT \
$DIB_DISTRIBUTION_MIRROR \
${DIB_DEBIAN_DEBOOTSTRAP_SCRIPT:-}"
echo "Customizing result for cloud use"
apt_get_bp_extra_opts=
if [ "$DIB_RELEASE" = "wheezy" ]; then
sudo sh -c "echo deb $DIB_DISTRIBUTION_MIRROR wheezy-backports main >> ${TARGET_ROOT}/etc/apt/sources.list"
cat << EOF | sudo tee -a ${TARGET_ROOT}/etc/network/interfaces
source /etc/network/interfaces.d/*
EOF
apt_get_bp_extra_opts="-t wheezy-backports"
fi
# Need to update to retrieve the signed Release file
sudo sh -c "http_proxy=$http_proxy chroot ${TARGET_ROOT} apt-get update"
CLOUD_INIT_PACKAGES="cloud-init cloud-utils cloud-initramfs-growroot"
sudo sh -c "http_proxy=$http_proxy chroot ${TARGET_ROOT} apt-get install -y $apt_get_bp_extra_opts $CLOUD_INIT_PACKAGES"
if [ "$DIB_DEBIAN_ALT_INIT_PACKAGE" != "sysvinit" ]; then
# To avoid a conflict against an essential package, we need to remove sysvinit first
sudo chroot ${TARGET_ROOT} dpkg --purge --force remove-essential sysvinit
sudo sh -c "http_proxy=$http_proxy chroot ${TARGET_ROOT} apt-get install -y $apt_get_bp_extra_opts $DIB_DEBIAN_ALT_INIT_PACKAGE"
sudo sh -c "printf \"Package: sysvinit\nPin: origin ""\nPin-Priority: -1\n\" > \
${TARGET_ROOT}/etc/apt/preferences.d/sysvinit > ${TARGET_ROOT}/etc/apt/preferences.d/sysvinit"
fi
sudo sed -i "s/PermitRootLogin yes/PermitRootLogin without-password/" $TARGET_ROOT/etc/ssh/sshd_config
sudo chroot ${TARGET_ROOT} adduser --gecos Debian-cloud-init-user --disabled-password --quiet debian
sudo install -d -m 0755 -o root -g root ${TARGET_ROOT}/etc/sudoers.d
sudo sh -c "echo 'debian ALL=(ALL) NOPASSWD:ALL' > ${TARGET_ROOT}/etc/sudoers.d/debian-cloud-init"
sudo chmod 0440 ${TARGET_ROOT}/etc/sudoers.d/debian-cloud-init
sudo sh -c "echo \"proc /proc proc nodev,noexec,nosuid 0 0
LABEL=${DIB_ROOT_LABEL} / ext4 errors=remount-ro 0 1
\" > ${TARGET_ROOT}/etc/fstab"
sudo sh -c "echo 'blacklist pcspkr' > ${TARGET_ROOT}/etc/modprobe.d/blacklist.conf"
sudo sh -c "echo 'debian' > ${TARGET_ROOT}/etc/hostname"
# specify a hostname so that cloud-init does not default to (None)
sudo sh -c "echo 'hostname: debian' > ${TARGET_ROOT}/etc/cloud/cloud.cfg.d/01_hostname.cfg"
# cloud images expect eth0 to use dhcp.
sudo mkdir -p ${TARGET_ROOT}/etc/network/interfaces.d
cat << EOF | sudo tee ${TARGET_ROOT}/etc/network/interfaces.d/eth0
# The primary network interface
auto eth0
iface eth0 inet dhcp
EOF
# generate the default en_US locale
sudo sed -i -e "0,/^[# ]*en_US.UTF-8 UTF-8 *$/ s/^[# ]*en_US.UTF-8 UTF-8 *$/en_US.UTF-8 UTF-8/" ${TARGET_ROOT}/etc/locale.gen
sudo chroot ${TARGET_ROOT} dpkg-reconfigure -f noninteractive locales
echo Caching debootstrap result in $DEBOOTSTRAP_TARBALL
sudo tar -C $TARGET_ROOT -zcf $DEBOOTSTRAP_TARBALL --exclude='./tmp/*' .
fi
sudo rm -f ${TARGET_ROOT}/.extra_settings

View File

@ -0,0 +1,35 @@
===========
debootstrap
===========
Base element for creating minimal debian-based images.
This element is incomplete by itself, you'll want to use the debian-minimal
or ubuntu-minimal elements to get an actual base image.
If necessary, a custom apt keyring and debootstrap script can be
supplied to the `debootstrap` command via `DIB_APT_KEYRING` and
`DIB_DEBIAN_DEBOOTSTRAP_SCRIPT` respectively. Both options require the
use of absolute rather than relative paths.
Use of this element will also require the tool 'debootstrap' to be
available on your system. It should be available on Ubuntu, Debian,
and Fedora.
The `DIB_OFFLINE` or more specific `DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE`
variables can be set to prefer the use of a pre-cached root filesystem
tarball.
The `DIB_DEBOOTSTRAP_EXTRA_ARGS` environment variable may be used to
pass extra arguments to the debootstrap command used to create the
base filesystem image. If --keyring is is used in `DIB_DEBOOTSTRAP_EXTRA_ARGS`,
it will override `DIB_APT_KEYRING` if that is used as well.
-------------------
Note on ARM systems
-------------------
Because there is not a one-to-one mapping of `ARCH` to a kernel package, if
you are building an image for ARM on debian, you need to specify which kernel
you want in the environment variable `DIB_ARM_KERNEL`. For instance, if you want
the `linux-image-mx5` package installed, set `DIB_ARM_KERNEL` to `mx5`.

View File

@ -0,0 +1,2 @@
dib-run-parts
dpkg

View File

@ -0,0 +1,34 @@
#!/bin/bash
#
# Copyright 2015 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
# It would be eversogreat if we didn't need to do crap like this
echo $DISTRO_NAME > /etc/hostname
# cloud images expect eth0 and eth1 to use dhcp.
mkdir -p /etc/network/interfaces.d
for interface in eth0 eth1; do
cat << EOF | tee /etc/network/interfaces.d/$interface
auto $interface
iface $interface inet dhcp
EOF
done

View File

@ -0,0 +1,30 @@
#!/bin/bash
#
# Copyright 2015 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
install -d -m 0755 -o root -g root /etc/sudoers.d
echo 'blacklist pcspkr' > /etc/modprobe.d/blacklist.conf
cat << EOF | tee /etc/fstab > /dev/null
proc /proc proc nodev,noexec,nosuid 0 0
LABEL=${DIB_ROOT_LABEL} / ext4 errors=remount-ro 0 1
EOF

View File

@ -1,7 +1,6 @@
adduser:
locales:
ca-certificates:
cloud-initramfs-growroot:
linux-image-generic:
locales:
lsb-release:
phase: pre-install.d

View File

@ -26,10 +26,26 @@ fi
[ -n "$DISTRO_NAME" ]
[ -n "$DIB_RELEASE" ]
[ -n "$DIB_DISTRIBUTION_MIRROR" ]
if [ $DISTRO_NAME = 'ubuntu' ] ; then
KERNEL='generic'
else
case $ARCH in
amd64) KERNEL='amd64' ; break ;;
i386) KERNEL='686' ; break ;;
arm7)
[ -n "$DIB_ARM_KERNEL" ]
KERNEL="$DIB_ARM_KERNEL"
break
;;
esac
fi
KERNEL_PACKAGE="linux-image-$KERNEL"
DIB_DEBIAN_COMPONENTS=${DIB_DEBIAN_COMPONENTS:-main}
DIB_DEBOOTSTRAP_EXTRA_ARGS=${DIB_DEBOOTSTRAP_EXTRA_ARGS:-}
DEBOOTSTRAP_TARBALL=$DIB_IMAGE_CACHE/debootstrap-${DISTRO_NAME}-${DIB_RELEASE}-${ARCH}.tar.gz
DIB_DISTRIBUTION_MIRROR=${DIB_DISTRIBUTION_MIRROR:-http://archive.ubuntu.com/ubuntu}
DEBOOTSTRAP_TARBALL=$DIB_IMAGE_CACHE/debootstrap-${DISTRO_NAME}-${DIB_RELEASE}-${ARCH}-${KERNEL}.tar.gz
http_proxy=${http_proxy:-}
if [ -n "$DIB_OFFLINE" -o -n "${DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE:-}" ] && [ -f $DEBOOTSTRAP_TARBALL ] ; then
@ -38,16 +54,16 @@ if [ -n "$DIB_OFFLINE" -o -n "${DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE:-}" ] && [ -f $
else
KEYRING_OPT=
if [ -n "${DIB_DEBIAN_KEYRING:-}" ] ; then
KEYRING_OPT="--keyring=${DIB_DEBIAN_KEYRING}"
if [ -n "${DIB_APT_KEYRING:-${DIB_DEBIAN_KEYRING:-}}" ] ; then
KEYRING_OPT="--keyring=${DIB_APT_KEYRING:-${DIB_DEBIAN_KEYRING:-}}"
fi
# Have to --include=python because of dib-run-parts
# Have to --include=sudo for pre-install.d use of sudoers files
sudo sh -c "http_proxy=$http_proxy debootstrap --verbose \
--variant=minbase \
--include=python,sudo \
--components=main,restricted,universe \
--include=python,sudo,$KERNEL_PACKAGE \
--components=${DIB_DEBIAN_COMPONENTS} \
--arch=${ARCH} \
$KEYRING_OPT \
$DIB_DEBOOTSTRAP_EXTRA_ARGS \
@ -56,28 +72,6 @@ else
$DIB_DISTRIBUTION_MIRROR \
${DIB_DEBIAN_DEBOOTSTRAP_SCRIPT:-}"
echo "Customizing result for cloud use"
sudo install -d -m 0755 -o root -g root ${TARGET_ROOT}/etc/sudoers.d
cat << EOF | sudo tee ${TARGET_ROOT}/etc/fstab > /dev/null
proc /proc proc nodev,noexec,nosuid 0 0
LABEL=${DIB_ROOT_LABEL} / ext4 errors=remount-ro 0 1
EOF
sudo sh -c "echo 'blacklist pcspkr' > ${TARGET_ROOT}/etc/modprobe.d/blacklist.conf"
# It would be eversogreat if we didn't need to do crap like this
sudo sh -c "echo 'ubuntu' > ${TARGET_ROOT}/etc/hostname"
# cloud images expect eth0 and eth1 to use dhcp.
sudo mkdir -p ${TARGET_ROOT}/etc/network/interfaces.d
for interface in eth0 eth1; do
cat << EOF | sudo tee ${TARGET_ROOT}/etc/network/interfaces.d/$interface
auto $interface
iface $interface inet dhcp
EOF
done
echo Caching debootstrap result in $DEBOOTSTRAP_TARBALL
sudo tar --numeric-owner -C $TARGET_ROOT -zcf $DEBOOTSTRAP_TARBALL --exclude='./tmp/*' .
fi

View File

@ -10,7 +10,7 @@ Create a minimal image based on Ubuntu. We default to trusty but DIB_RELEASE
is mapped to any series of Ubuntu.
If necessary, a custom apt keyring and debootstrap script can be
supplied to the `debootstrap` command via `DIB_DEBIAN_KEYRING` and
supplied to the `debootstrap` command via `DIB_APT_KEYRING` and
`DIB_DEBIAN_DEBOOTSTRAP_SCRIPT` respectively. Both options require the
use of absolute rather than relative paths.
@ -25,4 +25,4 @@ tarball.
The `DIB_DEBOOTSTRAP_EXTRA_ARGS` environment variable may be used to
pass extra arguments to the debootstrap command used to create the
base filesystem image. If --keyring is is used in `DIB_DEBOOTSTRAP_EXTRA_ARGS`,
it will override `DIB_DEBIAN_KEYRING` if that is used as well.
it will override `DIB_APT_KEYRING` if that is used as well.

View File

@ -1,2 +1 @@
dib-run-parts
dpkg
debootstrap

View File

@ -1,2 +1,4 @@
export DISTRO_NAME=ubuntu
export DIB_RELEASE=${DIB_RELEASE:-trusty}
export DIB_DEBIAN_COMPONENTS=${DIB_DEBIAN_COMPONENTS:-main,restricted,universe}
export DIB_DISTRIBUTION_MIRROR=${DIB_DISTRIBUTION_MIRROR:-http://archive.ubuntu.com/ubuntu}