Re-use tinyipa image for ansbile-deploy

It is possible to rebuild the pre-built tinyipa image available
at tarballs.openstack.org to make it usable with ansible-deploy driver.

The rebuild is rather fast, and only downloads 2 packages from TC mirrors
(SSH server and dependencies).

Change-Id: Ie39ce67dc93e7d53bf75937c7defacafad5fbfcf
Related-Bug: #1526308
This commit is contained in:
Pavlo Shchelokovskyy 2016-10-03 21:37:52 +03:00 committed by Pavlo Shchelokovskyy
parent d0b8994d51
commit afb682bf54
7 changed files with 502 additions and 0 deletions

5
imagebuild/tinyipa-ansible/.gitignore vendored Normal file
View File

@ -0,0 +1,5 @@
build_files/cache
rebuild/
*.gz
*.initramfs
*.vmlinuz

View File

@ -0,0 +1,13 @@
.PHONY: all dependencies rebuild clean
all: dependencies rebuild
dependencies:
./install-deps.sh
rebuild:
./rebuild-tinyipa.sh
clean:
sudo -v
sudo rm -rf rebuild
rm -f *.initramfs
rm -f *.gz
rm -rf build_files/cache/*

View File

@ -0,0 +1,87 @@
###################################################
TinyIPA image compatible with Ansible-deploy driver
###################################################
It is possible to rebuild the pre-built tinyipa ramdisk available from
http://tarballs.openstack.org/ironic-python-agent/tinyipa
to make it usable with Ansible-deploy driver.
Rebuilding TinyIPA
==================
#. Run the provided ``rebuild-tinyipa.sh`` script,
set environment variables as explained in `Build options`_.
#. Running this script will create a rebuilt ramdisk as
``ansible-<original-tinyipa-ramdisk-name>``.
That file must be uploaded to Glance as ARI image.
* If tinyipa kernel is not in Glance yet, an appropriate version can be
downloaded from tarballs.openstack.org and
uploaded to Glance as AKI image.
#. Update nodes that use ``*_ansible`` driver:
* Assign ramdisk uploaded in the previous step as
``driver_info/deploy_ramdisk``.
* The kernel image created during TinyIPA build
(``tinyipa[-branch_name].vmlinuz``) should be used as
``driver_info/deploy_kernel`` if not set yet.
* Set ``tc`` as ``driver_info/ansible_deploy_user``.
+ If you have used a custom ``SSH_PUBLIC_KEY`` specify it as
``driver_info/ansible_deploy_key_file``
* Ensure that the private SSH key file has correct ``600`` or ``400``
exclusive permissions for the user running the ironic-conductor process.
#. You can also assign the ramdisk created to other nodes that use
``IPA``-based ramdisks as ``driver_info/deploy_ramdisk`` to have a
unified deploy image for all nodes.
It should work for them the same as original tinyipa ramdisk.
Build options
-------------
#. If rebuilding an existing tinyipa ramdisk file, set the
``TINYIPA_RAMDISK_FILE`` environment variable to absolute path to
this file before running this script::
export TINYIPA_RAMDISK_FILE=</full/path/to/tinyipa-ramdisk-file>
#. When not provided with existing file, this script will rebuild the
tinyipa master branch build.
To use a stable branch, set ``BRANCH_PATH`` environment variable
(``master`` by default) before running the rebuild script accordingly.
Branch names for stable releases must be in the form ``stable-<release>``,
for example::
export BRANCH_PATH=stable-newton
Consult https://tarballs.openstack.org/ironic-python-agent/tinyipa/files/
for currently available versions.
#. By default, the script will bake ``id_rsa`` or ``id_dsa`` public SSH keys
of the user running the build into the ramdisk as authorized_keys for
``tc`` user.
To provide a custom key, set absolute path to it as ``SSH_PUBLIC_KEY``
environment variable before running this script::
export SSH_PUBLIC_KEY=<path-to-public-ssh-key>
Using Makefile
--------------
For simplified configuration, a Makefile is provided to use ``make`` for
some standard operations.
make
will install required dependencies and run the ``rebuild-tinyipa`` script
without arguments, downloading and rebuilding the image available at
https://tarballs.openstack.org
All customizations through environment variables still apply.
make clean
will cleanup temporary files and images created during build

View File

@ -0,0 +1,104 @@
#!/bin/sh
S="Linux"
N="box"
R="4.2.9-tinycore64"
P="unknown"
V="#777 SMP (2016-02-29)"
M="x86_64"
I="unknown"
O="GNU/Linux"
OPT_A=false
OPT_S=false
OPT_N=false
OPT_R=false
OPT_P=false
OPT_V=false
OPT_M=false
OPT_I=false
OPT_O=false
if [ -z "$1" ]; then
echo "-ASNRPVMIO"
exit 1
fi
while :; do
case $1 in
-a)
OPT_A=true
shift
;;
-s)
OPT_S=true
shift
;;
-n)
OPT_N=true
shift
;;
-r)
OPT_R=true
shift
;;
-p)
OPT_P=true
shift
;;
-v)
OPT_V=true
shift
;;
-m)
OPT_M=true
shift
;;
-i)
OPT_I=true
shift
;;
-o)
OPT_O=true
shift
;;
*)
if [ ! -z "$1" ]; then
echo "uname -asnrpvmio"
exit 1
fi
break
;;
esac
done
if $OPT_A; then
echo "$S $N $R $V $M $O"
exit 0
fi
string=''
if $OPT_S; then
string="$string $S"
fi
if $OPT_N; then
string="$string $N"
fi
if $OPT_R; then
string="$string $R"
fi
if $OPT_P; then
string="$string $P"
fi
if $OPT_V; then
string="$string $V"
fi
if $OPT_M; then
string="$string $M"
fi
if $OPT_I; then
string="$string $I"
fi
if $OPT_O; then
string="$string $O"
fi
echo $string

View File

@ -0,0 +1,53 @@
#NOTE(pas-ha)
# The first URL is the official TC repo,
# the rest of the list is taken from
# http://wiki.tinycorelinux.net/wiki:mirrors
# as of time of this writing.
# Only HTTP mirrors were considered with the following ordering
# - those that were unavailable are moved to the bottom of the list
# - those that already responded with 404 are moved to the very bottom
# List generated on 12-Dec-2016
TC_MIRRORS="http://repo.tinycorelinux.net
http://distro.ibiblio.org/tinycorelinux
http://mirror.cedia.org.ec/tinycorelinux
http://mirror.epn.edu.ec/tinycorelinux
http://mirrors.163.com/tinycorelinux
http://kambing.ui.ac.id/tinycorelinux
http://ftp.nluug.nl/os/Linux/distr/tinycorelinux
http://ftp.vim.org/os/Linux/distr/tinycorelinux
http://www.gtlib.gatech.edu/pub/tinycore
http://tinycore.mirror.uber.com.au
http://l4u-00.jinr.ru/LinuxArchive/Ftp/tinycorelinux"
function probe_url {
wget -q --spider --tries 1 --timeout 10 "$1" 2>&1
}
function choose_tc_mirror {
if [ -z ${TINYCORE_MIRROR_URL} ]; then
for url in ${TC_MIRRORS}; do
echo "Checking Tiny Core Linux mirror ${url}"
if probe_url ${url} ; then
echo "Check succeeded: ${url} is responding."
TINYCORE_MIRROR_URL=${url}
break
else
echo "Check failed: ${url} is not responding"
fi
done
if [ -z ${TINYCORE_MIRROR_URL} ]; then
echo "Failed to find working Tiny Core Linux mirror"
exit 1
fi
else
echo "Probing provided Tiny Core Linux mirror ${TINYCORE_MIRROR_URL}"
if probe_url ${TINYCORE_MIRROR_URL} ; then
echo "Check succeeded: ${TINYCORE_MIRROR_URL} is responding."
else
echo "Check failed: ${TINYCORE_MIRROR_URL} is not responding"
exit 1
fi
fi
}

View File

@ -0,0 +1,17 @@
#!/bin/bash
PACKAGES="wget unzip sudo"
echo "Installing dependencies:"
if [ -x "/usr/bin/apt-get" ]; then
sudo -E apt-get update
sudo -E apt-get install -y $PACKAGES
elif [ -x "/usr/bin/dnf" ]; then
sudo -E dnf install -y $PACKAGES
elif [ -x "/usr/bin/yum" ]; then
sudo -E yum install -y $PACKAGES
else
echo "No supported package manager installed on system. Supported: apt, yum, dnf"
exit 1
fi

View File

@ -0,0 +1,223 @@
#!/bin/bash
# Rebuild upstream pre-built tinyipa it to be usable with ansible-deploy.
#
# Downloads the pre-built tinyipa ramdisk from tarballs.openstack.org or
# rebuilds a ramdisk under path provided as first script argument
# During rebuild this script installs and configures OpenSSH server and
# makes required changes for Ansible + Python to work in compiled/optimized
# Python environment.
#
# By default, id_rsa or id_dsa keys of the user performing the build
# are baked into the image as authorized_keys for 'tc' user.
# To supply different public ssh key, befor running this script set
# SSH_PUBLIC_KEY environment variable to point to absolute path to the key.
#
# This script produces "ansible-<tinyipa-ramdisk-name>" ramdisk that can serve
# as ramdisk for both ansible-deploy driver and agent-based Ironic drivers,
set -ex
WORKDIR=$(readlink -f $0 | xargs dirname)
SSH_PUBLIC_KEY=${SSH_PUBLIC_KEY:-}
source ${WORKDIR}/build_files/tc-mirror.sh
TINYCORE_MIRROR_URL=${TINYCORE_MIRROR_URL:-}
BRANCH_PATH=${BRANCH_PATH:-master}
TINYIPA_RAMDISK_FILE=${TINYIPA_RAMDISK_FILE:-}
TC=1001
STAFF=50
REBUILDDIR="$WORKDIR/rebuild"
CHROOT_PATH="/tmp/overides:/usr/local/sbin:/usr/local/bin:/apps/bin:/usr/sbin:/usr/bin:/sbin:/bin"
CHROOT_CMD="sudo chroot $REBUILDDIR /usr/bin/env -i PATH=$CHROOT_PATH http_proxy=$http_proxy https_proxy=$https_proxy no_proxy=$no_proxy"
TC_CHROOT_CMD="sudo chroot --userspec=$TC:$STAFF $REBUILDDIR /usr/bin/env -i PATH=$CHROOT_PATH http_proxy=$http_proxy https_proxy=$https_proxy no_proxy=$no_proxy"
function validate_params {
echo "Validating location of public SSH key"
if [ -n "$SSH_PUBLIC_KEY" ]; then
if [ -r "$SSH_PUBLIC_KEY" ]; then
_found_ssh_key="$SSH_PUBLIC_KEY"
fi
else
for fmt in rsa dsa; do
if [ -r "$HOME/.ssh/id_$fmt.pub" ]; then
_found_ssh_key="$HOME/.ssh/id_$fmt.pub"
break
fi
done
fi
if [ -z $_found_ssh_key ]; then
echo "Failed to find neither provided nor default SSH key"
exit 1
fi
choose_tc_mirror
}
function get_tinyipa {
if [ -z $TINYIPA_RAMDISK_FILE ]; then
mkdir -p $WORKDIR/build_files/cache
cd $WORKDIR/build_files/cache
wget -N https://tarballs.openstack.org/ironic-python-agent/tinyipa/files/tinyipa-${BRANCH_PATH}.gz
TINYIPA_RAMDISK_FILE="$WORKDIR/build_files/cache/tinyipa-${BRANCH_PATH}.gz"
fi
}
function unpack_ramdisk {
if [ -d "$REBUILDDIR" ]; then
sudo rm -rf "$REBUILDDIR"
fi
mkdir -p "$REBUILDDIR"
# Extract rootfs from .gz file
( cd "$REBUILDDIR" && zcat "$TINYIPA_RAMDISK_FILE" | sudo cpio -i -H newc -d )
}
function prepare_chroot {
sudo cp $REBUILDDIR/etc/resolv.conf $REBUILDDIR/etc/resolv.conf.old
sudo cp /etc/resolv.conf $REBUILDDIR/etc/resolv.conf
sudo cp -a $REBUILDDIR/opt/tcemirror $REBUILDDIR/opt/tcemirror.old
sudo sh -c "echo $TINYCORE_MIRROR_URL > $REBUILDDIR/opt/tcemirror"
mkdir -p $REBUILDDIR/tmp/builtin/optional
$CHROOT_CMD chown -R tc.staff /tmp/builtin
$CHROOT_CMD chmod -R a+w /tmp/builtin
$CHROOT_CMD ln -sf /tmp/builtin /etc/sysconfig/tcedir
echo "tc" | $CHROOT_CMD tee -a /etc/sysconfig/tcuser
$CHROOT_CMD mkdir -p /usr/local/tce.installed
$CHROOT_CMD chmod 777 /usr/local/tce.installed
mkdir -p $REBUILDDIR/tmp/overides
sudo cp -f $WORKDIR/build_files/fakeuname $REBUILDDIR/tmp/overides/uname
trap "sudo umount $REBUILDDIR/proc" EXIT
# Mount /proc for chroot commands
sudo mount --bind /proc "$REBUILDDIR/proc"
}
function clean_up_chroot {
# Unmount /proc and clean up everything
sudo umount $REBUILDDIR/proc
# all went well, remove the trap
trap - EXIT
sudo rm $REBUILDDIR/etc/sysconfig/tcuser
sudo rm $REBUILDDIR/etc/sysconfig/tcedir
sudo rm -rf $REBUILDDIR/usr/local/tce.installed
sudo rm -rf $REBUILDDIR/tmp/builtin
sudo rm -rf $REBUILDDIR/tmp/tcloop
sudo rm -rf $REBUILDDIR/tmp/overides
sudo mv $REBUILDDIR/opt/tcemirror.old $REBUILDDIR/opt/tcemirror
sudo mv $REBUILDDIR/etc/resolv.conf.old $REBUILDDIR/etc/resolv.conf
}
function install_ssh {
if [ ! -f "$REBUILDDIR/usr/local/etc/ssh/sshd_config" ]; then
# tinyipa was built without SSH server installed
# Install and configure bare minimum for SSH access
$TC_CHROOT_CMD tce-load -wic openssh
# Configure OpenSSH
$CHROOT_CMD cp /usr/local/etc/ssh/sshd_config.orig /usr/local/etc/ssh/sshd_config
echo "PasswordAuthentication no" | $CHROOT_CMD tee -a /usr/local/etc/ssh/sshd_config
# Generate and configure host keys - RSA, DSA, Ed25519
# NOTE(pas-ha) ECDSA host key will still be re-generated fresh on every image boot
$CHROOT_CMD ssh-keygen -q -t rsa -N "" -f /usr/local/etc/ssh/ssh_host_rsa_key
$CHROOT_CMD ssh-keygen -q -t dsa -N "" -f /usr/local/etc/ssh/ssh_host_dsa_key
$CHROOT_CMD ssh-keygen -q -t ed25519 -N "" -f /usr/local/etc/ssh/ssh_host_ed25519_key
echo "HostKey /usr/local/etc/ssh/ssh_host_rsa_key" | $CHROOT_CMD tee -a /usr/local/etc/ssh/sshd_config
echo "HostKey /usr/local/etc/ssh/ssh_host_dsa_key" | $CHROOT_CMD tee -a /usr/local/etc/ssh/sshd_config
echo "HostKey /usr/local/etc/ssh/ssh_host_ed25519_key" | $CHROOT_CMD tee -a /usr/local/etc/ssh/sshd_config
fi
# setup new user SSH keys anyway
$CHROOT_CMD mkdir -p /home/tc
$CHROOT_CMD chown -R tc.staff /home/tc
$TC_CHROOT_CMD mkdir -p /home/tc/.ssh
cat $_found_ssh_key | $TC_CHROOT_CMD tee /home/tc/.ssh/authorized_keys
$CHROOT_CMD chown tc.staff /home/tc/.ssh/authorized_keys
$TC_CHROOT_CMD chmod 600 /home/tc/.ssh/authorized_keys
}
function install_packages {
if [ -f "$WORKDIR/build_files/rebuildreqs.lst" ]; then
while read line; do
$TC_CHROOT_CMD tce-load -wic $line
done < $WORKDIR/build_files/rebuildreqs.lst
fi
}
function fix_python_optimize {
if grep -q "PYTHONOPTIMIZE=1" "$REBUILDDIR/opt/bootlocal.sh"; then
# tinyipa was built with optimized Python environment, apply fixes
echo "PYTHONOPTIMIZE=1" | $TC_CHROOT_CMD tee -a /home/tc/.ssh/environment
echo "PermitUserEnvironment yes" | $CHROOT_CMD tee -a /usr/local/etc/ssh/sshd_config
echo 'Defaults env_keep += "PYTHONOPTIMIZE"' | $CHROOT_CMD tee -a /etc/sudoers
fi
}
function make_symlinks {
set +x
echo "Symlink all from /usr/local/sbin to /usr/sbin"
cd "$REBUILDDIR/usr/local/sbin"
for target in *
do
if [ ! -f "$REBUILDDIR/usr/sbin/$target" ]
then
$CHROOT_CMD ln -s "/usr/local/sbin/$target" "/usr/sbin/$target"
fi
done
echo "Symlink all from /usr/local/bin to /usr/bin"
# this also includes symlinking Python to the place expected by Ansible
cd "$REBUILDDIR/usr/local/bin"
for target in *
do
if [ ! -f "$REBUILDDIR/usr/bin/$target" ]
then
$CHROOT_CMD ln -s "/usr/local/bin/$target" "/usr/bin/$target"
fi
done
set -x
}
function rebuild_ramdisk {
# Rebuild build directory into gz file
ansible_basename="ansible-$(basename $TINYIPA_RAMDISK_FILE)"
( cd "$REBUILDDIR" && sudo find | sudo cpio -o -H newc | gzip -9 > "$WORKDIR/${ansible_basename}" )
# Output file created by this script and its size
cd "$WORKDIR"
echo "Produced files:"
du -h "${ansible_basename}"
}
sudo -v
validate_params
get_tinyipa
unpack_ramdisk
prepare_chroot
# NOTE (pas-ha) default tinyipa is built without SSH access, enable it here
install_ssh
# NOTE (pas-ha) allow installing some extra pkgs by placing 'rebuildreqs.lst'
# file in the 'build_files' folder
install_packages
# NOTE(pas-ha) default tinyipa is built with PYOPTIMIZE_TINYIPA=true and
# for Ansible+python to work we need to ensure that PYTHONOPTIMIZE=1 is
# set for all sessions from 'tc' user including those that are escalated
# with 'sudo' afterwards
fix_python_optimize
# NOTE(pas-ha) Apparently on TinyCore Ansible's 'command' module is
# not searching for executables in the '/usr/local/(s)bin' paths.
# Thus we need to have everything from there symlinked to '/usr/(s)bin'
# which is being searched, so that 'command' module picks full utilities
# installed by 'util-linux' instead of built-in simplified BusyBox ones.
make_symlinks
clean_up_chroot
rebuild_ramdisk