openstack/trove
Code Issues Proposed changes

Improve trove guest agent image building

Browse Source 
- Add a new element 'guest-agent' for image building. This element is
  used when dev_mode=false, so that the trove code is downloaded into
  the image during the building phase rather than during the guest
  agent initialization.
- Improve trovestack sub-command 'build-image'.

  ./trovestack build-image ${datastore_type} \
     ${guest_os} \
     ${guest_release} \
     ${dev_mode}

- Improve documentation.

Story: #2005387
Task: #30375

Change-Id: I9d7acbd6a97f8c01b48b0f2cf94398d549d89124
This commit is contained in:
Lingxian Kong 2019-04-07 00:27:15 +13:00
parent fc1c246516
commit 798af4d02a
26 changed files with 675 additions and 687 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
devstack/plugin.sh
View File

@ -631,6 +631,8 @@ function _setup_minimal_image {
if [ -d ${SSH_DIR} ]; then
cat ${SSH_DIR}/id_rsa.pub >> ${SSH_DIR}/authorized_keys
sort ${SSH_DIR}/authorized_keys | uniq > ${SSH_DIR}/authorized_keys.uniq
mv ${SSH_DIR}/authorized_keys.uniq ${SSH_DIR}/authorized_keys
else
mkdir -p ${SSH_DIR}
/usr/bin/ssh-keygen -f ${SSH_DIR}/id_rsa -q -N ""
@ -638,6 +640,13 @@ function _setup_minimal_image {
chmod 600 ${SSH_DIR}/authorized_keys
fi
# Make sure the guest agent has permission to ssh into the devstack host
# in order to download trove code during the service initialization.
home_keys=$HOME/.ssh/authorized_keys
cat ${SSH_DIR}/id_rsa.pub >> ${home_keys}
sort ${home_keys} | uniq > ${home_keys}.uniq
mv ${home_keys}.uniq ${home_keys}
echo "Run disk image create to actually create a new image"
disk-image-create -a amd64 -o "${VM}" -x ${QEMU_IMG_OPTIONS} ${DISTRO} \
vm cloud-init-datasources ${DISTRO}-guest ${DISTRO}-${RELEASE}-guest \
@ -723,6 +732,10 @@ if is_service_enabled trove; then
# Start the trove API and trove taskmgr components
echo_summary "Starting Trove"
start_trove
# Guarantee the file permission in the trove code repo in order to
# download trove code from trove-guestagent.
sudo chown -R $STACK_USER:$STACK_USER "$DEST/trove"
elif [[ "$1" == "stack" && "$2" == "test-config" ]]; then
echo_summary "Configuring Tempest for Trove"
configure_tempest_for_trove

446
doc/source/admin/building_guest_images.rst
View File

@ -85,12 +85,12 @@ Injected Configuration for the Guest Agent
When TaskManager launches the guest VM it injects the specific settings
for the guest into the VM, into the file /etc/trove/conf.d/guest_info.conf.
The file is injected one of three ways. If use_heat=True, it is injected
during the heat launch process. If use_nova_server_config_drive=True
it is injected via ConfigDrive. Otherwise it is passed to the nova
create call as the 'files' parameter and will be injected based on
the configuration of Nova; the Nova default is to discard the files.
If the settings in guest_info.conf are not present on the guest
The file is injected one of three ways.
If use_nova_server_config_drive=True, it is injected via ConfigDrive. Otherwise
it is passed to the nova create call as the 'files' parameter and will be
injected based on the configuration of Nova; the Nova default is to discard the
files. If the settings in guest_info.conf are not present on the guest
Guest Agent will fail to start up.
------------------------------
@ -108,135 +108,23 @@ Building Guest Images using DIB
A Trove Guest Image can be built with any tool that produces an image
accepted by Nova. In this document we describe how to build guest
images using the 'Disk Image Builder' (DIB) tool, and we focus on
building qemu images [1]_. DIB is an OpenStack tool and is available for
download at
https://git.openstack.org/cgit/openstack/diskimage-builder/tree/ or
https://pypi.org/project/diskimage-builder/0.1.38.
images using the
`'Disk Image Builder' (DIB) <https://docs.openstack.org/diskimage-builder/latest/>`_
tool, and we focus on building qemu images.
DIB uses a chroot'ed environment to construct the image. The goal is
to build a bare machine that has all the components required for
launch by Nova.
----------
Invocation
----------
----------------------------
Build image using trovestack
----------------------------
You can download the DIB tool from OpenStack's public git
repository. Note that DIB works with Ubuntu and Fedora (RedHat). Other
operating systems are not yet fully supported.
.. code-block:: bash
user@machine:/opt/stack$ git clone https://git.openstack.org/openstack/diskimage-builder
Cloning into 'diskimage-builder'...
remote: Counting objects: 8881, done.
remote: Total 8881 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (8881/8881), 1.92 MiB | 0 bytes/s, done.
Resolving deltas: 100% (4668/4668), done.
Checking connectivity... done.
user@machine:/opt/stack$ cd diskimage-builder
user@machine:/opt/stack/diskimage-builder$ sudo pip install -r requirements.txt
user@machine:/opt/stack/diskimage-builder$ sudo python setup.py install
Ensure that you have qemu-img [2]_ and kpartx installed.
The disk-image-create command is the main command in the DIB tool that
is used to build guest images for Trove. The disk-image-create command
takes the following options:
.. code-block:: bash
user@machine:/opt/stack/diskimage-builder$ disk-image-create -h
Usage: disk-image-create [OPTION]... [ELEMENT]...
Options:
-a i386|amd64|armhf -- set the architecture of the image(default amd64)
-o imagename -- set the imagename of the output image file(default image)
-t qcow2,tar -- set the image types of the output image files (default qcow2)
File types should be comma separated
-x -- turn on tracing
-u -- uncompressed; do not compress the image - larger but faster
-c -- clear environment before starting work
--image-size size -- image size in GB for the created image
--image-cache directory -- location for cached images(default ~/.cache/image-create)
--max-online-resize size -- max number of filesystem blocks to support when resizing.
Useful if you want a really large root partition when the image is deployed.
Using a very large value may run into a known bug in resize2fs.
Setting the value to 274877906944 will get you a 1PB root file system.
Making this value unnecessarily large will consume extra disk space
on the root partition with extra file system inodes.
--min-tmpfs size -- minimum size in GB needed in tmpfs to build the image
--no-tmpfs -- do not use tmpfs to speed image build
--offline -- do not update cached resources
--qemu-img-options -- option flags to be passed directly to qemu-img.
Options need to be comma separated, and follow the key=value pattern.
--root-label label -- label for the root filesystem. Defaults to 'cloudimg-rootfs'.
--ramdisk-element -- specify the main element to be used for building ramdisks.
Defaults to 'ramdisk'. Should be set to 'dracut-ramdisk' for platforms such
as RHEL and CentOS that do not package busybox.
--install-type -- specify the default installation type. Defaults to 'source'. Set
to 'package' to use package based installations by default.
-n skip the default inclusion of the 'base' element
-p package[,package,package] -- list of packages to install in the image
-h|--help -- display this help and exit
ELEMENTS_PATH will allow you to specify multiple locations for the elements.
NOTE: At least one distribution root element must be specified.
Examples:
disk-image-create -a amd64 -o ubuntu-amd64 vm ubuntu
export ELEMENTS_PATH=~/source/tripleo-image-elements/elements
disk-image-create -a amd64 -o fedora-amd64-heat-cfntools vm fedora heat-cfntools
user@machine:/opt/stack/diskimage-builder$
The example command provided above would build a perfectly functional
Nova image with the 64 bit Fedora operating system.
In addition to the -a argument which specifies to build an amd64 (64
bit) image, and the -o which specifies the output file, the command
line lists the various elements that should be used in building the
image. The next section of this document talks about image elements.
Building a Trove guest image is a little more involved and the standard
elements (more about this later) are highly configurable through the use
of environment variables.
This command will create a guest image usable by Trove:
.. code-block:: bash
# assign a suitable value for each of these environment
# variables that change the way the elements behave.
export HOST_USERNAME
export HOST_SCP_USERNAME
export GUEST_USERNAME
export CONTROLLER_IP
export TROVESTACK_SCRIPTS
export SERVICE_TYPE
export PATH_TROVE
export ESCAPED_PATH_TROVE
export SSH_DIR
export GUEST_LOGDIR
export ESCAPED_GUEST_LOGDIR
export DIB_CLOUD_INIT_DATASOURCES="ConfigDrive"
export DATASTORE_PKG_LOCATION
export BRANCH_OVERRIDE
# you typically do not have to change these variables
export ELEMENTS_PATH=$TROVESTACK_SCRIPTS/files/elements
export ELEMENTS_PATH+=:$PATH_DISKIMAGEBUILDER/elements
export ELEMENTS_PATH+=:$PATH_TRIPLEO_ELEMENTS/elements
export DIB_APT_CONF_DIR=/etc/apt/apt.conf.d
export DIB_CLOUD_INIT_ETC_HOSTS=true
local QEMU_IMG_OPTIONS="--qemu-img-options compat=1.1"
# run disk-image-create that actually causes the image to be built
$disk-image-create -a amd64 -o "${VM}" \
-x ${QEMU_IMG_OPTIONS} ${DISTRO} ${EXTRA_ELEMENTS} vm \
cloud-init-datasources ${DISTRO}-guest ${DISTRO}-${SERVICE_TYPE}
Trove provides a script called ``trovestack`` that could do most of the
management and test tasks. Refer to the "Build guest agent image" section
in
`trovestack document <https://github.com/openstack/trove/blob/master/integration/README.md>`_
for how to build trove guest agent images.
-----------------------------
Disk Image Builder 'Elements'
@ -248,106 +136,16 @@ scripts that are executed by DIB in a specific order to generate the
image. You provide the names of the elements that you would like
executed, in order, on the command line to disk-image-create.
Elements are executed within the chroot'ed environment while DIB is
run. Elements are executed in phases and the various phases are (in
order) root.d, extra-data.d, pre-install.d, install.d, post-install.d,
block-device.d, finalise.d [3]_, and cleanup.d [4]_. The latter
reference provides a very good outline on writing elements and is a
'must read'.
DIB comes with some
`built-in elements <https://docs.openstack.org/diskimage-builder/latest/elements.html>`_.
In addition, projects like
`TripleO <https://github.com/openstack/tripleo-image-elements>`_ provide
elements as well.
Some elements use environment.d to setup environment
variables. Element dependencies can be established using the
element-deps and element-provides files which are plain text files.
-----------------
Existing Elements
-----------------
DIB comes with some tools that are located in the elements directory.
.. code-block:: bash
user@machine:/opt/stack/diskimage-builder/elements$ ls
apt-conf dpkg ramdisk
apt-preferences dracut-network ramdisk-base
apt-sources dracut-ramdisk rax-nova-agent
architecture-emulation-binaries element-manifest redhat-common
baremetal enable-serial-console rhel
base epel rhel7
cache-url fedora rhel-common
centos7 hwburnin rpm-distro
cleanup-kernel-initrd hwdiscovery select-boot-kernel-initrd
cloud-init-datasources ilo selinux-permissive
cloud-init-nocloud ironic-agent serial-console
debian ironic-discoverd-ramdisk source-repositories
debian-systemd iso stable-interface-names
debian-upstart local-config svc-map
deploy manifests uboot
deploy-baremetal mellanox ubuntu
deploy-ironic modprobe-blacklist ubuntu-core
deploy-kexec opensuse vm
dhcp-all-interfaces package-installs yum
dib-run-parts pip-cache zypper
disable-selinux pkg-map
dkms pypi
In addition, projects like TripleO [5]_ provide elements as well.
Trove provides a set of elements as part of the trove [6]_
project which will be described in the next section.
Trove Reference Elements
========================
Reference elements provided by Trove are part of the trove project.
In keeping with the philosophy of making elements 'layered', Trove
provides two sets of elements. The first implements the guest agent
for various operating systems and the second implements the database
for these operating systems.
---------------------------
Provided Reference Elements
---------------------------
The Trove reference elements are located in the
trove/integration/scripts/files/elements directory. The elements
[operating-system]-guest provide the Trove Guest capabilities and the
[operating-system]-[database] elements provide support for each
database on the specified database.
.. code-block:: bash
user@machine:/opt/stack/trove/integration/scripts/files/elements$ ls -l
total 56
drwxrwxr-x 5 user group 4096 Jan 7 12:47 fedora-guest
drwxrwxr-x 3 user group 4096 Jan 7 12:47 fedora-mongodb
drwxrwxr-x 3 user group 4096 Jan 7 12:47 fedora-mysql
drwxrwxr-x 3 user group 4096 Jan 7 12:47 fedora-percona
drwxrwxr-x 3 user group 4096 Jan 7 12:47 fedora-postgresql
drwxrwxr-x 3 user group 4096 Jan 7 12:47 fedora-redis
drwxrwxr-x 3 user group 4096 Jan 7 12:47 ubuntu-cassandra
drwxrwxr-x 3 user group 4096 Jan 7 12:47 ubuntu-couchbase
drwxrwxr-x 6 user group 4096 Jan 7 12:47 ubuntu-guest
drwxrwxr-x 3 user group 4096 Jan 7 12:47 ubuntu-mongodb
drwxrwxr-x 4 user group 4096 Jan 7 12:47 ubuntu-mysql
drwxrwxr-x 4 user group 4096 Jan 7 12:47 ubuntu-percona
drwxrwxr-x 3 user group 4096 Jan 7 12:47 ubuntu-postgresql
drwxrwxr-x 3 user group 4096 Jan 7 12:47 ubuntu-redis
user@machine:/opt/stack/trove/integration/scripts/files/elements$
With this infrastructure in place, and the elements from DIB and
TripleO accessible to the DIB command, one can generate the (for
example) Ubuntu guest image for Percona Server with the command line:
.. code-block:: bash
${DIB} -a amd64 -o ${output-file} Ubuntu vm \
cloud-init-datasources ubuntu-guest ubuntu-percona
Where ${DIB} is the fully qualified path to the disk-image-create
command and ${output-file} is the name of the output file to be
created.
Trove also provides a set of its own elements. In keeping with the philosophy
of making elements 'layered', Trove provides two sets of elements. The first
implements the guest agent for various operating systems and the second
implements the database for these operating systems.
-------------------------------------------------------------------
Contributing Reference Elements When Implementing a New 'Datastore'
@ -357,81 +155,6 @@ When contributing a new datastore, you should contribute elements
that will allow any user of Trove to be able to build a guest image
for that datastore.
This is typically accomplished by submitting files into the
trove project, as above.
Getting the Guest Agent Code onto a Trove Guest Instance
========================================================
The guest agent code typically runs on the guest instance alongside
the database. There are two ways in which the guest agent code can be
placed on the guest instance and we describe both of these here.
----------------------------------------
Guest Agent Code Installed at Build Time
----------------------------------------
In this option, the guest agent code is built into the guest image,
thereby ensuring that all database instances that are launched with
the image will have the exact same version of the guest image.
This can be accomplished by placing suitable code in the elements for
the image and these elements will ensure that the guest agent code is
installed on the image.
--------------------------------------
Guest Agent Code Installed at Run Time
--------------------------------------
In this option, the guest agent code is not part of the guest image
and instead the guest agent code is obtained at runtime, potentially
from some well known location.
In devstack, this is implemented in trove-guest.upstart.conf and
trove-guest.systemd.conf. Shown below is the code from
trove-guest.upstart.conf (this code may change in the future and
is shown here as an example only). See the code highlighted below:
.. code-block:: bash
description "Trove Guest"
author "Auto-Gen"
start on (filesystem and net-device-up IFACE!=lo)
stop on runlevel [016]
chdir /var/run
pre-start script
mkdir -p /var/run/trove
chown GUEST_USERNAME:root /var/run/trove/
mkdir -p /var/lock/trove
chown GUEST_USERNAME:root /var/lock/trove/
mkdir -p GUEST_LOGDIR
chown GUEST_USERNAME:root GUEST_LOGDIR
chmod +r /etc/guest_info
# If /etc/trove does not exist, copy the trove source and the
# guest agent config from the user's development environment
if [ ! -d /etc/trove ]; then
-> sudo -u GUEST_USERNAME rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -avz --exclude='.*' HOST_SCP_USERNAME@NETWORK_GATEWAY:PATH_TROVE/ /home/GUEST_USERNAME/trove
mkdir -p /etc/trove
-> sudo -u GUEST_USERNAME rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -avz --exclude='.*' HOST_SCP_USERNAME@NETWORK_GATEWAY:/etc/trove/trove-guestagent.conf ~GUEST_USERNAME/
mv ~GUEST_USERNAME/trove-guestagent.conf /etc/trove/trove-guestagent.conf
fi
end script
exec su -c "/home/GUEST_USERNAME/trove/contrib/trove-guestagent -config-file=/etc/guest_info --config-file=/etc/trove/trove-guestagent.conf" GUEST_USERNAME
In building an image for a production Trove deployment, it is a very
bad idea to use this mechanism. It makes sense in a development
environment where the thing that you are developing is in Trove and
part of the Guest Agent! This is because you get to merely boot a new
Trove instance and the freshly modified code gets run on the
Guest. But, in any other circumstance, it is much better to have the
guest image include the guest agent code.
Considerations in Building a Guest Image
========================================
@ -522,120 +245,3 @@ repository, this would equate to:
.. code-block:: bash
DATASTORE_PKG_LOCATION=/path/to/new_db.deb ./trovestack kick-start new_db
Building Guest Images Using Standard Elements
=============================================
A very good reference for how one builds guest images can be found by
reviewing the trovestack script (trove/integration/scripts). Lower level
routines that actually invoke Disk Image Builder can be found in
trove/integration/scripts/functions_qemu.
The following block of code illustrates the most basic invocation of
DIB to create a guest image. This code is in
trove/integration/scripts/functions_qemu as part of the function
build_vm(). We look at this section of code in detail below.
.. code-block:: bash
# assign a suitable value for each of these environment
# variables that change the way the elements behave.
export HOST_USERNAME
export HOST_SCP_USERNAME
export GUEST_USERNAME
export CONTROLLER_IP
export TROVESTACK_SCRIPTS
export SERVICE_TYPE
export PATH_TROVE
export ESCAPED_PATH_TROVE
export SSH_DIR
export GUEST_LOGDIR
export ESCAPED_GUEST_LOGDIR
export DIB_CLOUD_INIT_DATASOURCES="ConfigDrive"
export DATASTORE_PKG_LOCATION
export BRANCH_OVERRIDE
# you typically do not have to change these variables
export ELEMENTS_PATH=$TROVESTACK_SCRIPTS/files/elements
export ELEMENTS_PATH+=:$PATH_DISKIMAGEBUILDER/elements
export ELEMENTS_PATH+=:$PATH_TRIPLEO_ELEMENTS/elements
export DIB_APT_CONF_DIR=/etc/apt/apt.conf.d
export DIB_CLOUD_INIT_ETC_HOSTS=true
local QEMU_IMG_OPTIONS="--qemu-img-options compat=1.1"
# run disk-image-create that actually causes the image to be built
$disk-image-create -a amd64 -o "${VM}" \
-x ${QEMU_IMG_OPTIONS} ${DISTRO} ${EXTRA_ELEMENTS} vm \
cloud-init-datasources ${DISTRO}-guest ${DISTRO}-${SERVICE_TYPE}
Several of the environment variables referenced above are referenced
in the course of the Disk Image Building process.
For example, let's look at GUEST_LOGDIR. Looking at the element
elements/fedora-guest/extra-data.d/20-guest-systemd, we find:
.. code-block:: bash
#!/bin/bash
set -e
set -o xtrace
# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER
# PURPOSE: stages the bootstrap file and upstart conf file while replacing variables so that guest image is properly
# configured
source $_LIB/die
[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set"
[ -n "${GUEST_USERNAME}" ] || die "GUEST_USERNAME needs to be set to the user for the guest image"
[ -n "${HOST_SCP_USERNAME}" ] || die "HOST_SCP_USERNAME needs to be set to the user for the host instance"
[ -n "${CONTROLLER_IP}" ] || die "CONTROLLER_IP needs to be set to the ip address that guests will use to contact the controller"
[ -n "${ESCAPED_PATH_TROVE}" ] || die "ESCAPED_PATH_TROVE needs to be set to the path to the trove directory on the trovestack host"
[ -n "${TROVESTACK_SCRIPTS}" ] || die "TROVESTACK_SCRIPTS needs to be set to the trove/integration/scripts dir"
[ -n "${ESCAPED_GUEST_LOGDIR}" ] || die "ESCAPED_GUEST_LOGDIR must be set to the escaped guest log dir"
sed "s/GUEST_USERNAME/${GUEST_USERNAME}/g;s/GUEST_LOGDIR/${ESCAPED_GUEST_LOGDIR}/g;s/HOST_SCP_USERNAME/${HOST_SCP_USERNAME}/g;s/CONTROLLER_IP/${CONTROLLER_IP}/g;s/PATH_TROVE/${ESCAPED_PATH_TROVE}/g" \
${TROVESTACK_SCRIPTS}/files/trove-guest.systemd.conf >
${TMP_HOOKS_PATH}/trove-guest.service
As you can see, the value of GUEST_LOGDIR is used in the extra-data.d
script to appropriately configure the trove-guest.systemd.conf file.
This pattern is one that you can expect in your own building of guest
images. The invocation of disk-image-create provides a list of
elements that are to be invoked 'in order'.
That list of elements is:
.. code-block:: bash
${DISTRO}
${EXTRA_ELEMENTS}
vm
cloud-init-datasources
${DISTRO}-guest
${DISTRO}-${SERVICE_TYPE}
When invoked to (for example) create a MySQL guest image on Ubuntu, we
can expect that DISTRO would be 'Ubuntu' and SERVICE_TYPE would be
MySQL. And therefore these would end up being the elements:
.. code-block:: bash
ubuntu From diskimage-builder/elements/ubuntu
vm From diskimage-builder/elements/vm
cloud-init-datasources From diskimage-builder/elements/cloud-init-datasources
ubuntu-guest From trove/integration/scripts/files/elements/ubuntu-guest
ubuntu-mysql From trove/integration/scripts/files/elements/ubuntu-mysql
References
==========
.. [1] For more information about QEMU, refer to http://wiki.qemu.org/Main_Page
.. [2] On Ubuntu, qemu-img is part of the package qemu-utils, on Fedora and RedHat it is part of the qemu package.
.. [3] User (especially in the USA) are cautioned about this spelling which once resulted in several sleepless nights.
.. [4] https://git.openstack.org/cgit/openstack/diskimage-builder/tree/README.rst#writing-an-element
.. [5] https://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/elements
.. [6] https://git.openstack.org/cgit/openstack/trove/tree/integration/scripts/files/elements

184
integration/README.md
View File

@ -1,93 +1,92 @@
## Integration dev scripts, tests and docs for Trove.
# Trove integration script - trovestack
***
## Steps to setup environment
### Steps to setup this environment:
Install a fresh Ubuntu 16.04 (xenial) image. We suggest creating a development virtual machine using the image.
Install a fresh Ubuntu 16.04 (xenial) image ( _We suggest creating a development virtual machine using the image_ )
#### Login to the machine as root
#### Make sure we have git installed:
1. Login to the machine as root
1. Make sure we have git installed
```
# apt-get update
# apt-get install git-core -y
```
#### Add a user named ubuntu if you do not already have one:
1. Add a user named ubuntu if you do not already have one:
```
# adduser ubuntu
# visudo
```
add this line to the file below the root user
Add this line to the file below the root user
ubuntu ALL=(ALL:ALL) ALL
ubuntu ALL=(ALL:ALL) ALL
**OR use this if you dont want to type your password to sudo a command**
Or use this if you dont want to type your password to sudo a command:
ubuntu ALL=(ALL) NOPASSWD: ALL
ubuntu ALL=(ALL) NOPASSWD: ALL
if /dev/pts/0 does not have read/write for your user
if /dev/pts/0 does not have read/write for your user
# chmod 666 /dev/pts/0
# chmod 666 /dev/pts/0
*Note that this number can change and if you can not connect to the screen session then the /dev/pts/# needs modding like above.*
> Note that this number can change and if you can not connect to the screen session then the /dev/pts/# needs modding like above.
#### Login with ubuntu:
1. Login with ubuntu and download the Trove code.
```shell
# su ubuntu
$ mkdir -p /opt/stack
$ cd /opt/stack
```
*Note that it is important that you clone the repository
here. This is a change from the earlier trove-integration where
you could clone trove-integration anywhere you wanted (like HOME)
and trove would get cloned for you in the right place. Since
trovestack is now in the trove repository, if you wish to test
changes that you have made to trove, it is advisable for you to
have your trove repository in /opt/stack to avoid another trove
repository being cloned for you.
#### Clone this repo:
> Note that it is important that you clone the repository
here. This is a change from the earlier trove-integration where
you could clone trove-integration anywhere you wanted (like HOME)
and trove would get cloned for you in the right place. Since
trovestack is now in the trove repository, if you wish to test
changes that you have made to trove, it is advisable for you to
have your trove repository in /opt/stack to avoid another trove
repository being cloned for you.
1. Clone this repo and go into the scripts directory
```
$ git clone https://github.com/openstack/trove.git
#### Go into the scripts directory:
$ cd trove/integration/scripts/
```
#### Running trovestack:
*Run this to get the command list with a short description of each*
## Running trovestack
Run this to get the command list with a short description of each
$ ./trovestack
#### Install all the dependencies and then install trove via trovestack.
*This brings up trove (rd-api rd-tmgr) and initializes the trove database.*
### Install Trove
*This brings up trove services and initializes the trove database.*
$ ./trovestack install
***
#### Connecting to the screen session
### Connecting to the screen session
$ screen -x stack
*If that command fails with the error*
If that command fails with the error
Cannot open your terminal '/dev/pts/1'
*If that command fails with the error chmod the corresponding /dev/pts/#*
If that command fails with the error chmod the corresponding /dev/pts/#
$ chmod 660 /dev/pts/1
#### Navigate the log screens
### Navigate the log screens
To produce the list of screens that you can scroll through and select
ctrl+a then "
Num Name
An example of screen list:
```
..... (full list ommitted)
20 c-vol
21 h-eng
22 h-api
@ -96,6 +95,7 @@ Num Name
25 tr-api
26 tr-tmgr
27 tr-cond
```
Alternatively, to go directly to a specific screen window
@ -103,58 +103,47 @@ Alternatively, to go directly to a specific screen window
then enter a number (like 25) or name (like tr-api)
#### Detach from the screen session
### Detach from the screen session
Allows the services to continue running in the background
ctrl+a then d
***
#### Kick start the build/test-init/build-image commands
### Kick start the build/test-init/build-image commands
*Add mysql as a parameter to set build and add the mysql guest image. This will also populate /etc/trove/test.conf with appropriate values for running the integration tests.*
$ ./trovestack kick-start mysql
*Optional commands if you did not run kick-start*
#### Initialize the test configuration and set up test users (overwrites /etc/trove/test.conf)
### Initialize the test configuration and set up test users (overwrites /etc/trove/test.conf)
$ ./trovestack test-init
#### Build the image and add it to glance
### Build guest agent image
The trove guest agent image could be created using `trovestack` script
according to the following command:
$ ./trovestack build-image mysql
```shell
PATH_DEVSTACK_OUTPUT=/opt/stack \
./trovestack build-image \
${datastore_type} \
${guest_os} \
${guest_os_release} \
${dev_mode}
```
***
- If the script is running as a part of DevStack, the viriable
`PATH_DEVSTACK_OUTPUT` is set automatically.
- if `dev_mode=false`, the trove code for guest agent is injected into the
image at the building time. Now `dev_mode=false` is still in experimental
and not considered production ready yet.
- If `dev_mode=true`, the script assumes to be running on the host of
trove-taskmanager, otherwise, `CONTROLLER_IP` needs to be specified
explicitly.
### Reset your environment
For example, build a Mysql image for Ubuntu Xenial operating system:
#### Stop all the services running in the screens and refresh the environment:
$ killall -9 screen
$ screen -wipe
$ RECLONE=yes ./trovestack install
$ ./trovestack kick-start mysql
or
$ RECLONE=yes ./trovestack install
$ ./trovestack test-init
$ ./trovestack build-image mysql
***
### Recover after reboot
If the VM was restarted, then the process for bringing up Openstack and Trove is quite simple
$./trovestack start-deps
$./trovestack start
Use screen to ensure all modules have started without error
$screen -r stack
***
```shell
$ ./trovestack build-image mysql ubuntu xenial false
```
### Running Integration Tests
Check the values in /etc/trove/test.conf in case it has been re-initialized prior to running the tests. For example, from the previous mysql steps:
@ -167,21 +156,44 @@ should be:
"dbaas_datastore": "mysql",
"dbaas_datastore_version": "5.5",
Once Trove is running on DevStack, you can use the dev scripts to run the integration tests locally.
Once Trove is running on DevStack, you can run the integration tests locally.
$./trovestack int-tests
This will runs all of the blackbox tests by default. Use the --group option to run a different group:
This will runs all of the blackbox tests by default. Use the `--group` option to run a different group:
$./trovestack int-tests --group=simple_blackbox
You can also specify the TESTS_USE_INSTANCE_ID environment variable to have the integration tests use an existing instance for the tests rather than creating a new one.
You can also specify the `TESTS_USE_INSTANCE_ID` environment variable to have the integration tests use an existing instance for the tests rather than creating a new one.
$./TESTS_DO_NOT_DELETE_INSTANCE=True TESTS_USE_INSTANCE_ID=INSTANCE_UUID ./trovestack int-tests --group=simple_blackbox
***
## Reset your environment
### VMware Fusion 5 speed improvement
### Stop all the services running in the screens and refresh the environment
$ killall -9 screen
$ screen -wipe
$ RECLONE=yes ./trovestack install
$ ./trovestack kick-start mysql
or
$ RECLONE=yes ./trovestack install
$ ./trovestack test-init
$ ./trovestack build-image mysql
## Recover after reboot
If the VM was restarted, then the process for bringing up Openstack and Trove is quite simple
$./trovestack start-deps
$./trovestack start
Use screen to ensure all modules have started without error
$screen -r stack
## VMware Fusion 5 speed improvement
Running Ubuntu with KVM or Qemu can be extremely slow without certain optimizations. The following are some VMware settings that can improve performance and may also apply to other virtualization platforms.
1. Shutdown the Ubuntu VM.
@ -203,7 +215,7 @@ INFO: /dev/kvm exists
KVM acceleration can be used
```
### VMware Workstation performance improvements
## VMware Workstation performance improvements
In recent versions of VMWare, you can get much better performance if you enable the right virtualization options. For example, in VMWare Workstation (found in version 10.0.2), click on VM->Settings->Processor.

6
integration/scripts/files/elements/guest-agent/README.rst Normal file
View File

@ -0,0 +1,6 @@
Element to install an Trove guest agent.
Note: this requires a system base image modified to include OpenStack
repositories
the ubuntu-guest element could be removed.

5
integration/scripts/files/elements/guest-agent/element-deps Normal file
View File

@ -0,0 +1,5 @@
dib-init-system
package-installs
pkg-map
source-repositories
svc-map

34
integration/scripts/files/elements/guest-agent/environment.d/99-reliable-apt-key-importing.bash Normal file
View File

@ -0,0 +1,34 @@
# sometimes the primary key server is unavailable and we should try an
# alternate. see
# https://bugs.launchpad.net/percona-server/+bug/907789. Disable
# shell errexit so we can interrogate the exit code and take action
# based on the exit code. We will reenable it later.
#
# NOTE(zhaochao): we still have this problem from time to time, so it's
# better use more reliable keyservers and just retry on that(for now, 3
# tries should be fine).
# According to:
# [1] https://www.gnupg.org/faq/gnupg-faq.html#new_user_default_keyserver
# [2] https://sks-keyservers.net/overview-of-pools.php
# we'll just the primary suggested pool: pool.sks-keyservers.net.
function get_key_robust() {
KEY=$1
set +e
tries=1
while [ $tries -le 3 ]; do
if [ $tries -eq 3 ]; then
set -e
fi
echo "Importing the key, try: $tries"
apt-key adv --keyserver hkp://pool.sks-keyservers.net \
--recv-keys ${KEY} && break
tries=$((tries+1))
done
set -e
}
export -f get_key_robust

28
integration/scripts/files/elements/guest-agent/extra-data.d/11-ssh-key Executable file
View File

@ -0,0 +1,28 @@
#!/bin/bash
set -e
set -o xtrace
# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER
# PURPOSE: creates the SSH key on the host if it doesn't exist. Then this copies the keys over to a staging area where
# they will be duplicated in the guest VM.
# This process allows the host to log into the guest but more importantly the guest phones home to get the trove
# source
source $_LIB/die
HOST_USERNAME=${HOST_USERNAME:-"ubuntu"}
SSH_DIR=${SSH_DIR:-"/home/${HOST_USERNAME}/.ssh"}
[ -n "${TMP_HOOKS_PATH}" ] || die "Temp hook path not set"
# copy files over the "staging" area for the guest image (they'll later be put in the correct location by the guest user
# not these keys should not be overridden otherwise a) you won't be able to ssh in and b) the guest won't be able to
# rsync the files
if [ -f ${SSH_DIR}/id_rsa ]; then
dd if=${SSH_DIR}/authorized_keys of=${TMP_HOOKS_PATH}/ssh-authorized-keys
dd if=${SSH_DIR}/id_rsa of=${TMP_HOOKS_PATH}/ssh-id_rsa
dd if=${SSH_DIR}/id_rsa.pub of=${TMP_HOOKS_PATH}/ssh-id_rsa.pub
else
die "SSH Authorized Keys file must exist along with pub and private key"
fi

38
integration/scripts/files/elements/guest-agent/install.d/guest-agent-source-install/75-guest-agent-install Executable file
View File

@ -0,0 +1,38 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
SCRIPTDIR=$(dirname $0)
GUEST_VENV=/opt/guest-agent-venv
# Create a virtual environment to contain the guest agent
${DIB_PYTHON} -m virtualenv $GUEST_VENV
$GUEST_VENV/bin/pip install pip --upgrade
$GUEST_VENV/bin/pip install -U -c /opt/upper-constraints.txt /opt/guest-agent
# Link the trove-guestagent out to /usr/local/bin where the startup scripts look
ln -s $GUEST_VENV/bin/trove-guestagent /usr/local/bin/guest-agent || true
mkdir -p /var/lib/trove /etc/trove/certs /var/log/trove
install -D -g root -o root -m 0644 ${SCRIPTDIR}/guest-agent.logrotate /etc/logrotate.d/guest-agent
case "$DIB_INIT_SYSTEM" in
upstart)
install -D -g root -o root -m 0644 ${SCRIPTDIR}/guest-agent.conf /etc/init/guest-agent.conf
;;
systemd)
install -D -g root -o root -m 0644 ${SCRIPTDIR}/guest-agent.service /usr/lib/systemd/system/guest-agent.service
;;
sysv)
install -D -g root -o root -m 0644 ${SCRIPTDIR}/guest-agent.init /etc/init.d/guest-agent.init
;;
*)
echo "Unsupported init system"
exit 1
;;
esac

18
integration/scripts/files/elements/guest-agent/install.d/guest-agent-source-install/guest-agent.conf Normal file
View File

@ -0,0 +1,18 @@
description "Start up the Trove Guest Agent"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
respawn limit 2 2
exec guest-agent --config-file /etc/trove/guest-agent.conf
post-start script
PID=`status guest-agent | egrep -oi '([0-9]+)$' | head -n1`
echo $PID > /var/run/guest-agent.pid
end script
post-stop script
rm -f /var/run/guest-agent.pid
end script

67
integration/scripts/files/elements/guest-agent/install.d/guest-agent-source-install/guest-agent.init Normal file
View File

@ -0,0 +1,67 @@
### BEGIN INIT INFO
# Provides: guest-agent
# Required-Start: $remote_fs $syslog $network
# Required-Stop: $remote_fs $syslog $network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Runs the Trove Guest Agent processes
# Description: This script runs Trove Guest Agent processes.
# This script will start the Guest Agent services
# and kill them.
### END INIT INFO
# Using the lsb functions to perform the operations.
. /lib/lsb/init-functions
# Process name ( For display )
NAME=guest-agent
# Daemon name, where is the actual executable
DAEMON=/usr/local/bin/guest-agent
# pid file for the daemon
PIDFILE=/var/run/guest-agent.pid
# If the daemon is not there, then exit.
test -x $DAEMON || exit 5
case $1 in
start)
# Checked the PID file exists and check the actual status of process
if [ -e $PIDFILE ]; then
status_of_proc -p $PIDFILE $DAEMON "$NAME process" && status="0" || status="$?"
# If the status is SUCCESS then don't need to start again.
if [ $status = "0" ]; then
exit # Exit
fi
fi
# Start the daemon.
log_daemon_msg "Starting the process" "$NAME"
# Start the daemon with the help of start-stop-daemon
# Log the message appropriately
if start-stop-daemon --start -m --quiet --oknodo --pidfile $PIDFILE --startas $DAEMON -- --config-file /etc/octavia/guest-agent.conf ; then
log_end_msg 0
else
log_end_msg 1
fi
;;
stop)
# Stop the daemon.
if [ -e $PIDFILE ]; then
status_of_proc -p $PIDFILE $DAEMON "Stopping the $NAME process" && status="0" || status="$?"
if [ "$status" = 0 ]; then
start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
/bin/rm -rf $PIDFILE
fi
else
log_daemon_msg "$NAME process is not running"
log_end_msg 0
fi
;;
restart)
# Restart the daemon.
$0 stop && sleep 2 && $0 start
;;
*)
# For invalid arguments, print the usage message.
echo "Usage: $0 {start|stop|restart|reload|status}"
exit 2
;;
esac

14
integration/scripts/files/elements/guest-agent/install.d/guest-agent-source-install/guest-agent.logrotate Normal file
View File

@ -0,0 +1,14 @@
/var/log/guest-agent.log {
daily
rotate 10
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
# Signal name shall not have the SIG prefix in kill command
# http://pubs.opengroup.org/onlinepubs/9699919799/utilities/kill.html
kill -s USR1 $(cat /var/run/guest-agent.pid)
endscript
}

14
integration/scripts/files/elements/guest-agent/install.d/guest-agent-source-install/guest-agent.service Normal file
View File

@ -0,0 +1,14 @@
[Unit]
Description=OpenStack Trove Guest Agent
After=network.target syslog.service
Wants=syslog.service
[Service]
ExecStart=/usr/local/bin/guest-agent --config-dir=/etc/trove/conf.d
KillMode=mixed
Restart=always
ExecStartPost=/bin/sh -c "echo $MAINPID > /var/run/guest-agent.pid"
PIDFile=/var/run/guest-agent.pid
[Install]
WantedBy=multi-user.target

62
integration/scripts/files/elements/guest-agent/package-installs.yaml Normal file
View File

@ -0,0 +1,62 @@
guest-agent:
installtype: package
build-essential:
installtype: source
libffi-dev:
installtype: source
libssl-dev:
installtype: source
python-dev:
installtype: source
python-sqlalchemy:
python-lxml:
python-eventlet:
python-webob:
python-httplib2:
python-iso8601:
python-pexpect:
python-mysqldb:
python-migrate:
acl:
acpid:
apparmor:
apparmor-utils:
apt-transport-https:
at:
bash-completion:
cloud-guest-utils:
cloud-init:
cron:
curl:
dbus:
dkms:
dmeventd:
ethtool:
gawk:
ifenslave:
ifupdown:
iptables:
iputils-tracepath:
irqbalance:
isc-dhcp-client:
less:
logrotate:
lsof:
net-tools:
netbase:
netcat-openbsd:
open-vm-tools:
openssh-client:
openssh-server:
pollinate:
psmisc:
rsyslog:
screen:
socat:
tcpdump:
ubuntu-cloudimage-keyring:
ureadahead:
uuid-runtime:
vim-tiny:
vlan:

26
integration/scripts/files/elements/guest-agent/pkg-map Normal file
View File

@ -0,0 +1,26 @@
{
"family": {
"redhat": {
"guest-agent": "openstack-trove-guest-agent",
"netcat-openbsd": "nmap-ncat",
"netbase": "",
"cron": "",
"ifenslave": "",
"iputils-tracepath": "",
"cloud-guest-utils": "",
"apparmor": "",
"dmeventd": "",
"isc-dhcp-client": "",
"uuid-runtime": "",
"ubuntu-cloudimage-keyring": "",
"vim-tiny": "",
"ureadahead": "",
"apt-transport-https": "",
"pollinate": "",
"ifupdown": ""
}
},
"default": {
"guest-agent": "guest-agent"
}
}

11
integration/scripts/files/elements/guest-agent/post-install.d/11-enable-guest-agent-systemd Executable file
View File

@ -0,0 +1,11 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
if [ "$DIB_INIT_SYSTEM" == "systemd" ]; then
systemctl enable $(svc-map guest-agent)
fi

39
integration/scripts/files/elements/guest-agent/post-install.d/12-ssh-key Executable file
View File

@ -0,0 +1,39 @@
#!/bin/bash
# CONTEXT: GUEST during CONSTRUCTION as ROOT
# PURPOSE: take "staged" ssh keys (see extra-data.d/62-ssh-key) and put them in the GUEST_USERS home directory
set -e
set -o xtrace
SSH_DIR="/home/${GUEST_USERNAME}/.ssh"
TMP_HOOKS_DIR="/tmp/in_target.d"
if ! id -u ${GUEST_USERNAME} >/dev/null 2>&1; then
echo "Adding ${GUEST_USERNAME} user"
useradd -G sudo -m ${GUEST_USERNAME} -s /bin/bash
chown ${GUEST_USERNAME}:${GUEST_USERNAME} /home/${GUEST_USERNAME}
passwd ${GUEST_USERNAME} <<_EOF_
${GUEST_USERNAME}
${GUEST_USERNAME}
_EOF_
fi
if [ -f "${TMP_HOOKS_DIR}/ssh-authorized-keys" ]; then
if [ ! -d ${SSH_DIR} ]; then
# this method worked more reliable in vmware fusion over doing sudo -Hiu ${GUEST_USERNAME}
mkdir ${SSH_DIR}
chown ${GUEST_USERNAME}:${GUEST_USERNAME} ${SSH_DIR}
fi
sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/authorized_keys conv=notrunc if=${TMP_HOOKS_DIR}/ssh-authorized-keys
if [ ! -f "${SSH_DIR}/id_rsa" ]; then
sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/id_rsa if=${TMP_HOOKS_DIR}/ssh-id_rsa
# perms have to be right on this file for ssh to work
sudo -Hiu ${GUEST_USERNAME} chmod 600 ${SSH_DIR}/id_rsa
sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/id_rsa.pub if=${TMP_HOOKS_DIR}/ssh-id_rsa.pub
fi
else
echo "SSH Keys were not staged by host"
exit -1
fi

11
integration/scripts/files/elements/guest-agent/post-install.d/99-clean-apt Executable file
View File

@ -0,0 +1,11 @@
#!/bin/sh
# CONTEXT: GUEST during CONSTRUCTION as ROOT
# PURPOSE: Delete contents of apt cache on guest (saves image disk space)
set -e
set -o xtrace
apt-get clean

3
integration/scripts/files/elements/guest-agent/source-repository-guest-agent Normal file
View File

@ -0,0 +1,3 @@
# This is used for source-based builds
guest-agent git /opt/guest-agent https://git.openstack.org/openstack/trove
upper-constraints file /opt/upper-constraints.txt https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt

3
integration/scripts/files/elements/guest-agent/svc-map Normal file
View File

@ -0,0 +1,3 @@
guest-agent:
default: guest-agent
redhat: trove-guest-agent

11
integration/scripts/files/elements/no-resolvconf/README.rst Normal file
View File

@ -0,0 +1,11 @@
This element clears out /etc/resolv.conf and prevents dhclient from populating
it with data from DHCP. This means that DNS resolution will not work from the
amphora. This is OK because all outbound connections from the guest will
be based using raw IP addresses.
In addition we remove dns from the nsswitch.conf hosts setting.
This has the real benefit of speeding up host boot and configutation times.
This is especially helpful when running tempest tests in a devstack environment
where DNS resolution from the guest usually doesn't work anyway. This means
that the guest never waits for DNS timeouts to occur.

19
integration/scripts/files/elements/no-resolvconf/finalise.d/99-disable-resolv-conf Executable file
View File

@ -0,0 +1,19 @@
#!/bin/bash
echo "" > /etc/resolv.conf
echo "" > /etc/resolv.conf.ORIG
if [ -d /etc/dhcp/dhclient-enter-hooks.d ]; then
# Debian/Ubuntu
echo "#!/bin/sh
make_resolv_conf() { : ; }" > /etc/dhcp/dhclient-enter-hooks.d/noresolvconf
chmod +x /etc/dhcp/dhclient-enter-hooks.d/noresolvconf
rm -f /etc/dhcp/dhclient-enter-hooks.d/resolvconf
else
# RHEL/CentOS/Fedora
echo "#!/bin/sh
make_resolv_conf() { : ; }" > /etc/dhclient-enter-hooks
chmod +x /etc/dhclient-enter-hooks
fi
if [ -e /etc/nsswitch.conf ]; then
sed -i -e "/hosts:/ s/dns//g" /etc/nsswitch.conf
fi

2
integration/scripts/files/elements/ubuntu-guest/extra-data.d/15-trove-dep
View File

@ -10,7 +10,7 @@ source $_LIB/die
BRANCH_OVERRIDE=${BRANCH_OVERRIDE:-default}
ADD_BRANCH=$(basename ${BRANCH_OVERRIDE})
REQUIREMENTS_FILE=${TROVESTACK_SCRIPTS}/files/requirements/ubuntu-requirements.txt
REQUIREMENTS_FILE=${TROVESTACK_SCRIPTS}/../../requirements.txt
[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set"
[ -e ${REQUIREMENTS_FILE} ] || die "Requirements not found"

2
integration/scripts/files/elements/ubuntu-guest/install.d/62-ssh-key
View File

@ -1,7 +1,7 @@
#!/bin/bash
# CONTEXT: GUEST during CONSTRUCTION as ROOT
# PURPOSE: take "staged" ssh keys (see extra-data.d/62-ssh-key) and put them in the GUEST_USERS home directory
# In future, this should be removed and use Nova keypair to inject ssh keys.
set -e
set -o xtrace

4
integration/scripts/files/elements/ubuntu-mysql/pre-install.d/10-percona-apt-key
View File

@ -6,8 +6,8 @@
set -e
set -o xtrace
[ -n "${GUEST_USERNAME}" ] || die "GUEST_USERNAME needs to be set to the user for the guest image"
[ -n "${RELEASE}" ] || die "RELEASE must be set to a valid Ubuntu release (e.g. trusty)"
GUEST_USERNAME=${GUEST_USERNAME:-"ubuntu"}
RELEASE=${DIB_RELEASE:-"xenial"}
# Add Percona GPG key
mkdir -p /home/${GUEST_USERNAME}/.gnupg

246
integration/scripts/functions_qemu
View File

@ -3,147 +3,118 @@
# Additional functions that would mostly just pertain to a Ubuntu + Qemu setup
#
# tmpfs dedicates half the available RAM to an internal cache to speed up
# image building. This can lead to failures when the data doesn't fit into
# the cache, for example when compiling large programs. This flag allows
# tmpfs to be turned off by the caller by setting USE_TMPF=false
USE_TMPFS=${USE_TMPFS:-true}
# Use the latest distro provided hardware enablement stack which is required
# by some platforms (ppc64el) for nested virtualization. Trovestack runs a
# VM (db guest image) in a VM (OpenStack+Trove)
DIB_USE_HWE_KERNEL=${DIB_USE_HWE_KERNEL:-true}
function build_vm() {
exclaim "Actually building the image, this can take up to 15 minutes"
# set variables here and ensure they are not changed during the duration of this script
readonly HOMEDIR=$1
readonly HOST_USERNAME=$2
GUEST_USERNAME=${GUEST_USERNAME:-$2}
HOST_SCP_USERNAME=${HOST_SCP_USERNAME:-$2}
VM=$3
DISTRO=$4
SERVICE_TYPE=$5
datastore_type=$1
guest_os=$2
guest_release=$3
dev_mode=$4
guest_username=$5
image_output=$6
readonly SSH_DIR=${KEY_DIR:-${HOMEDIR}/.ssh}
elementes="base vm"
trove_elements_path=${PATH_TROVE}/integration/scripts/files/elements
GUEST_IMAGETYPE=${GUEST_IMAGETYPE:-"qcow2"}
GUEST_IMAGESIZE=${GUEST_IMAGESIZE:-3}
GUEST_CACHEDIR=${GUEST_CACHEDIR:-"$HOME/.cache/image-create"}
GUEST_WORKING_DIR=${GUEST_WORKING_DIR:-"$HOME/images"}
export SSH_DIR=${SSH_DIR:-"$HOME/.ssh"}
export GUEST_USERNAME=${guest_username}
export HOST_SCP_USERNAME=$(whoami)
manage_ssh_keys
ARCH=amd64
if [ $DISTRO == 'ubuntu' ]; then
export DIB_RELEASE=$RELEASE
export DIB_CLOUD_IMAGES=cloud-images.ubuntu.com/$DIB_RELEASE/current
export DIB_USE_HWE_KERNEL
ARCH=$(dpkg --print-architecture)
elif [ $DISTRO == 'fedora' ]; then
EXTRA_ELEMENTS=selinux-permissive
# In dev mode, the trove guest agent needs to download trove code from
# trove-taskmanager host.
if [[ "${dev_mode}" == "true" ]]; then
host_ip=$(ip route get 8.8.8.8 | head -1 | awk '{print $7}')
export CONTROLLER_IP=${CONTROLLER_IP:-${host_ip}}
export PATH_TROVE=${PATH_TROVE}
export ESCAPED_PATH_TROVE=$(echo ${PATH_TROVE} | sed 's/\//\\\//g')
export GUEST_LOGDIR=${GUEST_LOGDIR:-"/var/log/trove/"}
export ESCAPED_GUEST_LOGDIR=$(echo ${GUEST_LOGDIR} | sed 's/\//\\\//g')
export TROVESTACK_SCRIPTS=${TROVESTACK_SCRIPTS}
export HOST_USERNAME=${HOST_SCP_USERNAME}
fi
case "$USE_TMPFS" in
false|FALSE|False|no|NO|n|N)
TMPFS_ARGS='--no-tmpfs'
;;
*)
TMPFS_ARGS=''
;;
esac
# For system-wide installs, DIB will automatically find the elements, so we only check local path
if [ "${DIB_LOCAL_ELEMENTS_PATH}" ]; then
export ELEMENTS_PATH=${trove_elements_path}:${DIB_LOCAL_ELEMENTS_PATH}
else
export ELEMENTS_PATH=${trove_elements_path}
fi
export HOST_USERNAME
export HOST_SCP_USERNAME
export GUEST_USERNAME
export CONTROLLER_IP
export TROVESTACK_SCRIPTS
export SERVICE_TYPE
export PATH_TROVE
export ESCAPED_PATH_TROVE
export SSH_DIR
export GUEST_LOGDIR
export ESCAPED_GUEST_LOGDIR
export ELEMENTS_PATH=$TROVESTACK_SCRIPTS/files/elements
export ELEMENTS_PATH+=:$PATH_DISKIMAGEBUILDER_ELEMENTS
export ELEMENTS_PATH+=:$PATH_TRIPLEO_ELEMENTS/elements
export DIB_RELEASE=${guest_release}
export DIB_CLOUD_INIT_DATASOURCES="ConfigDrive"
export DATASTORE_PKG_LOCATION
export BRANCH_OVERRIDE
export DIB_APT_CONF_DIR=/etc/apt/apt.conf.d
export DIB_CLOUD_INIT_ETC_HOSTS=true
local QEMU_IMG_OPTIONS="--qemu-img-options compat=1.1"
disk-image-create ${TMPFS_ARGS} -a ${ARCH} -o "${VM}" \
-x ${QEMU_IMG_OPTIONS} ${DISTRO} ${EXTRA_ELEMENTS} vm \
cloud-init-datasources ${DISTRO}-${RELEASE}-guest ${DISTRO}-${RELEASE}-${SERVICE_TYPE}
if [ "${GUEST_WORKING_DIR}" ]; then
mkdir -p ${GUEST_WORKING_DIR}
TEMP=$(mktemp -d ${GUEST_WORKING_DIR}/diskimage-create.XXXXXX)
else
TEMP=$(mktemp -d diskimage-create.XXXXXX)
fi
pushd $TEMP > /dev/null
elementes="$elementes ${guest_os}"
if [[ "${dev_mode}" != "true" ]]; then
elementes="$elementes pip-and-virtualenv"
elementes="$elementes pip-cache"
elementes="$elementes no-resolvconf"
elementes="$elementes guest-agent"
else
elementes="$elementes ${guest_os}-guest"
elementes="$elementes ${guest_os}-${guest_release}-guest"
fi
elementes="$elementes ${guest_os}-${datastore_type}"
elementes="$elementes ${guest_os}-${guest_release}-${datastore_type}"
# Build the image
disk-image-create -x \
-a amd64 \
-o ${image_output} \
-t ${GUEST_IMAGETYPE} \
--image-size ${GUEST_IMAGESIZE} \
--image-cache ${GUEST_CACHEDIR} \
$elementes
# out of $TEMP
popd > /dev/null
rm -rf $TEMP
exclaim "Image ${image_output}.${GUEST_IMAGETYPE} was built successfully."
}
function build_guest_image() {
exclaim "Building an image for use with development and integration tests."
if [ -z "$1" ]
exclaim "Building an image for Trove guest agent."
datastore_type=$1
guest_os=${2:-"ubuntu"}
guest_release=${3:-"xenial"}
dev_mode=${4:-"true"}
guest_username=${5:-"ubuntu"}
if [ -z "$datastore_type" ]
then
echo "You must pass an image type to build, like mysql"
exit 1
fi
SERVICE_TYPE=$1
VALID_SERVICES='mysql percona mariadb redis cassandra couchbase mongodb postgresql couchdb vertica db2 pxc'
if ! [[ " $VALID_SERVICES " =~ " $SERVICE_TYPE " ]]; then
if ! [[ " $VALID_SERVICES " =~ " $datastore_type " ]]; then
exclaim "You did not pass in a valid image type. Valid types are:" $VALID_SERVICES
exit 1
fi
GUEST_LOGDIR=$(iniget $PATH_TROVE/etc/trove/trove-guestagent.conf.sample DEFAULT log_dir)
GUEST_LOGFILE=$(iniget $PATH_TROVE/etc/trove/trove-guestagent.conf.sample DEFAULT log_file)
image_name=${guest_os}_${datastore_type}
image_folder=$HOME/images
mkdir -p $image_folder
image_path=${image_folder}/${image_name}
if [ -z $GUEST_LOGDIR ] || [ -z $GUEST_LOGFILE ]
then
exclaim "error: log_dir and log_file are required in: " $PATH_TROVE/etc/trove/trove-guestagent.conf.sample
exit 1
fi
# Always rebuild the image.
rm -rf ${image_folder}/*
ESCAPED_GUEST_LOGDIR=`echo $GUEST_LOGDIR | sed 's/\//\\\\\//g'`
USERNAME=`whoami`
# To change the distro, edit the trovestack.rc file
readonly IMAGENAME=${DISTRO}_${SERVICE_TYPE}
readonly VM_PATH=$USERHOME/images/${IMAGENAME}
readonly VM_PATH_NAME=${VM_PATH}/${IMAGENAME}
mkdir -p $VM_PATH
# If the path does not exist, build it, otherwise just upload it
# (unless we're explicitly told to rebuild it)
REBUILD_IMAGE=$(echo "${REBUILD_IMAGE}" | tr '[:upper:]' '[:lower:]')
if [ "${REBUILD_IMAGE}" = "true" ] || [ ! -d $VM_PATH ] || [ `ls -1 $VM_PATH | wc -l` -eq '0' ]
then
if [ "${REBUILD_IMAGE}" = "true" ]
then
exclaim "Rebuilding image"
rm -rf "${VM_PATH}"
fi
build_vm $USERHOME $USERNAME $VM_PATH_NAME $DISTRO $SERVICE_TYPE
touch -c "${VM_PATH}"
else
exclaim "Found image in $VM_PATH - using the qcow2 image found here..."
ELEMENTS_DIR="files/elements/${DISTRO}-${SERVICE_TYPE}"
ELEMENTS_DIR_GUEST="files/elements/${DISTRO}-guest"
# Print out a warning on all the elements files that are newer than the image.
# Directories are not excluded as that is the only way to determine if a file
# has been removed.
# The rebuild is not automatically triggered as there are valid reasons for a
# new file to be present (rollback of change, inadvertent .log files present,
# feature half implemented, etc.).
IMAGE_OLD=
while IFS= read -r -d '' ELEMENT_FILE
do
if [ "${ELEMENT_FILE}" -nt "${VM_PATH}" ]
then
IMAGE_OLD=true
exclaim "${COLOR_RED}WARNING: Element file '${ELEMENT_FILE}' is newer than cached image${COLOR_NONE}"
fi
done < <(find "${ELEMENTS_DIR}" "${ELEMENTS_DIR_GUEST}" -depth -print0)
if [ "${IMAGE_OLD}" = "true" ]
then
exclaim "${COLOR_RED}Use ${COLOR_NONE}REBUILD_IMAGE=True${COLOR_RED} to rebuild image${COLOR_NONE}"
fi
fi
build_vm ${datastore_type} ${guest_os} ${guest_release} ${dev_mode} ${guest_username} ${image_path}
}
function clean_instances() {
@ -151,42 +122,23 @@ function clean_instances() {
for i in $LIST; do sudo virsh destroy $i; done
}
# Trove doesn't support to specify keypair when creating the db instance, the
# ssh keys are injected when the image is built. This could be removed when
# we support keypair in the future.
function manage_ssh_keys() {
if [ -e ${SSH_DIR} ]; then
if [ -d ${SSH_DIR} ]; then
echo "${SSH_DIR} already exists"
else
echo "Creating ${SSH_DIR} for ${HOST_USERNAME}"
sudo -Hiu ${HOST_USERNAME} mkdir -m go-w -p ${SSH_DIR}
echo "Creating ${SSH_DIR} for ${HOST_SCP_USERNAME}"
sudo -Hiu ${HOST_SCP_USERNAME} mkdir -m go-w -p ${SSH_DIR}
fi
if [ ! -f ${SSH_DIR}/id_rsa.pub ]; then
sudo ${PKG_MGR} ${PKG_GET_ARGS} install expect
generate_empty_passphrase_ssh_key ${HOST_USERNAME}
/usr/bin/ssh-keygen -f ${SSH_DIR}/id_rsa -q -N ""
fi
add_host_key_to_authorizedkeys
}
function generate_empty_passphrase_ssh_key () {
echo "generating a empty passphrase DEV ONLY rsa key"
expect -c "
spawn sudo -Hiu ${HOST_USERNAME} /usr/bin/ssh-keygen -f ${SSH_DIR}/id_rsa -q
expect \"empty for no passphrase\"
send \n
expect assphrase
send \n
expect eof"
}
function add_host_key_to_authorizedkeys () {
# test to see if the host key is already in its own authorized_keys file - if not then add it. This is then later copied
# to the guest image
is_in_keyfile=`cat ${SSH_DIR}/id_rsa.pub | grep -f - ${SSH_DIR}/authorized_keys | wc -l`
if [ $is_in_keyfile == 0 ]; then
echo "Adding keyfile to authorized_keys, it does not yet exist"
cat ${SSH_DIR}/id_rsa.pub >> ${SSH_DIR}/authorized_keys
chmod 600 ${SSH_DIR}/authorized_keys
else
echo "Keyfile already exists in authorized_keys - skipping"
fi
cat ${SSH_DIR}/id_rsa.pub >> ${SSH_DIR}/authorized_keys
sort ${SSH_DIR}/authorized_keys | uniq > ${SSH_DIR}/authorized_keys.uniq
mv ${SSH_DIR}/authorized_keys.uniq ${SSH_DIR}/authorized_keys
chmod 600 ${SSH_DIR}/authorized_keys
}

56
integration/scripts/trovestack
View File

@ -97,10 +97,6 @@ KEYSTONE_AUTH_PORT=${KEYSTONE_AUTH_PORT:-35357}
GLANCE_HOSTPORT=${GLANCE_HOSTPORT:-$SERVICE_HOST:9292}
GLANCE_SERVICE_PROTOCOL=${GLANCE_SERVICE_PROTOCOL:-http}
# The following depends on whether neutron is used or nova-network
# neutron uses a bridge, nova-network does not
[[ $ENABLE_NEUTRON = true ]] && CONTROLLER_IP=$BRIDGE_IP || CONTROLLER_IP=$NETWORK_GATEWAY
# PATH_TROVE more than likely has file separators, which sed does not like
# This will escape them
ESCAPED_PATH_TROVE=$(echo $PATH_TROVE | sed 's/\//\\\//g')
@ -828,51 +824,52 @@ function cmd_test_init() {
function cmd_build_image() {
local IMAGE_DATASTORE_TYPE=${1:-'mysql'}
local ESCAPED_PATH_TROVE=${2:-'\/opt\/stack\/trove'}
local HOST_SCP_USERNAME=${3:-'ubuntu'}
local GUEST_USERNAME=${4:-'ubuntu'}
local IMAGE_GUEST_OS=${2:-'ubuntu'}
local IMAGE_GUEST_RELEASE=${3:-'xenial'}
local DEV_MODE=${4:-'true'}
exclaim "Ensuring we have all packages needed to build image."
sudo $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS update
sudo $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS install qemu
sudo -H $HTTP_PROXY pip install --upgrade pip dib-utils
pkg_install python-yaml
sudo $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS install qemu git kpartx debootstrap
sudo -H $HTTP_PROXY pip install diskimage-builder
install_devstack_code
cmd_clone_projects do_not_force_update $TROVESTACK_SCRIPTS/image-projects-list
exclaim "Use tripleo-diskimagebuilder to actually build the Trove Guest Agent Image."
build_guest_image $IMAGE_DATASTORE_TYPE
exclaim "Use diskimage-builder to actually build the Trove Guest Agent Image."
build_guest_image $IMAGE_DATASTORE_TYPE $IMAGE_GUEST_OS $IMAGE_GUEST_RELEASE $DEV_MODE
}
function cmd_build_and_upload_image() {
local DATASTORE_TYPE=$1
local RESTART_TROVE=${2:-$(get_bool RESTART_TROVE "true")}
local datastore_type=$1
local restart_trove=${2:-$(get_bool RESTART_TROVE "true")}
local guest_os=${3:-"ubuntu"}
local guest_release=${4:-"xenial"}
local dev_mode=${5:-"true"}
local guest_username=${6:-"ubuntu"}
if [ -z "${DATASTORE_TYPE}" ]; then
if [ -z "${datastore_type}" ]; then
exclaim "${COLOR_RED}Datastore argument was not specified.${COLOR_NONE}"
exit 1
fi
glance_imageid=$(openstack $CLOUD_ADMIN_ARG image list | grep "$datastore_type" | get_field 1)
echo "IMAGEID: $glance_imageid"
if [[ -z $glance_imageid ]]; then
build_guest_image ${datastore_type} ${guest_os} ${guest_release} ${dev_mode} ${guest_username}
GLANCE_IMAGEID=$(openstack $CLOUD_ADMIN_ARG image list | grep "$DATASTORE_TYPE" | get_field 1)
echo "IMAGEID: $GLANCE_IMAGEID"
if [[ -z $GLANCE_IMAGEID ]]; then
build_guest_image "${DATASTORE_TYPE}"
QCOW_IMAGE=`find $VM_PATH -name '*.qcow2'`
IMAGE_URL="file://$QCOW_IMAGE"
GLANCE_IMAGEID=`get_glance_id upload_image $IMAGE_URL`
[[ -z "$GLANCE_IMAGEID" ]] && echo "Glance upload failed!" && exit 1
echo "IMAGE ID: $GLANCE_IMAGEID"
image_folder=$HOME/images
qcow_image=`find $image_folder -name '*.qcow2'`
image_url="file://$qcow_image"
glance_imageid=`get_glance_id upload_image $image_url`
[[ -z "$glance_imageid" ]] && echo "Glance upload failed!" && exit 1
echo "IMAGE ID: $glance_imageid"
if [[ -f /etc/trove/cloudinit/mysql.cloudinit ]]; then
sudo cp /etc/trove/cloudinit/mysql.cloudinit /etc/trove/cloudinit/${DATASTORE_TYPE}.cloudinit
sudo cp /etc/trove/cloudinit/mysql.cloudinit /etc/trove/cloudinit/${datastore_type}.cloudinit
fi
fi
exclaim "Updating Datastores"
cmd_set_datastore "${GLANCE_IMAGEID}" "${DATASTORE_TYPE}" "${RESTART_TROVE}"
cmd_set_datastore "${glance_imageid}" "${datastore_type}" "${restart_trove}"
}
@ -1158,7 +1155,6 @@ function cmd_run_ci() {
set -e
cmd_install
cmd_test_init "${DATASTORE_TYPE}"
# The arg will be the image type
cmd_build_and_upload_image "${DATASTORE_TYPE}" "${RESTART_TROVE}"
# Test in fake mode.