Merge "Remove contrib/devstack/docker code in kuryr"

This commit is contained in:
Jenkins 2016-07-21 14:34:14 +00:00 committed by Gerrit Code Review
commit 4287cf791b
13 changed files with 0 additions and 592 deletions

View File

@ -1,39 +0,0 @@
FROM alpine:3.3
MAINTAINER Antoni Segura Puimedon "toni@kuryr.org"
WORKDIR /
RUN apk add --no-cache \
bash \
iproute2 \
openvswitch \
py-pip \
python \
uwsgi-python && \
apk add --no-cache --virtual build-deps \
gcc \
git \
linux-headers \
musl-dev \
python-dev && \
pip install -U pip setuptools && \
git clone https://github.com/openstack/kuryr && \
cd /kuryr && \
pip install . && \
cd / && \
rm -fr /kuryr && \
apk del build-deps
ENV SERVICE_USER="admin"
ENV SERVICE_TENANT_NAME="admin"
ENV SERVICE_PASSWORD="pass"
ENV IDENTITY_URL="http://127.0.0.1:35357/v2.0"
ENV OS_URL="http://127.0.0.1:9696"
ENV CAPABILITY_SCOPE="local"
ENV LOG_LEVEL="INFO"
ENV PROCESSES=2
ENV THREADS=2
VOLUME /var/log/kuryr
ADD run_kuryr.sh /usr/bin/run_kuryr.sh
CMD ["/usr/bin/run_kuryr.sh"]

View File

@ -1,84 +0,0 @@
=================================
Kuryr Docker libnetwork container
=================================
This is the container generation file for Kuryr's Docker libnetwork driver,
useful for single Docker engine usage as well as Docker Swarm usage.
How to build the container
--------------------------
If you want to build your own container, you can just build it by running the
following command from this same directory:
::
docker build -t your_docker_username/libnetwork:latest .
How to get the container
------------------------
To get the upstream docker libnetwork container with ovs, you can just do:
::
docker pull kuryr/libnetwork:latest
It is expected that different vendors may have their own versions of the
Kuryr libnetwork container in their docker hub namespaces, for example:
::
docker pull midonet/libnetwork:latest
The reason for this is that some vendors' binding scripts need different (and
potentially non-redistributable) userspace tools in the container.
How to run the container
------------------------
First we prepare Docker to find the driver
::
sudo mkdir -p /usr/lib/docker/plugins/kuryr
sudo curl -o /usr/lib/docker/plugins/kuryr/kuryr.spec \
https://raw.githubusercontent.com/openstack/kuryr/master/etc/kuryr.spec
sudo service docker restart
Then we start the container
::
docker run --name kuryr-libnetwork \
--net=host \
--cap-add=NET_ADMIN \
-e SERVICE_USER=admin \
-e SERVICE_TENANT_NAME=admin \
-e SERVICE_PASSWORD=admin \
-e IDENTITY_URL=http://127.0.0.1:35357/v2.0 \
-e OS_URL=http://127.0.0.1:9696 \
-v /var/log/kuryr:/var/log/kuryr \
-v /var/run/openvswitch:/var/run/openvswitch \
kuryr/libnetwork
Where:
* SERVICE_USER, SERVICE_TENANT_SERVICE_PASSWORD are OpenStack credentials
* IDENTITY_URL is the url to OpenStack Keystone
* OS_URL is the url to OpenStack Neutron
* k8S_API is the url to the Kubernetes API server
* A volume is created so that the logs are available on the host
* NET_ADMIN capabilities are given in order to perform network operations on
the host namespace like ovs-vsctl
Other options:
* CAPABILITY_SCOPE can be "local" or "global", the latter being for when there
is a cluster store plugged into the docker engine.
* LOG_LEVEL for defining, for example, "DEBUG" logging messages.
* PROCESSES for defining how many kuryr processes to use to handle the
libnetwork requests.
* THREADS for defining how many threads per process to use to handle the
libnetwork requests.
Note that the 127.0.0.1 are most likely to have to be changed unless you are
running everything on a single machine with `--net=host`.

View File

@ -1,17 +0,0 @@
#!/bin/bash
mkdir -p /etc/kuryr
cat > /etc/kuryr/kuryr.conf << EOF
[DEFAULT]
bindir = /usr/libexec/kuryr
capability_scope = $CAPABILITY_SCOPE
EOF
/usr/sbin/uwsgi \
--plugin /usr/lib/uwsgi/python \
--http-socket :2377 \
-w kuryr.server:app \
--master \
--processes "$PROCESSES" \
--threads "$THREADS"

View File

@ -1,66 +0,0 @@
vagrant-devstack-Kuryr
======================
Getting started
---------------
A Vagrant based kuryr,neutron,keystone and experimental docker system.
Steps to try vagrant image:
1. Install Vagrant on your local machine. Install one of the current
providers supported: VirtualBox, Libvirt or Vagrant
2. Git clone Kuryr repository.
3. Run `cd kuryr/contrib/vagrant`
4. Run `vagrant up`
It will take from 10 to 60 minutes, depending on your internet speed.
Vagrant-cachier can speed up the process [2].
5. `vagrant ssh`
You will get a VM with everything running.
You will get vm shell with keystone and neutron already running.
At this point you should have experimental docker, kuryr, neutron, keystone all
up, running and pointing to each other. Any docker network related commands can
be tried now as explained in [1].
References:
[1] https://github.com/openstack/kuryr/blob/master/doc/source/devref/libnetwork_remote_driver_design.rst#L64
[2] http://fgrehm.viewdocs.io/vagrant-cachier/
Vagrant Options available
-------------------------
You can set the following environment variables before running `vagrant up` to modify
the definition of the Virtual Machine spawned:
* **VAGRANT\_KURYR\_VM\_BOX**: To change the Vagrant Box used. Should be available in
[atlas](http://atlas.hashicorp.com).
export VAGRANT_KURYR_VM_BOX=centos/7
Could be an example of a rpm-based option.
* **VAGRANT\_KURYR\_VM\_MEMORY**: To modify the RAM of the VM. Defaulted to: 4096
* **VAGRANT\_KURYR\_VM\_CPU**: To modify the cpus of the VM. Defaulted to: 2
* **VAGRANT\_KURYR\_RUN\_DEVSTACK**: Whether `vagrant up` should run devstack to
have an environment ready to use. Set it to 'false' if you want to edit
`local.conf` before run ./stack.sh manually in the VM. Defaulted to: true.
See below for additional options for editing local.conf.
Additional devstack configuration
---------------------------------
To add additional configuration to local.conf before the VM is provisioned, you can
create a file called "user_local.conf" in the contrib/vagrant directory of
networking-kuryr. This file will be appended to the "local.conf" created during the
Vagrant provisioning.
For example, to use OVN as the Neutron plugin with Kuryr, you can create a
"user_local.conf" with the following configuration:
enable_plugin networking-ovn http://git.openstack.org/openstack/networking-ovn
enable_service ovn-northd
enable_service ovn-controller
disable_service q-agt
disable_service q-l3

View File

@ -1,47 +0,0 @@
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
VM_MEMORY = ENV.fetch('VAGRANT_KURYR_VM_MEMORY', 6144).to_i
VM_CPUS = ENV.fetch('VAGRANT_KURYR_VM_CPUS', 2).to_i
RUN_DEVSTACK = ENV.fetch('VAGRANT_KURYR_RUN_DEVSTACK', 'true')
config.vm.hostname = 'devstack'
config.vm.provider 'virtualbox' do |v, override|
override.vm.box = ENV.fetch('VAGRANT_KURYR_VM_BOX', 'ubuntu/trusty64')
v.memory = VM_MEMORY
v.cpus = VM_CPUS
end
config.vm.provider 'parallels' do |v, override|
override.vm.box = ENV.fetch('VAGRANT_KURYR_VM_BOX', 'boxcutter/ubuntu1404')
v.memory = VM_MEMORY
v.cpus = VM_CPUS
v.customize ['set', :id, '--nested-virt', 'on']
end
config.vm.provider 'libvirt' do |v, override|
override.vm.box = ENV.fetch('VAGRANT_KURYR_VM_BOX', 'celebdor/trusty64')
v.memory = VM_MEMORY
v.cpus = VM_CPUS
v.nested = true
v.graphics_type = 'spice'
v.video_type = 'qxl'
end
config.vm.synced_folder '../../devstack/', '/devstack'
# For CentOS machines it needs to be specified
config.vm.synced_folder '.', '/vagrant'
config.vm.provision :shell do |s|
s.path = 'vagrant.sh'
s.args = RUN_DEVSTACK
end
if Vagrant.has_plugin?('vagrant-cachier')
config.cache.scope = :box
end
config.vm.network :forwarded_port, guest: 80, host_ip: "127.0.0.1", host: 8080
end

View File

@ -1,6 +0,0 @@
export SERVICE_USER=admin
export SERVICE_PASSWORD=pass
export SERVICE_TENANT_NAME=admin
export SERVICE_TOKEN=pass
export IDENTITY_URL=http://127.0.0.1:5000/v2.0
export DOCKER_HOST="tcp://0.0.0.0:2375"

View File

@ -1,60 +0,0 @@
#!/bin/bash
set -e
BASHPATH=$(dirname "$0"\")
RUN_DEVSTACK="$1"
echo "Run script from $BASHPATH"
# Copied shamelessly from Devstack
function GetOSVersion {
if [[ -x $(which lsb_release 2>/dev/null) ]]; then
os_FAMILY='Debian'
elif [[ -r /etc/redhat-release ]]; then
os_FAMILY='RedHat'
else
echo "Unsupported distribution!"
exit 1;
fi
}
GetOSVersion
if [[ "$os_FAMILY" == "Debian" ]]; then
export DEBIAN_FRONTEND noninteractive
sudo apt-get update
sudo apt-get install -qqy git
elif [[ "$os_FAMILY" == "RedHat" ]]; then
sudo yum install -y -d 0 -e 0 git
fi
# determine checkout folder
PWD=$(su "$OS_USER" -c "cd && pwd")
DEVSTACK=$PWD/devstack
# check if devstack is already there
if [[ ! -d "$DEVSTACK" ]]
then
echo "Download devstack into $DEVSTACK"
# clone devstack
su "$OS_USER" -c "cd && git clone -b master https://github.com/openstack-dev/devstack.git $DEVSTACK"
echo "Copy configuration"
# copy local.conf.sample settings (source: kuryr/devstack/local.conf.sample)
cp /devstack/local.conf.sample $DEVSTACK/local.conf
# If local settings are present, append them
if [ -f "/vagrant/user_local.conf" ]; then
cat /vagrant/user_local.conf >> $DEVSTACK/local.conf
fi
chown "$OS_USER":"$OS_USER" "$DEVSTACK"/local.conf
fi
if $RUN_DEVSTACK; then
echo "Start Devstack"
su "$OS_USER" -c "cd $DEVSTACK && ./stack.sh"
else
echo "Virtual Machine ready. You can run devstack by executing '/home/vagrant/devstack/stack.sh'"
fi

View File

@ -1,10 +0,0 @@
#!/bin/sh
export OS_USER=vagrant
export OS_HOST_IP=172.68.5.10
# run script
bash /vagrant/devstack.sh "$1"
#set environment variables for kuryr
su "$OS_USER" -c "echo 'source /vagrant/config/kuryr_rc' >> ~/.bash_profile"

View File

@ -1,24 +0,0 @@
[[local|localrc]]
LOGFILE=stack.sh.log
LOG_COLOR=False
DATABASE_PASSWORD=pass
RABBIT_PASSWORD=pass
SERVICE_PASSWORD=pass
SERVICE_TOKEN=pass
ADMIN_PASSWORD=pass
enable_plugin kuryr https://git.openstack.org/openstack/kuryr
enable_service kuryr
enable_service etcd-server
enable_service docker-engine
# Use Neutron instead of nova-network
disable_service n-net
enable_service q-svc
enable_service q-dhcp
enable_service q-l3
disable_service heat
enable_service q-agt
disable_service tempest

View File

@ -1,147 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# Save trace setting
XTRACE=$(set +o | grep xtrace)
set +o xtrace
ETCD_VERSION=v2.2.2
function install_etcd_data_store {
if [ ! -f "$DEST/etcd/etcd-$ETCD_VERSION-linux-amd64/etcd" ]; then
echo "Installing etcd server"
mkdir $DEST/etcd
wget https://github.com/coreos/etcd/releases/download/$ETCD_VERSION/etcd-$ETCD_VERSION-linux-amd64.tar.gz -O $DEST/etcd/etcd-$ETCD_VERSION-linux-amd64.tar.gz
tar xzvf $DEST/etcd/etcd-$ETCD_VERSION-linux-amd64.tar.gz -C $DEST/etcd
fi
# Clean previous DB data
rm -rf $DEST/etcd/db.etcd
}
function check_docker {
if is_ubuntu; then
dpkg -s docker-engine > /dev/null 2>&1
else
rpm -q docker-engine > /dev/null 2>&1
fi
}
# main loop
if is_service_enabled kuryr; then
if [[ "$1" == "stack" && "$2" == "install" ]]; then
install_etcd_data_store
setup_develop $KURYR_HOME
elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
if [[ ! -d "${KURYR_ACTIVATOR_DIR}" ]]; then
echo -n "${KURYR_ACTIVATOR_DIR} directory is missing. Creating it... "
sudo mkdir -p ${KURYR_ACTIVATOR_DIR}
echo "Done"
fi
if [[ ! -f "${KURYR_ACTIVATOR}" ]]; then
echo -n "${KURYR_ACTIVATOR} is missing. Copying the default one... "
sudo cp ${KURYR_DEFAULT_ACTIVATOR} ${KURYR_ACTIVATOR}
echo "Done"
fi
if [[ ! -d "${KURYR_CONFIG_DIR}" ]]; then
echo -n "${KURYR_CONFIG_DIR} directory is missing. Creating it... "
sudo mkdir -p ${KURYR_CONFIG_DIR}
echo "Done"
fi
if [[ ! -f "${KURYR_CONFIG}" ]]; then
if [[ -f "${KURYR_DEFAULT_CONFIG}" ]]; then
echo -n "${KURYR_CONFIG} is missing. Copying the default one... "
sudo cp ${KURYR_DEFAULT_CONFIG} ${KURYR_CONFIG}
echo "Done"
else
echo -n "${KURYR_CONFIG} and the default config missing. Auto generating and copying one... "
cd ${KURYR_HOME}
oslo-config-generator --config-file=${KURYR_CONFIG_GENERATOR}
sudo cp ${KURYR_DEFAULT_CONFIG}.sample ${KURYR_DEFAULT_CONFIG}
sudo cp ${KURYR_DEFAULT_CONFIG} ${KURYR_CONFIG}
cd -
fi
fi
# Run etcd first
run_process etcd-server "$DEST/etcd/etcd-$ETCD_VERSION-linux-amd64/etcd --data-dir $DEST/etcd/db.etcd --advertise-client-urls http://0.0.0.0:$KURYR_ETCD_PORT --listen-client-urls http://0.0.0.0:$KURYR_ETCD_PORT"
# FIXME(mestery): By default, Ubuntu ships with /bin/sh pointing to
# the dash shell.
# ..
# ..
# The dots above represent a pause as you pick yourself up off the
# floor. This means the latest version of "install_docker.sh" to load
# docker fails because dash can't interpret some of it's bash-specific
# things. It's a bug in install_docker.sh that it relies on those and
# uses a shebang of /bin/sh, but that doesn't help us if we want to run
# docker and specifically Kuryr. So, this works around that.
sudo update-alternatives --install /bin/sh sh /bin/bash 100
# Install docker only if it's not already installed. The following checks
# whether the docker-engine package is already installed, as this is the
# most common way for installing docker from binaries. In case it's been
# manually installed, the install_docker.sh script will prompt a warning
# if another docker executable is found
check_docker || {
wget http://get.docker.com -O install_docker.sh
sudo chmod 777 install_docker.sh
sudo sh install_docker.sh
sudo rm install_docker.sh
}
# After an ./unstack it will be stopped. So it is ok if it returns exit-code == 1
sudo service docker stop || true
run_process docker-engine "sudo /usr/bin/docker daemon -H tcp://0.0.0.0:$KURYR_DOCKER_ENGINE_PORT --cluster-store etcd://localhost:$KURYR_ETCD_PORT"
fi
if [[ "$1" == "stack" && "$2" == "extra" ]]; then
# FIXME(limao): When Kuryr start up, it need to detect if neutron support tag plugin.
# Kuryr will call neutron extension api to verify if neutron support tag.
# So Kuryr need to start after neutron-server finish load tag plugin.
# The process of devstack is:
# ...
# run_phase "stack" "post-config"
# ...
# start neutron-server
# ...
# run_phase "stack" "extra"
#
# If Kuryr start up in "post-config" phase, there is no way to make sure
# Kuryr can start before neutron-server, so Kuryr start in "extra" phase.
# Bug: https://bugs.launchpad.net/kuryr/+bug/1587522
run_process kuryr "sudo PYTHONPATH=$PYTHONPATH:$DEST/kuryr SERVICE_USER=admin SERVICE_PASSWORD=$SERVICE_PASSWORD SERVICE_TENANT_NAME=admin SERVICE_TOKEN=$SERVICE_TOKEN IDENTITY_URL=http://127.0.0.1:5000/v2.0 python $DEST/kuryr/scripts/run_server.py --config-file /etc/kuryr/kuryr.conf"
neutron subnetpool-create --default-prefixlen $KURYR_POOL_PREFIX_LEN --pool-prefix $KURYR_POOL_PREFIX kuryr
fi
if [[ "$1" == "unstack" ]]; then
stop_process kuryr
stop_process etcd-server
stop_process docker-engine
fi
fi
# Restore xtrace
$XTRACE

View File

@ -1,22 +0,0 @@
KURYR_HOME=${KURYR_HOME:-$DEST/kuryr}
KURYR_ACTIVATOR_FILENAME=kuryr.spec
KURYR_DEFAULT_ACTIVATOR=${KURYR_HOME}/etc/${KURYR_ACTIVATOR_FILENAME}
KURYR_CONFIG_GENERATOR_FILENAME=kuryr-config-generator.conf
KURYR_CONFIG_GENERATOR=${KURYR_HOME}/etc/${KURYR_CONFIG_GENERATOR_FILENAME}
# See libnetwork's plugin discovery mechanism:
# https://github.com/docker/docker/blob/c4d45b6a29a91f2fb5d7a51ac36572f2a9b295c6/docs/extend/plugin_api.md#plugin-discovery
KURYR_ACTIVATOR_DIR=${KURYR_ACTIVATOR_DIR:-/usr/lib/docker/plugins/kuryr}
KURYR_ACTIVATOR=${KURYR_ACTIVATOR_DIR}/${KURYR_ACTIVATOR_FILENAME}
KURYR_CONFIG_FILENAME=kuryr.conf
KURYR_DEFAULT_CONFIG=${KURYR_HOME}/etc/${KURYR_CONFIG_FILENAME}
KURYR_CONFIG_DIR=${KURYR_CONFIG_DIR:-/etc/kuryr}
KURYR_CONFIG=${KURYR_CONFIG_DIR}/${KURYR_CONFIG_FILENAME}
KURYR_POOL_PREFIX=${KURYR_POOL_PREFIX:-10.10.0.0/16}
KURYR_POOL_PREFIX_LEN=${KURYR_POOL_PREFIX_LEN:-24}
KURYR_DOCKER_ENGINE_PORT=${KURYR_DOCKER_ENGINE_PORT:-2375}
KURYR_ETCD_PORT=${KURYR_ETCD_PORT:-4001}

View File

@ -1,10 +0,0 @@
#!/usr/bin/env bash
set -ex
VENV=${1:-"debug-py27"}
GATE_DEST=$BASE/new
DEVSTACK_PATH=$GATE_DEST/devstack
$BASE/new/devstack-gate/devstack-vm-gate.sh

View File

@ -1,60 +0,0 @@
#!/usr/bin/env bash
set -xe
KURYR_DIR="$BASE/new/kuryr"
TEMPEST_DIR="$BASE/new/tempest"
SCRIPTS_DIR="/usr/os-testr-env/bin/"
venv=${1:-"debug-py27"}
function generate_test_logs {
local path="$1"
# Compress all $path/*.txt files and move the directories holding those
# files to /opt/stack/logs. Files with .log suffix have their
# suffix changed to .txt (so browsers will know to open the compressed
# files and not download them).
if [[ -d "$path" ]] ; then
sudo find "$path" -iname "*.log" -type f -exec mv {} {}.txt \; -exec gzip -9 {}.txt \;
sudo mv "$path/*" /opt/stack/logs/
fi
}
function generate_testr_results {
# Give job user rights to access tox logs
sudo -H -u "$owner" chmod o+rw .
sudo -H -u "$owner" chmod o+rw -R .testrepository
if [[ -f ".testrepository/0" ]] ; then
".tox/$venv/bin/subunit-1to2" < .testrepository/0 > ./testrepository.subunit
$SCRIPTS_DIR/subunit2html ./testrepository.subunit testr_results.html
gzip -9 ./testrepository.subunit
gzip -9 ./testr_results.html
sudo mv ./*.gz /opt/stack/logs/
fi
if [[ "$venv" == fullstack* ]] ; then
generate_test_logs "/tmp/${venv}-logs"
fi
}
#owner=tempest
# Configure the api tests to use the tempest.conf set by devstack.
#sudo_env="TEMPEST_CONFIG_DIR=$TEMPEST_DIR/etc"
owner=stack
sudo_env=
# Set owner permissions according to job's requirements.
cd "$KURYR_DIR"
sudo chown -R $owner:stack "$KURYR_DIR"
# Run tests
echo "Running Kuryr $venv tests"
set +e
sudo -H -u "$owner" tox -e "$venv"
testr_exit_code=$?
set -e
# Collect and parse results
generate_testr_results
exit $testr_exit_code