Adding GBP integration and functional gate

This patch implements the hooks so that functional and integration
tests can be run in the group-based-policy repo.

Currently, the configured devstack for this job is enabled to run GBP
exercise scripts, which result in complete end-to-end integration
testing of the GBP service and client components along with integration
with Neutron and Nova.

A placeholder functional test case is also added. A functional test suite
will be added in a separate commit.

Change-Id: Id705e5755b9662f6e297f29836d6983d66f845a8

.testr.conf

@@ -2,6 +2,6 @@
 test_command=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
              OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
              OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \
-             ${PYTHON:-python} -m subunit.run discover -t ./ . $LISTOPT $IDOPTION
+             ${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./gbpservice/neutron/tests/unit} $LISTOPT $IDOPTION
 test_id_option=--load-list $IDFILE
-test_list_option=--list
+test_list_option=--list

gbpservice/tests/contrib/README

@@ -0,0 +1,3 @@
+The files in this directory are intended for use by the
+infra jobs that run the various functional test
+suite in the gate for the group-based-policy repo.

gbpservice/tests/contrib/filters.template

@@ -0,0 +1,20 @@
+# rootwrap command filters to support functional testing.  It
+# is NOT intended to be used outside of a test environment.
+#
+# This file should be owned by (and only-writeable by) the root user
+
+[Filters]
+# '$BASE_PATH' is intended to be replaced with the expected tox path
+# (e.g. /opt/stack/new/neutron/.tox/dsvm-functional) by the neutron
+# functional jenkins job.  This ensures that tests can kill the
+# processes that they launch with their containing tox environment's
+# python.
+kill_tox_python: KillFilter, root, $BASE_PATH/bin/python, -9
+
+# enable ping from namespace
+ping_filter: CommandFilter, ping, root
+
+# enable curl from namespace
+curl_filter: CommandFilter, curl, root
+tee_filter: CommandFilter, tee, root
+tee_kill: KillFilter, root, tee, -9

gbpservice/tests/contrib/gate_hook.sh

@@ -0,0 +1,49 @@
+#!/bin/bash
+
+set -ex
+
+TOP_DIR=$BASE/new/devstack
+cd $TOP_DIR
+sudo git remote add group-policy http://github.com/group-policy/devstack
+sudo git fetch group-policy
+sudo git checkout -t group-policy/kilo-gate
+#sudo cp devstack-vm-gate.sh $BASE/new/devstack-gate/devstack-vm-gate.sh
+
+CONTRIB_DIR="$BASE/new/group-based-policy/gbpservice/tests/contrib"
+
+#$BASE/new/devstack-gate/devstack-vm-gate.sh
+source $TOP_DIR/functions
+source $TOP_DIR/functions-common
+sudo pip install httplib2
+install_package openvswitch-switch
+pip_install -e /opt/stack/new/group-based-policy
+$TOP_DIR/stack.sh
+
+# Add a rootwrap filter to support test-only
+# configuration (e.g. a KillFilter for processes that
+# use the python installed in a tox env).
+FUNC_FILTER=$CONTRIB_DIR/filters.template
+sed -e "s+\$BASE_PATH+$BASE/new/group-based-policy/.tox/dsvm-functional+" \
+    $FUNC_FILTER | sudo tee /etc/neutron/rootwrap.d/functional.filters > /dev/null
+
+# Use devstack functions to install mysql and psql servers
+source $TOP_DIR/lib/config
+source $TOP_DIR/stackrc
+source $TOP_DIR/lib/database
+disable_service postgresql
+enable_service mysql
+initialize_database_backends
+install_database
+
+# Set up the 'openstack_citest' user and database in each backend
+tmp_dir=`mktemp -d`
+
+cat << EOF > $tmp_dir/mysql.sql
+CREATE DATABASE openstack_citest;
+CREATE USER 'openstack_citest'@'localhost' IDENTIFIED BY 'openstack_citest';
+CREATE USER 'openstack_citest' IDENTIFIED BY 'openstack_citest';
+GRANT ALL PRIVILEGES ON *.* TO 'openstack_citest'@'localhost';
+GRANT ALL PRIVILEGES ON *.* TO 'openstack_citest';
+FLUSH PRIVILEGES;
+EOF
+/usr/bin/mysql -u root < $tmp_dir/mysql.sql

gbpservice/tests/contrib/post_test_hook.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+
+set -xe
+
+GBP_DIR="$BASE/new/group-based-policy"
+TEMPEST_DIR="$BASE/new/tempest"
+SCRIPTS_DIR="/usr/local/jenkins/slave_scripts"
+
+function generate_testr_results {
+    # Give job user rights to access tox logs
+    sudo -H -u $owner chmod o+rw .
+    sudo -H -u $owner chmod o+rw -R .testrepository
+    if [ -f ".testrepository/0" ] ; then
+        .tox/dsvm-functional/bin/subunit-1to2 < .testrepository/0 > ./testrepository.subunit
+        .tox/dsvm-functional/bin/python $SCRIPTS_DIR/subunit2html.py ./testrepository.subunit testr_results.html
+        gzip -9 ./testrepository.subunit
+        gzip -9 ./testr_results.html
+        sudo mv ./*.gz /opt/stack/logs/
+    fi
+}
+
+
+function dsvm_functional_prep_func {
+    :
+}
+
+
+owner=stack
+prep_func="dsvm_functional_prep_func"
+
+# Set owner permissions according to job's requirements.
+cd $GBP_DIR
+sudo chown -R $owner:stack $GBP_DIR
+# Prep the environment according to job's requirements.
+$prep_func
+
+# Run tests
+echo "Running group-based-policy dsvm-functional test suite"
+set +e
+sudo -H -u $owner tox -e dsvm-functional
+testr_exit_code=$?
+set -e
+
+# Collect and parse results
+generate_testr_results
+exit $testr_exit_code

gbpservice/tests/functional/test_policy_mapping_driver.py

@@ -0,0 +1,28 @@
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+# NOTE: The purpose of this module is to provide a nop test to verify that
+# the functional gate is working.
+
+from neutron.tests.functional import base as functional_base
+
+
+class TestPolicyMappingDriver(functional_base.BaseSudoTestCase):
+
+    """Test policy mapping driver."""
+
+    # NOTE: Tests may be added/removed/changed, when this is fleshed out
+    # in future commits.
+
+    def test_policy_target_group_create(self):
+        """Test PTG creation."""
+        pass

test-requirements.txt

@@ -10,6 +10,7 @@ coverage>=3.6
 discover
 fixtures>=0.3.14
 hacking>=0.9.2,<0.10
+httplib2>=0.7.5
 mock>=1.0
 ordereddict
 oslosphinx>=2.2.0  # Apache-2.0

tox.ini

@@ -22,12 +22,12 @@ commands =
 setenv = VIRTUAL_ENV={envdir}
 
 [testenv:functional]
-setenv = OS_TEST_PATH=./neutron/tests/functional
+setenv = OS_TEST_PATH=./gbpservice/tests/functional
 commands =
   python -m neutron.openstack.common.lockutils python setup.py testr --slowest --testr-args='{posargs}'
 
 [testenv:dsvm-functional]
-setenv = OS_TEST_PATH=./neutron/tests/functional
+setenv = OS_TEST_PATH=./gbpservice/tests/functional
          OS_SUDO_TESTING=1
          OS_ROOTWRAP_CMD=sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
          OS_FAIL_ON_MISSING_DEPS=1