openstack/barbican
Code Issues Proposed changes

Split serial and parallel functional test runs

Browse Source 
The functional tests as written take almost an hour to run.  This long run
is on the border of timing out and failing the test due time.  The
tests are run twice: once in serial and then a second time in parallel.
While an added bonus to catch race conditions, the second run is proving
to be a burden when running in the gate, by bringing about the need for
frequent rechecks and reverifies.

This patch splits the test runs into two parts: serial and parallel.
For now, we'll only run the serial tests in the gate.

Change-Id: Ife616ed912cdaa43ef70ea99b320085acd1827e0
This commit is contained in:
Dave McCowan 2017-01-31 10:18:45 -05:00
parent 9a934e57eb
commit cca1a280e3
2 changed files with 35 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
functionaltests/run_tests.sh
View File

@ -31,14 +31,4 @@ testr slowest
coverage combine
coverage report -m
# run the tests in parallel
SKIP=^\(\?\!\.\*\(ProjectQuotasPagingTestCase\|QuotaEnforcementTestCase\|ListingCAsTestCase\|ProjectCATestCase\|GlobalPreferredCATestCase\|CertificateAuthoritiesTestCase\|ListingSecretsTestCase\)\)
testr init
testr run $SKIP --parallel --subunit | subunit-trace --no-failure-debug -f
retval=$(($retval || $?))
testr slowest
coverage combine
coverage report -m
exit $retval

35
functionaltests/run_tests_parallel.sh Normal file
View File

@ -0,0 +1,35 @@
#!/bin/bash
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# How many seconds to wait for the API to be responding before giving up
API_RESPONDING_TIMEOUT=20
if ! timeout ${API_RESPONDING_TIMEOUT} sh -c "while ! curl -s http://127.0.0.1:9311/ 2>/dev/null | grep -q 'v1' ; do sleep 1; done"; then
echo "The Barbican API failed to respond within ${API_RESPONDING_TIMEOUT} seconds"
exit 1
fi
echo "Successfully contacted the Barbican API"
# run the tests in parallel
SKIP=^\(\?\!\.\*\(ProjectQuotasPagingTestCase\|QuotaEnforcementTestCase\|ListingCAsTestCase\|ProjectCATestCase\|GlobalPreferredCATestCase\|CertificateAuthoritiesTestCase\|ListingSecretsTestCase\)\)
testr init
testr run $SKIP --parallel --subunit | subunit-trace --no-failure-debug -f
retval=$(($retval || $?))
testr slowest
coverage combine
coverage report -m
exit $retval