Add apache benchmark (ab) to end of perfload jobs

Start the process of reporting some concurrency numbers by including
a 500 x 10 'ab' run against the query URL used in each perfload job.

There's duplication removal that could be done here, but we leave
that until we've determined if this is working well.

The PLACEMENT_URL is updated to use 127.0.0.1 instead of localhost;
ab will attempt to use the IPV6 version of localhost if that's the
case, and we've not bound the placement server to that interface.

The timeout on the placement-nested-perfload job has been raised to
1 hour as the default 30 minutes is leading to a timeout. If that's
still not enough we'll explore lowering concurrency.

We will quite likely need to adapt the mysql configuration if we
intend to continue down this road.

Change-Id: Ic0bf2ab666dab546dd7b03955473c246fd0f380a

.zuul.yaml

@@ -92,4 +92,5 @@
     description: |
         A simple node on which to run placement with the barest of configs and
         make nested performance related tests against it.
+    timeout: 3600
     run: playbooks/nested-perfload.yaml

gate/perfload-nested-runner.sh

@@ -1,7 +1,7 @@
 #!/bin/bash -x
 WORK_DIR=$1
 
-PLACEMENT_URL="http://localhost:8000"
+PLACEMENT_URL="http://127.0.0.1:8000"
 LOG=placement-perf.txt
 LOG_DEST=${WORK_DIR}/logs
 # The gabbit used to create one nested provider tree. It takes
@@ -21,6 +21,11 @@ ITERATIONS=1000
 # Number of times to write allocations and then time again.
 ALLOCATIONS_TO_WRITE=10
 
+# Apache Benchmark Concurrency
+AB_CONCURRENT=10
+# Apache Benchmark Total Requests
+AB_COUNT=500
+
 # The number of providers in each nested tree. This will need to
 # change whenever the resource provider topology created in $GABBIT
 # is changed.
@@ -39,6 +44,13 @@ function time_candidates {
     ) 2>&1 | tee -a $LOG
 }
 
+function ab_bench {
+    (
+        echo "#### Running apache benchmark"
+        ab -c $AB_CONCURRENT -n $AB_COUNT -H 'x-auth-token: admin' -H 'openstack-api-version: placement latest' "${PLACEMENT_URL}/allocation_candidates?${PLACEMENT_QUERY}"
+    ) 2>&1 | tee -a $LOG
+}
+
 function write_allocation {
     # Take the first allocation request and send it back as a well-formed allocation
     curl -s -H 'x-auth-token: admin' -H 'openstack-api-version: placement latest' "${PLACEMENT_URL}/allocation_candidates?${PLACEMENT_QUERY}&limit=5" \
@@ -80,6 +92,7 @@ function check_placement {
     # log a message.
     if [[ $rp_count -ge $TOTAL_PROVIDER_COUNT ]]; then
       load_candidates
+      ab_bench
     else
         (
             echo "Unable to create expected number of resource providers. Expected: ${COUNT}, Got: $rp_count"

gate/perfload-runner.sh

@@ -36,13 +36,18 @@ aggregates, or traits.
 AGGREGATE="14a5c8a3-5a99-4e8f-88be-00d85fcb1c17"
 TRAIT="HW_CPU_X86_AVX2"
 PLACEMENT_QUERY="resources=VCPU:1,DISK_GB:10,MEMORY_MB:256&member_of=${AGGREGATE}&required=${TRAIT}"
-PLACEMENT_URL="http://localhost:8000"
+PLACEMENT_URL="http://127.0.0.1:8000"
 
 
 LOG=placement-perf.txt
 LOG_DEST=${WORK_DIR}/logs
 COUNT=1000
 
+# Apache Benchmark Concurrency
+AB_CONCURRENT=10
+# Apache Benchmark Total Requests
+AB_COUNT=500
+
 trap "sudo cp -p $LOG $LOG_DEST" EXIT
 
 function time_candidates {
@@ -53,6 +58,13 @@ function time_candidates {
     ) 2>&1 | tee -a $LOG
 }
 
+function ab_bench {
+    (
+        echo "#### Running apache benchmark"
+        ab -c $AB_CONCURRENT -n $AB_COUNT -H 'x-auth-token: admin' -H 'openstack-api-version: placement latest' "${PLACEMENT_URL}/allocation_candidates?${PLACEMENT_QUERY}"
+    ) 2>&1 | tee -a $LOG
+}
+
 function write_allocation {
     # Take the first allocation request and send it back as a well-formed allocation
     curl -s -H 'x-auth-token: admin' -H 'openstack-api-version: placement latest' "${PLACEMENT_URL}/allocation_candidates?${PLACEMENT_QUERY}&limit=5" \
@@ -95,6 +107,7 @@ function check_placement {
     # log a message.
     if [[ $rp_count -ge $COUNT ]]; then
       load_candidates
+      ab_bench
     else
         (
             echo "Unable to create expected number of resource providers. Expected: ${COUNT}, Got: $rp_count"

gate/perfload-server.sh

@@ -8,7 +8,7 @@ mysql-server mysql-server/root_password password secret
 mysql-server mysql-server/root_password_again password secret
 mysql-server mysql-server/start_on_boot boolean true
 MYSQL_PRESEED
-sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev jq parallel
+sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev jq parallel apache2-utils
 sudo mysql -uroot -psecret -e "DROP DATABASE IF EXISTS placement;"
 sudo mysql -uroot -psecret -e "CREATE DATABASE placement CHARACTER SET utf8;"
 sudo mysql -uroot -psecret -e "GRANT ALL PRIVILEGES ON placement.* TO 'root'@'%' identified by 'secret';"