Chris Dent 65da83aece Use sync_on_startup in placement-perfload job
Use the [placement_database]/sync_on_startup config setting to
have the database schema synchronized during web-service startup
rather than through a separate call to placement-manage.

This is done for two reasons:

* It provides a reaonable test that it works, which is not present
  in other integration tests.
* Until Id9bc515cee71d629b605da015de39d1c9b0f8fc4 merges it will
  demonstrate the bug described in the story linked below.

Couple of things to note:

* The tempest job will continue to exercise placement-manage, as it
  has always done.
* The bug (in the story) doesn't impact the behavior of the API, it
  merely impacts what is or is not logged. In the
  logs/placement-api.log generated in the perfload job for this change
  there will be an initial burst of DEBUG and INFO logging, but then
  only request logging. This should be corrected by
  Id9bc515cee71d629b605da015de39d1c9b0f8fc4

Change-Id: Ib7f5cdfa3b314af7681d594dccb553bddb764224
Story: 2005187
2019-03-09 12:24:22 +00:00

133 lines
6.1 KiB
YAML

- hosts: all
tasks:
# Borrowed from devstack tasks but we want it early.
- name: Ensure {{ ansible_user_dir }}/logs exists
become: true
file:
path: "{{ ansible_user_dir }}/logs"
state: directory
owner: "{{ ansible_user }}"
- name: start placement
args:
chdir: "{{ ansible_user_dir }}/src/git.openstack.org/openstack/placement"
shell:
executable: /bin/bash
cmd: |
set -x
# TODO(cdent): Presumably ansible can do this, perhaps with 'package'.
# create database
sudo debconf-set-selections <<MYSQL_PRESEED
mysql-server mysql-server/root_password password secret
mysql-server mysql-server/root_password_again password secret
mysql-server mysql-server/start_on_boot boolean true
MYSQL_PRESEED
sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev
sudo mysql -uroot -psecret -e "DROP DATABASE IF EXISTS placement;"
sudo mysql -uroot -psecret -e "CREATE DATABASE placement CHARACTER SET utf8;"
sudo mysql -uroot -psecret -e "GRANT ALL PRIVILEGES ON placement.* TO 'root'@'%' identified by 'secret';"
python -m virtualenv -p python3 .placement
. .placement/bin/activate
pip install . PyMySQL uwsgi
# set config via environment
export OS_PLACEMENT_DATABASE__CONNECTION=mysql+pymysql://root:secret@127.0.0.1/placement?charset=utf8
export OS_PLACEMENT_DATABASE__MAX_POOL_SIZE=25
export OS_PLACEMENT_DATABASE__MAX_OVERFLOW=100
export OS_PLACEMENT_DATABASE__SYNC_ON_STARTUP=True
export OS_DEFAULT__DEBUG=True
export OS_API__AUTH_STRATEGY=noauth2
uwsgi --http :8000 --wsgi-file .placement/bin/placement-api --daemonize {{ ansible_user_dir }}/logs/placement-api.log --processes 5 --threads 25
- name: placement performance
args:
chdir: "{{ ansible_user_dir }}/src/git.openstack.org/openstack/placement"
shell:
executable: /bin/bash
# TODO(cdent): Change this task to a role?
cmd: |
set -x
# Do some performance related information gathering for placement.
EXPLANATION="
This output combines output from placeload with timing information
gathered via curl. The placeload output is the current maximum
microversion of placement followed by an encoded representation of
what it has done. Lowercase 'r', 'i', 'a', and 't' indicate successful
creation of a resource provider and setting inventory, aggregates, and
traits on that resource provider.
If there are upper case versions of any of those letters, a failure
happened for a single request. The letter will be followed by the
HTTP status code and the resource provider uuid. These can be used
to find the relevant entry in logs/screen-placement-api.txt.gz.
Note that placeload does not exit with an error code when this
happens. It merely reports and moves on. Under correct circumstances
the right output is a long string of 4000 characters containing
'r', 'i', 'a', 't' in random order (because async).
After that are three aggregate uuids, timing information for the
placeload run, and then timing information for two identical curl
requests for allocation candidates.
If no timed requests are present it means that the expected number
of resource providers were not created. At this time, only resource
providers are counted, not whether they have the correct inventory,
aggregates, or traits.
"
# This aggregate uuid is a static value in placeload.
AGGREGATE="14a5c8a3-5a99-4e8f-88be-00d85fcb1c17"
TRAIT="HW_CPU_X86_AVX2"
PLACEMENT_QUERY="resources=VCPU:1,DISK_GB:10,MEMORY_MB:256&member_of=${AGGREGATE}&required=${TRAIT}"
LOG=placement-perf.txt
LOG_DEST={{ ansible_user_dir }}/logs
COUNT=1000
trap "sudo cp -p $LOG $LOG_DEST" EXIT
function check_placement {
local placement_url
local rp_count
local code
code=0
python -m virtualenv -p python3 .placeload
. .placeload/bin/activate
# install placeload
pip install 'placeload==0.3.0'
# get placement endpoint
placement_url="http://localhost:8000"
set +x
# load with placeload
(
echo "$EXPLANATION"
# preheat the aggregates to avoid https://bugs.launchpad.net/nova/+bug/1804453
placeload $placement_url 10
echo "##### TIMING placeload creating $COUNT resource providers with inventory, aggregates and traits."
time placeload $placement_url $COUNT
) 2>&1 | tee -a $LOG
rp_count=$(curl -H 'x-auth-token: admin' $placement_url/resource_providers |json_pp|grep -c '"name"')
# Skip curl and note if we failed to create the required number of rps
if [[ $rp_count -ge $COUNT ]]; then
(
echo "##### TIMING GET /allocation_candidates?${PLACEMENT_QUERY} twice"
time curl -s -H 'x-auth-token: admin' -H 'openstack-api-version: placement 1.21' "$placement_url/allocation_candidates?${PLACEMENT_QUERY}" > /dev/null
time curl -s -H 'x-auth-token: admin' -H 'openstack-api-version: placement 1.21' "$placement_url/allocation_candidates?${PLACEMENT_QUERY}" > /dev/null
) 2>&1 | tee -a $LOG
else
(
echo "Unable to create expected number of resource providers. Expected: ${COUNT}, Got: $rp_count"
echo "See job-output.txt.gz and logs/screen-placement-api.txt.gz for additional detail."
) | tee -a $LOG
code=1
fi
set -x
deactivate
exit $code
}
check_placement