This commit allows CI to run the collectd-openstack role directly
to do this I take the relevent variables from the group_vars/all.yml
and duplicate them in to the defaults. If you're not making mix and
match playbooks with browbeat roles and run the playbooks 'normally'
these defaults will never affect you in any way and will be overridden
by the group_vars/all.yml file without any interaction from you.
If on the other hand you want a playbook where you can toss collectd
install whereever you need it, this makes life much easier. Also note
that the trick I'm using to get the name of the executing hosts first
group could be used to make the collectd-openstack playbook run in
parallel across all hosts all at once rather than one group at a time.
As one final tweak this adds 10 minutesof retires to the epel install
role, specifically because two of it's tasks rely on external internet
access and fail often enough to cause trouble. Previously I would retry
the whole CollectD playbook when that happened, this is more efficient
Change-Id: Ia67ab2b4152a8d8e9f3ba1dabe1f4417d7b86233
This restructures some of the metadata to better reflect how builds
are tracked up and downstream. Upstream we have a delorean hash which
is used to track builds as they come down the pipeline, downstream we end
up with RHOS puddles, ideally we will track the transition between these two
since every puddle has a delorean hash but not the other way around. Right now
I'll settle for collecting the infromation.
Change-Id: I9a9030523ec5f586b7e696abe892e0c8167e9869
This updates the url building for the rhos pipeline builds
so that we grab the correct image ID when the promotion pipeline
attempts to promote a build.
Change-Id: I13da83b7ecdf4cd145e7ffc1ae2b9e6e0a9690f1
This commit enanbles Ansible linting and does some
minor refactoring to make existing Ansible roles
compatible with the new rules.
Several Ansible linting rules have been excluded to keep the number
of changes from being too onerous.
Also a new script in ci-scripts is used to check very config file
included in the Browbeat repo for validity using the template
Browbeat uses when it runs.
Here's a list of the new linting rules
* Ansible tasks must have names
* When you use shell you must use become not sudo
* Using become_user without using become is not allowed
* If a repo is pulled it must be a pinned version of commit, not latest
* Always_run is deprecated don't use it
* Variables without {{}} and not in when statements are deprecated don't use them
* No Trailing whitepaces
* YAML checking, catches big syntax errors but not less obvious ones
Change-Id: Ic531c91c408996d4e7d8899afe8b21d364998680
The internal pipeline image mirror has a slightly different
folder structure that doesn't include the /current-tripleo/ folder
anyways this was causing Ocata pipeline builds to fail.
Change-Id: I688218dda2c32222df3752bceb771427c2ba9425
As the latest version is moved from Master to Ocata some folder structure
changes are required to make sure we get the right image. Eventually master
will point to 12 instead of 11.
Change-Id: Ie83cf1add5418c77054b67610c4f35318420b93e
See here for more details
https://review.openstack.org/#/c/437946/
The reason the old pin isn't working is that --bootstrap in quickstart
doesn't work like it should and force the reinstallation of the virtualenv
so we carry over the prep internal script that now uses extras itself.
Change-Id: I5479738565017941df71313f10816f2c9a4debea
This commit sets the variables for network creation to the
appropriate values. Since the network details are internal, this CI
variables file is being moved to internal git.
Change-Id: Ib55d2896991c74562f01e3cc56117af7110dc403
StatsD support works, working on dashboards
As documented here http://docs.openstack.org/developer/ironic/deploy/metrics.html
Ironic allows performance metrics to be dumped via StatsD, this installs StatsD
on the undercloud via epel, sets up a service for it, enables it, and updates
Ironic to use it.
Change-Id: I793d4d3211ecf6113bd4863a0672ea0cb0de9dd3
Reverting Commit: 2ba6da9022491315cd0d5c21dc502adbed20860f
This needed much more testing before merging, my bad.
I strongly suggest we don't add this functionality
in a seperate commit again, it doesn't make sense
to reorg and test all of this and then the pip commit
right after. Just add it's functionality there.
Change-Id: Iee7aa439fbc077c3c71f67b625b67fc55a86f199
* if results and log path not found, create it.
* Also resolve $FOO in browbeat config paths -jkilpatr
Change-Id: Ie5ec32386ca0d6db9177d9a3a55387b5b1e88a69
While the Grafana infra is having trouble it's causing tests to fail
so until it's all squared away I'm going to make it non-mandatory
Change-Id: Iaa779f19325bb5aef830bd93086995cb2beef3b5
https://review.openstack.org/#/c/403677 The above commit is currently
required for the Microbrow CI to work well. It should be clean enough
to be mainlined now but no one wants to merge someone elses hack. Anyways
this just moves the pin forward for the sake of staying close to upstream.
Change-Id: I1d455e3ecc490c62289be25eaec4bfd0a124f346
In some cases the user might want to run the same scenario with the
same concurrency value multiple times. In that case currently Browbeat
exits with an ugly traceback since the test name would be based on the
concurrency and hyaving the same concurrency twice results in trying to
create a file that already exists.
This commit makes the code smart enough to recognize duplicate concurrency
values and name files accordingly. For example. having 16 twice as the
concurrency value for ceilometer_list_meters scenario produces two different
log files as 'results/20160718-202008/CeilometerMeters/ceilometer_list_meters/
20160718-202008-browbeat-ceilometer_list_meters-16-iteration-0.log' and
'results/20160718-202008/CeilometerMeters/ceilometer_list_meters/
20160718-202008-browbeat-ceilometer_list_meters-16-2-iteration-0.log.
Change-Id: Ie39c8c54ddf0435ff46975bfc4a5fd62995b2a32
Reflow some of the content so that it wraps nicely to 80 chars. Also adjust
some of the indenting, formatting, and general small tweaks to the wording.
Change-Id: I5397356b802c3a06762dfae524bb1ba1ea634972
Just moves a few lines from the JJB to here. I remember thinking
this wouldn't work for some reason, but now I can't remember why
and a quick test shows no problems. So here we go anyways.
Change-Id: I6e7ea058629234d63edadbd715d0cd762f6ccc04
The tripleo-quickstart-extras requirements file is now
imported by default, it also checks for duplicate imports
and fails on them, therefore if we continue to import the
extras file we fail out on a duplicate import error
Change-Id: If0f5067ed2a2bae6f6f4873819e0da229acef10b
So to enable both graphite and grafana in perf-ci you use the
grafana_enabled_template toggle, not the non-existant graphite_enabled_template
variable.
Change-Id: Ia0f6933c498068e1e7e85b5ac4b3c1db79459e26
'' is different from "" in Bash. Leaving this up for
whatever the next problem I find is so that I can stop
polluting the commit history with tiny fixes.
Change-Id: Ia1d53670347a3862c743a4a74b0f0d3eda62dbd8
adds set -eu to exit on undefined variable usage.
avoids exporting every variable.
Also adds case for downstream out of pipeline jobs that should fix them.
Change-Id: I588c6def5bd9d51f3b5c6fa2a8a8557d451e2086
this sets up pipeline jobs to actually use pipeline images, but in
such a way that they can still be kicked manually without problems
Change-Id: I03c6c77315ecf87021c851cf5632026b01c7b45b
Added this switch yesterday, should speed up my CI work in general.
Needs some quick testing to make sure we don't run into that timed out
waiting for sudo error that came up earlier.
Change-Id: I7cc64a48fe0e26427fed01260d232de22cf91a90
This adds support for Browbeat to run as a oooq role
none of the playbooks or the roles located in ansible/oooq
are usable if called without oooq, so maybe they should go
in the ci-scripts folder.
Change-Id: Iefcae69af591a790bdeb4acc3c85e75292b6b8ce
this could not quite follow the existing pattern due to
the fact that glance has multiple independent config files.
So this uses a fileglob loop to find all the configuration files
in the /etc/glance directory and load each of them in turn.
Change-Id: I259bef0d022500b3865926fc36f289c6a17ec5db
Since people don't seem to be paying attention to failed CI builds and
correcting the CI all.yml when they update install/all.yml this commit
uses ansible's run time variable loading to overwrite the relevent
vars from the all.yml
Change-Id: Icdc5ce33fcd1230ade39563824682134a9865f1d
Working:
* Benchmark output makes it to browbeat results directory
* v1.7 PerfKitBenchmarker is installed
* Many benchmarks are tested however there is still some that
require testing/configuration
* Example are provided in conf/perfkit-benchmarks.yml
Change-Id: I62dec86fe7e8c6f71b7c5654abfd4b2079904e4b
Upstream oooq has enabled tempest after overcloud
deployments by default now, since temptest coverage
has been lacking on builds for a while we are seeing
failures, since this isn't particularly important for
browbeat CI specifically this commit supresses running
the overcloud ping test and temptest
Change-Id: I9020bdd017e46dc529467141ec43516768c4b2a4
This commit adds logic to run Metadata at the beginning of every browbeat
run through browbeat.py. Also some imports have been fixed.
Co-Authored-By: Joe Talerico <jtaleric@redhat.com>
Change-Id: Ibc13a64710209b25a755f606ea7fddc80232cbc4
Changes and improvements to Shaker
Upgrade shaker to 0.0.15 and shaker_dns_nameservers option to image build
Change-Id: I0b3cd8ed8c656e757c0e9bbbec64eebaa8de88e1
Moves bug report save location to the playbook directory
on the machine running ansible, I feel that this is an odd
place to put anything but it seems to be intended behavior.
This commit changes where the bug report is saved and where
the CI script checks for the output file, until a consistent
version of this is checked in CI will fail often.
Change-Id: I94d221d718fb13779bf056ae83376851f82a355b
BASH is whitespace sensitive, so one part of the
conditional for checking changes to Browbeat.py and the
libs folder would always fail.
Change-Id: I090bcc87e5112a0d0c40796ed9f4ea470d7607fa
The rally version was added as a variable in the
install group vars file, but CI maintains it's own
version of this file that was not updated, therefore
the rally_version change was breaking to all CI builds.
It might be better for CI to move to overwriting specific
variables if possible.
Change-Id: Ib4b789cc2cdbe409c5a866444fe144328f086e8a
Will need to be updated once https://review.openstack.org/#/c/354885/
is merged.
The check playbook was integrated into the CI script
several versions ago but was removed when checks where
declared depricated. That effort fell though and checks
are now un-depricated, so adding them to CI is prudent.
Change-Id: I93d4b9fa6ad3e5e8fd98898886da42e7bd8266ea
If anything in the lib/ directory is modified rally will
now be run to test browbeat, same for Browbeat.py
Change-Id: Id4cb1a01d1940b718a85ac6dd58a560d2c3804cc
This commit adds basic CI scripts and CI integration
into Browbeat, this script is sourced from the
Triple O quickstart full-deploy.sh CI script
This script currently works with Rally and Shaker
once my other commit is in, perfkit support depends
on a bugfix in Centos or upstream DNF support in perfkit
Change-Id: I022f05e4408fe5466c3b1032cbbca7adc96d0527
