58 Commits

Author SHA1 Message Date
Monty Taylor
c7f8c6c1f6 Add retry message to gerrit-git-prep macro
So that we can track how often it happens.

Change-Id: I7daf5251dd95e07e98cff9845382cdbdd8bcabb5
2015-04-28 09:16:21 -04:00
Monty Taylor
9450ee37e3 Put retry loop around gerrit-git-prep
As an attempt to mitigate git mirror disconnects.

Change-Id: I7f2901580e8507a6455d013cd8929394e9ef0ab3
2015-04-28 08:53:09 -04:00
Michael Krotscheck
e6d20ad130 Add NPM Command Execution as a Build Target
- Add npm-run as a build macro.
- Add xvfb-start as a build macro for generic xvfb startup.
- Added gate-{project}-npm-run-{command} as a build template.
- Made that same build template non-voting.
- Added gate-storyboard-webclient-npm-run-lint as a first job.

Note that using this target presumes that you will pipe your output reports
into ./reports.

Change-Id: Ie772bf2c7b56aaed3c36fc1cb1cb970dfc3f93ff
2015-04-06 13:13:57 -07:00
Joshua Hesketh
077f49ce3d zuul-swift-devstack-logs includes console
Update the zuul-swift-devstack-log job to include the console.
This job isn't in use yet.

Change-Id: I85a6dd4258e04904355781e38abf376a6164b840
2015-04-02 11:59:02 +11:00
Joshua Hesketh
ca21f24944 Upload devstack-gate documentation with logs
We need to place the tempest log documentation in with the devstack
logs. We can do this as a publisher shell step.

The tempest overview is a little more complicated as it needs
placing in the top most job results directory. We can scp this for
the old log storage and then for zuul-swift-logs we pass it as a
file to be uploaded which will hence be in the master index (ie at
the top of our pseudo filesystem)

Change-Id: I54d2d2ba9b6532221ca3b376ab2e51bc5398444e
Depends-On: I8b26cc58c2ad6ca029726d227cf1d38d72465627
2015-04-02 11:58:58 +11:00
Jenkins
ed3ded0dbd Merge "Upload python test results to swift" 2015-03-16 16:58:08 +00:00
Jenkins
22c7b9333e Merge "Create and upload heat test images to tarballs" 2015-03-09 22:50:03 +00:00
Jenkins
bf9600f6e2 Merge "Add job runtime system package and database setup" 2015-03-09 20:54:36 +00:00
Steve Baker
213ec9d7fb Create and upload heat test images to tarballs
This change will only create a fedora image but a future change could
also generate images for Ubuntu and CentOS7.

build-heat-test-image.sh requires an installed diskimage-builder,
so the recipe for diskimage-builder-install was determined by running
build-heat-test-image.sh in a pristine trusty cloud image.

Change-Id: I03a9efe47fc3f21471a57e33ab09b0d7a13b219c
2015-03-09 10:26:33 +13:00
Joshua Hesketh
f8246628c3 Upload python test results to swift
Duplicate the test-results publisher for swift. This will capture
all logs from the job which is a superset of what was being
uploaded.

Change-Id: If1e7baf5dad8601c7489366688ccb02d7fc1c742
2015-03-04 09:33:21 +11:00
Ian Cordasco
4bd139d1f4 Fix os-ansible-deployment publishers one last time
Ant's SCP job uses DirSet which only allows for paths relative to the
directory from which Ant is working. It also does not search for
anything outside the current directory. After
Iee055beb853cbb76a073c8060d6920d317f5bd64 os-ansible-deployment will
simply be able to use the devstack-logs publisher.

This change needs to go in first so the os-ansible-deployment gate will
unblock.

Change-Id: If94e6b97c159000c83fadc6e4dd72d56ee90d3b9
2015-03-03 08:35:26 -06:00
Timothy Chavez
c5b0cd6e1b Print instance UUID to log
This macro allows a job running on a cloud-based instance to print the
UUID of that instance to the build log.  By the time it is determined
that a test failed due to a system or networking error, the instance
has already been deleted.  The UUID can be used to locate the physical
system the instance ran on so that analysis can be done at that level.
By providing it in the build log, users will not have to find someone
with access to the nodepool logs to get this level of support for this
type of issue.

Depends-On: Ib84ec30b5c4c5de5d5be80cae2b4230c1aba3bc6
Change-Id: Iddcd56e2e2f2a942573a9868d1c009106b38564a
2015-03-02 18:39:20 -06:00
Ian Cordasco
193f151c3b Fix glob pattern for os-ansible-logs publisher
Neither Ant's nor infra's documentation is clear, but it appears that
keep-hierarchy shouldn't be true when also specifying a pattern that
only looks for filenames that match a pattern. Instead, just copy all of
the sub-directories and their contents within /openstack/log to ensure
all log files are copied during os-ansible-deployment gate-commit-check
jobs.

Change-Id: I0b3d8c0b81a1cd26806be824843290f4153daee0
2015-02-27 15:22:04 -06:00
Tom Cameron
d523f5160e Improve gate-check logging
Add publisher os-ansible-logs to collect openstack log files
Add publisher to os-ansible-jobs to perform log collection

Change-Id: Iffe6812f5c596fe72b2de641f9acfc3ede68b275
Closes-Bug: 1425482
2015-02-26 15:07:32 -05:00
Monty Taylor
66eaf7473b Add ansible role for running puppet
Rather than embedding ansible roles inside of puppet modules (which is
mildly more than passing strange) make the ansible role a standalone
role that can be published to ansible galaxy.

Change-Id: I79a5a21278a1268e0e0011a874ff29c6980fd334
2015-02-24 18:07:25 -05:00
Jesse Pretorius
8c37394570 Change os-ansible-deployment checks to use scripts
This commit changes the os-ansible-deployment checks to exclusively
make use of scripts which are located in the repository which is
cloned. This is done to improve flexibility - if the check needs to
change, it can be done within the script instead of submitting a review
to the project-config.

This commit also implements the commit check as a voting check across
all branches, and also changes the merge check from the lint check to
the commit check.

Change-Id: I95d4f12e374b80be70f513429a5dc04b0bdf3534
2015-02-20 14:28:01 +00:00
Jeremy Stanley
0bd51ad9cd Add job runtime system package and database setup
This is an initial step toward combining the bare and devstack nodes
into a common type, by pulling package installation and database
configuration into the job runtime. It also adds an experimental job
to test-drive these features.

Change-Id: I47bd65e91323779a251ec231c748cb848d745e08
2015-02-18 00:03:31 +00:00
Jan Klare
6ed1a5723e move from macro install-chefdk to chef-bootstrap
* removed macro 'install-chefdk' in favor of 'chef-bootstrap'
* macro 'chef-bootstrap' executes a cookbook specific bootstrap
  scipt which preconfigures the testnode (install packages and
  dependencies) before running the tests from the Rakefile with the job
  template 'gate-{name}-chef-rake'
* currently the bootstrap.sh does not exist in all cookbooks, since this
  is a experimental job and the final bootstrap.sh will be defined after
  this patch is merged and then cloned/added to all cookbooks using this
  job

Change-Id: I4d622b7b594aa185dd1165c913764aaeced9faae
blueprint: bootstrap-infra-test-nodes
2015-02-09 20:12:16 +01:00
Jenkins
c79f59412c Merge "Update chef-rake jobs to use chefdk and run on trusty" 2015-02-06 06:23:27 +00:00
Jan Klare
b1fcf184a0 Update chef-rake jobs to use chefdk and run on trusty
* added macro for installing chefdk (currently with version 0.4.0-1
  since chef does not provide a 'latest' link)
* switched from using ruby-191-prep in gate-{name}-chef-rake job to installing
  chefdk which should include the whole environment needed
* changed chef-rake-test builder to execute rake inside of the chefdk
  environment (test is omitted, since its the default job)
* removed the pin on precise for the gate-{name}-chef-rake job to use
  the current default trusty instead

Change-Id: I09e7465068d467395aafccd2de027000a1ec4143
2015-02-04 09:58:43 +01:00
Clark Boylan
fbc4fb4fbd Revert "Revert "Push dg-tempest-dsvm-full logs to swift""
This reverts commit 9fe1a61101890af1b405c87ba244fb3cc044e4d9.

The original change was reverted because the virtualenv required to run
the swift upload script was not built with all dependencies. We have
since corrected the virtualenv builds and all current devstack-* images
that run these d-g jobs are up to date with properly built virtualenvs.

Change-Id: I217a2b92b80ce6cc3f286e11acea6455ec505655
2015-02-02 11:10:23 -08:00
Jeremy Stanley
23f67302d1 Merge "Revert "Revert "Start using the zuul-swift-upload venv""" 2015-01-29 16:09:30 +00:00
Jeremy Stanley
859cfef9b9 Handle missing dotfiles in static-publish-prep
Change-Id: I9e0d92c9bfa3b0691de4338a4222e30a3a3cc80d
2015-01-23 20:23:57 +00:00
Jeremy Stanley
a987a720ac Correct job parameter for static-publish-prep
Change-Id: I41567bb1c23effe48132c702e09405bf22929969
2015-01-23 19:45:38 +00:00
Jenkins
4d1ad2c1a1 Merge "Properly publish specs pages to top-level" 2015-01-23 19:09:15 +00:00
Jenkins
927573028b Merge "Moved 'rake test' out of the bundle for macro chef-rake-test" 2015-01-23 18:50:32 +00:00
Andreas Jaeger
1aec2939d0 Properly publish specs pages to top-level
The content is generated in specs/output and we need to publish it
directly in the specs directory. Move files around so that we can
publish to it since the scp publisher does not allow removal of paths.

Change-Id: Icd8af32b47c04ae207471007435c1dac45471a18
2015-01-22 20:50:41 +00:00
Marton Kiss
a90ce7d238 Added chrome to javascript selenium test
Selenium tests on Firefox 35.0 failing due to an open FF bug. Switching
to chrome solves this problem, and tests are passing. The FF bug not
affects usage, just the execution of the test environment.

This patch is related:
https://review.openstack.org/#/c/148928/

Change-Id: Iadd71846be7f64c51fb46bfb5c29f56a81c10e9e
2015-01-21 16:44:42 +01:00
Jan Klare
a7c3d44571 Moved 'rake test' out of the bundle for macro chef-rake-test
Since we are currently installing rake before we bundle and are acutally
bundling inside of the rake job, the macro should not use 'bundle exec
rake test' but 'rake test'

Change-Id: Ibce74d7527527bba0e3e167dab988745fc0ad97e
2015-01-20 09:59:54 +01:00
Jan Klare
25578620ee Added gate-{name}-chef-rake job-template
This job template is supposed to be the first try of implementing rake
for managing the tests and how they are run. Added installation of rake gem to
ruby-191-prep macro to make it available for chef-rake-test macro and
skip the usage of chef-bundler-prep (done in rake bundle_prep in
chef-rake-test). Added a new project-template chef-rake-jobs which can
be utilized by all cookbooks with rakefiles (for now only includes a
job in the experimental queue).

Change-Id: Ie3687796faa210656d0b9d929fbc9e07de0d4a91
Blueprint: rakefile
2015-01-17 02:04:21 +01:00
Jeremy Stanley
d96aacf251 Revert "Revert "Start using the zuul-swift-upload venv""
This reverts commit 48e627bf5141ee23708c9a16d37c317da64bbdd1.

Do not approve until Ie79b59e1c687cce2b435c9704959b02911f94778 and
new images are built.

Change-Id: I529956760c226ff5add7bbb80a7569a13d42dff9
2015-01-15 18:36:58 +00:00
Jeremy Stanley
48e627bf51 Revert "Start using the zuul-swift-upload venv"
This reverts commit fad2d7f7fbfce7c617d34fd8fe2522a47b26451a.

Conflicts:
	jenkins/jobs/macros.yaml

Change-Id: I398e6d689d878fbfc9755422948fd992494a1128
2015-01-15 18:35:54 +00:00
Jeremy Stanley
193465653e Correct typo in venv name for zuul-swift-upload
Change-Id: I07d1133a69d6084ffe284bee406d16827252c6e4
2015-01-15 16:46:20 +00:00
Jenkins
ca5d23bfe2 Merge "Start using the zuul-swift-upload venv" 2015-01-15 14:48:15 +00:00
Jeremy Stanley
64f23c918b Correct revoke-sudo to actually work
* jenkins/jobs/macros.yaml(revoke-sudo): Simplify the sudoers
include file deletion to not rely on a conditional check, and then
test that it actually worked. Previously, systems where
/etc/sudoers.d was non-world-readable caused it to be a silent
no-op.

Change-Id: Ie713482acbd454eeb58c3481e8b8820049daaab8
2015-01-07 21:45:44 +00:00
Joshua Hesketh
fad2d7f7fb Start using the zuul-swift-upload venv
Do this as a separate step so that slaves can rebuild in the meantime.

Change-Id: Ife20d633131a35eb9a07bec6ba875cfb00f55753
2015-01-07 10:39:02 +11:00
James Polley
aeadb73010 Always use sudo -H when pip installing
Recent versions of pip will create ~/.cache if it doesn't already exist.

If this happens while running "sudo pip" the resulting dir will be owned
by root and 700 - which breaks anything else on the system that wants to
use ~/.cache

I *think* this finds all instances of "sudo pip" in this repo and fixes
them to use -H. This should mean they always run in the right ~

Change-Id: I47ddb3b591df6ac2100f09b38c9b8a03cb1ba1ff
closes-bug: #1405732
2014-12-26 10:41:58 +01:00
Jeremy Stanley
9fe1a61101 Revert "Push dg-tempest-dsvm-full logs to swift"
This reverts commit 4bde0571f968909bd6c85bccdea95a1de6c8c7f0.

The upload script lacks the necessary requirements (the "magic"
library at least) in the system context on devstack-.* workers.

Change-Id: I358fb649bfee472d47641e2780208bac7521e0d3
2014-12-19 20:40:37 +00:00
Joshua Hesketh
4bde0571f9 Push dg-tempest-dsvm-full logs to swift
Test pushing devstack-gate logs to swift.

Change-Id: I40ddbbec1f1f7d1691607319c1ee3e218a31010b
2014-12-16 17:37:30 +11:00
Joshua Hesketh
93e4547577 Rename zuul-swift-upload jobs to use hypens
Be more consistent with other macros/jobs.

Change-Id: Ia54261e8fd4295b42bef0e2911628dddab778a42
2014-12-16 17:37:27 +11:00
Joshua Hesketh
da53e4d7e1 Create zuul_swift_upload publishers
Use postbuildscript publishers to capture jobs that have failed
by ensuring that logs are uploaded even if the build has exited.

Keep the build macros as the postbuildscript needs builders to
execute and they also aren't strictly just for use as a publisher.

Change-Id: I21463f89281911dff6efda31e754bdc7b3266e24
2014-12-09 15:01:21 +11:00
Jenkins
1e1fa7e0d7 Merge "Add Syntax job for os-ansible-deployment" 2014-12-04 18:36:40 +00:00
Jesse Pretorius
6d5f950547 Add deployment build check to os-ansible-deployment
This patch inserts an all-in-one server deployment as a non-voting check. It
will later be promoted to a gate once it's confirmed to be working as required.

Change-Id: I310bdf13e7c62c8f81d7a0effb8a7b53f683b36a
2014-12-04 15:25:09 +00:00
Hugh Saunders
441ce697f8 Add Syntax job for os-ansible-deployment
This patch inserts a syntax check (using ansible-playbook
--syntax-check) before the existing lint job. The lint tool throws
exceptions on invalid playbooks so it makes sense to check for syntax
first.

Exclusions from syntax checking:
 * Playbooks that use vars_prompt due to ansible bug
   https://github.com/ansible/ansible/issues/9607

 * os-service-config-update.yml as it attempts to include a role based on
   a variable that isn't available at syntax check time.

Change-Id: I724bf0b8b3dd1e4bf3b1bd5a24b2c62eee912efb
2014-12-04 09:11:33 +00:00
Jenkins
8fa462e39f Merge "Switch to using run-tox.sh everywhere." 2014-12-04 01:03:49 +00:00
Justin Shepherd
d027dc6157 Updating gating jobs for os-ansible repo
* Adding ansible gate job definition
* Adding ansible-lint job definition
* Updating zuul/layout with ansible gate jobs

Change-Id: I174f082e3991e1b9c933dad2db3f882bc2f6e0c7
2014-12-03 10:55:55 +00:00
Clark Boylan
5f549caaa8 Switch to using run-tox.sh everywhere.
Stop using run-unittests.sh. It creates confusion when people create
jobs. Instead standardize on run-tox.sh. This means rewriting all the
jobs that use run-unittests.sh to use run-tox.sh.

Note that this depends on the parent change to merge first and have all
our slave images rebuilt prior to merging this change.

Change-Id: Ib81403c25485b120f48080a59f1894dc2130c05f
2014-12-01 08:36:19 -08:00
Michael Krotscheck
8157a5723b Updated use of nodejs packages.
This patch removes the manual creation of the nodejs symlink,
and replaces it with the debian nodejs-legacy package which
does exactly the same thing, except better.

Change-Id: Ifbe915a019f0360740932989a6885c972ce6fd65
2014-11-06 12:14:46 +01:00
Jenkins
d31243fe9f Merge "Add missing dependency to Horizon integration tests" 2014-10-27 12:22:31 +00:00
Julie Pichon
3d64fd743f JSHint job updates
The JSHint job was added during Juno and the style-changes-only patches
that were written to make it pass are unlikely to be backported to the
previous stable branches. Stop running it against stable/icehouse.

Now that the job is passing both on master and stable/juno, it would
make sense to make it voting to keep things that way.

Finally, make sure to update the apt-get cache before installing npm
in order to be more future-proof against future package updates.

Change-Id: I0ff4bad8af7507d039390358bd43df1348ed2e98
2014-10-24 11:41:28 +01:00