Avoid these lines in every output of the layout gate job with
whitelisting bash:
WARNING:test command found but not installed in testenv
cmd: /bin/bash
env: /home/aj/vcs/OpenStack/openstack-infra/project-config/.tox/zuul
Maybe you forgot to specify a dependency? See also the
whitelist_externals envconfig setting.
Also, order list of external commands alphabetically.
Change-Id: I4d1fe347c8006e14bc0fe3c61b3783e9039648b7
Add a new script to build release notes including translation,
we will run this script as part of each release notes build.
The new script will build releasenotes and all translations for them.
It checks for available translations and only builds for these.
The translated releasenotes are published in-tree, so for
trove-dashboard this would publish:
/releasenotes/trove-dashboard # Original untranslated content
/releasenotes/trove-dashboard/ja/ # Japanese translation
/releasenotes/trove-dashboard/ko_KR/ # Korean translation
The index file gets automatically enhanced to include
links to the translated releasenotes.
A preview of the generated release notes are at:
http://users.suse.com/~aj/horizon-releasenotes/http://users.suse.com/~aj/trove-dashboard-releasenotes/
Include the file directly in JJB so that it can be changed anytime.
For details, see:
http://docs.openstack.org/infra/jenkins-job-builder/definition.html#module-jenkins_jobs.local_yaml
This needs a change for jenkins-projects-checks.py: The yaml is now an
extented yaml, use the JJB yaml loader to parse it.
For now, use a separate job gate-{name}-i18n-releasenotes-nv and add it
only to horizon and openstack-manuals so that we can evaluate that this
works as expected. A followup will add this for all jobs and the
publishing as well.
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Change-Id: Ic64d571a91bc1bd292d45c0f6ba1a67bfee75997
This adds a JJB module that validates the parameters we use for
the afs publisher. This publisher is not an actual Jenkins
publisher and only exists in Zuul v2.5's ansible launcher. Thus,
it has no XML output and is defined locally in this repo.
This also removes the run-layout script (which used to clone HEAD
of both jjb and zuul) in favor of installing these dependencies
via pip in the tox testenv. We are no longer tracking JJB master,
so pinning to the version of JJB we are using makes more sense.
Similary, it is not expected that we will need cutting edge
versions of zuul for this test in the immediate future (we may
want to revisit this when we approach zuulv3).
Change-Id: I4acd70c868cb6965d39f968b7582257b4ab21778
Check changes to the upstream URL for a repo that has option
track-upstream. The check currently is just cloning of the repository.
Since we currently have over 580 repos with track upstream, this check
compares the previous version of the file with the current one and only
tests for changed entries.
Change-Id: Ic716c685512ebe660876388d663314bf3b008b66
Release of today's PyPi module: "irc" broke the job:
gate-project-config-irc-access
This patch unpins jaraco.itertools which fixes the issue
(in fact, removes it from tox.ini since it is a transitive dep).
Change-Id: I196f31f4bf35c7faecb8fffa115fbfba5136b893
When running tox -e zuul locally, some files are not deleted which
results in strange or wrong results. Remove the generated data files
before each run.
Change-Id: Ic8bd50b4d969d27ac99d743cb0be94019691fd2b
Since projects is not used anymore (it was merged with the gerrit
environment), this can be removed now.
Change-Id: Ic9693c9660cd8ca1e29da0405b7f5704c908b778
Both gate-project-config-projects-yaml and gate-project-config-gerrit
test files in gerrit directory, both check ACLs a swell. Let's merge
those into one job since in most cases both need to be run anyway and
the name projects-yaml might be confused with
jenkins/jobs/projects.yaml.
Note that this one leaves the projects tox environment in so that
testing of this change passes.
Change-Id: I34324643c26d0bcf2d24ce1dc51250ad53f2d48f
Check the following common problems with gerrit ACL lines:
* Wrong path
* Non-existence of a referenced file
* Redundant setting of acl-config with default value
Remove redundant entries that the script found.
Add acl-config for freezer-specs since no ACL existed.
Add acl-config for vinz-webclient since no ACL existed,
this was in original change I1706332f911ad9c8887d1e40613b06cbeb5e1143.
Change-Id: I6b9fc6d2ff1cd5a1f871975608947d6fe763b500
The mass rename is done, we do not need these anymore in our repository, remove it again.
This reverts commit 246bbfe03bb5a04ee904e9d09f199dab615fd244.
Change-Id: I5caeabdde4988a987d3fd828605bf5cb73b1aefa
This reverts commit b70bbeb5c7a93a0ae0fad067359f35909dab8afe.
This is no longer needed with the dependent commit merged.
Change-Id: I22fad4995cb379009a1fac9be25dd580e1aacee5
Depends-On: Iacba1611e5c50fbccd6ae700b3456028b142fc5c
Rather than installing grafyaml from pypi, install it with
zuul-cloner so that it may be validated with the contents of
master (or even changes in review). We run grafyaml continuously
deployed from source rather than installing from pypi, so this
more closely matches what we actually want to validate.
Change-Id: I26663e5c70883b910fced6c32ae364734542b5a5
Use a single linting target for changes and run that instead of the
individual bashate and pep8 targets.
Fix name of pep8 job for project-config-example.
Create new gate-{name}-linters job-template for this.
The pep8 job was run also on project-config-examples, create a new job
for that repo. It now runs without the filters since with the amount
of changes to the repo, we do not need to optimize anything right now.
Change-Id: I2afa7931bf3e18798b95a8af33ccf5ef761c4ad5
Initial commit to validate some JJB files. This validates
for existence of required sections and where possible that no additional
ones are available.
It will catch some errors that the JJB parser misses, since JJB ignores
unknown entries.
Change-Id: I51606d7066af7d5761f09655eead48883f7f56dc
Co-Authored-By: Andreas Jaeger <aj@suse.com>
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
jaraco.itertools was released in the last 24hours.
---
jaraco.itertools-1.6.tar.gz : 2015-11-18T17:41:38
jaraco.itertools-1.6.1.tar.gz : 2015-11-18T17:52:02
---
it breaks check_irc_access with:
---
$ tools/check_irc_access.py -l accessbot/channels.yaml openstackinfra
Traceback (most recent call last):
File "tools/check_irc_access.py", line 19, in <module>
import irc.client
File "<snip>/irc/client.py", line 67, in <module>
from jaraco.itertools import always_iterable
---
Add a quick pin to unblock that gate check.
Change-Id: Ie37268d7003392e8865f6850e6345be0b6b4d5c7
This is a trivial change to keep things inline with system-config.
Additionally, this will allow a follow up patch to update pep8 to only
run when .py files changes.
Change-Id: I7981c118b2483147a145617a14a583fe80a20df7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Nodepool.yaml.erb in system-config no longer contains any secrets.
Let's move it to the project-config repo.
Add the nodepool validation jobs and tox environment, initially
as non-voting until it is boot strapped.
The version of nodepool.yaml proposed matches the version in system-config
depends-on patch. A follow-up patch will be submitted with any deltas needed
when the needed-by patch is ready to merge. Until that happens, this nodepool.yaml
file won't be used and is safe to merge.
Make both gate-project-config-nodepool and gate-system-config-nodepool
non-voting to aid in bootstrapping this change.
The nodepool logging template file and associated tool that generates the file
will remain in system-config. In the short term, updates to nodepool.yaml in
this repo may require a related change in system-config to update the logging
configuration. In the longer term, nodepool will be updated to automatically
log image creations without needing a customized logging configuration.
Needed-By: I2b45a7145805368b1598d3a3e8a94f0e4eb8cf2d
Depends-On: Ib045c104daac15360e62f46e3e5683c7a387bc96
Depends-On: I89207d100eb4b6bbb502a6ed38831f49e4b29096
Change-Id: I473a1b78acdb035eb379394a7ed5f771434dc942
This checks out the upstream git projects (unless we use track-upstream)
and prints some information about it.
Note this checks for all upstream repositories but since those are
cleaned up regularly, this should be fine.
It fails if the there is no remote master branch.
Reindent the original code as main function.
Change-Id: Ie469905f48e4bc1a408fb9e731f95b2b735a6a02
With the dependent change, this should be requested by
diskimage-builder itself.
Change-Id: Id45c0dfcae86672e4a0c6cdc7c057d9aa6d291b1
Depends-On: I3052f860cebc66844cd21f34be64aea8474d2248
This takes a list of projects from stdin in the format of
aeromancer
anvil
blazar
and renames stackforge/aeromancer, stackforge/anvil, and stackforge/blazar
to openstack/aeromancer, openstack/anvil, and openstack/blazar respectively
in gerrit/projects.yaml, gerritbot/channels.yaml, zuul/layout.html,
and every file in jenkins/jobs/ , as well as renaming acl files.
A side effect of this process is that comments will be stripped from
gerritbot/channels.yaml ; if you are committing the results of a run of
this script, please re-add the comments manually unless we come up with
a more elegant solution.
This script also has test coverage for verification and documentation.
Also included is a WIP list of renames in data/stackforge-renames, as
per
http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2015-10-13.log.html#t2015-10-13T20:10:17
and a WIP list of retirements in data/stackforge-retirements, which
is not used by the script in this change.
Co-Authored-By: Augustina Ragwitz <aragwitz+lp@pobox.com>
Change-Id: Iaf2a13d8b9b9a840d9fde07ec1ef65d1c9e90e8e
This is our first grafyaml dashboard which reproduces the current
graphs on status.o.o/zuul rendered by graphite. While we aren't
actually running grafyaml upstream yet, we can get started on building
our dashboards.
We also added a tox job to properly gate on the configuration.
Change-Id: Ia738bcb510e146ab38566f0c13ff483ec618a6ed
Depends-On: I16b9affd4402fe5d1637238a2e27f22fdd3986ff
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Add a "dib" tox environment to run "dib-lint", the linting tool
diskimage-builder runs over its included elements.
Add this as a non-voting job for now; there are issues in dib-lint and
our elements to sort out
(e.g. Ibba1dadb9e819f94294c9d583b83ff698252f93f, as a start).
Change-Id: I28459a9bbaaa6175d03dbe753498849a75942538
Ensure that the output of tools/normalize_channels_yaml.py matches
gerritbot/channels.yaml modulo any comments.
Change-Id: I3147cb094a795aeeffcceda182d065ed5145c2dd
Depends-On: Ia710997f37e8fba4a18b26577ecac42dd3137e2a
Since both pep8 and bashate are voting, we can merge the two lint
targets into the "common" pep8 lint target. There's no need for a
separate bashate run.
This change is the second part for this.
Since the jobs are changed, we can now remove the obsolete bashate
tox.ini target.
Change-Id: I572ba3fe8b3a9a05f5239a858d2f701bfd8f684d
This is a partial cargo-culting of the index job and associated
files from the specs.o.o site, to generate a simple top-level index
page for the http://docs.openstack.org/infra/ tree.
Change-Id: Idb5a8dba58a53c7c8ea1de69e9a4d9b5ef410c78
* Fix print syntax in check_valid_gerrit_projects.py
* tox.ini: jenkins-project env works also on Python 3, no need to force
Python 2.7
Change-Id: I3c643470c6b5db666aeab208c615382719f8768d
This is the first part of merging pep and bashate jobs: Remove
bashate job from project-config queues.
Since both pep8 and bashate are voting, we can merge the two lint
targets into the "common" pep8 lint target. There's no need for a
separate bashate run.
The second part will remove the bashate job from tox.ini. This cannot
be done as part of this change since it would break gating.
Change-Id: I9522b2b7478590bf41dc1dab763de01d38b70404
Add new test that checks that channels mentioned in gerritbot are also
in accessbot.
Enhance the irc tox target so that it runs the checks, and update Zuul
so that the job is run for changes to gerritbot.
Add channels openstack-mistral and puppet-openstack to accessbot,
openstackinfra now has access to the channels. These two channels are
already in gerritbot.
Make check_irc_access.py script executable.
Change-Id: Ic68ba9a1c60854e68817fcadb1a5df2144944af7
Use a YAML file listing the projects to generate the specs.openstack.org
root page and an OPML file for all of the available RSS feeds.
These changes are based on the work of Maish Saidel-Keesing
<maishsk@gmail.com> in https://review.openstack.org/#/c/140041/
Change-Id: I828d65f10fb7608b59e3561cedc9553d4d96ee75
This patch adds a check for Gerrit ACL configs. In this initial
version, the test being added checks that the config defines
at least one team that acts as a core/admin one.
The current check is intentially pretty loose to ensure some
flexibility.
Currently the check applies to stackforge and openstack-dev only.
Change-Id: I2b04f2d94130699d6fd1c16a9b00b9c81d1ba07d
Fix the python interpreter used by layout-checks.py so that the "tox -e
zuul" target works correctly on a local system.
Add the PyYAML dependency needed by layout-checks.py to the tox
configuration for the zuul target.
Add the zuul target to the default list for tox.
Change-Id: Ic4db9fc3bb236d36365abfe966eb9e7769896011
Rename check_upstream_url_scheme.py to check_valid_gerrit_projects.py
and extend the check so that:
* It shows all errors, not only the first one
* It checks that the keywords are valid
This was triggered by seen twice in a week the keyword "description"
misspelled.
Script now outputs in error case:
Error: Upstream URLs should use a scheme in ['https://', 'http://', 'git://'],
found git@://github.com/emonty/cookiecutter-openstack.git in
openstack-dev/cookiecutter
Error: Unknown keyword 'decription' in project stackforge/ec2-driver
Found 2 error(s) in /tmp/projects.yaml
Also add the usual license header.
Change-Id: I334dcadc6fd4472fb28d379aec317c1a16dee2e2
Add jenkins-project to the list of envs used when "tox" is run locally
without arguments. This should make it easier for someone working on a
change to test locally before submitting the patch.
Change-Id: I5f2732daa2509ff378429959bd15d0f2f7f98692
Example output of test:
$ tools/jenkins-projects-checks.py
Checking section 'OpenStack server projects'
Checking section 'OpenStack client projects'
Checking section 'oslo libraries'
Checking section 'Other OpenStack projects'
Wrong alphabetical order: tempest, qa-specs
Checking section 'OpenStack API projects'
Checking section 'OpenStack documentation projects'
Checking section 'OpenStack development projects'
Checking section 'OpenStack infrastructure projects'
Checking section 'Stackforge projects'
Found errors in jenkins/jobs/projects.yaml!
This patch fixes the order so that it passes.
It also creates a new job to test this.
Change-Id: I36fd522261363494f07b57e40f0d9b943cf08841
Also create tools/run-bashate.sh to wrap a find command to search for
files to run bashate on, and use it in tox.
Change-Id: I5ab07425f566a556f158a4b76b00aa35f9fb4385
Check that each project (with exception of z/tempest) has a merge-check
template.
Check projects - so far stackforge only - for alphabetical order.
The merge template check is taken from a test that Jeremy shared.
Change-Id: I2bc457ddfe0eca787bb88ceeb919f89d491978d7
Co-Authored-By: Jeremy Stanley <fungi@yuggoth.org>
Adjust tox environment, the check_projects_yaml_alphabetized.sh script
only works with gerrit/projects.yaml but not with layout.yaml, so remove
the call.
Change the job to use tox -e zuul.
Change-Id: I70d918ec1de94c6ec9aa0c2c6647aa8c791df303
