Background: during a cycle, we deploy OpenStack on centos7 from trunk
using RDO repositories, automatically built at each commit in OpenStack.
Currently, Puppet OpenStack CI manually promotes
puppet-openstack-integration every week, to latest URL. We would like to
automate this process with a daily periodic job so every day we would
have a promotion and we could detect which commit in OpenStack would
break our CI, and we could work on it quicker than usual, because
feedback would be daily.
This patch adds a periodic job, called
"propose-puppet-openstack-rdo-promote" that will execute
jenkins/scripts/propose_rdo_promote.sh script. The script has been
tested and already used manually.
Change-Id: I9b19a440325f609746a197676d3799aa0f60ce66
Remove wrong setup of temporary path. mktemp creates the directory in
/tmp, so no need to add current directory to it.
Change-Id: Ie8e9ed90d1bb038a877ff9caf6d2d339f04766f0
Translation proposals have been changing files in the VENV directory
since the pull_from_zanata function downloads all files and it's not
clear which directories to use. The find there is from the top level.
Move the VENV to outside of the git tree to the tmp directory - like
it's done in all other cases for jenkins/scripts/. This also allows to
simplify the upstream_translation_update script.
Change-Id: Idf344d89691189c7c9ed46ef885433d3db14dff2
Run each round of wheel building in a fresh virtualenv. This solves
needing to have pip installed into the system python3 module search
path.
Change-Id: I7c58826e12bd5747930510af0c5c8c4134c9b815
Closes-Bug: #2000593
Replace pip3 command with "python3 -m pip": the pip3 command doesn't exist on
the builder. Moreover, "python -m pip" is now recommended over "pip".
Replace also the pip command with "python -m pip".
Change-Id: Idbc6e53bf2f54a9da952fd940efb8e41e6b7c253
Related-Bug: 1582257
Currently, tox gets many prebuilt wheel packages on Python 2 which
make the creation of the Python 2 virtual environments fast. But
Python 3 requires to build much more dependencies and so the creation
of the Python 3 virtual environments is slower.
This change builds also wheel packages on Python 3.
Change-Id: I60c5b2c55eaf3145c3028ede1cf93a76e766b587
Related-Bug: 1582257
Complete a script to generate a clean Puppetfile, by adding a missing
$DIR so we generate Puppetfile with all modules.
Change-Id: I0d760e155f8b45aaab579323da5446a143e4f249
The script was generating a puppetfile in the workspace, and not in the
puppet-openstcak-integration, so git didn't find any diff after the
script run.
This patch aims to generate the Puppetfile in the right directory.
Change-Id: Iaac200fa5a520f79e102171a2a1b02ca2b3c3801
For testing purposes use the "test" profile to install packages. This
installs also any packages that have no selector.
Change-Id: Ia1890185d2cded484de5ceb6a4b4001510aa33da
Some projects like horizon have several modules and each contains the
same pot file, just in different locations:
* horizon/locale/djangojs.pot
* horizon/locale/django.pot
* openstack_dashboard/locale/djangojs.pot
* openstack_dashboard/locale/django.pot
so, the previous logic copied all in the exact same directory, just
resulting only in one of each - and not both.
Change the logic to use exactly same path for publishing. This is also
the path that is shown for each file in Zanata, so it's easy to
recognize and consistent accross git repository, Zanata and
tarballs.openstack.org.
Instead of a final find, run find only in the modules that we handle.
So, files are published to:
tarballs.openstack.org/translation-source/horizon/master/horizon/locale/djangojs.pot
etc
Change-Id: I2b5f6383b2d918ffdbbf4adc1dcccd06d721c65f
We want to publish to
tarballs.openstack.org/translation-source/PROJECT/VERSION . Correctly
construct path for it.
Change-Id: I9802f50cae5dd461f76d8ba016a5f404fe74f4f3
Since we publish the source translation files (*.pot) now at
tarballs.openstack.org, we can remove the storing of source translation
files in git completly.
Note that we still need to generate pot files - to check percentage of
translated files.
Change-Id: Ia9b3a65fbb902c2644e81d7567f5e5e166a66b6a
python setup.py --version fails on tacker repo since it has an extra
setup_hook. Instead of aborting the script, use "unknown" as version.
Change-Id: If67cd1c41e006a8505683a004c7cbce93064b3a1
When looking at the console log it's more human friendly to sort this output
then for example all the oslo libraries are togther.
Change-Id: Iefe9184c9746e0c6ad487418ceda578bf7b363ff
In Puppet OpenStack CI, we use r10k to clone Puppet OpenStack modules
and all their dependencies from different places.
The Puppetfile is useful for both unit and functional tests, so we have
a consistency in what we test.
Until now, we were managing this Puppetfile without automation.
From time to time, we were updating the file but it does not scale,
regarding the work that needs to be done and the amount of dependencies
that never stop to grow up.
We recently implemented a script that is called
generate_puppetfile.sh (see dependency). This script aims to generate a
Puppetfile by cloning OpenStack & External modules.
The script will fetch tags and take the most recent one.
The script works as it is now, please look:
https://review.openstack.org/300696
And the Puppetfile that is generated.
This patch is an attempt to automate the run of this script, by using
the periodic job that is already in place for many other projects.
More features will come in next iterations:
* support stable branches
* add more logic in the script to detect whether or not tags exist (all
our current modules have tags, but we never know...)
Change-Id: Id23ba86487d2223861b21941fa04acac0d49e3f3
Instead of using a hardcoded version of Babel, let's use the constrained
version. This is still 2.2.0 right now but once it changes, it changes
here as well. We can make this change now since Babel is tested as part
of the requirements repository.
When installing horizon, use the same branch as the project uses
and use constraints here.
Test babel changes in requirements now using a tox target. The job is
only run when upper-constraints or one of the test files is changed -
and only run for master and newton or newer.
Change-Id: I15fa6a706323615d6072d8acaa859fea9bf35883
Depends-On: Iab7844ba748ecd84ec07ad4280b2317bd626fe9a
Here we are making sure our tarballs that we produce are named
puppet-foo.tar.gz. Additionally, we don't need to rename tagged
tarballs, since puppet build module does the right thing.
Change-Id: Ib16327fe436e5dd84c773be19892f3eca4a09f55
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Publish all pot files to
tarballs.openstack.org/translation-source/{name}/VERSION
This way we have always the most recent pot files available for
reviewers and can remove them from the git repositories in a followup.
Change-Id: I3c5e145115206b773466a74d63fad84da17f25e5
This reverts commit 4f96b2727c2afd3c3d6b888c21c8d18ba1cb7376.
The correct fix for this is not to stop using the feature.
Change-Id: I4b35a6ca13153257984074b5145a5e691564e0ca
Currently failures of this job leave a broken comment that doesn't
match the test-result regexes, meaning that you can't see the result
in the callout box. See [1].
I863d88e9a7ed2fd41924b8fc4a12dbea3ee2b205 fixes the root cause, but
this message is still a bit wrong because it doesn't have the
[PASSED|FAILURE] bit.
I don't really see why we need this special result at all. If it
fails, you have to read the log anyway to figure out what is wrong, so
move the message pointing to the docs to where the failure actually
happens in the logs.
[1] https://review.openstack.org/#/c/292193/
Change-Id: Ie5e7d2318a510a5811c9131d56594e856da9dcc5
We created the job for puppet-aodh first, so we could test.
Now it's in place, we would like the jobs for all modules.
Change-Id: I03e0235b0cd8f216b952c03145d158577862c0fc
drop "-tarball" in MODULE_NAME:
Because we're cloning the module in puppet-<modulename>-tarball, and we
want the tarball puppet-<modulename>-<version>.tar.gz, we need to drop
"-tarball" from the basename.
Change-Id: I77680a3ed0647c458900f605f1e8e50bf70369d3
Merge extract_messages and extract_messages_log into one function
extract_messages_python since these are always used together.
Merge propose_python and propose_django in one function propose,
call the extract_messages_* function outside of it.
Refactor python and django handling in a single function that can
be invoked twice instead of duplicating lines.
Change-Id: Ib92df685f9268e2429a119f619c1ab7e9ff1b82a
extract_messages removes the pot file if it does not contain any
translations. But then git add will fail since nothing is to add.
This currently breaks keystone proposal:
http://logs.openstack.org/periodic/keystone-propose-translation-update/2ef243f
Add a test for empty directory and exist early if it exists.
Change-Id: I3cce7f81856d579dde36445f722ba513a29fae61
Since we have now first translations of releasenotes, we can test
downloading of these. Testing barbican, nova, and openstack-manuals on
my local system found a couple of problems that are fixed with this
patch.
Refactor how we handle releasenotes, let's handle them like python and
django projects with specific functions.
Run the same cleanup on releasenotes like on other pot and po
files.
Fix download of releasenotes translations for openstack-manuals, previously
zanata-cli complained about a missing rule for these.
Remove from setup_project the special reno_resource setup, this is not
needed, the general rule works just fine.
Also, openstack-manuals does not set up the venv, so no need to remove
it.
Include minor cleanup for cleanup_pot_files.
Change-Id: I8c9a018b859b983f39c5239224bb530ce7500cb3
The repos checkout needs to be in the PYTHONPATH for the
import of the xstatic module to work. Since we invoke
the xstatic_check_version.py by absolute path, Python
does not include the cwd() in the PYTHONPATH.
Change-Id: Idd4f8db6334c9f29168e3bc39de3ed95a4e1c60f
Remove untranslated and unused po/pot files for django modules - the
same way we do for python projects.
Change-Id: I06e71e29a727a40c960b3721333dda9e0bde7b11
pybabel does not set by default PROJECT, VERSION or BUG-address - unlike
"python setup.py extract". Set these values now.
Change-Id: I889899034208f2861216b0d7944db8480b2dd7fb
For horizon, we pushed 10+ files to Zanata, since all pot files in the
VENV were pushed as well.
Remove the VENV before we push to the translation server.
Change-Id: I236af881b2fa46bf1f676ebe06513fb27accaf5f