Since we do not store any pot files in git anymore, the optimization to
only push to the translation server if strings have changed, is not
possible anymore. Remove this code now, this includes removing
some git adds for pot files.
Change-Id: I28ad9edc49b0f840cd51354334d7ae08aadab87a
Add subunit testr output at end of run of the translation scripts, so
that healthcheck will have information about success/failure of the job.
This is implemented as an exit trap since we abort on every error and
need to provide the failure message.
Also, upload the healthcheck data to logs.openstack.org so that
healthcheck can import it.
This should be a temporary solution until healthcheck can get data from
Zuul about failure/success of job runs using database access.
Change-Id: I74bba9659a9767027e2daa649ec6f38bf822b65a
Translation proposals have been changing files in the VENV directory
since the pull_from_zanata function downloads all files and it's not
clear which directories to use. The find there is from the top level.
Move the VENV to outside of the git tree to the tmp directory - like
it's done in all other cases for jenkins/scripts/. This also allows to
simplify the upstream_translation_update script.
Change-Id: Idf344d89691189c7c9ed46ef885433d3db14dff2
Some projects like horizon have several modules and each contains the
same pot file, just in different locations:
* horizon/locale/djangojs.pot
* horizon/locale/django.pot
* openstack_dashboard/locale/djangojs.pot
* openstack_dashboard/locale/django.pot
so, the previous logic copied all in the exact same directory, just
resulting only in one of each - and not both.
Change the logic to use exactly same path for publishing. This is also
the path that is shown for each file in Zanata, so it's easy to
recognize and consistent accross git repository, Zanata and
tarballs.openstack.org.
Instead of a final find, run find only in the modules that we handle.
So, files are published to:
tarballs.openstack.org/translation-source/horizon/master/horizon/locale/djangojs.pot
etc
Change-Id: I2b5f6383b2d918ffdbbf4adc1dcccd06d721c65f
We want to publish to
tarballs.openstack.org/translation-source/PROJECT/VERSION . Correctly
construct path for it.
Change-Id: I9802f50cae5dd461f76d8ba016a5f404fe74f4f3
Instead of using a hardcoded version of Babel, let's use the constrained
version. This is still 2.2.0 right now but once it changes, it changes
here as well. We can make this change now since Babel is tested as part
of the requirements repository.
When installing horizon, use the same branch as the project uses
and use constraints here.
Test babel changes in requirements now using a tox target. The job is
only run when upper-constraints or one of the test files is changed -
and only run for master and newton or newer.
Change-Id: I15fa6a706323615d6072d8acaa859fea9bf35883
Depends-On: Iab7844ba748ecd84ec07ad4280b2317bd626fe9a
Publish all pot files to
tarballs.openstack.org/translation-source/{name}/VERSION
This way we have always the most recent pot files available for
reviewers and can remove them from the git repositories in a followup.
Change-Id: I3c5e145115206b773466a74d63fad84da17f25e5
Merge extract_messages and extract_messages_log into one function
extract_messages_python since these are always used together.
Merge propose_python and propose_django in one function propose,
call the extract_messages_* function outside of it.
Refactor python and django handling in a single function that can
be invoked twice instead of duplicating lines.
Change-Id: Ib92df685f9268e2429a119f619c1ab7e9ff1b82a
Since we have now first translations of releasenotes, we can test
downloading of these. Testing barbican, nova, and openstack-manuals on
my local system found a couple of problems that are fixed with this
patch.
Refactor how we handle releasenotes, let's handle them like python and
django projects with specific functions.
Run the same cleanup on releasenotes like on other pot and po
files.
Fix download of releasenotes translations for openstack-manuals, previously
zanata-cli complained about a missing rule for these.
Remove from setup_project the special reno_resource setup, this is not
needed, the general rule works just fine.
Also, openstack-manuals does not set up the venv, so no need to remove
it.
Include minor cleanup for cleanup_pot_files.
Change-Id: I8c9a018b859b983f39c5239224bb530ce7500cb3
For horizon, we pushed 10+ files to Zanata, since all pot files in the
VENV were pushed as well.
Remove the VENV before we push to the translation server.
Change-Id: I236af881b2fa46bf1f676ebe06513fb27accaf5f
We use already pybabel for django projects, let's use it also for python
projects. This works around a bug in Babel 2.3.3 with keyword option
parsing and unifies on using pybabel for both python and django.
Also extract the installation of horizon so that it's only done once for
modules that have multiple django modules.
This change uses Babel 2.2.0 now but works with 2.3.3 as well.
Reenable syncing of translation.
Change-Id: Ie220fc9418ae31941d182885e44237dc7fafe10a
Babel 2.3.3 has broken all python projects. I expect that with the way
Zanata works, we will get back all strings but right now all log-level
strings are extracted in the wrong way.
Temporarily disable translations completely. We do not loose any data
due to this and win time to discuss a proper plan.
Change-Id: Iae8d6b32736b1dd9cff3142c00a27399e4862304
At now, horizon repo has two python modules and we have
special handling in the translation job scripts.
This commit handles multiple python modules per repository
and locale directory will be prepared per python module.
After this we no longer need to have dedicated functions
for horizon.
Change-Id: Id32fbf37489c0056e5d7722fc3e7c71a95213e08
Remove now unused functions and rename new functions:
* Removed setup_project, renamed setup_django to setup_project
* Removed extract_messages, renamed extract_messages_new to
extract_messages
* Removed extract_messages_log, renamed extract_messages_log_new to
extract_messages_log.
Remove quotation marks around 0/1 for integer comparisons.
Change-Id: Iacdd8bd3a1f715bda14b402af7ba36044c345465
The python-client, networking, and neutron repos are setup
properly again, so we can enable translation syncing again.
Depends-On: I836f7a11943a3d76fb0a119b401ead112680d7a7
Change-Id: Id2a5c00ddf46d43330e28fa60f8850c96de13f0c
Story: 2000452
Also, remove liberty jobs for these so that we do not need to backport
or handle them extra.
Backporting would break packaging by third parties, therefore not an
option.
Designate-dashboard setup is broken in liberty anyhow, so no need to
handle it extra. django_openstack_auth is good translated, and we're at
the end of translation.
Besides re-enabling the translations using the new setup, we can now
remove the old function setup_django_openstack_auth since it's not used
anymore.
Story: 2000452
Change-Id: I8b72a148d0f0e89d61807eee50a772e1fbcecaa2
Use the new translation setup for all python projects. This now uses
it for most python projects and those will just work without any change
with the new setup since for them modulename and reponame are the same.
For example for nova, both are "nova".
In addition, enable new setup for vmware-nsx and ironic-inspector - the
projects are ready now.
Note this change temporarily disables python-novaclient and
python-neutronclient - instead of making the script complicated, we wait
until all python projects are ready and then reenable all.
This also removes the old propose_python function that is not needed
anymore and renames propose_python_new to propose_python.
Remove also some outdated comments.
Story: 2000452
Change-Id: I5b89f7b168a862773282dc0400763b1a94ea9e10
All changes for olso.* have been merged and the translations renamed
on our translation server, we can now handle translations the "new" way.
Remove temporary disabling of them and use the new functions.
This also adds a missing invocation of setup_log_level_vars to the setup
of python projects. Without it, extract_messages_log_level will fail.
Change-Id: I66311726c93b54099ad149e6d06a01ec2f9b2edc
Story: 2000452
Disable syncing of translations for django_openstack_auth and
designate-dashboard until we have renamed file in zanata and the repo for it.
Change-Id: Ie2462614ce92ee09a4d84481b4df9410f57f597b
Story: 2000452
New django translation setup scripts can applied to all
horizon related repositories. The exceptions are horizon
and django_openstack_auth.
This fixes the failure in the re-enabled trove-dashboard
translation jobs. We should do this before re-enabling it.
Change-Id: I90015ccf531db9f62a54d5b5c022886798db73be
Story: 2000452
These repositories need to get stopped as well, since we will rename
them also.
Before we can enable them again, we need to update Zanata and the
repositories itself.
Change-Id: I9986833819566f39280f5a3944a49c8f7de9ebe5
python-neutronclient and python-novaclient repos should work with
the new scripts
(though the current locale files still exist in the repo).
The current script will be replaced after the migration completes.
This script supports $modulename/locale/$modulename.pot as POT filename.
Part of infra-spec Improve Translation Setup
Story: 2000452
Change-Id: I67bfab533a8fd48f038c10e82509a2e626205a33
Disable syncing of translations for python-* and oslo.* projects until
we have renamed files in zanata for them.
Change-Id: I511d9bebda56cdc16b769cc5c0c60e62ae00e97c
Story: 2000452
New generic django translation functions can be used with magnum-ui.
magnum-ui specific functions are no longer required.
Also add a note on django_openstack_auth.
Change-Id: I71a1bc78612adefa5f81b89d9fe7e0a1517e08d5
- Create a new extract_django function that can be used
for all django repositories.
- Create a new setup_django function to setup translation server
for django repositories.
- Create a new propose_django function to propose translations.
- Extra a module name from setup.cfg.
Use these functions for murano-dashboard.
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Change-Id: I75c61ca709e05740c8932218b132752828b90b91
Story: 2000452
In case of new projects, the locale directory might not exist. Create it
so that the jobs do not fail.
Change-Id: I9b8af9ae61a026d351d00ae93f1d74877e5925b2
magnum-ui is a special snowflake in that its project name is magnum-ui,
but the directory base for source and translations is magnum_ui. Since
it's a Django project, it also isn't treated like a general Python
project anyway, so treat it like django_openstack_auth, by performing
a little refactoring.
Since the repo now contains files, and is marked for translations in
projects.yaml anyway, enable it for translations jobs.
Change-Id: I7eb73181cccae6d4bfb81848d39099d3502a044c
To allow us to check if a version exists for a Zanata project without
requiring us to create a zanata.xml first, add a small script that
uses the Zanata REST API to query for its existance.
Make use it instead of the zanata-cli in upstream_translation_update,
since we can't create a zanata.xml if the version doesn't exist.
Change-Id: I93a72845aa705ce6b72acec9b5d62b5565368ab0
As our release versions change keep Zanata up to date with those
changes by pushing them to specific versions in Zanata.
Note that because Zanata can't handle /'s in version names we replace
/'s with -'s and map between git and Zanata that way. We also will exit
0 for success if the current git branch does not have a corresponding
version in Zanata.
Change-Id: I806b2a01c8d4fec2d149be3647931f95b559e76f
Create extract_messages function that can be used by both python and
django_openstack_auth repositories.
This enables the extra keywords for django_openstack_auth as well.
Change-Id: Ica998700215db5a0eae6aa88f4462ae416af2bdd
Building on the excellent work done in
I54d560398daa67af8effeb794fce11e53affa1a6, collapse two case blocks
into one.
Change-Id: I9dfe79ed12fa93c30aab3e52a8357c44d1cba938
Merge upstream_translation_update_manuals.sh,
upstream_translation_horizon.sh, and
upstream_translation_django_openstack_auth.sh into
upstream_translation_update.sh.
Adjust job definition for these projects so that the new scripts
get used.
Adjust layout.yaml for change, remove now obsolete condition.
Change-Id: I54d560398daa67af8effeb794fce11e53affa1a6
Remove all invocations of transifex client "tx".
This removes also the script populate-zanata-project.sh that is not
needed anymore, since we have switch to Zanata and started translating
there. The script is broken with the removal of "tx" support in other
places and not needed anymore.
Change-Id: I74350c92da425a6bf32ea1debd217a531dc2f400
This removes the code from all four upstream_translation_* scripts
that pushes new .pot files up to Transifex, and forces the comments
around them to be in line.
This leaves Transifex setup in place for the moment, since the
propose jobs that pull from Transifex are still utilising it.
Change-Id: I702e6073f92faaa79a7aa53c44022f9e437721af
Some projects like oslo.messaging and oslo.reports have now empty PO
files. Pushing empty files to transifex results in an error.
Check before pushing of the main file whether it's empty and only
push if it's non-empty - like it's done for the log-level PO
files already.
Change-Id: Ia757e9d4b88bc8eb7b1ce800a25fe0ed1ebe3e76
Now that the self-signed certificate for the Zanata server is stored
in the default Java keystore, zanata-cli push for the translation
upstream update jobs is working nicely, so we can stop using if to
guard the non-zero exit status from the script.
Change-Id: I75d892c9fe207b19ef263bb09b9beaf25848fbaf
Running zanata-cli push on the proposal slave is turning up some
errors that require investigation. To give us some time to investigate
them, ignore errors from zanata-cli push.
Change-Id: I56a59027a9f8931e1a226383107b526fe7b90e68