Merge extract_messages and extract_messages_log into one function
extract_messages_python since these are always used together.
Merge propose_python and propose_django in one function propose,
call the extract_messages_* function outside of it.
Refactor python and django handling in a single function that can
be invoked twice instead of duplicating lines.
Change-Id: Ib92df685f9268e2429a119f619c1ab7e9ff1b82a
extract_messages removes the pot file if it does not contain any
translations. But then git add will fail since nothing is to add.
This currently breaks keystone proposal:
http://logs.openstack.org/periodic/keystone-propose-translation-update/2ef243f
Add a test for empty directory and exist early if it exists.
Change-Id: I3cce7f81856d579dde36445f722ba513a29fae61
Since we have now first translations of releasenotes, we can test
downloading of these. Testing barbican, nova, and openstack-manuals on
my local system found a couple of problems that are fixed with this
patch.
Refactor how we handle releasenotes, let's handle them like python and
django projects with specific functions.
Run the same cleanup on releasenotes like on other pot and po
files.
Fix download of releasenotes translations for openstack-manuals, previously
zanata-cli complained about a missing rule for these.
Remove from setup_project the special reno_resource setup, this is not
needed, the general rule works just fine.
Also, openstack-manuals does not set up the venv, so no need to remove
it.
Include minor cleanup for cleanup_pot_files.
Change-Id: I8c9a018b859b983f39c5239224bb530ce7500cb3
Remove untranslated and unused po/pot files for django modules - the
same way we do for python projects.
Change-Id: I06e71e29a727a40c960b3721333dda9e0bde7b11
We use already pybabel for django projects, let's use it also for python
projects. This works around a bug in Babel 2.3.3 with keyword option
parsing and unifies on using pybabel for both python and django.
Also extract the installation of horizon so that it's only done once for
modules that have multiple django modules.
This change uses Babel 2.2.0 now but works with 2.3.3 as well.
Reenable syncing of translation.
Change-Id: Ie220fc9418ae31941d182885e44237dc7fafe10a
Babel 2.3.3 has broken all python projects. I expect that with the way
Zanata works, we will get back all strings but right now all log-level
strings are extracted in the wrong way.
Temporarily disable translations completely. We do not loose any data
due to this and win time to discuss a proper plan.
Change-Id: Iae8d6b32736b1dd9cff3142c00a27399e4862304
At now, horizon repo has two python modules and we have
special handling in the translation job scripts.
This commit handles multiple python modules per repository
and locale directory will be prepared per python module.
After this we no longer need to have dedicated functions
for horizon.
Change-Id: Id32fbf37489c0056e5d7722fc3e7c71a95213e08
If there're no translations at all, the locale directory is empty
but we try to add content and that fails like:
+ git add 'ironic_ui/locale/*'
| fatal: pathspec 'ironic_ui/locale/*' did not match any files
Do not use a wildcard, just add the directory. That way the git command
and will not fail - and the rest of the script works fine.
Change-Id: I2a67b692f437a1bced9507e66cce217765d5d913
Remove now unused functions and rename new functions:
* Removed setup_project, renamed setup_django to setup_project
* Removed extract_messages, renamed extract_messages_new to
extract_messages
* Removed extract_messages_log, renamed extract_messages_log_new to
extract_messages_log.
Remove quotation marks around 0/1 for integer comparisons.
Change-Id: Iacdd8bd3a1f715bda14b402af7ba36044c345465
The python-client, networking, and neutron repos are setup
properly again, so we can enable translation syncing again.
Depends-On: I836f7a11943a3d76fb0a119b401ead112680d7a7
Change-Id: Id2a5c00ddf46d43330e28fa60f8850c96de13f0c
Story: 2000452
Also, remove liberty jobs for these so that we do not need to backport
or handle them extra.
Backporting would break packaging by third parties, therefore not an
option.
Designate-dashboard setup is broken in liberty anyhow, so no need to
handle it extra. django_openstack_auth is good translated, and we're at
the end of translation.
Besides re-enabling the translations using the new setup, we can now
remove the old function setup_django_openstack_auth since it's not used
anymore.
Story: 2000452
Change-Id: I8b72a148d0f0e89d61807eee50a772e1fbcecaa2
Use the new translation setup for all python projects. This now uses
it for most python projects and those will just work without any change
with the new setup since for them modulename and reponame are the same.
For example for nova, both are "nova".
In addition, enable new setup for vmware-nsx and ironic-inspector - the
projects are ready now.
Note this change temporarily disables python-novaclient and
python-neutronclient - instead of making the script complicated, we wait
until all python projects are ready and then reenable all.
This also removes the old propose_python function that is not needed
anymore and renames propose_python_new to propose_python.
Remove also some outdated comments.
Story: 2000452
Change-Id: I5b89f7b168a862773282dc0400763b1a94ea9e10
All changes for olso.* have been merged and the translations renamed
on our translation server, we can now handle translations the "new" way.
Remove temporary disabling of them and use the new functions.
This also adds a missing invocation of setup_log_level_vars to the setup
of python projects. Without it, extract_messages_log_level will fail.
Change-Id: I66311726c93b54099ad149e6d06a01ec2f9b2edc
Story: 2000452
Disable syncing of translations for django_openstack_auth and
designate-dashboard until we have renamed file in zanata and the repo for it.
Change-Id: Ie2462614ce92ee09a4d84481b4df9410f57f597b
Story: 2000452
Test imported files whether those are valid (can be compiled by
gettext) and do not propose them if they are invalid.
If there are invalid files, a proposal of valid files will still happen.
The script will fail at the end so that these invalid files can be
detected.
Change-Id: Ia48181bc79903dd47be054a6f7c7ee535506d282
New django translation setup scripts can applied to all
horizon related repositories. The exceptions are horizon
and django_openstack_auth.
This fixes the failure in the re-enabled trove-dashboard
translation jobs. We should do this before re-enabling it.
Change-Id: I90015ccf531db9f62a54d5b5c022886798db73be
Story: 2000452
These repositories need to get stopped as well, since we will rename
them also.
Before we can enable them again, we need to update Zanata and the
repositories itself.
Change-Id: I9986833819566f39280f5a3944a49c8f7de9ebe5
python-neutronclient and python-novaclient repos should work with
the new scripts
(though the current locale files still exist in the repo).
The current script will be replaced after the migration completes.
This script supports $modulename/locale/$modulename.pot as POT filename.
Part of infra-spec Improve Translation Setup
Story: 2000452
Change-Id: I67bfab533a8fd48f038c10e82509a2e626205a33
Disable syncing of translations for python-* and oslo.* projects until
we have renamed files in zanata for them.
Change-Id: I511d9bebda56cdc16b769cc5c0c60e62ae00e97c
Story: 2000452
New generic django translation functions can be used with magnum-ui.
magnum-ui specific functions are no longer required.
Also add a note on django_openstack_auth.
Change-Id: I71a1bc78612adefa5f81b89d9fe7e0a1517e08d5
- Create a new extract_django function that can be used
for all django repositories.
- Create a new setup_django function to setup translation server
for django repositories.
- Create a new propose_django function to propose translations.
- Extra a module name from setup.cfg.
Use these functions for murano-dashboard.
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Change-Id: I75c61ca709e05740c8932218b132752828b90b91
Story: 2000452
In case of new projects, the locale directory might not exist. Create it
so that the jobs do not fail.
Change-Id: I9b8af9ae61a026d351d00ae93f1d74877e5925b2
So far we have updated some po files (files in a specific language) whenever
we downloaded files from our translation server. This was done
for python projects only but not for manuals and horizon - and
was broken for django_openstack_auth.
It was done to ensure that POT files (translation source) and PO files
(translated files in specific language) are in sync. But since we're
pushing to Zanata and downloading *all* translation files from there,
Zanata does the update step for us and we get all changes.
Remove update_po now, it's not needed.
Change-Id: I9a0dac230dbb8c47ae6d82aebbf7727c54f2d2cd
magnum-ui is a special snowflake in that its project name is magnum-ui,
but the directory base for source and translations is magnum_ui. Since
it's a Django project, it also isn't treated like a general Python
project anyway, so treat it like django_openstack_auth, by performing
a little refactoring.
Since the repo now contains files, and is marked for translations in
projects.yaml anyway, enable it for translations jobs.
Change-Id: I7eb73181cccae6d4bfb81848d39099d3502a044c
To support multiple translation versions in Zanata we need to make the
reverse push into Gerrit branch aware as well. Do this by setting up the
existing propose translation jobs to accept a branch parameter which
tells them which version to pull from Zanata and where to push into
Gerrit.
This will add a new periodic jenkins job for each release/branch.
Change-Id: Ia206bfaa7dd6eaad6aa1f6e5ddbbbf9cdadf8db0
Merge the two pull_from_zanata functions. We do not need the special
50 % rule for manuals anymore.
Change-Id: I80ed31b5d59020ece3abcfc4bd9e6f8145dd566a
Create extract_messages function that can be used by both python and
django_openstack_auth repositories.
This enables the extra keywords for django_openstack_auth as well.
Change-Id: Ica998700215db5a0eae6aa88f4462ae416af2bdd
Merge
jenkins/scripts/propose_translation_update_django_openstack_auth.sh,
jenkins/scripts/propose_translation_update_horizon.sh, and
jenkins/scripts/propose_translation_update_manuals.sh into
jenkins/scripts/propose_translation_update.sh.
Update Jenkins and Zuul files, we can use now the same job
in all cases.
Change-Id: Idb9e8534dfc0454d75fe186e1a4a454832c8fdae
Remove all invocations of transifex client "tx".
This removes also the script populate-zanata-project.sh that is not
needed anymore, since we have switch to Zanata and started translating
there. The script is broken with the removal of "tx" support in other
places and not needed anymore.
Change-Id: I74350c92da425a6bf32ea1debd217a531dc2f400
Switch the translation scripts to only pull from Zanata, no matter
what extra arguments they are passed on the command line. This does
not completly remove support for Transifex, which will be cleaned up
in a future commit.
Change-Id: I21bd54d4205693315da1152a310ff4faf22531a5
The existing function pull_from_zanata would only work for the simple
Python project case of pulling translations from Zanata -- with only
locales directory and trivial rules to apply, with no real hope of
working for Horizon or OpenStack Manuals, given the current
functionality of the Zanata CLI utility.
Rewrite it to download every translation from Zanata and perform some
checks by hand if we keep the file or remove it. This also required
refactoring a large portion of cleanup_po_files so both functions could
leverage the same code. OpenStack Manuals treats downloaded
translations sufficiently different that a similar function was required.
This also adds full support for Horizon and OpenStack Manuals projects,
including jobs, as well as new job templates to make it easier to
transition projects to pulling from Zanata. Adding -zanata jobs for the
Python projects and django_openstack_auth is not required, they are
already present.
Change-Id: I5b9f2eca7f099c56057fc5d09b4bbfe5ea5c596d
To help move to Zanata, we should pull down translations from
the server. Add a function that should behave the same way as
the existing Transifex function, as well as changing two existing
scripts to call it based on a command-line parameter. Add two new
jobs with a -zanata suffix so we can switch projects one at a time.
This change deals with all of the Python projects and
django_openstack_auth. Horizon and OpenStack manuals will be dealt with
in a future patch.
Change-Id: I8003ac37aeb89d59ceeba2e75eed2f8100b82367
In preperation for switching projects one at a time to pulling
translations from Zanata, refactor the two calls to tx pull into
a common function.
Change-Id: I21051fbdcf7bd210606333ba46e5abb0c64e7540
Cleanup every use of `` for subshells in the jenkins/scripts
directory, replacing them with $(), and finally making the
scripts consistent.
Change-Id: Id3e45fb873c1ebeae6ab63a0caa4c427a7ccbb62
Remove .tx file at the end to have a clean repository.
Run setup.py with parameter --quiet, the long list of "extracting
messages from file XXX" are not needed.
Change-Id: I735f02a968aed8b362803781498ccc8085fa0e03
Today's nova import had some file changes where only the
POT-Creation-Date was changed which we don't need to import.
With the new compression step, we need to filter out commits *after* the
compression, otherwise counting is wrong.
Change-Id: Ib8fe6ca5f77c805147b73fe94905c72ccf3369d4
We have some strange files in the tree that contain no or little
translation after merging - and these are downloaded from transifex.
So, these files are not known to git and git rm fails.
To not special case every git rm and git reset for both old and new
files, re-add a "git add" at the beginning so that we can operate on
files known to git.
Note: This happens when we have a file like glance-log-critical.po that has no
strings - but we had to create it with initial content in transifex,
since empty files are not allowed there.
Change-Id: Ie9bb3e59560d50829e56928b2ed79e3999dd39cb