Switch the translation scripts to only pull from Zanata, no matter
what extra arguments they are passed on the command line. This does
not completly remove support for Transifex, which will be cleaned up
in a future commit.
Change-Id: I21bd54d4205693315da1152a310ff4faf22531a5
Translations proposals are being moved to being submitted by Zanata rather than
Transifex, reflect this in the Gerrit topic.
Change-Id: I9bd7dc8c6cd761cdcd50b7ec0dcd34e57b253f47
Since Zanata has trouble mapping .pot files that don't start with ./
for projects that set trans-dir and src-dir to ., make use of a new
variable to set them correctly for all manuals projects.
Change-Id: I729633db3e76cc07e0ec3d5083c15fa73770a21c
The default rule for django_openstack_auth does not match anything since
the .po files are not named the same as the .pot file, set the rule
manually.
Change-Id: I32971259531ed18141a2011621b69d12085cbe5f
The rule pattern for finding .pot files within Horizon is far too
locked down, and will not match anything. Correct it to use paths
that will match.
Change-Id: I7ca45dec0ea4067d8d88a6ac99f11cf9da4a8b35
The existing function pull_from_zanata would only work for the simple
Python project case of pulling translations from Zanata -- with only
locales directory and trivial rules to apply, with no real hope of
working for Horizon or OpenStack Manuals, given the current
functionality of the Zanata CLI utility.
Rewrite it to download every translation from Zanata and perform some
checks by hand if we keep the file or remove it. This also required
refactoring a large portion of cleanup_po_files so both functions could
leverage the same code. OpenStack Manuals treats downloaded
translations sufficiently different that a similar function was required.
This also adds full support for Horizon and OpenStack Manuals projects,
including jobs, as well as new job templates to make it easier to
transition projects to pulling from Zanata. Adding -zanata jobs for the
Python projects and django_openstack_auth is not required, they are
already present.
Change-Id: I5b9f2eca7f099c56057fc5d09b4bbfe5ea5c596d
This causes the Zanata CLI utility to print an error message about
accepted variables, and it may cause problems for the parser.
Change-Id: If46bd2baf8ee12394b61a9d505fd9f7e690f1b23
Since we're getting closer to switching over to Zanata, we want to
be notified of problems, and most, if not all projects do not fail
when generating the zanata.xml file, no longer mask the exit status
by using an if block.
Change-Id: I0e013df1048ab564ce72124dbdacce2065807d52
*/source/common/* will only match one path element either side, so
we need to open the glob a little wider to match correctly. It also
appears that the other manuals repos also have source/common as a
symlink, so remove the openstack-manuals guard.
Change-Id: Id6e883ec388d9d5f6d471fa8b23f8e3f2ed1885a
*/source/common in the openstack-manuals repository is a symbolic
link to common-rst, which results in pushing the files twice, and
may result in confusion. Add that path to the excludes for
openstack-manuals only.
Change-Id: I6067d94d71d3d025fc81cf331c84d5b214752218
Zanata 3.7.1 has moved much of the locale logic to the server, which
means we can no longer make use of client side transforms, such as
map-from, so set the locale to locale_with_underscore so dashes are
replaced with underscores.
Change-Id: I0bde5b84388f2be39c85600870ec530e8b5c07a5
To help move to Zanata, we should pull down translations from
the server. Add a function that should behave the same way as
the existing Transifex function, as well as changing two existing
scripts to call it based on a command-line parameter. Add two new
jobs with a -zanata suffix so we can switch projects one at a time.
This change deals with all of the Python projects and
django_openstack_auth. Horizon and OpenStack manuals will be dealt with
in a future patch.
Change-Id: I8003ac37aeb89d59ceeba2e75eed2f8100b82367
Fix last change to be applied at proper place.
The glossary test needs to be in the !RST case, move it over.
Change-Id: Ib629ad85edd7f57288c94056154d1359126a4eac
Transifex now reserves glossary as part of the name, so rename it (as
done in transifex) to glossary-1.
Glossary is only used for openstack-manuals, so we do this change only
at that place.
Change-Id: I6e0e678b9c19c3a5fec882ede40679a7c8b73c5a
For projects such as Manuals and Horizon, the source directory in
zanata.xml is set to '.', which may have the unintended side-effect
of the Zanata client finding .po or .pot files under dot directories.
Add a new option to create-zanata-xml, and use it for those two
projects.
Change-Id: I5fc26069c288eca130a45c6e9c5e3dd00ad2c784
In preperation for switching projects one at a time to pulling
translations from Zanata, refactor the two calls to tx pull into
a common function.
Change-Id: I21051fbdcf7bd210606333ba46e5abb0c64e7540
Building on existing work for supporting pushing translations
to Zanata for other projects, add support for pushing translations
for the four openstack-manuals projects.
Change-Id: Ic18331b7ee402c4c89b6dacdc4edc02c3ef07cdc
It turns out that Zanata can not share a project like Transifex
does, the client gets all jealous and removes all other translation
files when you push the second project. To that end, change the
project of django_openstack_auth to its own name rather than horizon.
Change-Id: I8ddcf18f3fd2549ee0d948041bb62da7f669e76b
Building on work landed in II3bfb188c8b0c0e65f22d7edc30721b163f084fff,
add support for pushing source documents of Horizon into Zanata.
This has also necessitated adding support for parsing and adding
rules for create-zanata-xml.py.
As a drive-by correct some incorrect docstrings in ZanataUtils, split
some ludicrously long lines calling create-zanata-xml.py, and correct
usage of an undefined variable in setup_django_openstack_auth.
Change-Id: Id7a2c82e979d1f878022b4982f51ef4ef06ecf4b
Horizon uses pot files now as all other projects, adjust scripts for the
change.
Add new Javascript translation for openstack_dashboard.
Remove now unused function compress_non_en_po_files.
Change-Id: Ibd7069c68e458d655a879e552f69f7146b47abc9
Depends-On: I2ae68207994ba549b5d8e551a40a468d63e0a4bb
searchlight project had only one pot file but not the other five that
are expected. The filter_commit function was filtering out the newly
added file and run "git checkout -- NEWFILE" which failed.
Make the script more robust so that it handles newly added files
correctly.
Change-Id: I4a7a8ab9b75f310992b7232920206d4ddc041268
In order to migrate to Zanata for translations, we need to modify
the proposal scripts to also upload translations to the Zanata
server. This commit supports most projects (those that make use of
upstream_translation_update.sh), as well as django_openstack_auth.
This does not impact the existing push and pull of
translations to Transifex, and also untouched are the translations
for Horizon or the manuals, which are all handled in separate scripts.
Co-Authored-By: stephane <stephane@alum.mit.edu>
Change-Id: I3bfb188c8b0c0e65f22d7edc30721b163f084fff
This fixes a problem with propose_translation_update.sh on
oslo.concurrency:
+ git reset -q oslo.concurrency/locale/en_GB/LC_MESSAGES/oslo.concurrency-log-error.po
fatal: ambiguous argument
'oslo.concurrency/locale/en_GB/LC_MESSAGES/oslo.concurrency-log-error.po': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Add the "--" to separate paths to handle deleted files as well.
Change-Id: Ifbce50bd851ea0028ebbb948e5173bda5777fc43
Subshells are not required for if statements, so remove them. This
also drops all usage of `` in scripts.
Change-Id: I44c476ddfb67e5b3ebd3aaba3b26561f02708477
Cleanup every use of `` for subshells in the jenkins/scripts
directory, replacing them with $(), and finally making the
scripts consistent.
Change-Id: Id3e45fb873c1ebeae6ab63a0caa4c427a7ccbb62
Changes like
https://review.openstack.org/#/c/185848/1/horizon/locale/ca/LC_MESSAGES/djangojs.po
just reordered entries with no content change. Pass --sort-output to
msgattrib to filter the entries in alphabetical order so that any
rearrangements of the source files will not have an effect on the order
of the entries.
Change-Id: I2d9c91f3d1fbcf61ae21521758836793742389ed
Using the construct of x$var != x is unnecessary, and has been
unnecessary for quite some time. We trust our shell, so switch to
using test -n.
Change-Id: I8eea943e5b641ba3d8781212877da7ddec167a6c
There are a number of calls to git config in the slave scripts are
all the same, and we should use a shell function to not repeat
ourselves.
Change-Id: Ib2fa3d5ca12d5d08e2d21a734fea13aa9848b19e
When we edit the commit message for sending the patch for updating
translations, we repeat the entire text of the commit message,
rather than using the existing variable (which is fine, because the
shell will expand the variable before assigning it).
Change-Id: I2840935d2174b1931de0cb782e52cfed4c049312
Do not import a file if the Last-Translator line has changed (together
with other ignored lines).
This will ignore imports like
https://review.openstack.org/#/c/183771/1/doc/glossary/locale/ja.po
If there are other changes, including real translations, the file will
get imported as usual.
Change-Id: I6f56f64c107e4f93b92460612e656b91341a8338
Today's propose_translation_update.sh run failed for ceilometer since
files were deleted but the loop in filter_commit considered these
as zero changing lines. Change the loop to check only new and modified
files.
Change-Id: I94f5389b9fe0c16d529860f57bb63ef8a3c5c64a
Remove .tx file at the end to have a clean repository.
Run setup.py with parameter --quiet, the long list of "extracting
messages from file XXX" are not needed.
Change-Id: I735f02a968aed8b362803781498ccc8085fa0e03
Do not import files if only content is deleted but no new translation
added. If there's at least one file with added translations, files with
removed content are also imported.
This should reduce the number of imports, since a project should see an
import only if new strings are in transifex.
Change-Id: Id521a2209bf9cad3b90acf81447db027beed7b64
Remove untranslated strings and location information from PO files
for horizon and django_openstack_auth. For horizon, the
location information is still in the English PO files as reference for easy
regeneration using "msgmerge EnglishPOFILE POFILE -o COMPLETEPOFILE".
Change-Id: Ibb440320e0fd882c86ff79ec295411c3d062a521
Not all manual repositories have a doc directory, adjust the paths.
Not all files should be compressed, we need to exclude in all
repositories - besides openstack-manuals - the glossary.
Change-Id: I358db42585699b05e893b66e1ac6d2927fb43ee3
Remove untranslated strings and location information from PO files. The
location information is still in the POT files as reference for easy
regeneration using "msgmerge POTFILE POFILE -o COMPLETEPOFILE".
Our tools handle the smaller PO files just fine.
Enable this for Documentation projects.
Also, quote file name arguments in cleanup_po_files.
Change-Id: I4649314f68332d9db46a10aac0d7550bddc46d0e
locale files in RST folders are under $DOCNAME/source/locale/, fix one
error.
Also, small refactor of an condition.
Change-Id: I9a61b2d0e26bd19e2b0c1b8b95b2369530b05fcf
The OpenStack manuals are switching to some RST files. Enhance our
scripts so that they can handle RST files in manual projects.
Change-Id: Ie3e9591d2aad88aced3092aa2a8919af186ce83d
Depends-On: I3124cf65b254c1c5aefce59447bb94240d96a742