89 Commits

Author SHA1 Message Date
Andreas Jaeger
a97afd8b83 Re-Enable django_openstack_auth/designate-dashboard translations
Also, remove liberty jobs for these so that we do not need to backport
or handle them extra.

Backporting would break packaging by third parties, therefore not an
option.

Designate-dashboard setup is broken in liberty anyhow, so no need to
handle it extra. django_openstack_auth is good translated, and we're at
the end of translation.

Besides re-enabling the translations using the new setup, we can now
remove the old function setup_django_openstack_auth since it's not used
anymore.

Story: 2000452
Change-Id: I8b72a148d0f0e89d61807eee50a772e1fbcecaa2
2016-02-03 08:02:48 +01:00
Jenkins
221de77380 Merge "Test translation imports" 2016-02-02 23:31:48 +00:00
Andreas Jaeger
cf5ca10c1a Ignore empty django files
We do not need to handle empty django*pot files. Check in
extract_messages_django whether files are empty and remove them.
This is the same logic that is done in extract_messages.

Story: 2000452
Change-Id: I8e36609d7286dc90393f02b0e7bf229a2e1f4420
2016-02-02 10:45:08 +01:00
Andreas Jaeger
aa3d342839 Test translation imports
Test imported files whether those are valid (can be compiled by
gettext) and do not propose them if they are invalid.

If there are invalid files, a proposal of valid files will still happen.
The script will fail at the end so that these invalid files can be
detected.

Change-Id: Ia48181bc79903dd47be054a6f7c7ee535506d282
2016-01-30 17:05:26 +01:00
Akihiro Motoki
f49e0112a4 Prepare new translation functions for python projects
python-neutronclient and python-novaclient repos should work with
the new scripts
(though the current locale files still exist in the repo).
The current script will be replaced after the migration completes.

This script supports $modulename/locale/$modulename.pot as POT filename.

Part of infra-spec Improve Translation Setup

Story: 2000452
Change-Id: I67bfab533a8fd48f038c10e82509a2e626205a33
2016-01-28 13:52:20 +01:00
Akihiro Motoki
db00502c88 magnum-ui use generic django translation processing
New generic django translation functions can be used with magnum-ui.
magnum-ui specific functions are no longer required.

Also add a note on django_openstack_auth.

Change-Id: I71a1bc78612adefa5f81b89d9fe7e0a1517e08d5
2016-01-26 06:29:52 +09:00
Andreas Jaeger
8af2b26682 Add new django extraction
- Create a new extract_django function that can be used
  for all django repositories.
- Create a new setup_django function to setup translation server
  for django repositories.
- Create a new propose_django function to propose translations.
- Extra a module name from setup.cfg.

Use these functions for murano-dashboard.

Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Change-Id: I75c61ca709e05740c8932218b132752828b90b91
Story: 2000452
2016-01-26 06:22:55 +09:00
Andreas Jaeger
edcac0194f Make translation scripts more robust
In case of new projects, the locale directory might not exist. Create it
so that the jobs do not fail.

Change-Id: I9b8af9ae61a026d351d00ae93f1d74877e5925b2
2016-01-24 19:55:24 +01:00
Andreas Jaeger
f5145cf8ee Unify usages of checking for existing changes (2/2)
Use function setup_commit_message everywhere to avoid duplication of
these lines.
Also, document the side effects of the function in the invocation.

Change-Id: I3b77a232467b536b62b5c61d30705deeb11e725e
2016-01-08 09:28:38 +01:00
Andreas Jaeger
b0ea5a3bbe Unify usages of checking for existing changes (1/2)
Create new function setup_commit_message in common.sh that sets the
commit message after checking for its existance. This check is
currently done in four places, the code has been copied around for this.
Let's unify this so that we only have a single piece of this code block.

For this change, use  the new function only for translation update.
The next change will update all other places. We're doing this in steps
in case of problems.

Change-Id: I80c7f81b0b7f225fbf1e1564239f40b5df8537d9
2015-12-27 20:47:06 +01:00
Jenkins
dd9b59848e Merge "Fix two errors in setup_training_guides" 2015-11-25 10:46:16 +00:00
Steve Kowalik
8f1d418735 Fix two errors in setup_training_guides
setup_training_guides was written assuming that the project would be
passed as the first argument, but it wasn't called like that. Change
the function to match others to hardcode the project, and look for the
version as the first argument.

Correct the path to the locale directory so that the Zanata client can
read and write translation files correctly.

Change-Id: Iacfd2e831d18af5870fc70a2a54e569d0924474e
2015-11-24 19:04:52 +11:00
Andreas Jaeger
d69c742245 Really skip directories for translation sync
This fixes a problem on stable/liberty for openstack-manuals where only
2 out of 12 directories should be translated. But we currently push all
12 source files to the server.

If we have directories that contain locale files, we need to explicitly
exclude them. Add the skip directories to the exclude list for
manual projects.

Change-Id: I1eb481772545261c293baa0eb35731a8ac92d53c
2015-11-23 09:31:34 +01:00
Steve Kowalik
61ff25ede0 Drop now unused PERC variables
Two functions defined in common_translation_update set $PERC variables,
which are no longer referenced since Transifex support was dropped,
remove them.

Change-Id: I32044abffca0321d31f21f63b2f97d5e78fe8dce
2015-11-23 13:47:56 +11:00
KATO Tomoyuki
cc392a5a8f Add training-guides translation jobs and scripts
Change-Id: I129197fa9a711d2d1ee6b6b24889331b96af1514
2015-11-16 07:40:39 +00:00
Jenkins
b5338c2202 Merge "Handle "No changes" in translation scripts" 2015-11-03 20:00:36 +00:00
Andreas Jaeger
4a7888b812 Handle "No changes" in translation scripts
Some translation proposal jobs currently fail since there are no changes
since the previous run like
https://jenkins.openstack.org/job/django_openstack_auth-propose-translation-update/32/console

Handle the situation to easily figure out real failures.

Change-Id: I1f1dc9b73578ee95e5a964c30313fe0421dbf90a
2015-11-03 07:09:42 +01:00
Steve Kowalik
00ad7446f1 Properly deal with magnum-ui for Zanata
magnum-ui is a special snowflake in that its project name is magnum-ui,
but the directory base for source and translations is magnum_ui. Since
it's a Django project, it also isn't treated like a general Python
project anyway, so treat it like django_openstack_auth, by performing
a little refactoring.

Since the repo now contains files, and is marked for translations in
projects.yaml anyway, enable it for translations jobs.

Change-Id: I7eb73181cccae6d4bfb81848d39099d3502a044c
2015-10-23 16:43:29 +11:00
Andreas Jaeger
3e7931bef3 Ignore empty log translation source files
We do not need to store any empty source translation files in git or in
Zanata. There are many empty LOG source translation files, so let's
remove them.

Note we always keep the "main" files, the "main" files should always
have content.

The change tests after generation of LOG source translation files
whether these are empty and deletes them from filesystem and git.

Change-Id: I0957c66fca5aa5ded93c858766a4c0a93f963032
2015-09-28 19:32:23 +02:00
Andreas Jaeger
e271cf246f Translations: Ignore Generated-By changes
Today's import includes some files where only the Generated-By line
changes since Babel updated from 2.0 to 2.1.1. Add line to blacklist so
that a review is not generated for files with just this single line
changed.

Change-Id: I8bd477882ca621ece95faa90b4edf2da39925785
2015-09-26 08:51:57 +02:00
Jenkins
463a2458ef Merge "Translations: Merge pull_from_zanata" 2015-09-25 18:49:05 +00:00
Clark Boylan
624832c5fa Make translation pushes to gerrit branch aware
To support multiple translation versions in Zanata we need to make the
reverse push into Gerrit branch aware as well. Do this by setting up the
existing propose translation jobs to accept a branch parameter which
tells them which version to pull from Zanata and where to push into
Gerrit.

This will add a new periodic jenkins job for each release/branch.

Change-Id: Ia206bfaa7dd6eaad6aa1f6e5ddbbbf9cdadf8db0
2015-09-23 10:35:01 -07:00
Andreas Jaeger
f40bcd2213 Translations: Merge pull_from_zanata
Merge the two pull_from_zanata functions. We do not need the special
50 % rule for manuals anymore.

Change-Id: I80ed31b5d59020ece3abcfc4bd9e6f8145dd566a
2015-09-23 08:40:02 +02:00
Clark Boylan
8ce91496d6 Support different translations versions
Add support for different translations versions but default to master.

Change-Id: I1c8f9d39496908ae3c2f7245602a5f95e2f5652d
2015-09-22 13:59:10 -07:00
Andreas Jaeger
56ccfa16cf Translations: Refaktor
Create extract_messages function that can be used by both python and
django_openstack_auth repositories.

This enables the extra keywords for django_openstack_auth as well.

Change-Id: Ica998700215db5a0eae6aa88f4462ae416af2bdd
2015-09-22 20:08:36 +02:00
Andreas Jaeger
3e9acdb725 Remove Transifex comments and special cases
Update comments for switch to Zanata.
Remove special handling for glossary that was needed by transifex and is
now a nop.

Change-Id: I74b40d15b652de036dd94a0c024aeee828d31236
2015-09-18 21:34:22 +02:00
Andreas Jaeger
0f78e7bf19 Remove usage of transifex client
Remove all invocations of transifex client "tx".

This removes also the script populate-zanata-project.sh that is not
needed anymore, since we have switch to Zanata and started translating
there. The script is broken with the removal of "tx" support in other
places and not needed anymore.

Change-Id: I74350c92da425a6bf32ea1debd217a531dc2f400
2015-09-18 21:29:59 +02:00
Jenkins
ae215070fd Merge "Remove support for pulling from Transifex" 2015-09-18 18:16:12 +00:00
Elizabeth K. Joseph
ddb85c41b7 Exclude glossary from translation updates
With the exception of openstack-manuals, we don't want glossary
changes proposed to Gerrit. Adjust common_translation_update.sh
to add the glossary directories to the excluded files.

Change-Id: Id732c048200f241f160188969737bd38b679ecf6
2015-09-16 15:11:42 -07:00
Andreas Jaeger
8278e18624 Translation update: Ignore X-Generator
Add X-Generator header to ignore list: If this variable changes, there's
no need to update the translations.
Refactor to write regex only once.

Change-Id: Ic4d1ef140c1419bf14e43bd89b371d8324c79193
2015-09-16 18:53:04 +02:00
Steve Kowalik
28eb4ce6f5 Remove support for pulling from Transifex
Switch the translation scripts to only pull from Zanata, no matter
what extra arguments they are passed on the command line. This does
not completly remove support for Transifex, which will be cleaned up
in a future commit.

Change-Id: I21bd54d4205693315da1152a310ff4faf22531a5
2015-09-16 15:53:14 +10:00
Elizabeth K. Joseph
d23e14688d Change translations proposal topic for zanata
Translations proposals are being moved to being submitted by Zanata rather than
Transifex, reflect this in the Gerrit topic.

Change-Id: I9bd7dc8c6cd761cdcd50b7ec0dcd34e57b253f47
2015-09-11 10:28:59 -07:00
Steve Kowalik
7b8769c2c9 Set DocFolder correctly for Zanata
Since Zanata has trouble mapping .pot files that don't start with ./
for projects that set trans-dir and src-dir to ., make use of a new
variable to set them correctly for all manuals projects.

Change-Id: I729633db3e76cc07e0ec3d5083c15fa73770a21c
2015-09-09 16:47:19 +10:00
Jenkins
b408a630ee Merge "Change rule for django_openstack_auth" 2015-09-09 04:03:21 +00:00
Steve Kowalik
bd1308d3e9 Change rule for django_openstack_auth
The default rule for django_openstack_auth does not match anything since
the .po files are not named the same as the .pot file, set the rule
manually.

Change-Id: I32971259531ed18141a2011621b69d12085cbe5f
2015-09-09 13:25:20 +10:00
Steve Kowalik
b8d6965fc5 Fix the rule pattern for Horizon
The rule pattern for finding .pot files within Horizon is far too
locked down, and will not match anything. Correct it to use paths
that will match.

Change-Id: I7ca45dec0ea4067d8d88a6ac99f11cf9da4a8b35
2015-09-09 13:07:31 +10:00
Steve Kowalik
f54eb1ed69 Redo function that pulls translations from Zanata
The existing function pull_from_zanata would only work for the simple
Python project case of pulling translations from Zanata -- with only
locales directory and trivial rules to apply, with no real hope of
working for Horizon or OpenStack Manuals, given the current
functionality of the Zanata CLI utility.

Rewrite it to download every translation from Zanata and perform some
checks by hand if we keep the file or remove it. This also required
refactoring a large portion of cleanup_po_files so both functions could
leverage the same code.  OpenStack Manuals treats downloaded
translations sufficiently different that a similar function was required.

This also adds full support for Horizon and OpenStack Manuals projects,
including jobs, as well as new job templates to make it easier to
transition projects to pulling from Zanata. Adding -zanata jobs for the
Python projects and django_openstack_auth is not required, they are
already present.

Change-Id: I5b9f2eca7f099c56057fc5d09b4bbfe5ea5c596d
2015-09-08 17:10:45 +00:00
Steve Kowalik
fdab16cc23 Remove errant } in create-zanata-xml for Horizon
This causes the Zanata CLI utility to print an error message about
accepted variables, and it may cause problems for the parser.

Change-Id: If46bd2baf8ee12394b61a9d505fd9f7e690f1b23
2015-09-03 14:46:53 +10:00
Steve Kowalik
029594205e Do not ignore the exit status creating Zanata XML
Since we're getting closer to switching over to Zanata, we want to
be notified of problems, and most, if not all projects do not fail
when generating the zanata.xml file, no longer mask the exit status
by using an if block.

Change-Id: I0e013df1048ab564ce72124dbdacce2065807d52
2015-08-28 14:37:16 +10:00
Steve Kowalik
b15242b47c Really add common exclusion for manuals repos
*/source/common/* will only match one path element either side, so
we need to open the glob a little wider to match correctly. It also
appears that the other manuals repos also have source/common as a
symlink, so remove the openstack-manuals guard.

Change-Id: Id6e883ec388d9d5f6d471fa8b23f8e3f2ed1885a
2015-08-27 17:25:08 +10:00
Steve Kowalik
f55193d3b9 Add common exclusion for Zanata openstack-manuals
*/source/common in the openstack-manuals repository is a symbolic
link to common-rst, which results in pushing the files twice, and
may result in confusion. Add that path to the excludes for
openstack-manuals only.

Change-Id: I6067d94d71d3d025fc81cf331c84d5b214752218
2015-08-26 16:12:35 +10:00
Steve Kowalik
770b73d35e Switch locale marker to locale_with_underscore
Zanata 3.7.1 has moved much of the locale logic to the server, which
means we can no longer make use of client side transforms, such as
map-from, so set the locale to locale_with_underscore so dashes are
replaced with underscores.

Change-Id: I0bde5b84388f2be39c85600870ec530e8b5c07a5
2015-08-20 13:39:11 +10:00
Jenkins
50eaf49461 Merge "Add support for pulling translations from Zanata" 2015-08-12 19:14:40 +00:00
Steve Kowalik
3ca25f7204 Add support for pulling translations from Zanata
To help move to Zanata, we should pull down translations from
the server. Add a function that should behave the same way as
the existing Transifex function, as well as changing two existing
scripts to call it based on a command-line parameter. Add two new
jobs with a -zanata suffix so we can switch projects one at a time.

This change deals with all of the Python projects and
django_openstack_auth. Horizon and OpenStack manuals will be dealt with
in a future patch.

Change-Id: I8003ac37aeb89d59ceeba2e75eed2f8100b82367
2015-08-06 18:12:50 +10:00
Andreas Jaeger
7268fd329d Fix glossary recognition
Fix last change to be applied at proper place.

The glossary test needs to be in the !RST case, move it over.

Change-Id: Ib629ad85edd7f57288c94056154d1359126a4eac
2015-08-05 20:32:13 +02:00
Andreas Jaeger
a9af3157bf Fix translation sync with Transifex
Transifex now reserves glossary as part of the name, so rename it (as
done in transifex) to glossary-1.

Glossary is only used for openstack-manuals, so we do this change only
at that place.

Change-Id: I6e0e678b9c19c3a5fec882ede40679a7c8b73c5a
2015-08-05 11:33:48 +02:00
Peng Wu
ce43c75f24 Update jenkins jobs for more-gettext-support blueprint
Refer URL:
https://blueprints.launchpad.net/oslo.i18n/+spec/more-gettext-support
https://review.openstack.org/#q,topic:bp/more-gettext-support,n,z

Implements: blueprint more-gettext-support

Depends-on: I258eac447ecc7b71fb02952077cf3ef3ecfe12bb

Change-Id: I493e4698cd430792395fd77325fc0efab0ea917d
2015-07-28 16:22:52 +08:00
Jenkins
d3dfbb8f88 Merge "Add excludes support for create-zanata-xml" 2015-07-14 22:00:24 +00:00
Steve Kowalik
990b4d7737 Add excludes support for create-zanata-xml
For projects such as Manuals and Horizon, the source directory in
zanata.xml is set to '.', which may have the unintended side-effect
of the Zanata client finding .po or .pot files under dot directories.
Add a new option to create-zanata-xml, and use it for those two
projects.

Change-Id: I5fc26069c288eca130a45c6e9c5e3dd00ad2c784
2015-07-13 15:24:05 +10:00
Steve Kowalik
85f99e2d18 Refactor pulling translations into a common func
In preperation for switching projects one at a time to pulling
translations from Zanata, refactor the two calls to tx pull into
a common function.

Change-Id: I21051fbdcf7bd210606333ba46e5abb0c64e7540
2015-07-09 16:45:42 +10:00