109 Commits

Author SHA1 Message Date
Andreas Jaeger
2451b23f54 Translations: Enabling syncing again
All projects are setup again, we can sync them again.

Change-Id: I3496993cb5704faa1c74eba15e7452075d48f62f
2016-04-12 13:06:27 +02:00
Andreas Jaeger
22d7f1fcb7 Translations: Temporarily stop master sync for python projects
To allow translation of release notes, we need to change the srcdir
argument for zanata. The locale files stay the same, just the way they
are referenced is different now.

First, we disable syncing of master for projects with one module
(most projects, exceptions include horizon and keystone) temporarily.

Then, a manual update of resources is done.

And finally syncing is enabled again.

This change disables syncing and lays the ground work for how the setup
will be later:
* stable/mitaka will use old setup.
* master for *all* projects will use same settings, so this unifies
  current django and python projects, independent of number of modules.

Change-Id: I9719e256bbd213d9b9265faaf4e4e32632c5d8d7
2016-04-12 13:05:57 +02:00
Andreas Jaeger
81d8c8449f Fix Babel to 2.2.0 for now for django translation
Looking at today's django imports for horizon like
https://review.openstack.org/#/c/303695 and
https://review.openstack.org/303807 , a lot of strings are removed
from pot files - which looks wrong. There was just a Babel 2.3.2
release, let's go back to 2.2.0 that does not remove these strings.

We need some more time to investigate this and fixing the version to
2.2.0 will give us this time. I fear the translated strings are already
removed from zanata but we might be able to get them back from git.

Change-Id: I952d842e4406b33257e8d715b7ac774a5f6c4f4f
2016-04-10 21:40:50 +02:00
Jenkins
c9b0a2b24f Merge "Handle python projects with multiple modules" 2016-04-09 19:43:55 +00:00
Andreas Jaeger
ff03a350a5 Handle python projects with multiple modules
Keystone has now two modules and our extraction generates the same
content for both modules.

To confirm, check current keystone directory, it contains the same
locale files:
$ l keystone_tempest_plugin/locale/keystone_tempest_plugin.pot \
keystone/locale/keystone.pot
-rw-r--r-- 1 aj users 44265 Apr  9 19:46 keystone/locale/keystone.pot
-rw-r--r-- 1 aj users 44265 Apr  9 19:46 keystone_tempest_plugin/locale/keystone_tempest_plugin.pot

Fix the extraction so that only the module directory is scanned for input,
and not anything outside of it. This will generate in this case file
with just the boilerplate in it since the plugin is not translated.

Change-Id: I1d19d8f5172fc44b686fd9f1912af2e88a769d7e
2016-04-09 19:44:04 +02:00
Andreas Jaeger
d29de9da3c Fix horizon releasenotes translation
Currently pushing of translated release fails for horizon
since a .tox directory exists that has pot files as well.

Add an exclude rule to zanata.xml so that .tox is not considered at all.

Example failure:
http://logs.openstack.org/fd/fd92d8ec59c2f4d91f4a1c5463d95d1237f07b21/post/horizon-upstream-translation-update/f5c2061/

Change-Id: I595bd1fbdaee14698e3205eb0370435551fc8ddf
2016-04-09 19:16:41 +02:00
KATO Tomoyuki
71cd4500f2 [i18n] Fix release notes path
Since Zanata client has an option "--srcdir $modulename/locale
--txdir $modulename/locale", we need "../../" prefix.

Change-Id: If196556581c0033262330a96fee28da1dd523cf6
2016-04-08 23:46:01 +09:00
Jenkins
63c197dfd3 Merge "Enable release notes translation for python projects" 2016-04-08 08:25:41 +00:00
KATO Tomoyuki
5888cda2e1 [i18n] update reno i18n filenames for consistency
Change-Id: I9bef390cf973b91ddf88b8ace21fdeec515cabb5
2016-04-06 16:54:30 +09:00
KATO Tomoyuki
5d37267ed5 Enable release notes translation for python projects
Change-Id: I4f3b5c90488f40ef703a50eadf1c64bc7f78b6b9
Implements: blueprint release-notes-translation
2016-04-06 16:51:06 +09:00
Andreas Jaeger
92b8fab718 Fix releasenotes translation rule
releasenotes are in the top-level directory, so no need to add
ZanataDocFolder (which is doc for most projects) here. Remove
it so that translation import works.

Change-Id: I1d192098670bd149b102a2be2f1541de252c4714
2016-04-03 08:48:09 +02:00
Andreas Jaeger
5f97cd26e5 Fix releasenotes translation
sphinx-build needs to be run inside a virtualenv. Use tox to set it up.

Change-Id: I63d7a00793c7f2d68ca1f8bb3d13400b7208bda1
2016-03-31 14:24:44 +02:00
KATO Tomoyuki
abe826e956 Add release notes translation scripts
As first step, extract translation resource,
upload it to Zanata for release notes on openstack-manuals.

Change-Id: I277ab4e8424f09994ec09b1cb83b2a515a71873d
Implements: blueprint release-notes-translation
2016-03-29 13:04:33 +00:00
Akihiro Motoki
6d5bd6adc8 Translation script: honor comments for translators
Zanata supports comments for translators and Horizon previously
honors comments for translators. This can be done by passing
--add-comments Translators: option.

Change-Id: I1986d14587ef7128dc0317e37786b4d23169fb5d
2016-03-07 00:51:54 +09:00
Steve Martinelli
17c210d620 do not propose a new keystone config file if already approved
Borrowing from common_translation_update.sh, the config file
proposal should behave the same way. if there is already an
approved job, then we can wait to propose a new revision once it
has been merged.

Moved the original code that perfoms the check into common.sh
and named it check_already_approved.

Change-Id: I48180f8580a54749248a25505edbc1303457e2e4
2016-03-03 03:28:57 -05:00
Akihiro Motoki
2ce411f7f6 Translation job scripts support multiple python modules
At now, horizon repo has two python modules and we have
special handling in the translation job scripts.
This commit handles multiple python modules per repository
and locale directory will be prepared per python module.

After this we no longer need to have dedicated functions
for horizon.

Change-Id: Id32fbf37489c0056e5d7722fc3e7c71a95213e08
2016-02-22 15:00:17 +09:00
Jenkins
5d5bb7706e Merge "Cleanup translation scripts" 2016-02-03 20:56:29 +00:00
Jenkins
1bfd806738 Merge "Re-Enable django_openstack_auth/designate-dashboard translations" 2016-02-03 20:35:34 +00:00
venkatamahesh
750c8a8b31 Fix some word spellings
Change-Id: Ia95837ef10546c786c57de8b6d91ce56c10f82bb
2016-02-03 12:46:20 +05:30
Andreas Jaeger
24cadad668 Cleanup translation scripts
Remove now unused functions and rename new functions:
* Removed setup_project, renamed setup_django to setup_project
* Removed extract_messages, renamed extract_messages_new to
  extract_messages
* Removed extract_messages_log, renamed extract_messages_log_new to
  extract_messages_log.

Remove quotation marks around 0/1 for integer comparisons.

Change-Id: Iacdd8bd3a1f715bda14b402af7ba36044c345465
2016-02-03 08:04:39 +01:00
Andreas Jaeger
a97afd8b83 Re-Enable django_openstack_auth/designate-dashboard translations
Also, remove liberty jobs for these so that we do not need to backport
or handle them extra.

Backporting would break packaging by third parties, therefore not an
option.

Designate-dashboard setup is broken in liberty anyhow, so no need to
handle it extra. django_openstack_auth is good translated, and we're at
the end of translation.

Besides re-enabling the translations using the new setup, we can now
remove the old function setup_django_openstack_auth since it's not used
anymore.

Story: 2000452
Change-Id: I8b72a148d0f0e89d61807eee50a772e1fbcecaa2
2016-02-03 08:02:48 +01:00
Jenkins
221de77380 Merge "Test translation imports" 2016-02-02 23:31:48 +00:00
Andreas Jaeger
cf5ca10c1a Ignore empty django files
We do not need to handle empty django*pot files. Check in
extract_messages_django whether files are empty and remove them.
This is the same logic that is done in extract_messages.

Story: 2000452
Change-Id: I8e36609d7286dc90393f02b0e7bf229a2e1f4420
2016-02-02 10:45:08 +01:00
Andreas Jaeger
aa3d342839 Test translation imports
Test imported files whether those are valid (can be compiled by
gettext) and do not propose them if they are invalid.

If there are invalid files, a proposal of valid files will still happen.
The script will fail at the end so that these invalid files can be
detected.

Change-Id: Ia48181bc79903dd47be054a6f7c7ee535506d282
2016-01-30 17:05:26 +01:00
Akihiro Motoki
f49e0112a4 Prepare new translation functions for python projects
python-neutronclient and python-novaclient repos should work with
the new scripts
(though the current locale files still exist in the repo).
The current script will be replaced after the migration completes.

This script supports $modulename/locale/$modulename.pot as POT filename.

Part of infra-spec Improve Translation Setup

Story: 2000452
Change-Id: I67bfab533a8fd48f038c10e82509a2e626205a33
2016-01-28 13:52:20 +01:00
Akihiro Motoki
db00502c88 magnum-ui use generic django translation processing
New generic django translation functions can be used with magnum-ui.
magnum-ui specific functions are no longer required.

Also add a note on django_openstack_auth.

Change-Id: I71a1bc78612adefa5f81b89d9fe7e0a1517e08d5
2016-01-26 06:29:52 +09:00
Andreas Jaeger
8af2b26682 Add new django extraction
- Create a new extract_django function that can be used
  for all django repositories.
- Create a new setup_django function to setup translation server
  for django repositories.
- Create a new propose_django function to propose translations.
- Extra a module name from setup.cfg.

Use these functions for murano-dashboard.

Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Change-Id: I75c61ca709e05740c8932218b132752828b90b91
Story: 2000452
2016-01-26 06:22:55 +09:00
Andreas Jaeger
edcac0194f Make translation scripts more robust
In case of new projects, the locale directory might not exist. Create it
so that the jobs do not fail.

Change-Id: I9b8af9ae61a026d351d00ae93f1d74877e5925b2
2016-01-24 19:55:24 +01:00
Andreas Jaeger
f5145cf8ee Unify usages of checking for existing changes (2/2)
Use function setup_commit_message everywhere to avoid duplication of
these lines.
Also, document the side effects of the function in the invocation.

Change-Id: I3b77a232467b536b62b5c61d30705deeb11e725e
2016-01-08 09:28:38 +01:00
Andreas Jaeger
b0ea5a3bbe Unify usages of checking for existing changes (1/2)
Create new function setup_commit_message in common.sh that sets the
commit message after checking for its existance. This check is
currently done in four places, the code has been copied around for this.
Let's unify this so that we only have a single piece of this code block.

For this change, use  the new function only for translation update.
The next change will update all other places. We're doing this in steps
in case of problems.

Change-Id: I80c7f81b0b7f225fbf1e1564239f40b5df8537d9
2015-12-27 20:47:06 +01:00
Jenkins
dd9b59848e Merge "Fix two errors in setup_training_guides" 2015-11-25 10:46:16 +00:00
Steve Kowalik
8f1d418735 Fix two errors in setup_training_guides
setup_training_guides was written assuming that the project would be
passed as the first argument, but it wasn't called like that. Change
the function to match others to hardcode the project, and look for the
version as the first argument.

Correct the path to the locale directory so that the Zanata client can
read and write translation files correctly.

Change-Id: Iacfd2e831d18af5870fc70a2a54e569d0924474e
2015-11-24 19:04:52 +11:00
Andreas Jaeger
d69c742245 Really skip directories for translation sync
This fixes a problem on stable/liberty for openstack-manuals where only
2 out of 12 directories should be translated. But we currently push all
12 source files to the server.

If we have directories that contain locale files, we need to explicitly
exclude them. Add the skip directories to the exclude list for
manual projects.

Change-Id: I1eb481772545261c293baa0eb35731a8ac92d53c
2015-11-23 09:31:34 +01:00
Steve Kowalik
61ff25ede0 Drop now unused PERC variables
Two functions defined in common_translation_update set $PERC variables,
which are no longer referenced since Transifex support was dropped,
remove them.

Change-Id: I32044abffca0321d31f21f63b2f97d5e78fe8dce
2015-11-23 13:47:56 +11:00
KATO Tomoyuki
cc392a5a8f Add training-guides translation jobs and scripts
Change-Id: I129197fa9a711d2d1ee6b6b24889331b96af1514
2015-11-16 07:40:39 +00:00
Jenkins
b5338c2202 Merge "Handle "No changes" in translation scripts" 2015-11-03 20:00:36 +00:00
Andreas Jaeger
4a7888b812 Handle "No changes" in translation scripts
Some translation proposal jobs currently fail since there are no changes
since the previous run like
https://jenkins.openstack.org/job/django_openstack_auth-propose-translation-update/32/console

Handle the situation to easily figure out real failures.

Change-Id: I1f1dc9b73578ee95e5a964c30313fe0421dbf90a
2015-11-03 07:09:42 +01:00
Steve Kowalik
00ad7446f1 Properly deal with magnum-ui for Zanata
magnum-ui is a special snowflake in that its project name is magnum-ui,
but the directory base for source and translations is magnum_ui. Since
it's a Django project, it also isn't treated like a general Python
project anyway, so treat it like django_openstack_auth, by performing
a little refactoring.

Since the repo now contains files, and is marked for translations in
projects.yaml anyway, enable it for translations jobs.

Change-Id: I7eb73181cccae6d4bfb81848d39099d3502a044c
2015-10-23 16:43:29 +11:00
Andreas Jaeger
3e7931bef3 Ignore empty log translation source files
We do not need to store any empty source translation files in git or in
Zanata. There are many empty LOG source translation files, so let's
remove them.

Note we always keep the "main" files, the "main" files should always
have content.

The change tests after generation of LOG source translation files
whether these are empty and deletes them from filesystem and git.

Change-Id: I0957c66fca5aa5ded93c858766a4c0a93f963032
2015-09-28 19:32:23 +02:00
Andreas Jaeger
e271cf246f Translations: Ignore Generated-By changes
Today's import includes some files where only the Generated-By line
changes since Babel updated from 2.0 to 2.1.1. Add line to blacklist so
that a review is not generated for files with just this single line
changed.

Change-Id: I8bd477882ca621ece95faa90b4edf2da39925785
2015-09-26 08:51:57 +02:00
Jenkins
463a2458ef Merge "Translations: Merge pull_from_zanata" 2015-09-25 18:49:05 +00:00
Clark Boylan
624832c5fa Make translation pushes to gerrit branch aware
To support multiple translation versions in Zanata we need to make the
reverse push into Gerrit branch aware as well. Do this by setting up the
existing propose translation jobs to accept a branch parameter which
tells them which version to pull from Zanata and where to push into
Gerrit.

This will add a new periodic jenkins job for each release/branch.

Change-Id: Ia206bfaa7dd6eaad6aa1f6e5ddbbbf9cdadf8db0
2015-09-23 10:35:01 -07:00
Andreas Jaeger
f40bcd2213 Translations: Merge pull_from_zanata
Merge the two pull_from_zanata functions. We do not need the special
50 % rule for manuals anymore.

Change-Id: I80ed31b5d59020ece3abcfc4bd9e6f8145dd566a
2015-09-23 08:40:02 +02:00
Clark Boylan
8ce91496d6 Support different translations versions
Add support for different translations versions but default to master.

Change-Id: I1c8f9d39496908ae3c2f7245602a5f95e2f5652d
2015-09-22 13:59:10 -07:00
Andreas Jaeger
56ccfa16cf Translations: Refaktor
Create extract_messages function that can be used by both python and
django_openstack_auth repositories.

This enables the extra keywords for django_openstack_auth as well.

Change-Id: Ica998700215db5a0eae6aa88f4462ae416af2bdd
2015-09-22 20:08:36 +02:00
Andreas Jaeger
3e9acdb725 Remove Transifex comments and special cases
Update comments for switch to Zanata.
Remove special handling for glossary that was needed by transifex and is
now a nop.

Change-Id: I74b40d15b652de036dd94a0c024aeee828d31236
2015-09-18 21:34:22 +02:00
Andreas Jaeger
0f78e7bf19 Remove usage of transifex client
Remove all invocations of transifex client "tx".

This removes also the script populate-zanata-project.sh that is not
needed anymore, since we have switch to Zanata and started translating
there. The script is broken with the removal of "tx" support in other
places and not needed anymore.

Change-Id: I74350c92da425a6bf32ea1debd217a531dc2f400
2015-09-18 21:29:59 +02:00
Jenkins
ae215070fd Merge "Remove support for pulling from Transifex" 2015-09-18 18:16:12 +00:00
Elizabeth K. Joseph
ddb85c41b7 Exclude glossary from translation updates
With the exception of openstack-manuals, we don't want glossary
changes proposed to Gerrit. Adjust common_translation_update.sh
to add the glossary directories to the excluded files.

Change-Id: Id732c048200f241f160188969737bd38b679ecf6
2015-09-16 15:11:42 -07:00
Andreas Jaeger
8278e18624 Translation update: Ignore X-Generator
Add X-Generator header to ignore list: If this variable changes, there's
no need to update the translations.
Refactor to write regex only once.

Change-Id: Ic4d1ef140c1419bf14e43bd89b371d8324c79193
2015-09-16 18:53:04 +02:00