Merge "Pseudo translation tool"

doc/source/contributing.rst

@@ -88,6 +88,7 @@ Once you've made your changes, there are a few things to do:
 
 * Make sure the unit tests pass: ``./run_tests.sh``
 * Make sure PEP8 is clean: ``./run_tests.sh --pep8``
+* Make sure your code is ready for translation: ``./run_tests.sh --pseudo de`` See the Translatability section below for details.
 * Make sure your code is up-to-date with the latest master: ``git pull --rebase``
 * Finally, run ``git review`` to upload your changes to Gerrit for review.
 
@@ -124,6 +125,42 @@ The community's guidelines for etiquette are fairly simple:
   a piece of code, it's polite (though not required) to thank them in your
   commit message.
 
+Translatability
+===============
+Horizon gets translated into multiple languages. The pseudo translation tool
+can be used to verify that code is ready to be translated. The pseudo tool
+replaces a language's translation with a complete, fake translation. Then
+you can verify that your code properly displays fake translations to validate
+that your code is ready for translation.
+
+Running the pseudo translation tool
+-----------------------------------
+
+#. Make sure your English po file is up to date: ``./run_tests.sh --makemessages``
+#. Run the pseudo tool to create pseudo translations. For example, to replace the German translation with a pseudo translation: ``./run_tests.sh --pseudo de``
+#. Compile the catalog: ``./run_tests.sh --compilemessages``
+#. Run your development server.
+#. Log in and change to the language you pseudo translated.
+
+It should look weird. More specifically, the translatable segments are going
+to start and end with a bracket and they are going to have some added
+characters. For example, "Log In" will become "[~Log In~您好яшçあ]"
+This is useful because you can inspect for the following, and consider if your
+code is working like it should:
+
+* If you see a string in English it's not translatable. Should it be?
+* If you see brackets next to each other that might be concatenation. Concatenation
+  can make quality translations difficult or impossible. See
+  https://wiki.openstack.org/wiki/I18n/TranslatableStrings#Use_string_formating_variables.2C_never_perform_string_concatenation
+  for additional information.
+* If there is unexpected wrapping/truncation there might not be enough
+  space for translations.
+* If you see a string in the proper translated language, it comes from an
+  external source. (That's not bad, just sometimes useful to know)
+* If you get new crashes, there is probably a bug.
+
+Don't forget to cleanup any pseudo translated po files. Those don't get merged!
+
 Code Style
 ==========
 
@@ -180,7 +217,7 @@ Required
   fragment and then append the fragment to the DOM in one pass instead of doing
   multiple smaller DOM updates.
 * Use “strict”, enclosing each JavaScript file inside a self-executing
-  function.  The self-executing function keeps the strict scoped to the file,
+  function. The self-executing function keeps the strict scoped to the file,
   so its variables and methods are not exposed to other JavaScript files in
   the product.
 
@@ -256,7 +293,7 @@ Required
        elements if dynamic content is required.
 
     3. Avoid using classes for detection purposes only, instead, defer to
-       attributes.  For example to find a div:
+       attributes. For example to find a div:
       .. code ::
 
        <div class="something"></div>
@@ -322,7 +359,7 @@ Required
   * Controllers and Services should not contain DOM references. Directives
     should.
   * Services are singletons and contain logic independent of view.
-* Scope is not the model (model is your JavaScript Objects).  The scope
+* Scope is not the model (model is your JavaScript Objects). The scope
   references the model.
 
   * Read-only in templates.
@@ -349,7 +386,7 @@ Required
   ways to do it.
 
   * Using ``gettext`` or ``ngettext`` function that is passed from server to
-    client.  If you're only translating a few things, this methodology is ok
+    client. If you're only translating a few things, this methodology is ok
     to use.
 
   * Use an Angular directive that will fetch a django template instead of a

run_tests.sh

@@ -18,6 +18,7 @@ function usage {
   echo "  --makemessages           Create/Update English translation files."
   echo "  --compilemessages        Compile all translation files."
   echo "  --check-only             Do not update translation files (--makemessages only)."
+  echo "  --pseudo                 Pseudo translate a language."
   echo "  -p, --pep8               Just run pep8"
   echo "  -8, --pep8-changed [<basecommit>]"
   echo "                           Just run PEP8 and HACKING compliance check"
@@ -83,6 +84,7 @@ with_coverage=0
 makemessages=0
 compilemessages=0
 check_only=0
+pseudo=0
 manage=0
 
 # Jenkins sets a "JOB_NAME" variable, if it's not set, we'll make it "default"
@@ -112,6 +114,7 @@ function process_option {
     --makemessages) makemessages=1;;
     --compilemessages) compilemessages=1;;
     --check-only) check_only=1;;
+    --pseudo) pseudo=1;;
     --only-selenium) only_selenium=1;;
     --with-selenium) with_selenium=1;;
     --selenium-headless) selenium_headless=1;;
@@ -444,6 +447,17 @@ function run_compilemessages {
   exit $(($HORIZON_PY_RESULT || $DASHBOARD_RESULT))
 }
 
+function run_pseudo {
+  for lang in $testargs
+  # Use English po file as the source file/pot file just like real Horizon translations
+  do
+      ${command_wrapper} $root/tools/pseudo.py openstack_dashboard/locale/en/LC_MESSAGES/django.po openstack_dashboard/locale/$lang/LC_MESSAGES/django.po $lang
+      ${command_wrapper} $root/tools/pseudo.py horizon/locale/en/LC_MESSAGES/django.po horizon/locale/$lang/LC_MESSAGES/django.po $lang
+      ${command_wrapper} $root/tools/pseudo.py horizon/locale/en/LC_MESSAGES/djangojs.po horizon/locale/$lang/LC_MESSAGES/djangojs.po $lang
+  done
+  exit $?
+}
+
 
 # ---------PREPARE THE ENVIRONMENT------------ #
 
@@ -511,6 +525,12 @@ if [ $compilemessages -eq 1 ]; then
     exit $?
 fi
 
+# Generate Pseudo translation
+if [ $pseudo -eq 1 ]; then
+    run_pseudo
+    exit $?
+fi
+
 # PEP8
 if [ $just_pep8 -eq 1 ]; then
     run_pep8

tools/pseudo.py

@@ -0,0 +1,79 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# Copyright 2015 IBM Corp.
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+import argparse
+
+import babel.messages.catalog as catalog
+import babel.messages.pofile as pofile
+
+
+def translate(segment):
+    prefix = u""
+    # When the id starts with a newline the mo compiler enforces that
+    # the translated message must also start with a newline.  Make
+    # sure that doesn't get broken when prepending the bracket.
+    if segment.startswith('\n'):
+        prefix = u"\n"
+    orig_size = len(segment)
+    # Add extra expansion space based on recommenation from
+    # http://www-01.ibm.com/software/globalization/guidelines/a3.html
+    if orig_size < 20:
+        multiplier = 1
+    elif orig_size < 30:
+        multiplier = 0.8
+    elif orig_size < 50:
+        multiplier = 0.6
+    elif orig_size < 70:
+        multiplier = 0.4
+    else:
+        multiplier = 0.3
+    extra_length = int(max(0, (orig_size * multiplier) - 10))
+    extra_chars = "~" * extra_length
+    return u"{0}[~{1}~您好яшçあ{2}]".format(prefix, segment, extra_chars)
+
+
+def main():
+    # Check arguments
+    parser = argparse.ArgumentParser()
+    parser.add_argument('pot_filename', type=argparse.FileType('r'))
+    parser.add_argument('po_filename', type=argparse.FileType('w'))
+    parser.add_argument('locale')
+    args = parser.parse_args()
+
+    # read POT file
+    pot_cat = pofile.read_po(args.pot_filename, ignore_obsolete=True)
+
+    # Create the new Catalog
+    new_cat = catalog.Catalog(locale=args.locale,
+                              last_translator="pseudo.py",
+                              charset="utf-8")
+    num_plurals = new_cat.num_plurals
+
+    # Process messages from template
+    for msg in pot_cat:
+        if msg.pluralizable:
+            msg.string = [translate(u"{}:{}".format(i, msg.id[0]))
+                          for i in range(num_plurals)]
+        else:
+            msg.string = translate(msg.id)
+        new_cat[msg.id] = msg
+
+    # Write "translated" PO file
+    pofile.write_po(args.po_filename, new_cat, ignore_obsolete=True)
+
+
+if __name__ == '__main__':
+    main()