Merge pull request #119 from GoogleCloudPlatform/i18n

Move sample from appengine-i18n-sample-python

appengine/i18n/README.md

@@ -0,0 +1,126 @@
+# App Engine Internationalization sample in Python
+
+A simple example app showing how to build an internationalized app
+with App Engine. The main purpose of this example is to provide the
+basic how-to.
+
+## What to internationalize
+
+There are lots of things to internationalize with your web
+applications.
+
+1.  Strings in Python code
+2.  Strings in HTML template
+3.  Strings in Javascript
+4.  Common strings
+    - Country Names, Language Names, etc.
+5.  Formatting
+    - Date/Time formatting
+    - Number formatting
+    - Currency
+6.  Timezone conversion
+
+This example only covers first 3 basic scenarios above. In order to
+cover other aspects, I recommend using
+[Babel](http://babel.edgewall.org/) and [pytz]
+(http://pypi.python.org/pypi/gaepytz). Also, you may want to use
+[webapp2_extras.i18n](http://webapp-improved.appspot.com/tutorials/i18n.html)
+module.
+
+## Wait, so why not webapp2_extras.i18n?
+
+webapp2_extras.i18n doesn't cover how to internationalize strings in
+Javascript code. Additionally it depends on babel and pytz, which
+means you need to deploy babel and pytz alongside with your code. I'd
+like to show a reasonably minimum example for string
+internationalization in Python code, jinja2 templates, as well as
+Javascript.
+
+## How to run this example
+
+First of all, please install babel in your local Python environment.
+
+### Wait, you just said I don't need babel, are you crazy?
+
+As I said before, you don't need to deploy babel with this
+application, but you need to locally use pybabel script which is
+provided by babel distribution in order to extract the strings, manage
+and compile the translations file.
+
+### Extract strings in Python code and Jinja2 templates to translate
+
+Move into this project directory and invoke the following command:
+
+    $ env PYTHONPATH=/google_appengine_sdk/lib/jinja2 \
+        pybabel extract -o locales/messages.pot -F main.mapping .
+
+This command creates a `locales/messages.pot` file in the `locales`
+directory which contains all the string found in your Python code and
+Jija2 tempaltes.
+
+Since the babel configration file `main.mapping` contains a reference
+to `jinja2.ext.babel_extract` helper function which is provided by
+jinja2 distribution bundled with the App Engine SDK, you need to add a
+PYTHONPATH environment variable pointing to the jinja2 directory in
+the SDK.
+
+### Manage and compile translations.
+
+Create an initial translation source by the following command:
+
+    $ pybabel init -l ja -d locales -i locales/messages.pot
+
+Open `locales/ja/LC_MESSAGES/messages.po` with any text editor and
+translate the strings, then compile the file by the following command:
+
+    $ pybabel compile -d locales
+
+If any of the strings changes, you can extract the strings again, and
+update the translations by the following command:
+
+    $ pybabel update -l ja -d locales -i locales/messages.pot
+
+Note: If you run `pybabel init` against an existant translations file,
+you will lose your translations.
+
+
+### Extract strings in Javascript code and compile translations
+
+    $ pybabel extract -o locales/jsmessages.pot -F js.mapping .
+    $ pybabel init -l ja -d locales -i locales/jsmessages.pot -D jsmessages
+
+Open `locales/ja/LC_MESSAGES/jsmessages.po` and translate it.
+
+    $ pybabel compile -d locales -D jsmessages
+
+
+## How it works
+
+As you can see it in the `appengine_config.py` file, our
+`main.application` is wrapped by the `i18n_utils.I18nMiddleware` WSGI
+middleware. When a request comes in, this middleware parses the
+`HTTP_ACCEPT_LANGUAGE` HTTP header, loads available translation
+files(`messages.mo`) from the application directory, and install the
+`gettext` and `ngettext` functions to the `__builtin__` namespace in
+the Python runtime.
+
+For strings in Jinja2 templates, there is the `i18n_utils.BaseHandler`
+class from which you can extend in order to have a handy property
+named `jinja2_env` that lazily initializes Jinja2 environment for you
+with the `jinja2.ext.i18n` extention, and similar to the
+`I18nMiddleware`, installs `gettext` and `ngettext` functions to the
+global namespace of the Jinja2 environment.
+
+## What about Javascript?
+
+The `BaseHandler` class also installs the `get_i18n_js_tag()` instance
+method to the Jinja2 global namespace. When you use this function in
+your Jinja2 template (like in the `index.jinja2` file), you will get a
+set of Javascript functions; `gettext`, `ngettext`, and `format` on
+the string type. The `format` function can be used with `ngettext`ed
+strings for number formatting. See this example:
+
+    window.alert(ngettext(
+        'You need to provide at least {0} item.',
+        'You need to provide at least {0} items.',
+        n).format(n);

appengine/i18n/app.yaml

@@ -0,0 +1,24 @@
+application: i18n-sample-python
+runtime: python27
+api_version: 1
+version: 1
+threadsafe: true
+
+
+handlers:
+- url: /favicon\.ico
+  static_files: favicon.ico
+  upload: favicon\.ico
+
+- url: /static
+  static_dir: static
+- url: /.*
+  script: main.application
+
+libraries:
+- name: webapp2
+  version: latest
+- name: jinja2
+  version: latest
+- name: webob
+  version: 1.2.3

appengine/i18n/appengine_config.py

@@ -0,0 +1,35 @@
+#!/usr/bin/env python
+#
+# Copyright 2013 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""App Engine configuration file for applying the I18nMiddleware."""
+
+
+from i18n_utils import I18nMiddleware
+
+
+def webapp_add_wsgi_middleware(app):
+    """Applying the I18nMiddleware to our HelloWorld app.
+
+    Args:
+        app: The WSGI application object that you want to wrap with the
+            I18nMiddleware.
+
+    Returns:
+        The wrapped WSGI application.
+    """
+
+    app = I18nMiddleware(app)
+    return app

appengine/i18n/i18n_utils.py

@@ -0,0 +1,207 @@
+#!/usr/bin/env python
+#
+# Copyright 2013 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""A small module for i18n of webapp2 and jinja2 based apps.
+
+The idea of this example, especially for how to translate strings in
+Javascript is originally from an implementation of Django i18n.
+"""
+
+
+import gettext
+import json
+import os
+
+import jinja2
+
+import webapp2
+
+from webob import Request
+
+
+def _get_plural_forms(js_translations):
+    """Extracts the parameters for what constitutes a plural.
+
+    Args:
+        js_translations: GNUTranslations object to be converted.
+
+    Returns:
+        A tuple of:
+            A formula for what constitutes a plural
+            How many plural forms there are
+    """
+    plural = None
+    n_plural = 2
+    if '' in js_translations._catalog:
+        for l in js_translations._catalog[''].split('\n'):
+            if l.startswith('Plural-Forms:'):
+                plural = l.split(':', 1)[1].strip()
+                print "plural is %s" % plural
+    if plural is not None:
+        for raw_element in plural.split(';'):
+            element = raw_element.strip()
+            if element.startswith('nplurals='):
+                n_plural = int(element.split('=', 1)[1])
+            elif element.startswith('plural='):
+                plural = element.split('=', 1)[1]
+                print "plural is now %s" % plural
+    else:
+        n_plural = 2
+        plural = '(n == 1) ? 0 : 1'
+    return plural, n_plural
+
+
+def convert_translations_to_dict(js_translations):
+    """Convert a GNUTranslations object into a dict for jsonifying.
+
+    Args:
+        js_translations: GNUTranslations object to be converted.
+
+    Returns:
+        A dictionary representing the GNUTranslations object.
+    """
+    plural, n_plural = _get_plural_forms(js_translations)
+
+    translations_dict = {'plural': plural, 'catalog': {}, 'fallback': None}
+    if js_translations._fallback is not None:
+        translations_dict['fallback'] = convert_translations_to_dict(
+            js_translations._fallback
+        )
+    for key, value in js_translations._catalog.items():
+        if key == '':
+            continue
+        if type(key) in (str, unicode):
+            translations_dict['catalog'][key] = value
+        elif type(key) == tuple:
+            if key[0] not in translations_dict['catalog']:
+                translations_dict['catalog'][key[0]] = [''] * n_plural
+            translations_dict['catalog'][key[0]][int(key[1])] = value
+    return translations_dict
+
+
+class BaseHandler(webapp2.RequestHandler):
+    """A base handler for installing i18n-aware Jinja2 environment."""
+
+    @webapp2.cached_property
+    def jinja2_env(self):
+        """Cached property for a Jinja2 environment.
+
+        Returns:
+            Jinja2 Environment object.
+        """
+
+        jinja2_env = jinja2.Environment(
+            loader=jinja2.FileSystemLoader(
+                os.path.join(os.path.dirname(__file__), 'templates')),
+            extensions=['jinja2.ext.i18n'])
+        jinja2_env.install_gettext_translations(
+            self.request.environ['i18n_utils.active_translation'])
+        jinja2_env.globals['get_i18n_js_tag'] = self.get_i18n_js_tag
+        return jinja2_env
+
+    def get_i18n_js_tag(self):
+        """Generates a Javascript tag for i18n in Javascript.
+
+        This instance method is installed to the global namespace of
+        the Jinja2 environment, so you can invoke this method just
+        like `{{ get_i18n_js_tag() }}` from anywhere in your Jinja2
+        template.
+
+        Returns:
+            A 'javascript' HTML tag which contains functions and
+            translation messages for i18n.
+        """
+
+        template = self.jinja2_env.get_template('javascript_tag.jinja2')
+        return template.render({'javascript_body': self.get_i18n_js()})
+
+    def get_i18n_js(self):
+        """Generates a Javascript body for i18n in Javascript.
+
+        If you want to load these javascript code from a static HTML
+        file, you need to create another handler which just returns
+        the code generated by this function.
+
+        Returns:
+            Actual javascript code for functions and translation
+            messages for i18n.
+        """
+
+        try:
+            js_translations = gettext.translation(
+                'jsmessages', 'locales', fallback=False,
+                languages=self.request.environ[
+                    'i18n_utils.preferred_languages'],
+                codeset='utf-8')
+        except IOError:
+            template = self.jinja2_env.get_template('null_i18n_js.jinja2')
+            return template.render()
+
+        translations_dict = convert_translations_to_dict(js_translations)
+        template = self.jinja2_env.get_template('i18n_js.jinja2')
+        return template.render(
+            {'translations': json.dumps(translations_dict, indent=1)})
+
+
+class I18nMiddleware(object):
+    """A WSGI middleware for i18n.
+
+    This middleware determines users' preferred language, loads the
+    translations files, and install it to the builtin namespace of the
+    Python runtime.
+    """
+
+    def __init__(self, app, default_language='en', locale_path=None):
+        """A constructor for this middleware.
+
+        Args:
+            app: A WSGI app that you want to wrap with this
+                middleware.
+            default_language: fallback language; ex: 'en', 'ja', etc.
+            locale_path: A directory containing the translations
+                file. (defaults to 'locales' directory)
+        """
+
+        self.app = app
+        if locale_path is None:
+            locale_path = os.path.join(
+                os.path.abspath(os.path.dirname(__file__)), 'locales')
+        self.locale_path = locale_path
+        self.default_language = default_language
+
+    def __call__(self, environ, start_response):
+        """Called by WSGI when a request comes in.
+
+        Args:
+            environ: A dict holding environment variables.
+            start_response: A WSGI callable (PEP333).
+
+        Returns:
+            Application response data as an iterable. It just returns
+            the return value of the inner WSGI app.
+        """
+        req = Request(environ)
+        preferred_languages = list(req.accept_language)
+        if self.default_language not in preferred_languages:
+            preferred_languages.append(self.default_language)
+        translation = gettext.translation(
+            'messages', self.locale_path, fallback=True,
+            languages=preferred_languages, codeset='utf-8')
+        translation.install(unicode=True, names=['gettext', 'ngettext'])
+        environ['i18n_utils.active_translation'] = translation
+        environ['i18n_utils.preferred_languages'] = preferred_languages
+
+        return self.app(environ, start_response)

appengine/i18n/js.mapping

@@ -0,0 +1 @@
+[javascript: **.js]