Saturday, January 12, 2013

Handing off Armenian localization to others

For a very long time I have been involved with the Armenian localization process for Mozilla but since I got married I had to re-prioritize where I would put my time into.

In this post, I will describe what pieces of knowledge I gathered over the years and try to write something that others can use. Please forgive the lack of structure and whatever mistakes I make.

NOTE: "L10n" is shorthand for Localization; which is the process of translating texts plus everything else involved to get the products adjusted for a region and releasing to users.

To learn more you can read the official documentation in here:
https://developer.mozilla.org/en-US/docs/Localization_Quick_Start_Guide

To ask questions you can visit the Google Group for L10n:
https://groups.google.com/forum/?fromgroups#!forum/mozilla.dev.l10n

 Concepts involved

  • Mozilla's development cycle
    • every 6 weeks there are new strings to be translated
    • The strings travel through 4 different versions of Firefox
      • Nightly -> Aurora -> Beta -> Release
    • We do translations in the Aurora cycle which it's supposed to be the first cycle frozen with regards to new texts to be translated
  • Product translations
    • How to translate Firefox, Thunderbird and Firefox for Android
    • Every 6 weeks (matching the opening of an Aurora cycle), I would remind contributors with an email or a Facebook post with instructions on what to translate
  • Web translations
    • Web pages translated into Armenian
  • Bugzilla
    • Issue tracking system
  • Mozilla.am and Facebook
  • Accounts

1 - Mozilla's development cycle

Here's a brief explanation on how developers change Firefox and include texts to be translated. Developer's create code changes and add texts to be translated while Firefox is on the first six weeks' development cycle. This period is called Trunk, Nightly (since every night a new update is generated) or the latest version of the product or mozilla-central (central point of integration).

After six weeks of changes the code is taken from Trunk/Nightly to Mozilla Aurora. In this cycle translations are not to be changed. It is at this point in time that we want to translate texts since there should be no new texts to be added on this six weeks cycle. The code and translations on Aurora are generated every morning for every language and for every platform. If your team introduces new translations it will be integrated on the next morning's update. On this 6 week period, you are expected to do your translations, submit them and then sign them off (more details later).

After this Aurora period passes, the translations move into the Beta cycle where a much larger audience tests all products. At this point, missed translations can still be taken if necessary. Approximately one beta is generated every week.

After this Beta development cycle, the beta product reaches the Release cycle where millions of users get to use it. The majority of users don't know any other cycles besides the Release one and they are not aware that there is such a concept as the release version. No translations are to happen on this period either. One version is produced at the beginning of the cycle and security releases only occur due to major security issues. 

Here's a summary:
  • Nightly: First 6 weeks, new texts appear on Nightly
  • Aurora: Second 6 weeks, all texts are ready to be translated
  • Beta: Third 6 weeks period, you should have translated all your texts on the Aurora period. The translations from the Aurora period are kept on this cycle.
  • Release: The last 6 weeks are exactly the same as the Beta period. No translations. They are kept exactly the same

2- Product translations

There are several ways of translating the various Mozilla products [1]. One of them is Narro, which a web tool. My recommendation is to abandon such project and continue with Pootle's version. The guys running it are very responsive and are much more involved with the translations they support. We have had way too many issues with Narro in the last 2 years. I'm not sure if they have support for all three products or not. Visit the Armenian page they set up for us: http://mozilla.locamotion.org/hy_AM.

Nevertheless, here are the instructions on how I used to do it with Narro.
A complete set of tutorials are documented in here:
https://developer.mozilla.org/en-US/docs/Localizing_with_Narro 

[1] https://developer.mozilla.org/en-US/docs/Localization_Quick_Start_Guide/Translation_phase#Mozilla_applications

2.1 - Translate with Narro

Every 6 weeks you want to encourage your team of contributors to translate the new set of texts available. Narro should automatically import into its database the translations from Mozilla's Mercurial:
http://hg.mozilla.org/releases/l10n/mozilla-aurora/hy-AM

Roberto Reyes from the Tagalog team has created a simple video showing how to translate in Narro:
http://www.youtube.com/watch?v=EtoLPL_JIG8

I also wrote a more in-depth blog post about the topic:
http://armenzg.blogspot.ca/2010/10/how-we-use-narro-to-localize-firefox.html

2.1.1 - Review other people's translations

Some users in Narro can actually review and approve other contributor's suggestions. Only reviewed translations should make it into the Narro exports.

2.2 - Export Narro into Mercurial

The translations of your team won't be official until they are committed/submitted to Mozilla's Mercurial servers. Mercurial is a version control system that can store the translated files. Narro exports the translations in a zip file that has the same structure as the files in Mercurial.
Every 6 weeks (or every time you get a bunch of translations) you should export and commit your translations into Mercurial. Don't leave it until the end of the 6 weeks cycle of Narro could fail you. The next day your translations will be included on the Armenian Aurora version of Firefox for desktop and mobile plus Thunderbird:
http://www.mozilla.org/en-US/firefox/all-aurora.html#hy-AM (Aurora is the second cycle out of the four development cycles)

Steps (rough outline):
  • Load each Narro project: Firefox Aurora, Fennec Aurora & Thunderbird Aurora
  • Load the “Export” tab for each one and download the zip file for each
  • Download the zip file and unzip unto your Mercurial checkout
  • Make sure that you run “hg st” to know if you have any files not going in
  • Watch the video and read this post for more details: https://blog.mozilla.org/l10n/2012/07/16/tips-tricks-cleaning-up-narro-exports

On this video I show how I export the translations and I clean up the code before submitting (committing) them to the Mercurial servers:
 

You can see a post explaining how to clean up your export:
https://blog.mozilla.org/l10n/2012/07/16/tips-tricks-cleaning-up-narro-exports/

Narro trick

The following post explains how to prevent Narro from exporting specific files. This is useful for some files that require review from the L10n driving team.
http://armenzg.blogspot.ca/2011/12/how-to-exclude-file-from-being-exported.html

2.3 - Sign off on the L10n Dashboard
To communicate to the L10n team that your translations are ready for a development cycle you have to sign off. To sign off is to use a webtool to mark a specific submission to Mercurial as ready to be released.
After we import the changes into Mercurial in 15-20 minutes we should see the results on this dashboard:
https://l10n.mozilla.org/teams/hy-AM
    • We only care about “fx_aurora” (Firefox), “fennec_aurora” (Firefox Mobile) and “tb_aurora” (Thunderbird) when we want to take care of current needed translations
    • Once your commits have greened out you have to click on the icon that looks like a magnifying glass (This is to mark a sign off)
    • Only people who are signed in and have the right privileges can submit a sign-off.

    2.4 - Example of status communication

    Every 6 weeks blog/email the group of potential localizers so they can do the translations:
    Hi,
    In 6 weeks version 20 of the various Mozilla products we translate into Armenian will be released to the beta audience.


    We're not doing great for the Firefox Desktop version. If you could think of who could help us do translations please let us know.


    For translations click on the following:


    For reviews (some of you can actually do reviews):
    You can tests these translated versions in here:
    • Desktop (Windows, Mac & Linux)
    • Android - Look for the apk that says "hy-AM"


    Best regards,
    Armen

    3 - Web translations

    If you look at the L10n dashboard:
    https://l10n.mozilla.org/teams/hy-AM#webdashboard
    You will notice that there is a web dashboard that leads you to all web translations still needed:
    https://l10n.mozilla-community.org/webdashboard/?locale=hy-AM

    If you follow the link of missing strings it will take you here:
    http://l10n.mozilla-community.org/~pascalc/langchecker/?locale=hy-AM#main.lang

    On the side you will see instructions on how to attach translations:

    IdenticalTranslatedMissing
    61020
    Original English source file
    Your translated file
    Attach your updated file to Bugzilla

    4 - Bugzilla

    In the software industry, issues are tracked with issue tracking systems which are also known as bugs. Mozilla has a database called Bugzilla which tracks issues of all sorts. In the case of L10n, there are bugs that are specific to your language (a.k.a. locale).

    On the dashboard you can find a summary of the bugs for your locale:
    https://l10n.mozilla.org/teams/hy-AM#bugzilla

    Each bug have different instructions on what to do.

    5 - Mozilla.am and Facebook

    Mozilla.am is the Armenian site that promotes Mozilla and I believe it is still maintained by Robert Sargsyan
    http://www.mozilla.am

    I have also found that Facebook is a good way to engage people (better than email). I have in the past pasted an etherpad with web translations and asked for help:
    https://www.facebook.com/groups/firefox.in.armenian
    https://etherpad.mozilla.org

    6 - Accounts

    What sort of accounts could a locale's administrator need?
    • A project admin account on Narro.
    • SVN access
      • A bug needs to be filed.
      • pascalc can also land things by asking him on bugs
      • A SVN account expires after 6 months of not being used
    • Mercurial access
      • A bug needs to be filed.
      • The people from Narro and Pootle can help you land your code 
    • Sign-off privileges
      • A bug needs to be filed (I assume so. I don't know under which component)
      • Ask on the L10n newsgroup or directly talk with Pike
    I hope this blog post is somehow useful for the continuation of the Armenian localization.



    Creative Commons License
    This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

    Thursday, January 10, 2013

    Running Gaia UI b2g panda tests on the Cedar branch

    Back in November, Release Engineering tackled with IT, the A-team and the B2G team, the undertaking of running in production the Gaia UI tests on the b2g pandas [1].

    Back in December 7th, we managed to get all the piping done for re-imaging pandas and assigning dummy jobs to them (see "B2G test jobs running on panda boards on the Cedar branch" post for details).

    Last week, after I came back from holidays, I was pleased to see that one of the last blocking bugs had been solved (bug 820617 - Add a hook to make NetworkManager not manage offline status and use it in Marionette for B2G CI) and I could now try to run the tests.

    Last Friday, we landed the code to actually run the tests [2]. This is simply run with a mozharness script like this:
    /tools/buildbot/bin/python scripts/scripts/b2g_panda.py --cfg b2g/panda_releng.py
    
    At that point in time we stopped running dummy jobs and started running tests for the first time.

    Since then, then test jobs have been failing but we are working on the last few bugs to get them to be green. See bug bug 829053 (Gaia UI tests failing with JavascriptException: TypeError: settings is null) if you're curious.

    Thirty five fixed dependent bugs later (and 2 bugs left to be fixed) I can say that it has been a great joint project and I have personally learned a lot through it.

    It would have great to have completed it a while ago but too many difficulties and code problems were found.

    I hope to write to you soon once the tests go green and we start running them every b2g branch.

    [1] https://bugzilla.mozilla.org/show_bug.cgi?id=802317
    [2] http://hg.mozilla.org/build/mozharness/rev/caf3b92e7207


    Creative Commons License
    This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.