Friday, May 15, 2009

L10n nightly updates starts with...

The main bug that I am currently working on is to generate MAR files for the different locales we have. This will be one more step towards having nightly updates for all the different locales.

A MAR file (Mozilla ARchive) is basically a zip file but custom made for Mozilla purposes. This type of file is downloaded by Firefox through the update mechanisms; These update systems are used by Firefox to know where the next available update will be.

These MAR files can be partial or complete updates. What does this mean? When you are in the previous release to the new update you just receive a small/partial MAR file which only contains the differences between your current release and the new one. When you skip an update falling behind one or more releases you will get a full/complete MAR file with all the files for the new release meaning that Firefox will download a larger file.

Currently I have been able to create locally with the help of a script a complete MAR file for the French locale. Check the code that I used at the end of this post.

What comes next? I have to make changes to the factory that produces the l10n nightly builds to generate them with the MAR files.

There are many more things to consider but I will skip the details for when I make a little bit more of progress.

To keep yourself up-to-date with this work, CC yourself to the following bug:
Bug 480081 - Generate partial & complete mars for nightly l10n builds

Here is a document explaining what a MAR file is:
https://wiki.mozilla.org/Software_Update:MAR

NOTE: Remove "dist/l10n-stage" if you run into weird problems

Here is the script I used to create the MAR file (check under dist/update):
#!/bin/bash
# Author: Armen Zambrano Gasparnian
# Contact: armenzg@mozilla.com
# Purpose: Repackage a locale in hg
#
export BRANCH='mozilla-central'
#export BRANCH='mozilla-1.9.1'
export REPO_PATH='mozilla-central'
#export REPO_PATH='releases/mozilla-1.9.1'
export L10N_REPO_PATH='l10n-central'
#export L10N_REPO_PATH='releases/l10n-mozilla-1.9.1'
export LOCALE='fr'
export EN_US_BINARY_URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-$BRANCH/"
# 1) Get ready
set -ex
# Let's isolate our work depending on the branch
mkdir -p $BRANCH && cd $BRANCH
mkdir -p l10n
rm -rf $BRANCH/dist/upload
# 2) checkout the main repo
if [ -d $BRANCH ]; then
hg -R $BRANCH pull -r tip ;
else
hg clone http://hg.mozilla.org/$REPO_PATH/ ;
fi
hg -R $BRANCH update

# 3) checkout the locale's repo
cd l10n
if [ -d $LOCALE ]; then
hg -R $LOCALE pull -r tip ;
else
hg clone http://hg.mozilla.org/$L10N_REPO_PATH/$LOCALE/ ;
fi
hg -R $LOCALE update
cd ..
# 4) generate the configure file
cd $BRANCH
autoconf-2.13
cd js/src
autoconf-2.13
cd ../..
# 5) configure
./configure --enable-application='browser' --with-l10n-base=../l10n --enable-update-packaging
# 5.A) tools needed
make -C nsprpub
make -C config
make -C modules/libmar
# 6) get the en-US binary
make -C browser/locales wget-en-US
# 7) generate the xpi and the installers
make -C browser/locales installers-$LOCALE MOZ_MAKE_COMPLETE_MAR=1
# 8) rearrange the packages in the correct structure for a nightly on ftp
make -C browser/locales prepare-upload-latest-$LOCALE
# 9) list the packages in the correct place, the correct naming and the correct chmod
ls -l dist/upload/latest
ls -l dist/update
# I do not have an ftp server so I won't be running this command
# sh -c 'scp -i $KEY_PATH -r * $USER@$SERVER:$UPLOAD_PATH'





Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Thursday, May 14, 2009

HOWTO: Paste to pastebin from the command line

You need the script "paste2oastebin.pl" from http://de.pastebin.ca/tools.php.

Here is what I needed to do on Mac OS X to make it work:
  • I needed to have WWW::Mechanize in my Perl modules so I used CPAN's interactive shell to do it:
sudo /usr/bin/cpan
install WWW:Mechanize
  • NOTE: The script points to a specific perl path and that implies a specific CPAN. Remember that we have MacPorts taking preference over default MacOS X installed packages. Use whereis to find this out and compare with the first line of the script.
  • Put the script in a folder which is part of your PATH and set the right permissions

Here is how to use it:
  • paste2pastebin file_to_submit
It will return you the URL for it. The script has more advanced options.


Errors I found which required installing WWW::Mechanize and the second one I am ignoring it every time:
Can't locate WWW/Mechanize.pm in @INC (@INC contains: /System/Library/Perl/5.8.8/darwin-thread-multi-2level /System/Library/Perl/5.8.8 /Library/Perl/5.8.8/darwin-thread-multi-2level /Library/Perl/5.8.8 /Library/Perl /Network/Library/Perl/5.8.8/darwin-thread-multi-2level /Network/Library/Perl/5.8.8 /Network/Library/Perl /System/Library/Perl/Extras/5.8.8/darwin-thread-multi-2level /System/Library/Perl/Extras/5.8.8 /Library/Perl/5.8.6 /Library/Perl/5.8.1 .) at /Users/armenzg/scripts/paste2pastebin.pl line 23.
BEGIN failed--compilation aborted at /Users/armenzg/scripts/paste2pastebin.pl line 23.
Use of uninitialized value in string ne at /Library/Perl/5.8.8/HTML/Form.pm line 704, <> line 555.





Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Monday, May 11, 2009

My first day as a full-timer

I am really excited to become part of Mozilla as a full-time employee. For those who don't know about me here is a real short summary:
  • I am from Spain
  • Moved to Canada 3 years ago
  • Studied at Seneca College
  • Took the two open-source courses in my 3rd year of the Software Development Degree
  • I did an internship in the Release Engineering team during the summer
  • I have been working on improving the release infrastructure for making different languages of Firefox faster
  • I kept on working as a consultant during my last of my degree
  • and today I am starting my first day at Mozilla!! (one year and 2 weeks after the beginning on my internship)
You can find me on IRC as "armenzg" on the #build and #seneca channels!