Thursday, January 10, 2013

Running Gaia UI b2g panda tests on the Cedar branch

Back in November, Release Engineering tackled with IT, the A-team and the B2G team, the undertaking of running in production the Gaia UI tests on the b2g pandas [1].

Back in December 7th, we managed to get all the piping done for re-imaging pandas and assigning dummy jobs to them (see "B2G test jobs running on panda boards on the Cedar branch" post for details).

Last week, after I came back from holidays, I was pleased to see that one of the last blocking bugs had been solved (bug 820617 - Add a hook to make NetworkManager not manage offline status and use it in Marionette for B2G CI) and I could now try to run the tests.

Last Friday, we landed the code to actually run the tests [2]. This is simply run with a mozharness script like this:
/tools/buildbot/bin/python scripts/scripts/b2g_panda.py --cfg b2g/panda_releng.py
At that point in time we stopped running dummy jobs and started running tests for the first time.

Since then, then test jobs have been failing but we are working on the last few bugs to get them to be green. See bug bug 829053 (Gaia UI tests failing with JavascriptException: TypeError: settings is null) if you're curious.

Thirty five fixed dependent bugs later (and 2 bugs left to be fixed) I can say that it has been a great joint project and I have personally learned a lot through it.

It would have great to have completed it a while ago but too many difficulties and code problems were found.

I hope to write to you soon once the tests go green and we start running them every b2g branch.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=802317
[2] http://hg.mozilla.org/build/mozharness/rev/caf3b92e7207


Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

No comments:

Post a Comment