Facebook’s mobile apps no longer run merely on web technology. As an alternative, they run as native software loaded onto iPads, iPhones, Android devices and other mobile gear. The advantage is that the app is much quicker and more responsive than before. The problem, for Facebook, is that testing the app is tricky. The iPhone application is different from the Android application, and so on.
Companies have built a tool called Airlock to help out ease the agony of testing its diverse pieces of mobile software with real-world users. It’s part of a rising number of tools that manage what’s called A/B testing on mobile phones.
The method is like focus group testing. Facebook designers and developers make alterations to the app – some small and some major – and then roll these differences out to only a little fraction of the people employing the thing in the real world. The transformations that work best can later on be rolled out to one and all.
It’s fairly simple to do A/B testing with web applications – alterations can be brought directly from Facebook’s servers to your browser – but it can be tougher to do this with native apps. You would have to submit each variation of an app to every app store. Then you have to wait for Apple and Google to support all variations. That is not practical for a company like Facebook that carries out A/B tests on such a huge scale. In addition, there is no way to tell the app stores to allocate a novel version of the app to, say, only 10% of users.
That set Facebook’s developers in a bind. “Testing is an important part of our development, and we run hundreds of tests on Facebook — most of which are rolled out to a sample of people to test their impact,” Facebook developers Ari Grant and Kang Zhang wrote in their blog post. “Not every test makes it into production, but even failed tests help us understand how to improve. Losing some of this ability became a challenge.”
To resolve the problem, Facebook developed Airlock. The developers are competent to stack all the diverse variations they would like to test into a single version of the app. Once a user downloads the app from the app store, the Facebook server notifies the app which variation to show.
Facebook kicked off by employing Airlock to verify variations on employees’ phones, but afterward graduated to employing it to test across all its users, a Facebook spokesperson tells us. One of the major features tested with Airlock thus far is the new navigation pane that emerges in the Android and iPhone versions of the Facebook app.
“This tool has allowed us to now scale the framework to support 10 or 15 different variations of a single experiment and put it in the hands of millions of people using our apps,” the developers write.
Facebook is keeping Airlock to itself at the present though a spokesperson says the company is assessing whether it will give worth to the open source community. In the interim, a team of ex-Googlers has publicized a mobile A/B testing platform called Leanplum. But a little open source competition would be a great thing.
No comments:
Post a Comment