TechCrunch » android
TestFairy Opens Its Android App Testing Platform To All, Making Even Bad Beta Testers Useful
Beta testing mobile apps becomes only more challenging as the crop of devices and mobile OS versions continues to expand, and Israel-based TestFairy, a new startup launching its product to the public today, is trying to tackle an aspect of the process that often results in beta users not being very useful to developers, thanks to a platform that gathers crucial feedback even if a user isn’t supplying it.
TestFairy does this without even requiring that a developer integrate any SDK or APIs into their app builds, which co-founder and CEO Yair Bar-On explained is one of the platform’s key advantages over competitors like TestFlight and others. A developer simply uploads their APK (Android application file) to the TestFairy platform, and then the company’s tools takes it apart, inserts the necessary hooks to gather information in the proper places, and then puts it back together for distribution to beta testers.
Not only is TestFairy’s solution easier for developers to use than competing services, Bar-On says, but it’s also designed to provide more feedback. The chief problem with running beta tests, according to Bar-On, is that your pool of testers is usually a massively mixed bag in terms of the quality of input you receive. Often you can find a decent crop of people to test from out of your pool of acquaintances friends, family and social media contacts, but getting those people to consistently tell you about bugs they’ve experienced, or about UX issues is another story entirely.
With TestFairy, users receive a ton of information to their own dashboard, feeding back data from the app about hardware details like memory and CPU usage, network bandwidth consumption, battery performance and phone signal, so that you can really optimize your app for a range of devices, and be incredibly specific about what happens and when. You can view a sample dashboard here, simulating live data exactly as a developer would see on their own back end. And there’s no waiting for the stream of information, either. You can even watch a video of exactly what a user has done in your app, complete with taps displayed on screen.
“It doesn’t take any time to process the results of the test,” Bar-On says. “If you have a test of test of an app that takes you two hours, 2 seconds after you start it starts sending you feedback to the developer.”
The entire TestFairy process is designed to require as light a touch as possible from the developer themselves, and from their beta testers. But it’s better than a fully automated platform that handles the testing in lab facilities, according to Bar-On, because those can never replicate true real-world conditions and usage scenarios. They can’t reflect real coverage areas and dead zones, for instance, or exactly mimic the types of things users will do with their smartphones when left to their own devices.
TestFairy arrives just as Google has introduced its own beta tools to the Google Play store for interested Android developers, but this isn’t a threat to their business, Bar-On suggests, since what they offer is far more comprehensive in terms of the information it provides. In the short term, TestFairy will be free, but eventually the company plans to charge based on volume usage. TestFairy is self-funded, but plans to seek funding once the product is out the door, and will also add more features like path analysis down the road.
