TestFlight Your Apps

You've been working on your new app for ages and it's finally ready for beta testing. Prior to iOS 4.0, it was a considerable pain in the neck to get a build on to your testers devices. You needed to package up your apps and ad hoc profile, send them to each tester, and then they needed to use iTunes to install the profile and app on their device. It seemed like an art more than a science getting an app installed, never mind an updated build later. It was a very clunky system, and it didn't always go smoothly.

With iOS 4.0, Apple made it much easier to install test apps on a device. It's now possible to install profiles and apps without going through iTunes. Using XCode's Build and Archive option, you can create an .ipa file that embeds the ad hoc profile. Just put the .ipa file on a web server, (or your public Dropbox folder), and send the link to your testers. They can install the build by clicking the link in mobile Safari.

You're still required to collect your users UDID's, add their devices to your developer account and create an ad hoc profile that includes their devices.

Last week, TestFlight, a new service for distributing your test builds to your beta testers, went live. TestFlight promises to revolutionize the way developers beta test their apps, and after testing it out for a bit, I'm pretty sure they're going to do just that.

The free service, found at testflightapp.com, allows a developer to invite people to become a beta tester. The tester creates an account through the web site in Mobile Safari on their device, and then they register their device with the system. This process involves installing a configuration profile (different from an ad hoc profile, but listed in the same area on the device's settings). This gathers the device UDID and reports it back to the server. A tester can register more than one device. The tester, and her device(s) then show up in the developer's account. As a developer, you can export the UDID's of all your testers and import them into Apple's iOS Provisioning Portal. One issue I found here though, is that if you attempt to import a file of UDID's that contain a record that you've already added to the Portal, it will reject the entire file, instead of just ignoring that record.

Once you've imported all the new UDID's into the iOS Provisioning Portal, you can update your ad hoc profile to include all of your testers. Then, use it to create a test build of your app. You still need to use XCode's Build and Archive Option, and then use it's sharing wizard to create the .ipa file (all in all, a simple process, more details here: http://iphonedevelopment.blogspot.com/2010/05/xcode-32-build-and-archive.html).

Next, you upload your .ipa file to the TestFlight web site, and select the testers you'd like to notify of the new build. TestFlight emails each of them a link to a page that lets them install the build easily.

TestFlight lets you see how many times the .ipa file was downloaded (though, not by whom for some reason). I had one person delete it from their phone and redownload it, and it counted as another download. So a list of 5 people, with 5 downloads, doesn't really mean they all downloaded it. But it will likely be close.

I tested this entire process with some extremely non-technical people, not one had an issue. They were all able to create their accounts, register their devices, and install my test app with ease.

If you need more beta testers, TestFlight offers a recruitment tool. You can start recruiting random people through Twitter, your web site, or wherever. It's just a link you post. When people sign up to be a beta tester for you, you can accept or reject them based on whatever criteria you like.

Overall, I'm very impressed with the initial service offering TestFlight has. And, remember, it's entirely free (for now) for developers. They are charging for enterprise accounts.

I do have to point out one concern I have with TestFlight however. During my limited testing of the service over the last couple of days, I uploaded a few builds, sent them out to some testers, including myself, and then deleted the builds. When I went back to one of my test devices, I still had the link open in Safari to do the install, for a build that I had deleted in the TestFlight dashboard. So, I tested, and tried to install the build, that should have been deleted. It installed perfectly. This is my concern, because I had deleted the build! Which means, that TestFlight, is not actually deleting the bits of the builds you tell it to delete, it's just removing them from your dashboard. Whether or not you consider this acceptable is up to you (and your personal level of paranoia) . But remember, that when you upload your .ipa to TestFlight, you're letting unknown people view and test your ideas (yes, the TestFlight people can view and run your .ipa files, if they so choose).

If you are concerned, you can use Hockey to manage your beta installs. It's not as easy to manage as TestFlight, but you con trol everything on your servers so it's arguably more secure.

You can even mix and match some of the TestFlight features (UDID collection, recruitment etc), and then deliver the actual builds via Hockey.

Paranoia aside, TestFlight is excellent so far, and will change the way you deliver your test builds to your testers. Down the road, they plan to support additional features, such as adding analytics so you can view what your testers tested and for how long.