TestFlight your Apps

Posted by on
  • Font size: Larger Smaller
  • Hits: 17107
  • 11 Comments

You've been working on your new app for ages and it's finally ready for beta testing. Prior to iOS 4.0, it was a considerable pain in the neck to get a build on to your testers devices. You needed to package up your apps and ad hoc profile, send them to each tester, and then they needed to use iTunes to install the profile and app on their device. It seemed like an art more than a science getting an app installed, never mind an updated build later. It was a very clunky system, and it didn't always go smoothly.

With iOS 4.0, Apple made it much easier to install test apps on a device. It's now possible to install profiles and apps without going through iTunes. Using XCode's Build and Archive option, you can create an .ipa file that embeds the ad hoc profile. Just put the .ipa file on a web server, (or your public Dropbox folder), and send the link to your testers. They can install the build by clicking the link in mobile Safari.

You're still required to collect your users UDID's, add their devices to your developer account and create an ad hoc profile that includes their devices.

Last week, TestFlight, a new service for distributing your test builds to your beta testers, went live. TestFlight promises to revolutionize the way developers beta test their apps, and after testing it out for a bit, I'm pretty sure they're going to do just that.

The free service, found at testflightapp.com, allows a developer to invite people to become a beta tester. The tester creates an account through the web site in Mobile Safari on their device, and then they register their device with the system. This process involves installing a configuration profile (different from an ad hoc profile, but listed in the same area on the device's settings). This gathers the device UDID and reports it back to the server. A tester can register more than one device. The tester, and her device(s) then show up in the developer's account. As a developer, you can export the UDID's of all your testers and import them into Apple's iOS Provisioning Portal. One issue I found here though, is that if you attempt to import a file of UDID's that contain a record that you've already added to the Portal, it will reject the entire file, instead of just ignoring that record.

Once you've imported all the new UDID's into the iOS Provisioning Portal, you can update your ad hoc profile to include all of your testers. Then, use it to create a test build of your app. You still need to use XCode's Build and Archive Option, and then use it's sharing wizard to create the .ipa file (all in all, a simple process, more details here: http://iphonedevelopment.blogspot.com/2010/05/xcode-32-build-and-archive.html).

Next, you upload your .ipa file to the TestFlight web site, and select the testers you'd like to notify of the new build. TestFlight emails each of them a link to a page that lets them install the build easily.

TestFlight lets you see how many times the .ipa file was downloaded (though, not by whom for some reason). I had one person delete it from their phone and redownload it, and it counted as another download. So a list of 5 people, with 5 downloads, doesn't really mean they all downloaded it. But it will likely be close.

I tested this entire process with some extremely non-technical people, not one had an issue. They were all able to create their accounts, register their devices, and install my test app with ease.

If you need more beta testers, TestFlight offers a recruitment tool. You can start recruiting random people through Twitter, your web site, or wherever. It's just a link you post. When people sign up to be a beta tester for you, you can accept or reject them based on whatever criteria you like.

Overall, I'm very impressed with the initial service offering TestFlight has. And, remember, it's entirely free (for now) for developers. They are charging for enterprise accounts.

I do have to point out one concern I have with TestFlight however. During my limited testing of the service over the last couple of days, I uploaded a few builds, sent them out to some testers, including myself, and then deleted the builds. When I went back to one of my test devices, I still had the link open in Safari to do the install, for a build that I had deleted in the TestFlight dashboard. So, I tested, and tried to install the build, that should have been deleted. It installed perfectly. This is my concern, because I had deleted the build! Which means, that TestFlight, is not actually deleting the bits of the builds you tell it to delete, it's just removing them from your dashboard. Whether or not you consider this acceptable is up to you (and your personal level of paranoia) . But remember, that when you upload your .ipa to TestFlight, you're letting unknown people view and test your ideas (yes, the TestFlight people can view and run your .ipa files, if they so choose).

If you are concerned, you can use Hockey to manage your beta installs. It's not as easy to manage as TestFlight, but you con trol everything on your servers so it's arguably more secure.

You can even mix and match some of the TestFlight features (UDID collection, recruitment etc), and then deliver the actual builds via Hockey.

Paranoia aside, TestFlight is excellent so far, and will change the way you deliver your test builds to your testers. Down the road, they plan to support additional features, such as adding analytics so you can view what your testers tested and for how long.


This post is part of iDevBlogADay, a group of indie iPhone development blogs featuring two posts per day. You can keep up with iDevBlogADay through the web site, RSS feed, or Twitter.

Comments

  • Guest
    Dave Newman Sunday, 23 January 2011

    Automate Build and Archive

    You can also automate the "Build and Archive" part which you'll want to do if you're running continuous integration. I wrote a quick how-to here: http://whatupdave.com/post/2343427882/auto-packaging-ios-apps-for-wireless-install" rel="nofollow">http://whatupdave.com/post/2343427882/auto-packaging-ios-apps-for-wireless-install

    I'm looking forward to giving Test flight a go on my next project

  • Guest
    @GiovaMaster Sunday, 23 January 2011

    ;)

    Excellent Article, thumbs up! (I yet know the subject but you've been really clear at explaining the topic)

  • Guest
    Trystan Kosmynka Monday, 24 January 2011

    TestFlight Response

    Thanks for the article and providing feedback. We can and will be removing the binaries upon deletion. Thank you for pointing this out. Also, we will be launching other security features to address some of the other concerns. We take privacy and security seriously so we will continue to look for ways to make TestFlight a service everyone can trust.

  • Guest
    Trystan Kosmynka Monday, 24 January 2011

    RE:TestFlight Response

    We just pushed an update which includes the removal of binaries upon build deletion. Thanks again for pointing this out!

  • Guest
    Fraser Monday, 24 January 2011

    Are you sure they can run your app?

    How exactly are the people at TestFlight able to run your app? Surely they would need to have a device with it's UDID included in the provisioning profile?

  • Dave Wood
    Dave Wood Monday, 24 January 2011

    @frazer

    Yes, they can definitely run your apps. There are at least two ways to do this, 1) resign the .ipa you've uploaded with another ad hoc profile, or 2) install the .ipa on a jailbroken device in the same manner that they can run pirated apps, using the same tools to by pass the signature etc.

    I haven't actually tested either of those two methods so I can't give you exact instructions, but I've seen enough evidence of it happening (by others, not TestFlight) to know it can be done.

    More info on #1 can be found here: http://www.tuaw.com/2009/06/24/dev-corner-signing-iphone-apps-for-informal-distribution/" rel="nofollow">http://www.tuaw.com/2009/06/24/dev-corner-signing-iphone-apps-for-informal-distribution/

  • Guest
    http://David McGraw Monday, 24 January 2011

    Signing

    @frazer: I have been in the position several times where i would get an app, or send an app to someone, and resign it with my own profile. If your dealing with other devs, it's just far easier.

  • Guest
    Andreas Linde Sunday, 30 January 2011

    Hockey

    We are working hard to improve the Hockey framework further. The framework was just declared as version 1 and version 2 with lots of new features is coming very soon. There are two teaser images of the new client UI on dribbble: http://dribbble.com/therealkerni" rel="nofollow">http://dribbble.com/therealkerni

    One major feature Hockey makes unique, is In-App-Updates. If you integrate the client, it will update itself, and developers do not need to send lots of emails around. Testers will be able to update to a new version from inside your own app. Using the client it is also possible to see which tester is using which device, os version, app version and when it was started the last time (if updating checking is not set to manual by the tester).

    The repository also contains a shell script to automate the build and distribution process from within Xcode.

  • Guest
    Tim Miller Tuesday, 01 February 2011

    Safari cannot download this file

    Thanks for the article, Dave. I was just trying this out by putting the .ipa file in my dropbox to test it out, but when I click on the link in mobile safari, I get an error that says "Safari cannot download this file". I created the .ipa correctly as near as I can tell. Any thoughts on why Safari says it can't download the file?

  • Guest
    Toru Thursday, 23 June 2011

    RE:Safari cannot download this file

    Having the same problem, did you figure this one out?

  • Dave Wood
    Dave Wood Saturday, 25 June 2011

    RE:RE:Safari cannot download this file

    This generally means the .IPA was not created correctly.

Leave your comment

Guest Sunday, 23 November 2014