Distributing your app builds to end users via one of the app stores is the tip of the iceberg when it comes to build distribution. For every “final” build you upload and release, your team has likely generated tens or even hundreds of pre-production builds at various stages of the development lifecycle – while, ideally, distributing those builds into each other’s hands for testing. Pre-prod and beta distribution is an important part of a successful mobile continuous delivery setup, and it can have a big positive impact on developer experience and app quality.
But actually setting your team up for streamlined app distribution can be tricky. Aside from technical challenges around provisioning, codesigning, and upload and storage, it can be especially difficult to get the user experience and human factors around build distribution right. Installation flows can be convoluted, and it’s hard to handle the mix of different build flavors and audiences involved: from early development builds and one-offs that product managers or designers need to check out, to nightly builds and betas for the wider team to grab, to release candidates (RCs) that QA will run regression testing on.
Thankfully there are a handful of pre-prod and beta app distribution tools that can help. In this post we’ll first highlight some of the features and factors to consider when looking for an app distribution platform, then cover the most popular options when it comes to specific mobile distribution tools.
Key features: how to compare pre-prod/beta app distribution tools
Build distribution is not only technically complex but also requires thoughtful UX and meaningful integration with the rest of your stack and development process. That’s why it’s important to look at a platform’s complete offering across multiple key areas when considering the right mobile distribution tool for your team.
Grouping and separation of build flavors
Without clear separation between different build types in your distribution platform, it’s hard for team members – technical and non-technical alike – to be sure they’re grabbing the right builds. Best case this wastes time, and worst case it results in builds making it in front of the wrong audience – even shipped to prod prematurely. If there’s confusion around distribution, the natural tendency is to simply avoid doing much with distributed builds beyond a couple of basic flavors (RCs and perhaps nightly builds).
Different tools vary in how much they let you group and separate distinct flavors of distributed builds, but unfortunately the norm tends to look like one long, combined list of builds that your team will have to sift through. Instead of allowing for clear differentiation between types of builds, most tools seem to skip a step and focus only on grouping from the user side, allowing for the creation of different test groups and audiences. If your team distributes more than a single type of build (hint: you should!) look for the ability to logically group and separate build flavors so they’re not stored in a single, jumbled pile.
Build installation and tester experience
A better installation and tester experience has a direct impact on the “success” of your team’s mobile distribution. It means your team is more likely to grab distributed builds early and often during the development lifecycle, with the resulting positive effects on app quality and DevEx.
Typical installation flows directly link to builds for authorized testers to open on their own devices, often surfacing them via chat notifications (e.g. Slack) or email. Some tools even offer a native app for testers to install from. Beyond installation, there are other tester experience features to be on the lookout for, including in-app feedback mechanisms, session recording, and install metrics. Plus, distribution platforms can surface differing amounts of helpful context alongside each build — from nothing beyond a build number, to commit hash, to a full diff showing changed code or even project management ticket info.
Codesigning and provisioning restrictions
When it comes to exactly what types of builds can be distributed via a given app distribution tool, there are some constraints depending on whether the tool is third-party or an “official” option from Apple or Google. In short, you’ll be limited in your ability to distribute builds signed for the stores through third-party options and, on the flipside, you often can’t distribute non-store signed builds through the official platforms.
Tester limits and external review
App distribution platforms sometimes impose caps on the numbers of testers you can distribute builds to, and these limits can vary depending on the flavor of build as well. In certain cases providers will approve increased limits on a case-by-case basis, but at a certain point a ceiling may be reached so large orgs will want to be especially mindful. In addition to caps on numbers, certain distribution tools apply further guardrails, subjecting builds to review prior to distributing to particular kinds of audiences (e.g. external groups of testers where applicable).
Because app distribution touches so many team members and parts of the development lifecycle, the tool you use for it will likely become quite an important piece of your overall toolchain. As such, you’ll want to consider how exactly it can integrate with the rest of your stack.
For example, you might need to add upload steps to your continuous integration (CI) pipeline to bridge the gap from build creation to distribution, or perhaps the distribution tool in question offers an out-of-the-box integration that can automatically grab artifacts generated by CI. Other integrations to look for might be notifications and alerting to Slack, for example, connections with source control and project management platforms to provide color on the work contained in distributed builds, and SSO/SAML based authentication to streamline and secure tester management.
Although the underlying idea of pre-prod app distribution is universal – get more builds into more people’s hands more easily – there are differences between the major platforms your team is likely building and distributing for that you’ll want to keep in mind. Apple and Google worlds differ when it comes to signing and other restrictions around distribution, and what you can do with a given tool will depend on whether it is third-party or from Apple or Google themselves. There are also various sub-platforms to factor in – from iOS to macOS to tvOS, Android to Android TV to Wear OS, and beyond.
It’s especially worth considering whether it’s important to your mobile org to have all of your apps and platforms distributed via the same tool or not, as certain options are limited to only one platform.
The top tools for pre-prod and beta app distribution
Now that you have a sense for what to look out for when comparing app distribution tools, let’s take a look at all the top options you’ll be comparing between.
Runway Build Distro
Hey, that’s us! The newest and (in our clearly unbiased opinion) best build distribution option for most teams is our own. We launched Build Distro to offer a more complete app distribution option, catering to both Android and iOS while addressing frustrating gaps found in existing distribution tools.
One of the biggest differentiators that sets Build Distro apart is our emphasis on keeping distinct flavors of builds clearly grouped and separated. Builds are automatically channeled to different “buckets” which you define using various combinations of branch, pull request base or head, author, continuous integration (CI) workflow, and certain other filters. None of the other options allow for this, instead dumping your builds into big lists with nothing to set them apart from each other.
With Build Distro, testers know exactly where to look for a particular kind of build – from one-off feature builds to nightly to RC – and you can then also attach different individuals or groups of testers per bucket, with no limits on either. Most other settings also live at bucket level, so you can tailor things like notifications and additional automations based on types of builds, instead of needing to operate on a single, jumbled list of various types.
Testers can be notified about new builds in Slack and there are shortcuts to install builds via QR or direct link, both of which lead to a streamlined mobile web experience (no native apps…yet!). And drawing on Runway’s breadth of integrations, Build Distro gives testers more helpful context alongside each build. Instead of just a commit hash or message, you can see a complete diff not only of code changes contained in the build, but also corresponding project management tickets, depending on what integrations you choose to hook up.
Getting up and running is designed to be especially low-lift. Instead of requiring that you add an upload script or step as part of your CI pipeline, you just need to get Build Distro talking to your CI provider using one of the out-of-the-box integrations (Oauth or API key, depending). Build Distro will then grab and securely store builds produced by any relevant CI runs, saving you precious CI build minutes and avoiding the need to babysit flaky uploads. Plus, with no changes to the existing integration or any other extra overhead, you can easily spin up new build buckets by simply entering a definition for them in the user-friendly Build Distro UI.
Let’s compare this to the other options.
TestFlight was a third-party platform that Apple acquired some years ago, and it’s now an Apple-native beta distribution tool integrated directly into App Store Connect. Builds can be distributed to individual testers or groups of testers, and there’s support for two main kinds of testers and groups – “internal” or “external” – with pros and cons to each. Internal testers receive new builds immediately, but a maximum of 100 internal testers is enforced. By contrast, you can have up to 10,000 external testers, but the first build of each new version must go through a beta review process by Apple. Upon request, Apple will sometimes grant larger companies increases to these limits, and some teams use Enterprise distribution as a workaround, but the latter is a co-opted use of Enterprise certs that Apple doesn’t condone (and may one day clamp down on).
Where TestFlight shines is its streamlined installation and update flows, thanks to the tight integration with the Apple ecosystem. Testers get a nice native iOS app which surfaces new builds and testing release notes (“What to Test”), and there are shortcuts for sending beta feedback from the app to the developers. Testers can even take screenshots of issues and easily send them along with feedback. Apple also offers a native TestFlight macOS app, which is a nice alternative to App Store Connect on the web when it comes to managing testers and builds.
TestFlight has a few significant downsides. The obvious one is that it only caters to the Apple world, which means you’ll need two separate distribution platforms if your mobile org also ships Android apps. Also, TestFlight doesn’t offer a good way to group and separate different flavors of builds – every distributed build lands in the same big pile, so folks need to hunt around for the right builds and you run the risk of accidentally mixing pre-prod or staging builds with prod. Finally, because the tester experience is tied to Apple ID, it can be hard for testers to keep personal and work accounts separated.
Google Play Console (testing tracks)
Google’s closest analogue to TestFlight is a system of testing tracks offered within Google Play Console. Teams can distribute different flavors of Android builds using internal, closed and open testing tracks, with individual testers (via Gmail email address) and/or testing groups (via Google Group) added to tracks as needed. There are some limits involved. Internal testing tracks can have 100 testers only but builds aren’t subject to any review process. By contrast, limits for closed testing tracks are much higher (2,000 individual users, or unlimited if using Google Groups), but builds distributed to closed (and open) testing tracks are subject to Google’s review process, which can slow down the distribution of new builds to your testers.
The tester experience with Play Console testing tracks is relatively simple. For internal and closed tracks, once they have accepted an initial invitation email, testers can simply download new builds directly from the Play Store as they would for any other app. In the case of the open testing track, anyone and everyone who opts into being a beta tester for your app can also just download new beta builds from your Play Store listing. There’s a feedback flow for testers to provide their thoughts as well.
Here again, keep in mind that the Play Console testing tracks option will only cater to your Android team, so you’ll need to maintain different tooling for the iOS side of the house. Testing tracks do a decent job when it comes to bucketing different flavors of builds – you can spin up custom internal tracks as needed – but the interface and management of tracks can be a bit cumbersome. Not unlike TestFlight, the “native” experience comes at the expense of few integrations with other tools in your stack.
Firebase App Distribution
Google offers another distribution tool under their Firebase umbrella: Firebase App Distribution. Most notably, it adds support for distribution of iOS builds in addition to Android, and there are more features around the tester experience compared to Play Console’s testing tracks approach. Builds are distributed to different testing groups managed from the Firebase App Distribution dashboard. A maximum of 500 testers can be added to a given Firebase project, and no more than 200 testers can be added to the app distribution groups you can create to organize audiences. There’s no formal review process, so builds distributed through Firebase App Distribution become available to testers immediately.
For testers on Android, there’s a native app for viewing and installing new distributed builds. On iOS, lightweight functionality is offered via a Web Clip. There’s an optional SDK that developers can include in their app to automatically alert users about new versions and, though there’s no integrated tester feedback system, complementary pieces of the Firebase ecosystem (Crashlytics, Performance, and Test Lab) ensure you’re able to monitor stability and performance and run tests along the way.
While you can assign different audiences to builds, distributed builds land in one long list of “releases”, so there’s no distinction between or grouping of different flavors of builds (not unlike TestFlight). You’ll need to hunt through any and all types of builds you are uploading to Firebase in order to take next steps and get a given kind of build distributed to certain folks.
Microsoft App Center Distribute
Microsoft’s beta distribution product is officially called App Center Distribute but is more commonly referred to by its parent platform’s name (just “App Center”). It was partially born out of Microsoft’s acquisition of a different, much-loved distribution tool called HockeyApp. The big draw of App Center Distribute is its multi-platform support not only for both Android and iOS, but also Windows, macOS, and even React Native CodePush. Builds can be distributed to different testing groups managed via the App Center platform, and there are no limits on the number of testers.
The App Center Distribute tester experience is relatively basic, with no native apps and a somewhat clunky interface. However there is an SDK which allows for in-app alerting around new versions, and Microsoft has also implemented an auto-provisioning system which makes the process of registering devices and updating provisioning when new testers are added much easier.
In line with the recurring theme you may have picked up on by now, App Center Distribute lacks any kind of grouping or separation of builds. Every build you upload to App Center will be surfaced in a single list of “Releases” and it’s up to you and your team to distinguish between different flavors of builds. Something else to be aware of is that Microsoft seems to have put App Center Distribute into some form of maintenance mode as of 2022, so the product might not be at the forefront of options out there, increasingly so as time goes by.
TestFairy is an enterprise-geared distribution choice from Sauce Labs (perhaps best known for their continuous testing products). Both Android and iOS are supported, plus cross-platform frameworks both common (React Native, Flutter) and less common (Xamarin, Cordova). In addition to fully featured tester and test groups management, TestFairy offers a way to attach “tags” to builds to help categorize them – although uploaded builds still all land in a single list, which you’ll then have to filter down.
On the tester experience side, TestFairy offers install flows via email and auto-generated landing pages that you can customize and scope access to. The TestFairy SDK not only enables prompts for users on outdated versions to update, but it also bundles additional functionality around in-app bug reporting, session recording, and crash reporting.
TestFairy offers a handful of helpful integrations with other tools – for example, a Jira integration for attaching session info to bug tickets, and Slack and Microsoft Teams integrations for notifications. However, it’s worth keeping an eye on TestFairy’s packages and plugins as some aren’t super up-to-date. For enterprise folks, TestFairy’s focus on security and different SSO/SAML options are another draw.
Appaloosa is another distribution tool that tends to cater to larger enterprise customers, with a particular focus on mobile device management (MDM). For app distribution specifically, Appaloosa offers a system of internal app stores that you can create via their UI in order to keep different kinds of builds separated – helpful for “main” build flavors like betas and RCs, but a bit heavy-handed at any kind of scale for more ad hoc dev and feature builds. Testers and groups of testers can be assigned within each store, though note that pricing scales proportionally with user count.
Unsurprisingly given Appaloosa’s MDM roots, tester management and the overall tester experience is quite full-featured. Their internal app stores can be surfaced via the web or a native app, and testers will be alerted about new builds via email and/or push notifications accordingly. The Appaloosa SDK has been deprecated so testers leave feedback via the internal app store, and digests and stats are available to developers on the Appaloosa dashboard.
Appaloosa offers a few plugins to make integration with common CI providers a bit easier, and true to any enterprise offering, SAML-based SSO is available on higher tiers. Be aware that documentation is a bit sparse and certain plugins aren’t maintained super actively.
Your choice of app distribution tooling matters: get the right builds into the right teammates’ hands, easily and often
Creating a streamlined way for your team to get builds into each others’ hands throughout the entire development process – from early dev builds to nightly builds to RCs – has a big impact on app quality, developer experience, and productivity. As a result, your team’s choice of pre-production and beta distribution tooling is quite important. At a minimum, an app distribution tool will facilitate the process of getting binaries onto team members’ devices. But a good distribution tool will do more than just the minimum: it’ll also make it easier to distribute more kinds of builds more continuously, manage audiences for those builds, and collect feedback along the way. All of which can and will have a cascade of positive outcomes for your mobile org.