What’s the hardest part about monitoring mobile growth?
There’s nothing more frustrating than being in a meeting with folks from the growth team, the marketing team, the user acquisition team, and management, and trying to pinpoint the cause of a drop in app install volume.
We’ve seen it many times: large, global mobile brands can spend months and months trying to understand their mobile growth performance with nothing more than a long list of hypotheses that change based on the person/team coming up with them.
At the end of the day, a mobile-only (or a mobile-first) company lives by its performance in the App Store. After all, that’s the only source for new app users. The app stores are full of apps and games that were abandoned and marked as a failure after they couldn’t figure out how to maintain long-term, positive growth.
As Adjust once showed in their Zombie App Report, more than 80% of apps never get meaningful growth traction and remain mostly “invisible” in the app stores. We believe that the number is much higher these days.
This problem, like any problem in business, can be broken into small pieces to come up with a solution.
- How do you define a change in your growth performance?
- What are the core types of change events that happen?
- How do you identify a change in real-time?
- How do you get to the root cause of a change?
How do you define a change in mobile install growth?
When you think about it, change is not a straight-forward thing to define. The app stores are noisy environments. Almost every metric is aggressively moving up and down and you can’t conclude that every pop or drop is in fact a change in the overall trend.
A trend of what? Basically every metric that’s important to you (app store impressions, installs, referral traffic, conversion rates, etc.) and every KPI that you’re trying to improve has a set of metrics that influences it. Understanding changes in metrics that are predictors of the KPI change would help you get a head start before the KPI itself starts to drop.
On the other hand, you wouldn’t want to find yourself weeks into a declining trend and only then conclude that something has changed. You want to stay on top of these changes so you can get to the bottom of them and attempt to do something about it.
So to start, we need to limit our definition of ‘change’ to only those changes that are statistically significant in the rate of growth over the selected period of time.
Why is the selected period of time important? Because if you look at the data daily, you’d conclude that every day was a change! It works like this:
- Look at past data and ‘draw’ a metric growth trend line.
- Create a range. If the metric continues to move within the bounds of that range, you’ll conclude you’re still on the same trend.
- Collect data every single day and add it to your model of that trend, until enough data shows that the metric ‘got away’ from that trend.
- ‘Draw’ out a new trend line.
For example, tracking organic impressions of your App Store listing in search results and on the to measure your app discoverability would ensure that you know as soon as something happened and changed the trend for the best or for the worse.
If you had that ‘magic sense,’ you could start analyzing every factor impacting your discoverability to correct course.
Change doesn’t have to be negative, by the way. You want to know when something ‘good’ happened to improve your growth trend so you can identify and do more of it.
What core types of root causes for change events exist?
After identifying a change, you need to dig into its root cause. In the mobile app industry, there are several groups of potential root causes for change that are worth knowing:
- Featuring event: Getting featured on the App Store or Google Play Store can drive a radical increase in browse/explore traffic and brand awareness.
- Version releases: Releasing a new version can significantly influence top of the funnel metrics and growth, especially in the Google Play Store where you get penalized for having a high crash/app-not-responding event rate.
- Changes to paid user acquisition: Changes in paid UA strategy usually create a sizable ripple effect. Driving a significant volume of installs through paid ads can affect both discoverability (category/top charts ranking and search rankings) as well as brand awareness. This includes changes in paid search ads within the App Store.
- Changes in offline marketing: Any type of TV campaign, billboard campaign, or non-digital marketing campaign can affect brand awareness and, in turn, branded search traffic.
- Changes in App Store creative: App Store creative changes that influence conversion rates can create change for both paid and organic performance.
- Changes in App Store keyword strategies: Changing keyword strategies will often influence the volume and the mix of search traffic you’re getting.
- Changed to/by competitors: A new competitor in your category, or a sudden increase in UA budgets at a competitor, can also lead to changes in organic/paid performance.
- Changes to App Store algorithms: The algorithms impact the rankings in top charts/categories, the personalized sections of the stores, or search rankings and any changes in these algorithms can influence performance.
This is not a comprehensive list, but after identifying a change in the performance of a metric you track, you would ideally map it to one or more root causes, which helps you get to an actionable next step.
What’s the problem with monitoring change events?
They’re tough to monitor. Data doesn’t sit in one place, and getting all the data you need requires creating a single source of truth for the company.
For example, if you’re tasked with improving the discoverability of your app, you need data from the App Store consoles, market intelligence data to track rankings, and even attribution data to track the level of paid traffic from the different channels.
Let’s say you’re trying to maximize your search performance using paid App Store search ads. You’ll need to track not only paid search data coming in from the search ads console but also total search data from Remove featured image App Store Connect as well as ranking data to make sure you allocate budgets accordingly and avoid cannibalization of organic search.
Only one single source of truth allows for true monitoring of the metrics that matter.
How do I know which change-point I got hit with?
After a change was identified, how do you know which potential root cause has happened?
Please note that running analysis from top to bottom and going through every potential cause isn’t feasible over time. A better alternative would be to create a ‘map’ which ties different types of changes to different metrics to potential root causes.
The bottom line is that the App Store and Google Play ecosystems are pretty closed. In any closed system—no matter how complicated it is (such as a car’s engine)—there is an inherent cause and effect relationship under the hood (no pun intended; well, it was intended, sorry).
So it’s as if your engine overheated: a mechanic would immediately check two or three things that might have caused it. The same rule applies with drops to organic app impression volume, for instance, or a ranking drop.
It isn’t easy to create such a map, but it’ll save you a ton of time in the long run.
After creating this map, it’s only a matter of validating which of the potential causes actually occurred.
What framework should be used to combat organic and paid performance drops in mobile growth?
A strong framework includes four steps.
- Identify the metrics that affect the KPIs you’re trying to drive.
- Start monitoring these metrics and create alerts every time one of them starts changing trends.
- Using your cause and effect map, drill down to the root cause of the change.
- Based on the root cause strategize, fix the problem and go back to monitoring.
Following such a methodology ensures that you’ll very rarely experience the worst kind of growth problem: the problem that could’ve been avoided if only, only, someone would’ve looked in the right place at the right time.