+6 votes

Assuming anything is possible and in the context of non-game subscription apps...

  1. Top 3-5 key metrics you absolutely want in there for each channel?
  2. Things you typically shouldn't bother with at a high level (and can look up in each UA platform interface)?
  3. Anything that was life changing when you implemented it in terms of answering channel profitability questions?
  4. Any blind spots you've seen people ending up with resulting from non-optimal reporting implementation?

If you have any resource recommendations on creating the right dashboard(s) (e.g. in Tableau - but the tool doesn't really matter) to monitor/optimize UA, please share as well.

Thanks!

by (290 points)

2 Answers

+2 votes
Best answer

This is a great question!

This is what I consider to be a "minimum viable report" for mobile UA (each of these would be presented by ad set / campaign / channel, as available):

  • Some selection of DX ROAS, given the date range being considered (eg. Day 1, 3, 7 if I'm looking at the last 10 days of data);
  • Predicted DX ROAS for whatever timeline I am buying against (eg. Day 180);
  • A time series of whatever optimization metric is being acquired against (eg. Cost per Payer, Cost per DX Retained, Cost per Install);
  • Spend / install metrics (if I have a report with a filter console, I want to be able to very quickly toggle cost / installs metrics between some traffic filter and the topline, overall numbers);
  • I like to see the composition of DNU over time -- the % of DNU represented by the various channels. If there's a change in composition, that change can manifest in other metrics (which is why it's important to not look at topline metrics in isolation in the first place);
  • I like to see ROAS curves for monthly cohorts going back some months (eg. 6) to know if ROAS is degrading over time. These are all indexed from Day 0 so they can be easily compared along the X axis 
by (12.4k points)
selected by
And is there a tool that makes looking at a report like this easy?
You can do this in Tableau or even Google Sheets. It's really just a set of graphs and then a data table with ad sets running along the Y axis and the various metrics running across the X axis
Thanks a lot for your detailed answer Eric, very insightful. I do have follow up questions that would help understand further.

1. Would you have this separately for iOS and Android?

2. By "presented by ad set / campaign / channel" do you mean that you would for example have a "view by" filter that lets you see each chart mentioned either:
- at the channel level (e.g. comparing selected channels)
- at the campaign level (e.g. comparing selected campaigns from selected channels)
- at the ad set level (e.g. comparing ad sets from selected campaigns from selected channels)
This seems like it would be hard to read quickly.
Or can you "only" compare (i.e have on a chart at the same time) campaigns from 1 channel, or ad sets from 1 campaign?
(if this is not clear, can you just expand on "presented by ad set / campaign / channel"?)

3. Regarding the "time series"
If you have different campaigns optimizing towards different metrics (different objectives), would you then have a time series with all these metrics (e.g. Cost per Install, Cost per Registration, Cost per Purchase)? Or would you just focus on the one key metric regardless of what you're optimizing for in each campaigns (e.g. Cost per Purchase if that's what matters the most)?

4. Regarding "Spend / install metrics" toggle
Could you detail what you mean there? Do you mean being able to overlay spend and/or installs over the time series of optimization metric mentioned above (for example # of Purchases - I'm assuming this would be more relevant than vs. Cost per Purchase) so you can understand correlations with performance of the optimization metric?

5. ROAS curves
Something like this (from the Liftoff Dashboard)? https://liftoff.io/wp-content/uploads/2019/02/ROAS_by_day.png

6. Organic
I assume that when you were talking about installs it was paid UA installs. Would you also monitor organic as well somewhere? Maybe have a way to compare your optimization metric (e.g. # of Purchases) for "paid only", organic and total (paid + organic).

7. Finally, it's my understanding that you would leave things like CTR, CVRs, etc. out of this high-level report and to be looked at in each platform when optimizing campaigns, ad sets, ads. Is that correct?
1. Would you have this separately for iOS and Android?

-> Yes! Absolutely.

2. By "presented by ad set / campaign / channel" do you mean that you would for example have a "view by" filter that lets you see each chart mentioned either:
- at the channel level (e.g. comparing selected channels)
- at the campaign level (e.g. comparing selected campaigns from selected channels)
- at the ad set level (e.g. comparing ad sets from selected campaigns from selected channels)
This seems like it would be hard to read quickly.
Or can you "only" compare (i.e have on a chart at the same time) campaigns from 1 channel, or ad sets from 1 campaign?
(if this is not clear, can you just expand on "presented by ad set / campaign / channel"?)

-> In general, I'd have this at the lowest level of granularity -- there's no point in having a campaign vs. campaign view. What I like to have is a data table that breaks out every single Ad Set by some set of metrics. These don't need to be grouped at any higher level of hierarchy because what I really care about is ad set performance.

3. Regarding the "time series"
If you have different campaigns optimizing towards different metrics (different objectives), would you then have a time series with all these metrics (e.g. Cost per Install, Cost per Registration, Cost per Purchase)? Or would you just focus on the one key metric regardless of what you're optimizing for in each campaigns (e.g. Cost per Purchase if that's what matters the most)?

-> Ultimately I only really care about ROAS! Deciding what to optimize for requires analysis, which is different than a report. I might include those metrics in a data table or chart but I only am focusing on ROAS.

4. Regarding "Spend / install metrics" toggle
Could you detail what you mean there? Do you mean being able to overlay spend and/or installs over the time series of optimization metric mentioned above (for example # of Purchases - I'm assuming this would be more relevant than vs. Cost per Purchase) so you can understand correlations with performance of the optimization metric?

-> Yes, I mean I want to see spend and install counts filtered by some feature (eg. iOS, FB, US, yesterday), but I also want to be able to roll all of that up into topline values for tracking against my target budget

5. ROAS curves
Something like this (from the Liftoff Dashboard)? https://liftoff.io/wp-content/uploads/2019/02/ROAS_by_day.png

-> Yes, that looks reasonable

6. Organic
I assume that when you were talking about installs it was paid UA installs. Would you also monitor organic as well somewhere? Maybe have a way to compare your optimization metric (e.g. # of Purchases) for "paid only", organic and total (paid + organic).

-> I usually break Organic out into its own filter so I can segment that traffic and look at its specific quality metrics

7. Finally, it's my understanding that you would leave things like CTR, CVRs, etc. out of this high-level report and to be looked at in each platform when optimizing campaigns, ad sets, ads. Is that correct?

-> Yes, most likely, those aren't very relevant at a high level (I'd want those in the ad set level reporting)
Thank you Eric!
+1 vote
1. Mine would be (per ad set/campaign)

- blended return/active ROAS at various points (3d, 7d, 14d, but various a lot depending on the title and UA strategy)

- Outbound CTR

- CPM

- CPI

- IR

- CPP

- # Unique payers

2. If running worldwide UA, often various country splits just muddle the picture.

3. For me that was tracking daily blended return for every single ad set (see below) -- it improved the agility of decision making.

4. Depending on the selection of metrics, I've seen people killing good campaigns and dragging the bad campaigns for longer. The most commonly overlooked metric is the number of unque payers, when one random whale "feeds" the losing campaign for weeks.

I've been using a very simple template to track daily ROAS by manual entry from the FB Ads Manager: https://docs.google.com/spreadsheets/d/1KbRzSemRn5ROYsl1jEfiodHmZI73IqF40woa2Es4tnI/edit#gid=1292744512

The template tracks blended return/active ROAS at various days which different depending on your title. I used it daily to assess the state of my campaigns and to make changes, which are also logged in this file, so you can see all the historical changes in there.

I know this might not be enough for everyone, but especially if your UA strategy is ROAS-centric and you know your KPIs, I found it really useful as a reporting and decision making tool. Ideally this one would be coupled with some of the more visual tools that ingest data directly from your accounts via API to give's you a bird's eye view on various metrics.
by (180 points)