If you've jumped into ads manager and cranked up a Facebook Ads campaign, you've certainly noticed the 'delivery' column which gives you insight into whether your adset has completed the 'learning phase' or not.
Marketers disagree on how much impact completing the learning phase has on result... if any.
I want to know!
The Learning Phase & Theory
In theory the learning phase is supposed to be the period in time when Facebook's machine learning algorithm is 'learning' to optimize for your chosen optimization event. After completing learning you should (according to Facebook's info) see more stable results and a lower CPA.
To complete learning, you need around 50 optimization events within a week.
How The Test Would Be Set Up on Facebook Ads
Let's jump into it!
I needed an adset that would have a high enough budget to get the conversions required to complete the learning phase and then test that against an adset, exactly the same, that couldn't complete the learning phase because it was limited by budget.
This is important for data integrity... I would let them run until one adset exited the learning phase, and then compare the results for the time period that one adset was out of learning phase while the other was till in the learning phase.
###
I had an adset in a campaign targeting conversions, that had ran successfully, but it never got out of the learning phase.
Users would click through the Facebook Ad, hit my landing page, then fill out a lead form on the landing page to request an estimate for remodeling services. The conversion event was reaching a thank you page after submitting the form.
I duplicated the adset above and used the feature to create a split test for the adset.
I then set the daily budget to $75 for the one of the adsets and then $25 for the other. I would adjust this up as necessary to get the adset with the larger budget to complete learning.
Just to clarify...
- The audience was exactly the same (9.9 million).
- The ads were exactly the same.
- The ads used the same Post ID, so any likes/comments/shares (social synergy) would apply to each variant.
On October 21st I started the test.
This is big.
Most 'marketers', try to tweak their ads way too much and subsequently.
The more experienced (and successful) I've become with Facebook advertising, the less editing I do.
Day 9: Adset Exits Learning Phase
Finally on October 29th, the higher budget adset 'completed the learning phase with 62 conversions events. At this point $822.04 had been spent.
As you can see, the two of them performed about the freaking same! One got conversions at $10.00 while the other got conversions at $10.03.
While interesting, it's irrelevant. What matters is how the 2 compare with one having completed the learning phase and the other still stuck in the learning phase.
The period that will matter is October 30th and onward, so let's go on.
Results After 14 Days Since Completing Learning Phase
Fast forward to November 12th and the test has been running for 14 days since one adset completed the learning phase. Now we have something valuable to look at!
Here goes the money shot.
The adset that had completed the learning phase got conversions at $9.64 while the adset that was perpetually stuck in the learning phase got conversions at $13.00.
That's a material difference!
We could say;
> The adset that completed the learning phase got us results 26% cheaper.
or
> The adset that was stuck in learning phase was costing us 35% more to generate the same results.
It's clear, completing the learning phase does have a benefit.
Going forward...
STOP MAKING EDITS AND LET YOUR ADSETS COMPLETE THE LEARNING PHASE.
Making little changes here and there that are negligible really won't move the needle anyhow. At the core, direct response marketing is a factor of the audience and the offer. Facebook's algorithm will probably do a better job of putting your offer in front of the right people if you let it do the work.
Related:
(yes that is 2 Comma Club award over my shoulder)