Google right rail top 4

What We Learned About Google’s Top 4 Change From 90,000 PPC Campaigns

In February, Google began slowly making changes to the way ads are displayed on their desktop search results pages (SERPs). By the end of the month, ads in the right side had all but disappeared, and had been replaced by four ads at the top of the page, and, occasionally, two at the bottom, below the organic results. This is, perhaps, the most significant change to the way ads are displayed on SERPs ever.

We had several PPC experts give us their prognostications on what might happen for PPC advertisers — especially those with smaller budgets — as soon as it happened, and now that we’ve had a chance to really look at the data, we’re sharing our findings with you.

To assess the impact of these changes, Acquisio has extracted data from approximately 90,000 campaigns running on the Acquisio platform, with a combined spend of about $50 milllion per month. We examined the four weeks before the change became prevalent (January 21 – February 17, 2016), and the two weeks following (February 19 – March 3, 2016).

Our conclusions are:

  • Impressions and clicks below the top 4 have dropped significantly
  • CPC for spots below the Top 4 has increased
  • CPC for ads in the Top 4 has increased by as much as 10.5%
  • CTR for ads in the Top 4 has increased by as much as 4.5%
  • PPC campaigns optimized with machine learning algorithms outperform all other campaigns

The data we have gathered shows that competition for ads in the top 4 position has gotten much stronger. Those ads perform at much higher rates than those in the bottom spots, but advertisers will have to pay more for the privilege of showing up in those spots.

We also examined campaigns running with Acquisio’s Bid & Budget Management (BBM) machine learning optimization, and found that those campaigns outperformed all other campaigns. It’s one more example of how machine learning will outperform humans, and the importance of optimizing campaigns with the highest level of intelligence available.

Let’s take a look at the data and see how Google’s new ad display scheme on SERPs is affecting advertisers.

The methodology we used to assess the data

First of all, to reiterate, we used data from approximately 90,000 campaigns, so, at this point, we are very confident in the data. The results are from a very large, varied and representative sample of adwords advertisers. The numbers you will see in this section are percentage changes in absolute values.

We were curious to see, not just how all campaigns performed, but how our bid and budget management feature performed under these new conditions. We looked at:

  •        BBM optimized campaigns vs campaigns not optimized by BBM
  •        Before time frame:  21 Jan -17 Feb 2016 (4 weeks)
  •        After time frame:  19 Feb – 3 Mar 2016 (2 weeks)

We looked at campaign average daily values of: Impressions, Clicks, Spend, Position, CTR, and CPC for each campaign. In our spreadsheet, it broke down like this:

  1.      Average values for all key values for all of these cases
  1.      Comparing difference percentage of before and after: for each of the other 4 cases:
  1.      Comparing difference of Top 4 and bottom for the 4 other cases:
  1.      Comparing % change of the before and after change for each of these cases:

And here’s what we found out.

Impressions, clicks, and ad spend below position 4 dropped off significantly

Regardless of whether or not campaigns were running on BBM, we saw very significant drop-offs in the number of impressions and clicks below position four. The number of campaigns running in those spots also dropped off. These are the numbers for the two-week period after the change, as compared to the period previous to the change.

Results Without BBM

For regular campaigns not running with BBM optimization, we saw the following for ads at the bottom:

 Impressions -32.0%
 Clicks -44.8%
 Spend -38.3%
 Campaigns below Top 4 -27.4%

And here’s where it gets really interesting: CPC for spots below position 4 went up, while CTR dropped dramatically.

 CPC +13.6%
 CTR -20.0%

Results With BBM

The results for campaigns optimized by BBM were actually not that different until we get to CPC and CTR. For BBM, the results at the bottom went like this:

 Impressions -49.7%
 Clicks -49.7%
 Spend -47.3%
 Campaigns below Top 4 -21.9%

The difference starts to be really noticeable when we look at CPC and CTR. This was actually a very positive finding for us, given the numbers we see above. The CPC did increase, but only by a small margin, and while the CTR did drop, the percentage by which it dropped is so negligible as to be unnoticeable.

 CPC +4.9%
 CTR -0.057%

We already knew that ads on the right rail were only getting about 15% of the total clicks on SERPs, so it should not come as too much of a surprise that ads that are actually being displayed at the bottom of SERPs should get such low scores.

What we were pleased by, obviously, was that BBM is seeking to optimize CPC and CTR despite the drop in impressions and clicks. What we all want to really know about is how the change affected those who are managing to make it to those Top 4 spots.

CPC and total spend in the Top 4 went up significantly

We start to see more differences in the way regular campaigns behave as compared to BBM-optimized campaigns when we look at the Top 4. For non-BBM campaigns, we saw a tiny drop in impressions, just a little increase in clicks, and a big jump in spend. Again, we’re comparing the period after the change to the period before.

Results without BBM:

Here’s what we saw in the Top 4 positions for non-BBM campaigns:

 Impressions -1.5%
 Clicks +3.0%
 Spend +13.7%
 Campaigns in Top 4 +2.44%

The only thing that has changed significantly here is the amount of money that people are spending on getting their ads in the Top 4 positions. For those same campaigns, we found that their average position dropped by only 1.9% — basically zero — so they’re spending more to just stay in the game, and they’re losing ground while they do so. The competition is really heating up for those spots.

The price for a click has gone up for ads in the Top 4, but so has CTR. Here’s what we found:

 CPC +10.5%
 CTR +4.5%

The lift in CTR (4.5%) more than makes up for the 1.5% loss in impressions, however the CPC increase is so much higher that in the end advertisers get fewer click for, essentially, the same amount of money.

Results with BBM

Machine learning is definitely keeping these campaigns in the race. While ad spend increased, so did impressions and clicks. And the number of campaigns with ads in the Top 4 increased significantly.

 Impressions +9.8%%
 Clicks +8.0%
 Spend +15.8%
 Campaigns in Top 4 +12.3%

These are really significant jumps from campaigns that are not running on BBM. In this case we do see that the spend is up, but that is accounted for by the increased number of clicks, as well as, based on the data from regular campaigns, an overall increase in ad spend.

We also saw an increase in CPC with BBM, but not as much as non-optimized campaigns (41% lower price increase, +6.6% vs +10.5%), as well as a slight drop in CTR.

 CPC +6.6%
 CTR -1.4%

What this means is that if you spent $1,000 on advertising on February 17, 2016, then on February 19 you would have spent $1,105 without BBM. If you were a user of BBM, you would have only spent $1,066 on February 19, saving your $39 that day. That’s your 41% savings on the increase in CPC due to the new configuration. So, over the course of 30 days, that would translate into a savings of $1,170.

What this really means is that anyone who is advertising on large scales and is not using machine learning to optimize campaigns will be missing out on a vast amount of clicks.

Also, in contrast to non-BBM campaigns, the average position for these ones actually increased by 0.94% — which is to say that it really didn’t change at all. Let’s call this number zero and say that there was no change.

Looking at the overall difference machine learning makes

One of the most important lessons from this Top 4 change is the stark difference the 2 possible ad locations:

  • Top 4 positions
  • Bottom positions

When examining the data both before and after the change the production from the Top 4 totally dominates, and after the change this is accentuated. For this reason we decided to analytically examine this issue.

Looking at the ratio of values between Top 4 and Bottom positions finds:

BBM Status Before/After Ratio[Imp] Ratio[Click] Ratio[Spend ($USD)] Ratio[#Campaigns]
Not on BBM  Before  10.76  26.76  23.01  4.04
Not on BBM  After  15.61  49.93  42.40  5.71
BBM  Before  6.61  14.18  14.91  2.79
BBM  After  14.42  30.48  32.78  4.02

The value formulated for each of these cases is:


This shows number of impressions and clicks from the Top 4 positions is vastly larger than the Bottom with similar comments about spend and number of campaigns. After the change to the new Top 4 configuration without a right rail the number become even more exaggerated.  The percentage change of the the Top 4 to Bottom ratio after the change:

BBM Status Per[Ratio[Imp]] Per[Ratio[Click]] Per[Ratio[Spend ($USD)]] per[Ratio[#Campaigns]]
Not on BBM  45.01%  86.58%  84.25%  41.21%
On BBM  118.10%  114.92%  119.93%  43.79%

The formula used is:


These values show that campaigns that are not optimized have significant growth in the Top 4 to Bottom ratio in all cases. The BBM case is even more dramatic with the ratios for impressions, clicks and spend all more than doubling.

The BBM ratio growth ratio growth compared to the non-optimized case shows:

  • 162% higher Impression ratio growth
  • 33% higher Click ratio growth
  • 42% higher Spend ratio growth
  • 6.3% higher campaign ratio growth

The BBM ratios show a strong shift after the Top 4 change to greatly extend its campaigns’ ownership and presence in this prime Top 4 advertising real estate. BBM puts more of your ad spend where it performs best: in the Top 4.

What we’re really seeing here is what everyone expected. The vast majority of impressions, clicks, and spend are in the Top 4 spots. Saying this would happen before the change was a  bit like predicting that the ground will be wet soon when standing under a rain cloud. The good news is that it seems everyone was right.

But what really gets us excited is the performance of BBM in this new environment. Machine learning optimization has adapted and automatically evolved under the new paradigm, and you can clearly see that BBM is outperforming all other campaigns where it matters, and by very wide margins.

Staying in the competition for the Top 4

For the most part, campaigns are costing more, and when compared to the cost, most advertisers are not getting enough clicks to justify the increased spend. For advertisers not using machine learning to optimize campaigns, they will continue to find that:

  • Impressions and clicks below the Top 4 spot have dropped by almost half
  • Clicks for ads in the Top 4 have increased slightly, but spend has increased dramatically
  • CTR for ads in the Top 4 has increased slightly, but CPC has increased dramatically

The results are interesting, whether you’re using BBM or not. With clicks costing more and ad spend going up, Quality Score as a metric is more important than ever, and advertisers need to put more and more effort into making sure that they’re getting as many conversions as possible on landing pages.

Advertisers who run brand awareness campaigns will be forced to attempt to do more with less, and may seek to leverage social media channels and virality in an attempt to get more out of their campaigns.

What is absolutely clear is that machine learning optimization for PPC is far and away outperforming humans at this point, and taking advantage of optimization like this will be necessary to compete in the future.

For the benefit of clarity, let’s compare the resulting increase to a tax imposed by Google for appearing in the Top 4 results. The good news is that users of BBM will pay 41% less tax, to continue with the analogy.

One more time: BBM-optimized campaigns were faced with a slightly higher CPC than before the change, but a 41% lower CPC increase than the campaigns that were not optimized.

Anyone interested in learning more about how machine learning can optimize PPC campaigns should check out this webinar with Bryan Minor: 2016 The Year in Machine Learning. Bryan explains how our machine learning optimization works, and shows you real results. Make sure to check it out!

Mark John Hiemstra

Mark John Hiemstra

Mark John Hiemstra is a content and social media marketing consultant to Acquisio. He got involved in digital marketing purely by accident, and has spent the last several years learning about building relationships with consumers online through the many channels available to those who are willing to get involved. He is loathe to discuss himself in the third person, but can be persuaded to do so, from time to time.

The First Machine Learning Marketing Platform
Built to Scale Search for Local Resellers & Agencies

Automate, optimize and track more campaigns, more profitably.