The Economics of Quality Score (Revoked)

Welcome to week two of my mea culpa tour. Last week I revealed an error from an earlier post on how quality score takes search queries into account. Today I’ll talk about some new facts regarding the most popular post I’ve ever written – The Economics of Quality Score.

An Economist Walks Into A Bar…

The original Economics of Quality Score post describes the impact of quality score on CPC. What was interesting about it, I think, is that it included two tables that purported to quantify the actual economic impact of quality score – how much CPC decreases when a keyword earns a 10 and how much extra you pay if a keyword only gets a 3, for example. The original calculation was based on visual quality score (see the guest post I recently wrote about visual quality score over at PPC Hero). Doing the math while assuming that quality scores are really whole numbers between 1 and 10 produced the tables included in the original post. Working with these numbers resulted in dramatic results and a powerful graphic that has been borrowed and republished in many blog posts and used in quality score seminars. The story the numbers told was that earning a quality score 10 gets you a 30% discount on every click, while suffering with a quality score for costs you a 75% CPC premium – to take just two examples. This calculation made the risks and rewards of quality score very clear. Or so it seamed. It didn’t take too long after the original post went up, to realize the mistake in these calculations. Quality score isn’t really a whole number between 1 and 10. So these results must be inaccurate. Oops. A disclaimer was added to the original post. The disclaimer explained the mistaken assumption, and concluded by saying that while the actual numbers in the chart were undoubtedly wrong, the point remains true – the positive and negative effects of qualty score did apply – and ‘hopefully the numbers are roughly proportional’. Which brings us to the new information. They’re not proportional.

Quadratic, I Didn’t Even Factor

There are many differences between visible quality score and the quality score number used to calculate CPC. Visible quality score is a whole number between 1 and 10. Quality score for CPC is a real number and the scale is non-linear. The premise of the calculations I did in the ‘economics’ post was that the distance between the numbers was known and constant, and if you divide any number by 7 and then divide that same number by 10, you will always get a 30% difference in your answer. This was intended to be revealing in terms of quality score. But since the math that drives your CPC involves numbers that aren’t between 1 and 10, and don’t have a predictible relationship to each other – and are a secret held inside a big blue safe in Building 47 on the Google campus – it turns out we can’t reasonably calculate or estimate the actual impact of quality score in CPC. We can’t calculate or estimate how much a 10 saves you vs a 7. We can’t calculate or estimate how much extra you pay for keywords with poor quality scores such as 3. Google hasn’t shared enough information for us to know.

Why Did The Quality Score Cross The Road?

There are at least three morals to this story. The first is that we still don’t know how any increase or decrease in quality score economically impacts your account. I suppose we could track individual keywords and try to find instances where quality score goes from X to Y while position remains constant and calculate the size of that change, and then after doing this a great many times build a new table based on observation. Of course, there are so many other variables in the system (different search queries, geographies, competitors, etc.) that it would take a huge amount of data to even have a chance at accuracy and in the end we’d never know. The second is that I should better verify the veracity of the information I post. I’ll work on that. The third is that Google is really good at hiding their secrets. The ability to actually know the amount of money a change in quality score was worth seemed like such a big deal because it represented a rare bit of clarity in the sea of uncertainty. We orient well around something as clear and familiar as a 1-10 rating system, but when we stop and think about it:

  • We don’t know the CTR’s that achieve any given ranking,
  • We don’t know how many auctions we’re being ruled ineligible for because of our score,
  • It’s extremely hard to know how queries or geography or ad performance impacts our score, and
  • While we know ‘higher is better and lower is worse’ we have no way of knowing how much better or how much worse.

It’s like the perfect carnival game – it seems like getting the coin to land on the plate is easy and the variables are within our control… So in the end, another mystery not solved. I promise that the new book does get to the bottom of a few. Quality Score in High Resolution New 250-pg paperback by Craig Danuloff   Pre-Publication Learn more and order your copy today.

Acquisio Admin

Acquisio Admin

The First Machine Learning Marketing Platform
Built to Scale Search for Local Resellers & Agencies

Automate, optimize and track more campaigns, more profitably.