Quality score overview

Auctions have long proven to be an efficient way to balance incentives between buyers and sellers by ensuring that goods go to the highest bidder. This method works well in one-time transactions between buyers and a seller, where price alone determines the winner.

Online advertising is different. Unlike a traditional auction where a single item is sold, digital ad auctions happen continuously, triggered every time a user visits a publisher’s site. Here, there is a third stakeholder in the marketplace, the user, whose sustained participation is critical to the long-term success of the ads program.

Not all ads are created equal— some engage users and connect them with the products and services they need, while others annoy and frustrate, wasting advertising spend in the process. This can be especially true within commerce media networks, where site experience is paramount to ensuring not just ad sales, but ecommerce purchases at large. 

Without safeguards, ad auctions prioritize only the highest bid, creating the potential for intrusive or irrelevant ads to dominate advertising. Over time, this degrades the user experience, reducing engagement and ultimately driving users away. This harms both commerce media networks and advertisers, causing users to abandon the publisher altogether and ruining any opportunity a relevant advertiser would have to promote their product or service.

To prevent this, commerce media networks and ad platforms introduce Quality Score—a necessary extension of auction theory that accounts for user experience alongside price and competition. By incorporating ad relevance, expected engagement, and performance into ranking, Quality Score ensures that the best ads—not just the highest bids—win. 

What is a quality score?

Quality Score is a machine learning or rules-based mechanism for including ad relevance and expected performance in auction-based advertising platforms to determine ad placement and cost-per-click (CPC). The goal of quality scoring is to reward relevant ads with a better placement and lower cost. 

Let’s take the example of a user, Fred, who opened up his food delivery app in search of a “healthy lunch.” A national fast food chain specializing in greasy food bids $6 on Fred’s search because they have the advertising dollars to spend, while a local restaurant selling quinoa bowls can only afford a $4 bid. If price alone dictates the winner, the national chain shows their ad to Fred. Even if he clicks on the ad, he is unlikely to purchase from this chain. Frustrated because he cannot easily find any healthy options on the food delivery app, he closes out the app altogether and opens another with a better user experience. 

Several things happen in this case:

• Fred isn’t finding what he’s looking for, so he’s likely to try a different food delivery app, which is bad for the food delivery app’s business.
• The fast food chain is spending too much for an ad that isn’t likely to convert.
• The better ad in this case–the local healthy restaurant–is suppressed, creating further inefficiency in the system.

In this example it’s clear that if we had an effective way to represent Fred’s interests in the auction, the healthy option would win. This is precisely what Quality Score is designed to do.

At its core, Quality Score helps balance the incentives of a three-sided marketplace, ensuring that publishers, advertisers, and users all benefit from the auction dynamic. Without it, auctions would devolve into pure bidding wars where only the highest-spending advertisers win, often at the expense of user experience and publisher value. 

The history of quality scoring

The concept of Quality Score was first introduced by Google in 2005 as a way to improve ad relevance and overall user experience within its search advertising platform, Google AdWords (now Google Ads). Before this, the ad ranking system relied primarily on maximum bid amounts, meaning advertisers with the highest budgets could dominate search results, even if their ads were irrelevant or low-quality.

Google’s introduction of Quality Score fundamentally changed search advertising by incorporating ad relevance, click-through rate (CTR), and landing page experience into the auction mechanism. This shift favored ads delivering better user experiences, not just those backed by deep pockets.

The move was a response to several key challenges:

• User experience concerns: Poorly targeted ads led to low engagement and frustration.
• Advertiser inefficiency: High CPCs without relevance-based adjustments meant advertisers often overpaid for underperforming placements. 
• Auction sustainability: A relevance-based system created a more balanced and competitive marketplace where advertisers could optimize their strategies beyond just raising bids to win.

Since then, Quality Score has become a cornerstone of modern ad auctions, influencing not only search ads but also display, video, and commerce media. Today, leading ad platforms and commerce media networks prioritize partners who deliver high-precision, AI-driven Quality Scoring models to maintain both advertiser performance and user experience.

How quality scoring works

Most modern platforms compute a Quality Score in real time for every advertiser participating in an auction.

The particulars of Quality Score on any given platform are typically trade secrets, but often include:

• Expected click-thru-rate (CTR). CTR is an estimation of how likely the user is to click on the ad, and hence often a useful component of ad relevance.
• Ad Relevance with respect to search terms. If a user searches for hammer, they don’t want Arm & Hammer baking soda, or hammerhead shark toys, so these ads should be ranked lower.
• Other engagement metrics. Aside from CTR, conversion rate, dwell time, and other post-click metrics can help hone relevance. 

Once Quality Scores have been produced, they are combined with advertiser bids to produce what is called the ad rank defined by the following equation:

ad rank = Quality Score × Bid

The auction rank is then determined by ad rank instead of bid. Let’s take a look at an example where we have four advertisers participating in an auction. 

In the table, we’ve taken the bids and multiplied by Quality Scores to obtain ad rank. If this were a traditional auction, the highest bid, $4.00, would win. However, by accounting for Quality Score and re-ranking by ad rank we see the new winner is actually the lowest bid, $1.98, and the highest bid actually ends up ranking last because the associated Quality Score was so low. 

While this dynamic, where the top and bottom bids effectively switch places, is not the most common scenario, the example serves to illustrate some of the impact Quality Score can have on auction dynamics. 

Why it matters that quality scoring is done right

Quality Score is more than just a mechanism for ranking ads in auctions–it’s the foundation of a sustainable, high-performing advertiser ecosystem.

From the publisher perspective, Quality Score:

• Maximizes revenue while protecting the user experience. It prioritizes high-quality, relevant ads so that publishers can achieve higher user engagement, which leads to better long-term monetization.
• Encourages healthy auction competition. Quality Score prevents a race to the bottom where only the highest bidding advertisers win by allowing small but relevant advertisers to compete effectively.
• Protects brand integrity. Poor-quality ads drive users away. A good quality score ensures the ads on the page align with user expectations.

Advertisers, meanwhile, can expect:

• More meaningful engagement. Ads served based on relevance to the user result in more clicks and conversions.
• Better budget utilization. A well-implemented Quality Score ensures a fair and consistent auction, allowing advertisers to confidently invest in campaigns knowing that high-quality ads will be rewarded with stronger visibility and performance.

Last, but not least, end-users benefit from a better browsing and shopping experience. Putting the focus on the user experience leads to replacing intrusive and irrelevant ads, with useful and engaging experiences.

So does quality scoring mean publishers make less from each auction?

At a first glance, it may seem like Quality Score lowers CPCs for high-quality ads, but in reality it creates the conditions for long-term value in marketplace health. Here’s how:

• High-quality ads increase user engagement, which means more clicks and conversions. 
• An engaged user base drives better ROI for advertisers, who then invest more in the platform.
• More advertiser investment leads to stronger competition and a more durable marketplace, which is the foundation for sustained long-term publisher revenue.

Going back to our first example—Fred, hungry for a quinoa bowl, opens the food delivery app and gets hit with an ad for a fast-food burger instead. Maybe he clicks, maybe he doesn’t. If he does, the marketplace squeezes out a couple more dollars from the fast-food chain. More often than not, though, Fred doesn’t click. He gets frustrated, closes the app, and doesn’t come back the next time he’s hungry. No one makes money and the platform loses a user over a bad experience. Short-term, the extra $2 from an irrelevant ad might seem like a win, but it’s a losing strategy in the long-term.

Challenges publishers face when launching quality scoring

Implementing a great Quality Score is technically complex and resource-intensive. While the benefits are clear, there are multiple algorithmic, infrastructure, and operational hurdles to overcome to do it effectively.

Creating, deploying, and maintaining a great Quality Score model that evolves with your business is not a one-and-done activity. Instead it requires consistent investment in tech and machine learning talent and infrastructure for a few reasons:

• Data complexities. The very first challenge lies in collecting and combining the appropriate inventory, campaign, user, and other data in a form that’s suitable for machine learning.   
• Algorithmic complexity. AI and machine learning are rapidly evolving fields. While it’s possible to get up and running with simple models, the ability to train and deploy more sophisticated models and ensembles is the difference between a decent Quality Score and a really great Quality Score. 
• Low latency. Every publisher knows that the first step in creating an amazing user experience is ensuring the site is responsive and users aren’t waiting for ads to load. Ensuring that real time models such as Quality Score are both fast and accurate requires investments in state-of-the art machine learning techniques as well as infrastructure to support them.
• Continuous learning. Businesses experience a number of seasonalities ranging from holidays to time-of-day, depending on vertical. Ensuring your Quality Score model remains performant in these changing backgrounds requires constant model updates and re-deployments.
  

The importance of being able to impact your quality score

While Quality Score is foundational for a robust marketplace, ensuring you get the most out of your ads programs requires injecting your specific business logic into the system while maintaining a high bar for user relevance. A common example of this is giving preference to higher margin items, sacrificing a little ad revenue in the process. The ability to inject this type of logic, as well as the method for doing this varies widely across platforms. Integrating your own quality score or publisher scoring process within an external platform’s quality scoring algorithm can optimize performance even further.

Working with Koddi to launch highly-relevant ads

Quality scoring is essential to maximizing each commerce media network, ensuring highly-relevant ads that promote a positive user experience while also balancing publisher and advertiser priorities. 

Many tech providers offer quality score models that can be effective, but don’t offer the ability to inject your own business logic. This means commerce media networks ultimately lose control of their site relevancy, and aren’t able ensure their site ad experience is as personalized as their organic experience. On the other hand, simply injecting your own scores without considering other ad factors such as user propensity, margin, or predictive modeling means you’re missing out on critical information from users, publishers, and advertisers.

Koddi’s auction utilizes our AI/ML based quality score based on ad signals, but also allows publishers to submit their own quality score to incorporate unique business logic for best results within a generalized second-price auction. This combination ensures you are in control of your site experience, and are promoting the best possible ads to customers at each step of their buying journey. To learn more, contact us today.