On February 18, 2025 Google’s Hyung-Jin Kim was interviewed about Google’s ranking signals. Below are notes from that interview.

„Hand Crafting“ of Signals

Almost every signal, aside from RankBrain and DeepRank (which are LLM-based) are hand-crafted and thus able to be analyzed and adjusted by engineers.

• To develop and use these signals, engineers look at data and then take a sigmoid or other function and figure out the threshold to use. So, the „hand crafting“ means that Google takes all those sigmoids and figures out the thresholds.
  • In the extreme hand-crafting means that Google looks at the relevant data and picks the mid-point manually.
  • For the majority of signals, Google takes the relevant data (e.g., webpage content and structure, user clicks, and label data from human raters) and then performs a regression.

Navboost. This was HJ’s second signal project at Google. HJ has many patents related to Navboost and he spent many years developing it.

ABC signals. These are the three fundamental signals. All three were developed by engineers. They are raw, …

• Anchors (A) – a source page pointing to a target page (links). …
• Body (B) – terms in the document …
• Clicks (C) – historically, how long a user stayed at a particular linked page before bouncing back to the SERP. …

ABC signals are the key components of topicality (or a base score), which is Google’s determination of how the document is relevant to a query.

• T* (Topicality) effectively combines (at least) these three ranking signals in a relatively hand-crafted way. … Google uses to judge the relevance of the document based on the query term.
• It took a significant effort to move from topicality (which is at its core a standard „old style“ information retrieval („IR“) metric) … signal. It was in a constant state of development from its origin until about 5 years ago. Now there is less change.
  • Ranking development (especially topicality) involves solving many complex matheivlatical problems.
  • For topicality, there might be a team of … engineers working continuously on these hard problems within a given project.

The reason why the vast majority of signals are hand-crafted is that if anything breaks Google knows what to fix. Google wants their signals to be fully transparent so they can trouble-shoot them and improve upon them.

• Microsoft builds very complex systems using ML techniques to optimize functions. So it’s hard to fix things – e.g., to know where to go and how to fix the function. And deep learning has made that even worse.
• This is a big advantage of Google over Bing and others. Google faced many challenges and was able to respond.
  • Google can modify how a signal responds to edge cases, for example in response to various media/public attention challenges …
  • Finding the correct edges for these adjustments is difficult, but would be easy to reverse engineer and copy from looking at the data.

Ranking Signals „Curves“

Google engineers plot ranking signal curves.

The curve fitting is happening at every single level of signals.

lf Google is forced to give information on clicks, URLs, and the query, it would be easy for competitors to figure out the high-level buckets that compose the final IR score. High- level buckets are:

• ABC — topicality
  • Topicality is connected to a given query
• Navboost
• Quality
  • Generally static across multiple queries and not connected to a specific query.
  • However, in some cases Quality signal incorporates information from the query in addition to the static signal. For example, a site may have high quality but general information so a query interpreted as seeking very narrow/technical information may be used to direct to a quality site that is more technical.

Q* (page quality (i.e., the notion of trustworthiness)) is incredibly important. lf competitors see the logs, then they have a notion of “authority” for a given site.

Quality score is hugely important even today. Page quality is something people complain about the most.

• HJ started the page quality team ~ 17 years ago.
• That was around the time when the issue with content farms appeared.
  • Content farms paid students 50 cents per article and they wrote 1000s of articles on each topic. Google had a huge problem with that. That’s why Google started the team to figure out the authoritative source.
  • Nowadays, people still complain about the quality and AI makes it worse.

Q* is about … This was and continues to be a lot of work but could be easily reverse engineered because Q is largely static and largely related to the site rather than the query.

Other Signals

• eDeepRank. eDeepRank is an LLM system that uses BERT, transformers. Essentially, eDeepRank tries to take LLM-based signals and decompose them into components to make them more transparent. HJ doesn’t have much knowledge on the details of eDeepRank.
• PageRank. This is a single signal relating to distance from a known good source, and it is used as an input to the Quality score.
• … (popularity) signal that uses Chrome data.

Search Index

• HJ’s definition is that search index is composed of the actual content that is crawled – titles and bodies and nothing else, i.e., the inverted index.
• There are also other separate specialized inverted indexes for other things, such as feeds from Twitter, Macy’s etc. They are stored separately from the index for the organic results. When HJ says index, he means only for the 10 blue links, but as noted below, some signals are stored for convenience within the search index.
• Query-based signals are not stored, but computed at the time of query.
  • Q* – largely static but in certain instances affected by the query and has to be computed online (see above)
• Query-based signals are often stored in separate tables off to the side of the index and looked up separately, but for convenience Google stores some signals in the search index.
  • This way of storing the signals allowed Google to …

User-Side Data

By User Side Data, Google’s search engineers mean user interaction data, not the content/data that was created by users. E.g., links between pages that are created by people are not User Side data.

Search Features

• There are different search features – 10 blue links as well as other verticals (knowledge panels, etc). They all have their own ranking.
• Tangram (fka Tetris). HJ started the project to create Tangram to apply the basic principle of search to all of the features.
• Tangram/Tetris is another algorithm that was difficult to figure out how to do well but would be easy to reverse engineer if Google were required to disclose its click/query data. By observing the log data, it is easy to reverse engineer and to determine when to show the features and when to not.
• Knowledge Graph. Separate team (not H/’s) was involved in its development.
• Knowledge Graph is used beyond being shown on the SERP panel.
  • Example — “porky pig” feature. If people query about the relation of a famous person, Knowledge Graph tells traditional search the name of the relation and the famous person, to improve search results – Barack Obama’s wife’s height query example.
• Self-help suicide box example. Incredibly important to figure it out right, and tons of work went into it, figuring out the curves, threshold, etc. With the log data, this could be easily figured out and reverse engineered, without having to do any of the work that Google did.

Reverse Engineering of Signals

There was a leak of Google documents which named certain components of Google’s ranking system, but the documents don’t go into specifics of the curves and thresholds.

The documents alone do not give you enough details to figure it out, but the data likely does.

Time has grown more scarce after having a child, so I rarely blog anymore. Though I thought it probably made sense to make at least a quarterly(ish) post so people know I still exist.

One of the big things I have been noticing over the past year or so is an increasing level of automation in ways that are not particularly brilliant. 😀

Just from this past week I’ve had 3 treat encounters on this front.

One marketplace closed my account after I made a bunch of big purchases, likely presuming the purchases were fraudulent based on the volume, new account & an IP address in an emerging market economy. I never asked for a refund or anything like that, but when I believe in something I usually push pretty hard, so I bought a lot. What was dumb about that is they took a person who would have been a whale client & a person they were repeatedly targeting with ads & turned them into a person who would not recommend them … after being a paying client who spent a lot and had zero specific customer interactions or requests … an all profit margin client who spent big and then they discarded. Dumb.

Similarly one ad network had my account automatically closed after I had not used it for a while. When I went to reactivate it the person in customer support told me it would be easier to just create a new account as reactivating it would take a half week or more. I said ok, went to set up a new account, and it was auto-banned and they did not disclose why. I asked feedback as to why and they said that they could not offer any but it was permanent and lifetime.

A few months go by and I wondered what was up with that and I logged into my inactive account & set up a subaccount and it worked right away. Weird. But then even there they offer automated suggestions and feedback on improving your account performance and some of them were just not rooted in fact. Worse yet, if they set the default targeting options to overly broad it can cause account issues in a country like Vietnam to where if you click to approve (or even auto approve!) their automated suggestions you then get notifications about how you are violating some sort of ToS or guidelines … if they can run that logic *after* you activate *their* suggestions, why wouldn’t they instead run that logic earlier? How well do they think you will trust & believe in their automated optimization tips if after you follow them you get warning pop overs?

Another big bonus recently was a client was mentioned in a stray spam email. The email wasn’t from the client or me, but the fact that a random page on their site was mentioned in a stray spoofed email that got flagged as spam meant that when the ticket notification from the host sent wounded up in spam they never saw it and then the host simply took their site offline. Based on a single email sent from some other server.

Upon calling the host with a friendly WTF they explained to the customer that they had so many customers they have to automate everything. At the same time when it came time to restoring hosting that the client was paying for they suggested the client boot in secure mode, run Apache commands x and y, etc. … even though they knew the problem was not with the server, but an overmalicious automated response to a stray mention in a singular spam email sent by some third party.

When the host tried to explain that they „have to“ automate everything because they have so many customers the customer quickly cut them off with „No, that is a business choice. You could charge different prices or choose to reach out to people who have spent tens of thousands on hosting and have not had any issues in years.“ He also mentioned how emails can be sent to spam, or be sent to an inbox on the very web host that went offline & was then inaccessible. Then the lovely customer support person stated „I have heard that complaint before“ meaning they are aware of the issue, but do not see it as an issue for them. When the customer said they should follow up any emails with an SMS for servers going offline the person said you could do it on your end & then later sent them a 14-page guide for how to integrate the Twillio API.

Nothing in the world is fair. Nothing in the world is equal. But there are smart ways to run a business & dumb ways to run a business.

If you have enough time to write a 14-page integration guide it probably makes sense to just incorporate the feature into the service so the guide is unneeded!

Businesses should treat their heavy spenders or customers with a long history of a clean account with more care than a newly opened account. I had a big hedge fund as a client who would sometimes want rush work done & would do stuff like „hey good job there, throw in an extra $10,000 for yourself as a bonus“ on the calls. Whenever they called or emailed they got a quick response. 😀

I sort of get that one small marketplace presuming my purchases might have been a scam based on how many I did, how new my account was, and how small they were, but the hosting companies & ad networks that are worth 9 to 12 figures should generally do a bit better. Though in many ways the market cap is a sign the entity is insulated from market pressures & can automate away customer service hoping that their existing base is big enough to offset the customer support horror stories that undermine their brand.

It works.

At least for a while.

A parallel to the above is my Facebook ad account, which was closed about a half decade or so ago due to geographic mismatch. That got removed, but then sort of only half way. If I go to run ads it says that I can’t, but then if I go to request an account review to once again explain the geographic difference I can’t even get the form to submit unless I edit the HTML of the page on the fly to seed the correct data into the form field as by default it says I can not request a review since I have no ad account.

The flip side of the above is if that level of automation can torch existing paid accounts you have to expect the big data search & social companies are taking a rather skeptical view of new sites or players wanting to rank freely in their organic search results or social feeds. With that being the case, it helps to seed what you can to provide many signals that may remove some of the risks of getting set in the bad pile.

I have seen loads of people have their YouTube or Facebook or whatever such account get torched & only override the automated technocratic persona non grata policies by having followers in another channel who shared their dire situation so it could get flagged for human review and restoration. If that happens to established & widely followed players who have spent years investing into a platform the odds of it happening to most newer sites & players is quite high.

You can play it safe and never say anything interesting, ensuring you are well within the Overtone Window in all aspects of life. That though also almost certainly guarantees failure as it is hard to catch up or build momentum if your defining attribute is being a conformist.

Kent Walker promotes public policies which advantage the Google monopoly.

His role doing that means he has to write some really bad hot takes that lack context or intentionally & dishonestly redirect attention away from core issues – that’s his job.

With that in mind, his most recent blog post defending the Google monopoly was exceptional.

Force Ranking of Inferior Search Results

„When you have an urgent question — like “stroke symptoms” — Google Search could be barred from giving you immediate and clear information, and instead be required to direct you to a mix of low quality results.“

On some search queries users get a wall of Google ads, the forced ranked Google insert (or sometimes multiple of them with local & ecommerce) and then there can even be a „people also ask“ box above the first organic result.

The idea that organic results must be low quality if not owned & operated indicates 1 of the following 3 must be true:

• they should not be in search
• their content scraping & various revenue shifting scams with their ad tech stack demonetized legit publishers
• their forced rank of their own content is stripping them of the signals needed to rank websites & pages

Whenever Google puts a „people also ask“ box above the first organic result that is them saying they did not know what to rank, or they are just trying to create a visual block to push the organic result set down the page and user attention back up toward the ads.

The solution to Google’s claims is easy to solve. Either of the following would work.

• Have an API that allows user choice (to set rich snippet or vertical defaults in various categories), or
• If the vertical inserts remain Google-only then for Google to justify force ranking their own results above the organic result set Google should also be required to rank those same results above all of their ads, so that Google is demonetizing Google along with the rest of the ecosystem, rather than just demonetizing third parties.

If the thesis that this information needs to be front and center & that is a matter of life or death, then asking searchers to first scroll past a page or two of ads is not particularly legitimate.

Spam & Security

„when you use Google Search or Google Play, we might have to give equal prominence to a raft of spammy and low-quality services.“

Many of the worst versions of spam that have repeatedly made news headlines like fake tech support, fake government document providers, and fake locksmiths were buying distribution through Google Ads or were featured in the search results through Google force ranking their own local search offering even though they knew the results were vastly inferior to Yelp.

If Google did not force rank Google local results above the rest of the organic result set then the fake locksmiths would not have ranked.

I have lost count of how many articles I have read about hundreds or thousands of fake apps in the Google Play store which existed to defraud advertisers or commit identity theft, but there have been literally thousands of such articles. I see a similar headline at least once a month without eve looking for them. Here is one this week for scammers monetizing the popularity of Wordle with fake apps.

Making matters worse, some of the tech support scams showed the URL of a real business and rerouted the call through a Google number directly to a scammer. A searcher who trusted Google & sees Apple.com or Dell.com on Google Ads in the search results then got connected with a scammer who would commit identity theft or encrypt their computer then demand ransom cryptocurrency payments to decrypt it.

After making the ads harder to run for scammers Google decided the problem was too hard & expensive to sort out so they also blocked legitimate computer repair shops.

Sometimes Google considers something spam strictly due to financial considerations.

Their old remote rater documents stated *HELPFUL* hotel affiliate websites should be labeled as spam.

Years later the big OTAs are complaining about Google eating their lunch as well as Google is twice as big as the next player.

At one point Google got busted for helping an advertiser route around the automated safety features built into their ad network so that they could pay Google to run ads promoting illegal steroids.

With cartels, you can only buy illegal goods and services from the cartel if you don’t want to suffer ill consequences. The same appears to be true here.

The China Problem

„Handicapping America’s technology leaders would threaten our leading sources of research and development spending — just as bipartisan voices in Congress are recognizing the need to increase American R&D investment to stay competitive in the global race for AI, quantum, and other advanced technologies.“

We are patriotic, and, but China… is a favorite misdirection of a tech monopolist.

The problem with that is while Eric Schmidt warns it is a national emergency if China overtakes the US in AI tech, Google also operates an AI tech lab in China.

In other words, Eric Schmidt is trying to warn you about himself and his business interests at Google.

Duplicitous? Absolutely.

Patriotic? Less than Chamath!

Who the fuck did this? pic.twitter.com/BD4NKpila6— Girolamo Carlo Casio (Free Twatter) (@INArteCarloDoss) January 19, 2022

Inflation

„the online services targeted by these bills have reduced prices; these bills say nothing about sectors where prices have actually been rising and contributing to inflation.“

Technology is no doubt deflationary (moving bits on an optical line is cheaper than printing out a book and shipping it across the world) BUT some dominant channels have increased the cost of distribution by increasing the chunk size of information and withholding performance information.

Before Google Analytics was „free“ there was a rich and vibrant set of competition in web analytics software with lots of innovation from players like ClickTracks.

Most competing solutions went away.

Google moved away from an installed licensing model to a hosted service where they can change the price upon contract renewal.

Search hid progressively more performance information over time, only sampled data from larger data sets, & now you can sign up for Google Analytics 360 starting at only $150,000 per year.

The hidden search performance data also has many layers to that onion. Not only does Google not show keyword referrers on organic search, but they often don’t show your paid search keywords either, and they keep extending out keyword targeting broader than advertisers intend.

Yesterday’s announcement on match type changes had me crawling through query data this morning. I’m staring at many 2-3 word exact match keywords that are matching to 8-word queries. G thinks ‚deck paint‘ and ‚how do i put paint on my deck‘ mean the exact same thing. CPA is 10x.— Brad Geddes (@bgtheory) February 5, 2021

Google used to pay Brad Geddes to run official Google AdWords ad training seminars for advertisers, so the idea that *he* has to express his frustrations on Twitter is an indication of how little effort Google is putting into having open communications channels or caring about what their advertisers think.

This is in accordance with the Google customer service philosophy:

he told her that the whole idea of customer support was ridiculous. Rather than assuming the unscalable task of answering users one by one, Page said, Google should enable users to answer one another’s questions.

Those who were paying for ads get the above „serve yourself“ treatment, all the while Google regularly resets user default ad settings to extend out ad distribution, automatically ad keywords, shift to enhanced AdWords ad campaigns, etc.

Then there are other features which would be beneficial and offered in a competitive market that have been deprioritized. Many years ago eBay did a study which showed their branded Google AdWords ad buys were cannibalistic to eBay profits. Google maintained most advertisers could not conduct such a study because it would be too expensive and Google does not make the feature set available as part of their ad suite.

Missing Information

„When you search for local businesses, Google Search and Maps may be prohibited from highlighting information we gather about hours of operation, contact information, and reviews. That could hurt small businesses and local retailers, as well as their customers.“

Claiming reviews or an attempt to offer a comprehensive set of accurate review data as a strong point would be economical with the truth.

Back when I had a local business page my only review was from a locksmith spammer / scammer who praised his own two businesses, trashed a dozen other local locksmiths, crapped on a couple local SEO services, and joked about how a local mover smashed the guts out of his dog. Scammer fake reviewer’s name was rather sophisticated … it was … Loop Dee Loop

About a decade back when Google was clearly losing Google took Yelp reviews wholesale (sometimes without even attributing them to Yelp!) and told Yelp that if they did not want Google stealing their work and displacing them with a copy of it then they should block GoogleBot. Google offered the same sort of advice / threat to TripAdvisor.

A few years before that Google temporarily „forgot“ to show phone numbers on local listings.

After Yelp turned down an acquisition offer by Google & Yelp did a great job making some people aware of how Google was stealing their reviews wholesale without attribution Google bought Zagat & Fromer’s to augment the Google local review data and then sold those businesses off.

This is sort of the same playbook Google has run in the past elsewhere. After Groupon said no to Google’s acquisition offer, Google quickly provided daily deal ads to over a dozen Groupon competitors to help commoditize the Groupon offering and market position.

Ultimately with the above sort of stuff Google is primarily a volume aggregator or has lower editorial costs than pure plays due to the ability to force bundle their own distribution. And they use the ability to rank themselves above a neutral algorithmic position as a core part of their biz dev strategy. When shopping search engines were popular Google kept rewording the question set they sent remote raters to justify rank demotion for shopping search engines & Google also came up with innovative ranking „signals“ like concurrent ranking of their own vertical search offering whenever competitors x or y are shown in the result set & rolled out a „diversity“ algorithm to limit how many comparison shopping sites could appear in the search results. The intent of the change was strictly anti-competitive:

„Although Google originally sought to demote all comparison shopping websites, after Google raters provided negative feedback to such a widespread demotion, Google implemented the current iteration of its so-called ‚diversity‘ algorithm.“

As a matter of fact, part of one of many document dumps in recent years went further than the old concurrent ranking signal to a rank x above y feature which highlights how YouTube can be hard coded at a number 1 ranking position.

Part of that guide highlighted how to hardcode ranking YouTube #1.

If you re-represent content & can force rank yourself #1 (with larger listings) that can be used to force other players onto your platform on your terms. Back when YouTube was must less of a sure thing Google suggested they could threaten to change copyright.

This same approach to „relevancy“ is everywhere.

Did you watermark your images? Well shame on you, as that is good for a rank demotion

And if there are photos which are deemed illegal Google will make you file an endless series of DMCA removal requests even though they already had the image fingerprinted.

Now there are some issues where there is missing information. These areas involve original reporting on local politics & are called news deserts. As the ad pie has consolidated around Google & Facebook that has left many newspapers high and dry.

Private equity players like Alden Global Capital buy up newspapers, fire journalists, and monetize brand equity as they drive the papers into the ground.

If you are sub-scale maybe Google steals your money or hits you with a false positive algorithm flag that has you seeking professional mental health help.

Big players get a slower blood letting.

Google has maintained they do not make any money from news search, but the states lawsuit around ad tech made it clear Google promoted AMP for anti-competitive purposes to block header bidding, lied to news publishers to get them to adopt AMP and eat the tech costs of implementation, did a deal with their biggest competitor in online advertising Facebook to maintain the status quo, charge over double what their competitors do for ad tech, and had a variety of bid rigging auction manipulation algorithms they used to keep funneling more money to themselves.

Internally they had an OKR to make *most* search clicks land on AMP pages within a year of launch

„AMP launched as an open source project in October 2015, with 26 publishers and over 40 publications already publishing AMP files for our preview demo. Our team built g.co/ampdemo and is now racing towards launching it for all of our users. We’re responsible for the AMP @ Google integrations, particularly focusing on Search, our most visible product. We have a Google-wide 2016 OKR to deliver! By the end of 2016, our goal is that 50%+ of content consumed through Search is being consumed through AMP.“

You don’t get over half the web to shift to a proprietary version of HTML in under a year without a lot of manipulation.

So, when Google tells buyers an ad sold for one price and they tell sellers it sold for a lower price, isn’t that just plain old fraud? I mean, on top of the anti-competitive tying and all that, fraud is illegal, isn’t it?— Jerry Neumann (@ganeumann) January 14, 2022