The Google Search Quality Evaluator Guidelines is the instruction document used by the human quality raters who help Google figure out what the “digital fingerprint” of websites and webpages looks like.
The most recent copy of which was published on March 6, 2024 , and is 170 pages long.
Jump ahead to:
Why does the Google Search Quality Evaluator Guidelines matter for website owners and SEOs?
I’m starting with this as unless you are a search quality evaluator, this is what you care about.
This is the document in which Google defines what they mean by “quality content”. It is literally instructions to the search quality raters on how to distinguish between high quality and low quality.
And while there is a noticeable lag between Google saying they are updating the algorithm in a certain way and them actually doing it, they eventually do it.
So if you want the rankings of your sites to not degrade after a future algorithm update, you need to follow their rules.
Who are the Google Search Quality Evaluators?
There are about 16,000 such quality raters around the world, they directly work for third party companies whom Google hires for this function.
The two companies mentioned most are Appen in Australia and Lionbridge in the USA.
FWIW, I once applied to be a search quality evaluator with Lionbridge and went through a fairly extensive “interview” process. I quote the word interview as I never spoke to a person but rather took a series of online tests.
At the end of the process I was informed I qualified to be placed on a waiting list and that I would be considered for a job when there were openings.
I don’t recall hearing back, but I’m not remembering if they failed to ever contact me or I indicated somehow that I was no longer interested.
But I did learn that the job seemed fairly high stress. A rater is shown a split screen that shows two SERPs or two webpages and has to apply the guidelines from the document to make a “better than” decision in a matter of about a minute.
What is this document to them?
It’s their instruction manual, and it is incredibly detailed.
Below are two images of sections of the table of contents to demonstrate the level of detail in which they define the following:
- Lowest Quality Pages
- Low Quality Pages
- Medium Quality Pages
- High Quality Pages
- Highest Quality Pages
- Etc…
What do they do?
Google need software to “decide” what the quality of webpages and websites are, but computers aren’t good at subjective criteria.
So Google needs to turn the subjective assessment of “this is good” or “this is not good” into something their software can recognize.
The quality evaluators help them do this.
They provide the subjective analysis which helps Google determine the “digital fingerprints” for various levels of site and page quality.
And they do this via a never ending series of human assessments about the relative quality of SERPs, sites, and pages.
In addition to an assessment of the quality of websites and pages, they also (based on criteria in the document) assess:
- The purpose of the website
- Whether they see misinformation
- The overall reliability and reputation of the site
- The overall trustworthiness of the site
- And how well the site matches the search query for it was presented to the user
How do they do this?
The document breaks their work down into a sequence of steps, as follows:
- Understanding the query (which may involve understanding where the user is in the world).
- Understanding what the user meant (some queries have multiple meanings: Apple the fruit or Apple the company?).
- Understanding user intent (know something, do something, looking for a specific website, looking for a specific business or organization).
- Understanding results
- Assessing how well the website and the landing page meets the users needs
How does Google use their work to make search results better?
My understanding is there are two ways.
What is described above is used to asses websites and pages.
But Google also uses these workers to asses Search Engine Result Pages (SERPs) changes prior to releasing algorithm updates. In this use, the search evaluators “which is better? A or B?” assessment in an experimental search result.
And algorithm changes are pushed into production only after search quality evaluators assess it as being better.
Featured image credit: https://searchengineland.com/google-updates-search-quality-evaluator-guidelines-317189