Assessor is a person who, on the instructions of the developers of the search engine, evaluates how much the found document matches the query. Most often, assessors are not professionally familiar with information retrieval. Their qualification approaches the knowledge of average Internet users. One of the first labor assessors started using Google in 2003. Yandex assessors appeared in 2006.
Assessor workflow
At first, an automatic search takes place, and the search engine forms a sequence of sites according to their relevance, calculated by the robot. Then the assessor, who at that moment can be located anywhere in the world, using a special program and ordinary logic, evaluates the result. The rating scale is developed by each search engine independently. It usually contains six or more items. The work of assessors differs by an error of approximately 5%.
The assessor sends the result of his activity to the headquarters of the search engine, where, based on the joint analysis of the data of the search robot and the information of the assessor, the final rating of the document is calculated. The result, whichhelps to achieve the assessor is a significant increase in the objectivity of the assessment of documents, the exclusion from the rating of resources erroneously included in the rating list on the basis of formal features.
All search engines have their own staff of assessors, which constantly rotates. The most suitable assessor is a user with an average level of Internet proficiency. The purpose of the work that he does is to improve the functioning of search engines so that they answer user questions as accurately as possible.
Assessors who evaluate search results perform tasks. Their content is a keyword, a link, and instructions for evaluating the correspondence of a link to a given word. The given word according to the instruction to which the assessor must adhere is an action of the form "go", "do" or "learn".
The assessor must decide if the keyword corresponds to a specific action taken by the user (making a purchase, watching a movie, listening to music) or some data that interests him.
Types of assessor grades in Yandex
Yandex assessors consistently give two types of grades:
1. Preliminary estimate. Whether this document refers to pornography and does not contain malicious code. If the answer is “yes”, then the evaluation of the document stops.
2. Relevance assessment (compliance). This assessment is non-quantitative. The assessor gives his assessment by assigning the document to any category:
- "Vital"– if it is an official site or an official answer to a question.
- "Useful" - a document containing data that exactly matches the search query.
- "Relevant+" - a document that matches the search query.
- "Relevant-" - a document that does not exactly match the search query.
- "Irrelevant" - a document that does not match the search query.
- "Spam" - a document with signs of black optimization (attempts to deceive the search engine).
- “Not about that” is a category that is designed to separate concepts that are similar for a robot, but fundamentally different for a person. So, for the search query "Leo Tolstoy", the search engine should not return documents about fat people and animals as results.
Meaning of assessor grades
The work of assessors helps in assessing the degree of search accuracy and training the search robot. Assessors are not able to influence the positions occupied by a particular site.
Assessors of sites in their work are guided by clear instructions. It has become quite a large and complex document and is constantly updated with new requirements.