01:198:170 Chapter Notes - Chapter 5: Anchor Text, Bing, Site Analysis

• School: Rutgers University
• Department: Computer Science
• Course: 01:198:170
• Professor: G Fuentes

Document Summary

For each token, the crawler creates a list of urls associate with that token: words in the anchor text each get the link"s url added to their token list too. Query processor: second part of a search engine is query processing, query processor looks up tokens in index. User presents tokens that is, search terms to the query processor. Multiword searches: and-query returned results should be associated with all of those words. In summary indexed searches is very powerful because the computer take the time to crawl the data (web pages) and build an index first. Then all it needs to do it find the index entries for each word and intersect the lists to find the information for an and-query. Intersects (the lists) - to locate pages containing multiple words, the query processor simply fetches the index lists for each of the terms and find urls that are in all of the lists.