ARIN2610 Lecture Notes - Lecture 2: Web Crawler, Googlebot, Semantic Search
7 Aug 2018
School
Department
Course
Professor
Document Summary
Search: the process of categorisation, structuring, indexing, storage, analysis, filtering and retrieval of information. Search is a technology of control in an era of information abundance - allowed for the globalisation of industry and mass consumption. Ability to retain control will be directly proportional to the development of its informational technologies (beninger, 1986) Deep web 96% of digital universe - lies in databases, legal/medical records, organisational processes, financial records. Dark web - silk road, drug trafficking. Attention economy: an economic system that acknowledges that our attention is scarce. Search engines are aggregators of wealth through advertising. Server farm - barn of large amount of servers runner. Googlebot - a web crawler application that finds and fetches web pages. The indexer - sorts every word on every page and stores the resulting index of words in a huge database. Knowledge graph - detailed semantic search information about topics explored.
