Search Engines

By
Satyabrata Garanayak

Introduction
There are millions of websites on the World Wide
Web (WWW). How we can search a particular
web page out of billion of pages ???

Answer is !!!

Search Engine vs. Web Browser

Web Browser

Search Engine

Application Program that allows you

to view information on the web

Specialized Websites that help you

find what you are looking for on the
web

You can install in your system

Cant install; access in the Internet

With the help of Web

Browser we can access
Search Engine

What Exactly Search

Engines Are?
A search engine/web search engine is a software
system that is designed to search for information
on the World Wide Web.

Users

Web Browser

Search Engine

World Wide Web

It uses the keywords to search for documents that

relate to these keywords and then puts the result in
order of relevance to the topic that was searched for.

 The search results are generally presented in

a line of results often referred to as Search
Engine Results Pages (SERPs)
If you want to understand it in simple terms:
Search engines are basically a web based
tool that enables the users to find information
on the World Wide Web

Search Engine Result Page (SERP)

Features of Search Engines

Language Translation
Update Information
Quick search
Relevant result
Offers excellent spell checking, easy access to
dictionary definitions, integration of stock quotes,
street maps and more.
Option to find more web pages, i.e. similar pages
Each search engine has its own catalog or database of
collected web pages, so you will get different
results/hits by using different search engines.

Types of Search Engine

CRAWLER based search engines

DIRECTORIES based search engines
HYBRID search engines
META search engines
SPECIALIZED search engines

Crawler Based Search Engines

Crawler-based search engines What most of us
familiar Google, Bing etc.
They are called Crawler because the software produced
crawls the web like a spider, automatically updating and
adding new pages to its search index as it goes.
Crawler-based search engines are good when you have a
specific search topic in mind and can be very efficient
in finding relevant information in this situation.
However, when the search topic is general, crawler-base
search engines may return hundreds of thousands of
irrelevant responses to simple search requests.

Continue
Three major components of crawler based search engine are1.The Crawler (Spider)
The crawler/spider visits a web page, reads it, and then follows links to
other pages within the site. The spider will return to the site on a regular
basis, such as every month or every fifteen days, to look for changes.
2.The Index
Everything the spider finds goes into the second part of the search
engine, the index. The index will contain a copy of every web page that
the spider finds. If a web page changes, then the index is updated with
new information.
3.The Search Engine Software
This is the software program that accepts the user-entered query,
interprets it, and shifts through the millions of pages recorded in the
index to find matches and ranks them in order of what it believes is most
relevant and presents them in a customizable manner to the user.

How Crawler Based Search Engine Works?

Continue
World Wide
Web

Crawler

Downloader

Indexer

Database

Search
Interface
(Web
Browser)

Query Engine
(Search Engine)

Continue
Crawler-based search engines are constantly
searching the Internet for new web pages and
updating their database of information with
these new or altered pages.
Examples of crawler-based search engines are:

Directories based Search Engines/ Human- Powered

Directory
A directory uses human editors who decide what
category the site belongs to; they place websites within
specific categories in the directories database. The human
editors comprehensively check the website and rank it,
based on the information they find, using a pre-defined set
of rules.
There are two major directories at the time of writing:
Yahoo Directory (www.yahoo.com)
Open Directory (www.dmoz.org)
Note: Since late 2002 Yahoo has provided search results
using crawler-based technology as well as its own
directory.

DMOZ Directory

Yahoo! Directory

Hybrid Search Engines

Hybrid search engines use a combination of
both crawler-based results and directory-based
results. More and more search engines these
days are moving to a hybrid-based model.
Examples of hybrid search engines are:

Meta Search Engines

Meta search engines take the results from
all the other search engines results, and
combine them into one large listing.
Examples of Meta search engines include:
Dogpile, Metacrawler, Mamma etc.

Meta-Search Engine Architecture

Specialized Search Engines

Specialized search engines have been developed to cater for the
demands of niche areas (Specific areas). There are hundreds of
specialized search engines, including:
Images (PicSearch.com)
Shopping (shopping.yahoo.com)
Flights / Travel (SkyScanner.net)
Blogs (BlogPulse.com)
People (Pipl.com)
Forums (BoardReader.com)
Music (SongBoxx.com)
Audio & Video (PodScope.com, Blinkx.com)
Resources (FileDigg.com [.ppt and .pdf])
Private Search (DuckDuckGo.com)

Evaluation of Search Engine

1. Core Technology
2. Scalability
3. Connectors
4. Content Processing
5. Indexing
6. Query Functionality
7. Search Relevancy
8. Security
9. User Interface
10. Administration, Monitoring, and Maintenance