Components of Search Engine

Broadly search engines are divided into 2 main categories:
a). Crawler based search engines b). Human powered directories
Any crawler based search engine is made up of 3 basic components.
a). Crawler or Spider
b). Index
c). Search engine software
All these components work one after one and list the page on search engines. Search engines find websites in 2 ways:
1. By accepting listings sent by webmasters 2. By crawlers that roam the internet storing links and information about each page they visit.
Once the site is found by the search engine, crawlers scan the entire site. While scanning, the crawler visits the web page, reads it and then follows link to other pages within the site. Major search engines like Google, Yahoo and MSN use multiple search engines simultaneously.
Google uses 4 spiders which crawl over 100 pages per second and generating around 600KBs of data each second.
Then index program starts after the crawler. Once a webpage is crawled, it is necessary to transfer them to the database. The index contains a copy of each web pages scanned by the crawler. If the webpage is changed, the index is updated with the new information. It is very important that your pages are added to the index. Until and unless it is indexed, it is not available to those searching with the search engines.
The search engine software performs a task of relevant listings. It searches the entire database i.e. indexed pages and matches it with the search. It then ranks and lists the most relevant matches. These listings are done on how the search engine software is programmed. It delivers listings according to what it believes the most relevant content is!

Read More

Directories

A Web Directory is a web search tool compiled manually by human editors. Once websites are submitted with information such as a title and description, they are assessed by an editor and, if deemed suitable for addition, will be listed under one or more subject categories. Users can search across a directory using keywords or phrases, or browse through the subject hierarchy. Best examples of a directory are Yahoo and the Open Directory Project.
The major difference between search engine and directory is the human factor. A web site search directory indexes a web site based on an independent description of a site. While directories perform many of the same functions of a web page search engine, although their indexing format is different. The main difference is that directories do not spider your site to gather information about it. Instead they rely on a few text entries, typically a site title, domain name, and description to determine which keywords describe your site. While sites in the search engines are scanned and resulted by program (crawler), they are edited manually in directories. Directories contain groups of websites according to theme or industry i.e. automobile related sites are placed in one sub-directory, sports sites are placed into the other sub-directory and so on. Directories do effectively help organize thousands of web sites together. A directory contained inside another directory is called a subdirectory of that directory. Together, the directories form a hierarchy, or tree structure.
There are 5 types of directories namely Human Edited, User Categorized, User Classified, Independently Classified and Pay Per Click (PPC). DMOZ and Yahoo! are the largest directories in the world today.

Read More

Main Search Engines

1. Google:
• Google's main search results are provided solely from Google's search technology,offering results from no other engine or source.
• The Google Directory is comprised of listings from The Open Directory Project (ODP,DMOZ). • Google provides results to AOL, Netscape, IWon, Teoma, AskJeeves and Yahoo! Web Results.
2. Yahoo!:
• Paid and free submissions. - Paid results come from Overture. - Provides main results to HotBot, Excite, Go.com, MSN, Excite, Infospace. • About and backup results to LookSmart, and Overture.
3. MSN:
• MSN provides sponsored results from paid advertising sources. • MSN provides primary results from LookSmart. • Secondary results are provided from Inktomi.
4. AOL:
• AOL results for "Recommended Sites" are listings that have been hand picked by AOL editors. • AOL "Sponsored Sites" are supplied by Google AdWords. • AOL "Matching Sites" are supplied by Google results. The results in AOL may not always match the results in Google as Google often updates their database more frequently. • AOL directory listings are provided by the ODP.

5. Alta Vista:
• Alta Vista receives sponsored listings from Overture and Alta Vista's own advertisers. • Alta Vista will use results from their own database for the main search results. • Alta Vista obtains its directory results from LookSmart.
6. HotBot:
• HotBot results contain three categories: Top 10 Results, Directory Results & General Web Results. • Top 10 results include popular sites and searches. • Directory results are hand-picked by human editors. • Web Results are provided by Inktomi. • HotBot offers the capability to search the HotBot database, Lycos, Google and / or AskJeeves all from one location with the click of a button.
7. IWon:
• IWon Spotlight results are comprised of web pages found within IWon or web sites that IWon has a direct business partnership with. • IWon Sponsored Listings are provided by a variety of paid advertisements through third party pay for performance listings including Google, AdWords and Overture. • IWon Web Site Listings are powered by Google. • IWon Shopping Listings are provided by Dealtime.com
8. Lycos:
• Lycos provides directory results from The Open Directory Project. • Lycos provides sponsored listings from Overture. • Lycos provides Web Results from Fast and from the Lycos network.
9. Netscape:
• Netscape's sponsored links are provided by Google AdWords. • Netscape's matching results include sites that are handpicked by ODP editors mixed with results powered by Google.
10. AllTheWeb:
• AllTheWeb crawls and index ODP results. • AllTheWeb powers the main search results in Lycos. • AllTheWeb provides results from Lycos. • AllTheWeb also powers the Lycos advanced search feature, the FTP search feature and their MP3 specialty engine.
Directories:
1. Dmoz: Directory listings are provided to AOL, Google, Lycos and Netscape and many other web sites, directories & portals.
2. Yahoo!: Yahoo! Directory listings are supplied by Yahoo! editors and require a fee for commercial sites.Yahoo directory results are provided to Alta Vista.

Read More

Introduction to Search Engines

As the Internet started to grow and became an integral part of day-to-day work, it became almost impossible for a user to fetch the exact or relevant information from such a huge web.This is the main reason why ‘Search Engines’ were developed. Search engines became so popular that now more than 80% of web-site visitors come from them. What exactly is a Search Engine? According to webopedia, a “Search Engine” is a program that searchesdocuments for specified keywords and returns a list of the documents where the keywords were found”.

Search engine market share:

Four times voted as Most Outstanding Search Engine, Google is an undisputed market leader of the search engine industry. Google is a crawler based search engine, which is known for providing both comprehensive coverage of web pages and most relevant information. It attracts the largest number of searches and the number goes up to 250 million searches
everyday.

Major Search Engines-

(1) Google
(2) Yahoo
(3) MSN
(4) Ask Jeeves / Teoma

Read More

About SEO

SEO,search engine optimization is process by which we increase the popularity & also improving ranking of any website in any search engine basically in Google search engine.In Internet marketing strategy, SEO considers as that how search engines work and also what people search for.Search engine optimizers may be offer SEO as a stand-alone services or as a part of a broader marketing campaign.An effective SEO may be require changes to HTML source code of any site, SEO technique may be incorporated to web site development and design.
SEO does help in the promotion of sites and at the same time it requires technical knowledge – at least familiarity with basic HTML language.sometimes it is also called SEO because most of techniques that are used to promote any site.SEO helps to increase the traffic to any site, SEO is not advertisement.

Read More