Search engines are the primary tools of Internet users for finding products, services and information over the web. Search engines allow people to search the entire Web (or at least those pages of the Internet that are in the search engine’s database.)
How Does a Search Engine Index Web Pages?
There are four parts to an engine that you need to know about for optimization purposes:
- The spider is a program that goes out across the internet, looks for and gathers up web pages.
- The database is where the spider will store the pages that it finds.
- The search engine website, e.g. google.com, is where searchers go to pull up information from the database.
- The algorithms are programs that determine which sites will come up when searchers type in a query at the search engine website.
There are two ways that your site can get into the database:
- The spider will automatically find your site from a link on someone else’s site which is the path we recommend if you can get 1 or more quality inbound link(s).
- You submit your URL so that the spider will come out and find it.
What Happens When I Submit my URL to a Search Engine?
First, the search engine’s spider visits your URL immediately and schedules your page for inclusion in the search engine’s database.
Second, usually within a few weeks, the spider comes along and places your page(s) into its database.
There is no telling how many pages deep the spider will crawl or how many pages it will place in the database. Usually, on the first time around, it will be only a few pages -possibly only the home page. Third, the spider revisits your page(s) to grab any changes you’ve made. (The old term for this was “automatic update.”) Once a page is in the database, the spider usually revisits every few weeks. The
spider will also begin to crawl your site more deeply and place more and more of your pages into the database.
Fourth, when people use a search engine, they type keywords into a search box on the search engine’s website. They are submitting a query. The search engine, depending on algorithms, will pull up all of the sites relevant to that query.
What is the Difference between a Search Engine and a Directory?
A search engine is a machine – or a robot. A human may program algorithms for a search engine, but humans have nothing to do with your site when the spider is visiting your site or when the engine is ranking your pages. Google.com is an example of a search engine.
For the most part, humans compile directories. Dmoz.org (Open Directory Project) is an example of a directory. When you submit your site to Dmoz, a human will review your site for consideration in the Dmoz directory of web sites.
A search engine has a very large database because it will store several pages of every web site it indexes. A directory will only store a link to the home page of each site and a description.
Search engines will take the description either from some of the sentences on the web page or from the description meta tag. A directory will take the description from your submission information.
Each major search engine is usually associated with a directory. For instance, when you go to Google and you type in a search, you are getting results from all the web sites stored in Google’s database.
It is believed that Google’s algorithms are also programmed to place emphasis on sites that are also listed in the Dmoz directory. However many top SEO’s argue this holds less weight today.
It is a good idea to get listed in as many directories as possible. This simple piece of software can save you a lot of time and effort: Directory Inclusion.
What is Stemming?
Some engines use stemming technology. This means that sometimes a search engine will not only search for the words people type in, but also for words that are similar. For instance, if you type in “educational wooden toys,” the engine might also look for “educational wood toy.” The engine may do this if it cannot find good results for the terms that were queried.
Variables That Affect Ranks
This list includes most of the variables currently and previously known to affect search engine ranks. You should understand that some of these variables are more important than others. Variables that affect ranks in a positive way (these factors probably will not change over time, these are some of the most important variables):
- Keywords in your copy.
- Link popularity.
- Keywords in the title tag.
- Listings in directories.
Variables that affect ranks in a negative way (these factors probably will not change over time):
- Spamming by using the same word or phrase several times in your title, meta tags, or text.
- Spamming by putting words or phrases into your meta tags or title that have nothing to do with the actual content people see on your web page.
- Using text the same color as the background.
- Using tiny text (font size “-1” or smaller) as a way to cram keywords into a page.
- Linking out to sites that have nothing to do with the focus or niche of your site.
- Linking out to link farms or free-for-all (FFA) link pages. (Sites that contain pages just for the purpose of exchanging links with other sites without concern for content. Generally link farms or FFA link pages have thousands of links and the links are added by means of a program not a human.)
- Links coming in from link farms or FFA link pages.
Variables that have been thought to affect ranks positively in the past (these factors may change over time depending on the way the algorithm is programmed):
- Keywords in the domain name
- Bolding keywords, e.g. software development
- Using keywords in heading tags, e.g.
Software Development in London
- Keywords closest to the top of the page.
- Keywords in the meta description tag.
- Keywords in the meta keywords tag.
- Keywords in the names of linked pages and in the linked words, e.g. link to software development.
- Keywords in alt tags.
- Keywords as names of images, e.g. <img src=”software-development.gif alt=”software development outsourcing company”>.
- Getting listings in Pay-Per-Click search engines like Google Adwords or Overture.
After a user searches on a set of keywords on Google they are presented with a page of search results composed of sponsored listings and in the main section of the page, the organic listings. Organic search engine optimization (SEO) is about increasing the number of visitors that your site gets from a search engine. This is achieved by getting more documents indexed on the search engine and by attaining a higher position in the organic results. The problem for many marketers and traditional advertising agencies lies in the technical nature of search engines and the difficulty in understanding the underlying algorithms that produce the rankings for particular keyword searches.
The most popular search engine is Google which accounts for the greatest number of search requests.
This is not to say that other search engines are not important or that they use the same algorithm but that Google’s success has ensured that many search engines are moving in the same direction and since Google has filed their search patent we do not have to rely on the typical conjecture in this field.
Importantly, Google ensures that there is a level of feedback from users through the Google toolbar that enables Google to refine their search results over time. Also, a significant part of Google’s algorithm is to protect its search engine against search SPAM which is the use of techniques to improve a site’s rankings normally through the artificial creation of links, so we have outlined the main
areas to watch out for.
We suggest an approach that focuses on continuous improvement in a structured way rather than anything that the search engines could interpret as trying to manipulate the results. We believe that the best way to reach the top spots is to align with the search engines’ goal which is to deliver the best and most relevant results for their users, we call it ‘alignment’ strategy. This will ensure that you consistently rank highest amongst your competitors and do not get black-listed. In the following chapters, we are going to clarify the elements that are analyzed by Google. Using this information it starts to become clear what is needed for organic search engine success.
Search Engine Optimization is both an art and a science. It’s ever changing as Search Engines progress. Read the books mentioned above and gain a solid foundation. Then keep learning. Participate in forums, read the latest SEO articles and talk with fellow webmasters.
What worked today may hurt you tomorrow so keep up with it. Remember that SEO success isn’t achieved overnight.
Within this eGuide I’ve suggested many tools, programs and pieces of information that have helped me achieve top rankings and increase my traffic exponentially.
These can help you tremendously as well but they aren’t magic.
You still have to put in the effort.
So arm yourself with the right SEO knowledge and tools and get your site ready for waves of new traffic!