The mechanics of search engines

Before we plunge into the nebulous craft of search engine optimization, let's review how the Innertubes work. People dial in seeking two things: entertainment and information. Typically, they start hunting with a search engine, like Google, Yahoo or Microsoft Live. These deploy robotic "spiders" to look for content. They examine a given page and then move on to process pages linked to it, thus crawling the Internet.

 

How Amanda childishly likes to envision the spiders...

The bots have a hit-list of URLs, called "the seeds". That's why submitting your site to search engines is Very Good Thing: it lands you on the crawl frontier (which is possibly the most awesome jargon of this week's lecture). On Google's About Page, look under the yellow header "For Site Owners," top right. Click "submit your content to Google". Other engines, like Yahoo, have similar submission links.

Once spiders target your site, they romp all over the sucker. Their goal is to determine authenticity (that it's not a link farm) and suss the level of authority. These factors drive where your blog appears when people come sleuthing. In other words, they influence your ranking.

The science of this remains very hush hush, because search engines don't want spammers and scammers to stack the decks. Google's algorithm allegedly weighs 200 factors. Hackers waste a lot of brain cells trying to divine these trade secrets (but hey, like Mike, they have IQ to burn). Some general rules are universal, however.

Spiders give more weight to terms in:

• The URL
• Titles
• Headers
• The top of the page's content

They also measure frequency. So good web writers stress the terms important to readers – and ones they're likely to search upon. A search engine analyzes how often these keywords appear in relation to other words in a web page. This principle – pushed too far – is why a lot of web text is cringeworthy. Marketers, in particular, grow obsessed with repetition and other strategies for search engine optimization (SEO). They're writing for robots, not people – and it shows.

Don't be that author. No one likes a keyword stuffer. In fact, some engines kick the worst offenders – spamdexers – right off their indices.