Must.....Document.....Google's.....Crawlers!
Of the major disciplines in SEO, architecture is perhaps the most important of all. This is because if you don’t take into account Google crawling behavior and Google can’t properly spider your website and discover your content, then it won’t matter if it’s the right content, whether it’s optimized, or whether you have obtained any links to it. It may as well not exist if Google can’t discover it.
You’re probably familiar with Google’s spider program, Googlebot, but there are many other ways that Google visits your website. This posting will attempt to pull them all together, and raises some open questions about Google crawling behavior for the community because some aspects of Google’s numerous spiders are still relatively mysterious. Read on »
