Thursday, January 31, 2008

Googlebot

GET /articles_htm/2007/030707%20stories.pl.htm HTTP/1.1
Host: www.xxxxx.net
Connection: Keep-alive
Accept: */*
From: googlebot(at)googlebot.com
User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Accept-Encoding: gzip
If-Modified-Since: Fri, 25 Jan 2008 13:01:58 GMT


is this how googlebot index your website?