I was curious, so I looked through my web logs.
Web spiders from Google, Microsoft and Inktomi (Yahoo's search engine) downloaded 1602 pages since the 6th this week. The previous week, 1394. Google alone hit me for 469 pages this week, and 594 last week. They can't possibly have the bandwidth to do this to every page they index. I'm getting spidered every day. There is nothing on my web site but photographs. Nobody links to me but me. Why am I special?
Some purty graphs:
![]() Robot visits, by week |
![]() Hits by robots, by week |