Comments on: Are you writing targeted crawlers? Ever crush a site you’re crawling? Help is here! http://www.brokenbuild.com/blog/2006/02/08/are-you-writing-targeted-crawlers/ Databases, code and all things agile. Thu, 09 Feb 2012 02:03:24 +0000 http://wordpress.org/?v=2.6.3