[IP] Re: New whitehouse.gov robots.txt file

[David Farber <dave@farber.net>]

Begin forwarded message:

From: Dave CROCKER <dhc2@dcrocker.net>
Date: January 21, 2009 10:47:09 AM EST
To: dave@farber.net
Cc: ip <ip@v2.listbox.com>, Joseph Lorenzo Hall <joehall@gmail.com>,  
Declan McCullagh <declan.mccullagh@cnet.com>
Subject: Re: [IP] New whitehouse.gov robots.txt file
Reply-To: dcrocker@bbiw.net



> Begin forwarded message:
> Here's another fascinating sign of increased transparency in the new
> administration:
> The whitehouse.gov robots.txt file -- a file that specifies what areas
> of a web site that web spiders may crawl[1] -- has gone from 2400
> lines to just two lines:


Declan McCullagh has just posted an article that explains the error in  
this overly simplistic metric:

Much ado about Whitehouse.gov's new openness
<http://news.cnet.com/8301-13578_3-10146802-38.html?part=rss&tag=feed&subj=News-PoliticsandLaw 
>

d/
--

 Dave Crocker
 Brandenburg InternetWorking
 bbiw.net




-------------------------------------------