Re: yahoo crawlers hammering us

[Valdis.Kletnieks () vt edu]

On Wed, 08 Sep 2010 02:21:31 PDT, Bruce Williams said:
> > I *am* curious--what makes it any worse for a search engine like Google
> > to fetch the file than any other random user on the Internet
>
> Possibly because that other user is who the customer pays have their
> content delivered to?

Seems to me that if you're doing content-for-pay and are upset when
some(body|thing) downloads it without paying for it, you shouldn't be leaving
gigabytes of said content where even a stupid bot can find it and download it
without paying for it.  Just sayin'.

Attachment: _bin
Description: