Re: covert crawlers: I wonder how "nobody" came up with

[Dude VanWinkle <dudevanwinkle () gmail com>]

On 1/15/06, Dr. Neal Krawetz <hf () hackerfactor com> wrote:
> On Sun Jan 15 01:09:39 2006, Gadi Evron wrote:
> > Subject: [funsec] covert crawlers: I wonder how "nobody" came up with
> > this before?
> >
> > http://www.wired.com/news/technology/0,70016-0.html?tw=rss.index
>
> Actually, people have come up with it before.  :-)


How long would it take a "real human" to browse the entire web?
Clicking on all the links, reading every page, etc., etc.. It would
seem to me that the bandwidth "wasted" by webbots would me minimized
in direct proportion to their performance and functionality (assuming
their function is to crawl the _entire_ web).  "polite" webbot
shouldnt be as much of a problem or nusiance to those who dont like
the bot.

-JP
"except for ad's that pay-per-click ;-)"
-JP


-jp

_______________________________________________
Fun and Misc security discussion for OT posts.
https://linuxbox.org/cgi-bin/mailman/listinfo/funsec
Note: funsec is a public and open mailing list.