Web search engines / robots / spiders?

["Joseph S. D. Yao" <jsdy () cospo osis gov>]

We're looking to get some kind of a Web search engine that would walk
our intranet/extranet/whatever inside its firewall boundaries, ignoring
that Internet stuff outside the firewalls, and give us a way to search
the whole intra-web from one place.  I had thought there were several
announced products out there - Alta Vista, Excite, whatever.  Strangely,
the only one we can find anything on is Alta Vista, and there is more
info outside of DEC than in.  I also found references to an Open Source
package called web-search, which sure sounds like it might be something
like what we're looking for.  [Searching for a downloadable copy on
sunsite as I write.]

Does anybody have any suggestions / pointers / warnings of doom?

Thanks.

--
Joe Yao                         jsdy () cospo osis gov - Joseph S. D. Yao
COSPO Computer Support                                          EMT-A/B
-----------------------------------------------------------------------
This message is not an official statement of COSPO policies.