Re: httpspider lib and hostnames with special characters

[Djalal Harouni <tixxdz () opendz org>]

> > > On Mon, Mar 05, 2012 at 03:30:43PM +0100, Gutek wrote:
> Thanks Djalal.
> It sounds to me like a weakness in httpspider's efficiency. Let's
> consider a practical example with h-online.com and a
> httpspider-dependant script, let's say http-backup-finder.
> With a simple command like nmap -v -Pn -p80 -n --script
> http-backup-finder www.h-online.com  it (silently) won't work because a
> debug reveals that every link will be discarded, maybe fooling the user
> into thinking that no backup was found:
> - ----------
> NSE: httpspider: Spidering limited to: maxdepth=3; maxpagecount=20;
> withinhost=www.h-online.com
> ...
> NSE: httpspider: Link is not within host:
> http://www.h-online.com/nettools/tools/spam-list-query
> NSE: httpspider: Link is not within host:
> http://www.h-online.com/security/services/Reserved-IPv4-addresses-732899.html
> NSE: httpspider: Link is not within host:
> http://www.h-online.com/Contact-273335.html
> NSE: httpspider: Link is not within host:
> http://www.h-online.com/Privacy-Policy-of-h-online-com-273337.html
> - -----------
>
> Now, with a script arg to override this withinhost issue, it will work
> as intended:
> nmap -v -Pn -p80 -n --script http-backup-finder --script-args
> http-backup-finder.withindomain=www.h-online.com -d2 www.h-online.com
>
> - -----------
> NSE: httpspider: Spidering limited to: maxdepth=3; maxpagecount=20;
> withindomain=h-online.com
> - -----------
>
> As I understand it, a withindomain argument is mandatory when users want
> to deal with hyphened hostnames ? if it's an intented behavior and not a
> bug, maybe this should be explicitely stated in the documentation ?
Gutek this is clearly a bug, this should work by default.

httpspider must handle it, from rfc952:
<name> ::= <let>[*[<let-or-digit-or-hyphen>]<let-or-digit>]

but it was updated later in rfc1123 to allow names to start with digits.

Thanks.

-- 
tixxdz
http://opendz.org
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/