Re: Duplicate targets - was: Feature request: scanning an AS

[Djalal Harouni <tixxdz () opendz org>]

On 2011-04-27 20:17:28 -0700, David Fifield wrote:
> On Mon, Apr 18, 2011 at 09:12:56PM +0200, John Bond wrote:
> > another update as per the comments on secwiki to add newtargets functionality
> >
> > ---
> > -- @args asn-to-prefix.asn The asn number to search
> > -- @args asn-to-prefix.whois_server whois server to use default is
> > asn.shadowserver.org
> > -- @args asn-to-prefix.whois_port whois port to conect to default is 43
> > -- @args newtargets prefixes discovered will be added to the nmap scan
> > --
> > -- @usage
> > -- nmap --script asn-to-prefix --script-args
> > asn-to-prefix.asn={65000,65001}[asn-to-prefix.whois_server=asn.shadowserver.org,asn-to-prefix.whois_port=43,newtargets]
> > --
> > -- @output
> > -- 53/udp open  domain  udp-response
> > -- | asn-to-prefix:
> > -- |_    127.0.0.0/8
> >
> > I was originally skeptical about how useful this script would be but i
> > have been using it at least every week so look forward to seeing it in
> > current :)
>
> Okay, I think this is a nice script. Here are a few things I'd like to
> see:
>
> We should get permission from the operators of the whois server, like we
> did for the asn-query script. We just don't want them to be mad if the
> script being distributed with Nmap causes them a lot of extra load.
> Would you email them and ask if they object to the script.
>
> The script needs to go into the "external" category.
>
> I think this is unnecessary:
>         newtargets = stdnse.get_script_args('asn-to-prefix.newtargets')
> Just plain "newtarget" already sets target.ALLOW_NEW_TARGETS.
The newtarget feature is really helpful for this script, but the
current duplicate filtering engine is weak, it just compare the added
strings.

A better engine will do:
* Use a separate queue to store the added targets that are in a form
  of hostnames. (call this the hostname queue).

* Use another queue to store the added targets that are in a form of an
  IP address or a network. (call this the network queue).

* A cache to store the scanned targets in a form of _networks_ with
  their cidr/ranges. currently we just cache the new targets as strings
  regardless of their type: hostname, IP, network ... (Nmap targets).


Nmap will pop targets from the hostname and network queues to scan them,
and the cache will be used to store the final scanned targets.

* If the new target is an IP address or a network then check if it
  was already added, or if it overlaps with one of the targets that are
  in the cache. Of course we should also combine networks and update the
  appropriate cidr, this way the more we add networks (targets) the more
  we reduce the size of the cache.

* Nmap will pop the targets that are in the hostname queue and resolve
  them, and before any scan it will try to see if the current IP address
  is alreay in the cache, if not then update the cache and scan it,
  otherwise just ignore this IP.


Finally I think that we can make Nmap remember all the scanned targets
even the cmdline targets.

Thanks.

-- 
tixxdz
http://opendz.org
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/