Re: massping migration and you

[David Fifield <david () bamsoftware com>]

On Thu, Aug 30, 2007 at 05:59:46PM +0000, Brandon Enright wrote:
> On Thu, 30 Aug 2007 10:43:15 -0600 plus or minus some time David Fifield
> <david () bamsoftware com> wrote:
> > > Nmap done: 186336 IP addresses (11554 hosts up) scanned in 9040.909
> > > seconds
> > >
> > > real    150m40.914s
> > > user    21m38.227s
> > > sys     2m26.036s
> >
> > Wow, that's alarming. Your scan is one I would expect the migrated host
> > discovery to do well at. Although I've never tested it on such a large
> > group of hosts.
>
> There are many times when I want to know all machines with a particular
> port open on our public /16s and private /12.  I always scan
> with -T5 as a base template and generally add --max-retries 1 and
> - --min-hostgroup 2048.

I've been looking at the code, and I can't see that --min-hostgroup has
ever affected ping scans. That affects the group size for things like
port scanning, but it only has an effect after hosts come out of
nexthost, after massping has already seen them.

Which is all the more puzzling because I thought your --min-hostgroup
option might be the source of the speed discrepancy. Just for the heck
of it, can you try a scan using --min-parallelism 2048 instead?

> I used to do these scans with -P0 because in my own twisted logic "it's
> much faster to only send 1 or 2 SYNs than to have to ping/send other probes
> first before sending the SYNs.  By the time you've determined the host is
> up, you could have already determined if the port is up."
>
> Of course, when I actually tested it, it was between 5x and 10x faster to
> use -P A<short list of ports> before sending the single port probes.
>
> I always attributed this to the speed over reliability of massping() versus
> the reliability over speed for ultrascan().  I don't have real test results
> handy but I can run some scans to illustrate this if you're interested.

The majority of the massping migration work has been in trying to make
ultra_scan appropriate for these types of scans. That's just what we're
trying to overcome. There's no real reason why

        nmap -sP -PS80

should be different from

        nmap -p0 -sS -p80

> > Can you send me the times from scanning just one of your /16 address
> > spaces? Maybe there's something that's making the scan scale
> > non-linearly. Also, please try it again with -T4. That increases the
> > congestion window recovery speed, which will help if you're getting lots
> > of drops.
>
> Okay, I ran:
>
> ...
>
> > ultra_scan is much more cautious in the face of drops than massping was.
> > Are you getting many? You can find out by running with -d2 and grepping
> > the log file for "DROPPED".
>
> I know hitting drops and timeouts unnecessarily can severely hurt
> performance.  It seems to me that for 57k scanned hosts, even 106 drops is
> a drop in the bucket for total probes sent.

Actually that many drops can hurt a lot. That's (kind of) like getting
100 drops during a 64 K port scan of a single host, after which you
would expect Nmap to slow down considerably. However, if the slowness
doesn't increase accuracy, then there's no reason for it.

> I've always been under the impression that timing options like -T# didn't
> affect "ping scans" at all.  Did it used to or does it only now affect them
> because of the migration to ultrascan()?

I don't think it used to, but it surely does with ultra_scan.

> I'm going to re-run my 3 /16 net scans with -T4 and -T5 to see if that puts
> us back into the 25 minute range.
>
> Also, I've re-run the scan that crashed yesterday many times and it hasn't
> crashed again.  I'll keep trying.

Okay, thanks. Your detailed reports have been helpful so far. Sorry for
the late reply.

David Fifield

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org