Re: Desired improvements in Nmap performance? [FASTER IS SLOWER]

[David Fifield <david () bamsoftware com>]

On Tue, Dec 02, 2008 at 11:18:44PM +0000, Brandon Enright wrote:
> > > I'm starting a project to improve Nmap's performance and the
> > > predictability of the length of its scans. This may involve tuning
> > > performance parameters, adjusting the congestion control mechanism,
> > > or other things not thought of yet.
> > >
> > > Write back with possibilities you see for improvement. I'd like any
> > > changes we make to be in response to actual user concerns.
>
> This one is going to take a little bit of work to explain but the basic
> idea behind it is that sometimes Nmap severely penalizes itself for
> packet drop and never properly recovers from that penalty.
>
> This manifests itself as very poor performance scanning Windows
> machines when I scan from a highly tuned Linux scanning box but decent
> performance when scanning from a zero-tuned Linux box.
>
> When I dig into the problem, it seems that Windows (with the firewall
> off) has a maximum outstanding RST limit (or max RST rate) that causes
> it to drop a bunch of incoming SYNs if scanned too fast.  From a
> well-tuned box Nmap goes over that rate immediately and slows way
> down.  From a non-tuned box Nmap never goes above that rate and the
> Windows machine scans quickly.
>
> Here is an example from the tuned Linux machine:
>
> $ time sudo nmap -T5 -PN -n -v -d -p- --open 128.54.184.49
>
> Starting Nmap 4.76 ( http://nmap.org ) at 2008-12-02 23:10 GMT
> - --------------- Timing report ---------------
>   hostgroups: min 1, max 100000
>   rtt-timeouts: init 250, min 50, max 300
>   max-scan-delay: TCP 5, UDP 1000
>   parallelism: min 0, max 0
>   max-retries: 2, host-timeout: 900000
>   min-rate: 0, max-rate: 0
> - ---------------------------------------------
> Initiating SYN Stealth Scan at 23:10
> Scanning 128.54.184.49 [65535 ports]
> Packet capture filter (device eth0): dst host 132.239.1.114 and (icmp or ((tcp or udp) and (src host 128.54.184.49)))
> Increased max_successful_tryno for 128.54.184.49 to 1 (packet drop)
> Increased max_successful_tryno for 128.54.184.49 to 2 (packet drop)
> Increasing send delay for 128.54.184.49 from 0 to 5 due to 60 out of 149 dropped probes since last increase.
> Stats: 0:00:04 elapsed; 0 hosts completed (1 up), 1 undergoing SYN Stealth Scan
> SYN Stealth Scan Timing: About 0.76% done
> Current sending rates: 136.30 packets / s, 5986.87 bytes / s.
> ... scan will take a long time, killed ...

I think the killer here is scan delay. Once that delay goes from 0 to 5 ms, 
you're stuck with a 200 packet/s maximum for the rest of the scan.
Currently there is no way for the scan delay to ever decrease. Jason
DePriest mentioned this problem.

I discussed this with Fyodor and we agree that the scan delay mechanism
should be made more dynamic. It could grow in a way more sophisticated
than just doubling and be able to decrease also.

The congestion control on the tuned Linux box has correctly sensed great
network conditions a low RTT and so it starts blasting packets really
fast. But in doing so it unwittingly runs into a different limit, that
of Windows putting some restriction on its RSTs. Maybe instead of
increasing the scan delay we could artificially increase the estimated
RTT?

Usually the question with network congestion control is not "how fast
can I send packets?" but "how many packets can the network hold?". But
in the case of rate limiting by end hosts, we really do care about the
absolute rate at which we send packets. Scan delay and the congestion
window are attempts to solve two different problems. So as tricky as
scan delay is, I don't think we can get rid of it and rely solely on the
congestion window.

With the --max-rate infrastructure already in place, we could get rid of
scan delay entirely and replace it with absolute rate limits. This would
accomplish the same purpose but would be more granular. In the cases
where we currently increase the scan delay, we could instead drop the
maximum rate by half. Then it would creep upwards again like the
congestion window in congestion avoidance mode, eventually oscillating
around the correct rate. Thus we would have a pair of limiters: the
congestion window and the maximum scanning rate, both growing in a
similar fashion.

Thanks for your suggestions so far Brandon, they have been
thought-provoking.

David Fifield

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org