Re: [RFC] Mass rDNS performance tweak

[doug () hcsw org]

Hi jah,

Thank you for your analysis and work on the rdns system. Your
explanation of the algorithm is dead on and it looks like you
have some great tuning results here.

I was never completely satisfied with the resolver's performance
and I think there's lots of room for improvement. You may already
have seen them but here are my design notes and early benchmarks
for the system:

http://hcsw.org/nmap/rdns/

On Wed, Jan 14, 2009 at 09:40:13AM +0000 or thereabouts, jah wrote:
> [#: 1, OK: 171, NX: 0, DR: 79, TO: 363, SF: 0, TR: 534, CN: 0]

I like the TO timeouts stat. That sounds like it will be very
helpful when tuning.

> For the DNS servers I have use of, this algorithm is too aggressive and
> something between me and the server drops requests - I imagine when they
> are sent too quickly.

Interesting. Your transmission stats do seem to suggest the system
is a bit aggressive for your setup. For what it's worth I did most
of the development and got the best results with a local resolver,
ie BIND running on my machine or a machine on my LAN. Thinking about
it now it might be best to tune nmap for using your ISP's resolver
as that is probably what most people do.

> If I set CAPACITY_MIN to 2 and CAPACITY_MAX to about 6 or 7 I get very
> nearly 100% accuracy and the total time for resolution isn't hugely more
> than it is currently.

This sounds like a good improvement. It should be as accurate as possible.
Might also be a good idea to check that this also doesn't hurt performance
much when your resolver is local (if you haven't already).

> Another issue is that timed out requests aren't necessarily an indicator
> of the need to reduce capacity because they may have timed out for other
> reasons such as a non responsive nameserver.
> It's basically very difficult to determine the optimum capacity!

Agreed. There are so many variables that determine how long a DNS
request takes to fill that congestion heuristics like those used
for TCP aren't as effective here. Not only do you have the round
trips to the nameserver, but also all round trips between the
nameserver and all the other nameservers it needs to query, as
well as whether any of the records are cached or not. And don't
forget that each nameserver has its own timeout, load balancing
and re-transmission system too, just like nmap.

> What this means is depending on the drop ratio, we'll increase capacity
> by a maximum of 5 during any timeslot which allows for timed-out
> requests to balance the capacity with decreases.  This happens in small
> steps and the theory is that we should gradually approach the optimum
> capacity and then wobble fairly close to it thereafter.

Interesting approach. If you have the time you should experiment with
the following:

A) Distributing the load over many nameservers. This is difficult
   because it's hard to know when to consider a nameserver unable
   to resolve a record and move it to another server and also because
   some nameservers might have different capacities of how many
   resolutions on the wire they can handle vs others.

B) Find the nameservers that are authoritative for some large
   blocks of IPs:

doug@eclipse:~$ host google.com
google.com has address 209.85.171.100
...
doug@eclipse:~$ dig +trace -x 209.85.171.100
...
171.85.209.in-addr.arpa. 86400  IN      NS      ns1.google.com.
171.85.209.in-addr.arpa. 86400  IN      NS      ns3.google.com.
171.85.209.in-addr.arpa. 86400  IN      NS      ns4.google.com.
171.85.209.in-addr.arpa. 86400  IN      NS      ns2.google.com.
;; Received 127 bytes from 192.26.92.32#53(HENNA.ARIN.NET) in 26 ms

100.171.85.209.in-addr.arpa. 86400 IN   PTR     cg-in-f100.google.com.
...

  Then use these nameservers (ie ns1.google.com, ns2.google.com, etc)
  in your benchmarks. In theory Nmap should be able to do this just
  as fast or faster than a local BIND instance but I think BIND has
  better heuristics for handling timeouts and shuffling between servers
  so nmap currently slightly underperforms a local BIND.


Anyways keep up the good work. I haven't found much info online
about optimising large batches of DNS queries but it is fairly
important to me so I will be following all results closely.

Hope this helps,

Doug

Attachment: _bin
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org