Re: Performance thoughts

[Fyodor <fyodor () insecure org>]

On Tue, Jul 04, 2006 at 08:46:25AM +0200, majek04 wrote:

>       Hmm. UltraScanInfo::numIncompleteHosts is taking the most of
>       resources.
>       What's this function doing you ask?
>       unsigned int numIncompleteHosts()
>                       { return incompleteHosts.size();}

Good find.  Sheesh, that size() performance is pathetic.  Especially
since the STL guarantees that size() on a list is a constant-time
operation.  So implementations are required to store the size as a
variable rather than iterate through the list each time you ask for
the size.  Does anyone know what is going on here?  If you compile
with -O2, and don't include -g, do you still see this huge performance
difference?

If there is no way to fix the STL list implementation, your patch
which stores the size outside the list is a good option.  But I hate
to do it if we don't have to because it is just asking for bugs to
come later when someone adds/removes from the list and then forgets to
update the associated size variable.  Maybe it would be better to just
use our own efficient list implementation.  nsock has one named
gh_list.  We could potentially move that to Nbase and then use it in
Nmap too.

Admittedly I can't blame everything on STL.  It looks like
numIncompleteHosts() is called 626 million times, which seems a bit
excessive.

>       http://ai.pjwstk.edu.pl/~majek/private/nmap/nm-after.png

I like these graphs.  What are you using to create them?

>       
>       Most of cpu is consumed by getting random numbers "RAND_bytes".

Interesting.  We assumed that the OpenSSL random number generator
would be better and so we use that if it is available.  But it sounds
like it isn't as efficient as using Linux's /dev/urandom .  I'll
remove that (openssh random number) functionality for ALPHA3.

o Nmap no longer gets random numbers from OpenSSL when it is available
  because that turned out to be slower than Nmap's other methods
  (e.g. /dev/urandom on Linux, /dev/arandom on OpenBSD, etc.).  Thanks
  to Marek Majkowski for reporting the problem.


Cheers,
Fyodor


_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev