Nmap Development mailing list archives
Re: Timing race cars with a sundial (-sV match performance)
From: Brandon Enright <bmenrigh () ucsd edu>
Date: Sat, 9 May 2009 00:46:57 +0000
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 8 May 2009 17:27:58 -0700 Fyodor <fyodor () insecure org> wrote:
On Fri, May 08, 2009 at 10:07:03PM +0000, Brandon Enright wrote:A few weeks ago I embarked on a (mostly) futile attempt to measure the performance of applying a PCRE match expression in ourThanks for sending the results, even if they weren't all you hoped.
Actually the results were better than I hoped. I was under the mistake impression that a lot of our more inefficient match lines actually translated to slow matching. Instead of finding out that we really need to fix hundreds of matches we learned that we probably only need to fix ~5 :-)
As for optimizing these matches, for the LPD match we really need to add an anchor. I did some checking and all UCSD hosts that match the service can safely add '^' and still match.Sounds like a good idea.The '^.*' usage in the ser2net match is counter-productive. I don't have any matching services so I don't know what can be done to improve on '^.*' besides just removing it.Removing the "^.*" sounds like the way to go, as it seems completely pointless to me.
Will do. It won't actually make the match any faster but it will at least make study go faster.
The Xylan PizzaSwitch telnetd match is pretty zealous in its use of .* early in the match. Telnet services often match the start and then print a large amount of data (banners, abuse warnings, etc). '.*' is consuming all of that data on all telnet services and then backtracking a byte at a time. We should make the '.*' lazy by changing it to ".*?". Even better would be to add a few more matching bytes to match the telnet control bytes before using '.*' but we may not have enough data to do this.I'll forward you some Xylan PizzaSwitch signatures in a few minutes in case it helps to have the extra data.
(I just got them, thanks!)
The Cyrus Murder matches look like a '^' can be added. The protocol looks like IMAP and it is safe to anchor the \* in IMAP with '^'. UCSD doesn't have any Cyrus Murder installs for me to test. I'd suggest we add the anchor and then wait for new submissions if it doesn't match.Sounds reasonable.
Another user (thanks!) shared with me their server output which looked something like: * AUTH "GSSAPI" * PARTIAL-UPDATE * OK MUPDATE "hostname" "Cyrus Murder" "v2.3.13" "(master)" This means that although Cyrus Murder is in the spirit of IMAP it isn't exact and a ^ won't work. We can still make a failing match /much/ faster though. We need to tell PCRE that instead of skipping bytes that couldn't possibly match (aka .*) the format is "\* something\r\n". I propose: m|^(\* [^\r\n]+\r\n)*?\* OK... PCRE should be able to fail quickly rather than trying to apply the match to every byte offset in the service.
I'm happy to submit a patch that does all of the above if it sounds reasonable.Yeah, please do create such a patch and check it in.
I'll leave the Murder changes above out unless you think it is okay to put such a regex in the probes file.
We might also think about adding Nmap internal performance statistics logging to Nmap proper similar to my addition of log_write(LOG_STATS, ...) in this branch. I feel like sometimes using - -d3 or more is too much data when all you want to do is measure performance stats.Have written up anywhere how this feature works? We do try to avoid filling Nmap with too much debugging code, but sometimes it is definitely worthwhile.
Okay I'll come up with a proposal. Brandon -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) iEYEARECAAYFAkoE0ocACgkQqaGPzAsl94IHqwCeLnOEa5JDu5gIkhGBB+USofav 6ykAn0jp0OU1hhdsVwvuoxkzMtoCCotV =OlkQ -----END PGP SIGNATURE----- _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://SecLists.Org
Current thread:
- Timing race cars with a sundial (-sV match performance) Brandon Enright (May 08)
- Re: Timing race cars with a sundial (-sV match performance) Fyodor (May 08)
- Re: Timing race cars with a sundial (-sV match performance) Brandon Enright (May 08)
- Re: Timing race cars with a sundial (-sV match performance) Brandon Enright (Jun 08)
- Re: Timing race cars with a sundial (-sV match performance) Fyodor (May 08)