Nmap Development mailing list archives
Re: NSE: odd output, testing, etc
From: "Patrick Donnelly" <batrick.donnelly () gmail com>
Date: Sun, 28 Dec 2008 20:46:21 -0700
On Sun, Dec 28, 2008 at 6:34 PM, David Fifield <david () bamsoftware com> wrote:
On Sun, Dec 21, 2008 at 05:10:59AM -0700, Patrick Donnelly wrote:Brandon, Ron, and I have been discussing these problems privately by email to track the causes. Here are the results:* NSE is overly aggressive with parallelism. It isn't unusual for NSE to report more than 2000 active NSE scripts. When this happens Lua seems to thrash and NSE scanning slows to a crawl. I think this has the ability to trigger the "lock, (null), <int>, tcp, ERROR" errors describe below.This shouldn't be a problem as the default of 10 script threads may use network resources, all the other 1990 threads are blocked. Memory is also not a concern as each thread only consumes about 1 KB of memory. NSE shouldn't slow to a crawl. I assume most of the scripts were blocked on a socket. I haven't observed any problems with NSE encountering a deadlock or infinite loop since the problems discussed in [1][2].So it sounds like the scripts are running and blocking fine. The only problem is that they tend to time out before they get a chance to do anything. Brandon said that after one round of scanning with more than 300 script threads running, in the next round NSE locks immediately. Could something about timing out scripts be corrupting the Lua state? The problem goes away when r11122 is reverted. That was the change that made use of a persistent Lua state.
I need more information than NSE locking up on the second scan to determine the cause. As far as I know Brandon hasn't had this problem anymore after I corrected the threads improperly finishing because of time outs. I could speculate that some of the scripts may use mutexes with the same object as the last round of scripts. If those last scripts timed out with a lock on the needed mutex then there would be no progress. A garbage collection cycle in between scans may fix the problem to allow all the mutex functions to be collected (assuming no script thread saved a persistent reference somewhere). -- -Patrick Donnelly "One of the lessons of history is that nothing is often a good thing to do and always a clever thing to say." -Will Durant _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://SecLists.Org
Current thread:
- NSE: odd output, testing, etc Brandon Enright (Dec 17)
- Re: NSE: odd output, testing, etc Ron (Dec 17)
- Re: NSE: odd output, testing, etc Kris Katterjohn (Dec 17)
- Re: NSE: odd output, testing, etc M M (Dec 17)
- Re: NSE: odd output, testing, etc Patrick Donnelly (Dec 21)
- Re: NSE: odd output, testing, etc David Fifield (Dec 28)
- Re: NSE: odd output, testing, etc Patrick Donnelly (Dec 28)
- Re: NSE: odd output, testing, etc Brandon Enright (Dec 28)
- Re: NSE: odd output, testing, etc David Fifield (Dec 28)
- Re: NSE: odd output, testing, etc Patrick Donnelly (Dec 29)
- Re: NSE: odd output, testing, etc David Fifield (Dec 29)
- Re: NSE: odd output, testing, etc David Fifield (Dec 28)
- Re: NSE: odd output, testing, etc Ron (Dec 17)