Nmap Development mailing list archives
Re: Weird Crash - "WAITING_TO_RUNNING"
From: Nathan <nathan.stocks () gmail com>
Date: Wed, 3 Nov 2010 11:07:26 -0600
On Tue, Nov 2, 2010 at 4:45 PM, David Fifield <david () bamsoftware com> wrote:
On Tue, Nov 02, 2010 at 04:24:41PM -0600, Nathan wrote:On Tue, Nov 2, 2010 at 2:17 PM, Patrick Donnelly <batrick () batbytes com> wrote:Hi Nathan, On Mon, Nov 1, 2010 at 4:12 PM, Nathan <nathan.stocks () gmail com> wrote:Anyone know why I would get the following crash using the options "-sS -sV -T4 -p 1-65535" with nmap 5.35DC1? "nse_restore: WAITING_TO_RUNNING error! not enough memory" I've got 16GB of RAM on the box (2.6.23 linux kernel), with 10GB free...so "not enough memory" must not be talking about total system RAM...It is referring to system memory. I don't know why you got that error though. Do you have ulimit turned on? Are you able to reproduce this error repeatedly and reliably?To my knowledge, ulimit is not turned on in any fashion: $ ulimit unlimited I can indeed reproduce this error repeatedly and reliably. Here's the full output of it crashing three times in a row. Note that it takes about 13 minutes until it crashes. ====> Here's what I did to watch the memory during the third run. You'll notice that while a ton of memory was used, there was still over 2GB completely free. There was also a lot of memory in the disk cache that could have been used if necessary <==== $ while /bin/true ; do sleep 1 && echo "---------" && date && cat /proc/meminfo | grep MemFree ; done [snipped about 13 minutes worth of output right here] --------- Tue Nov 2 16:04:55 MDT 2010 MemFree: 2365096 kB <==== (crash!) --------- Tue Nov 2 16:04:56 MDT 2010 MemFree: 5509820 kB5509820 KB - 2365096 - KB is right at 3 GB, so you could be running out of per-process address space. Is this on a 32-bit machine? It looks like there's a loop somewhere that is allocating memory without bound. I'd like you to try running each potential script individually to see which one is responsible. One way is to run Nmap with the -d2 option, wait five minutes or so, and then press Enter in the terminal. With -d, Enter causes Nmap to print a stack trace of all running scripts. The stack trace will look like this: NSE: Running: 'html-title' (thread: 0x8feea48) stack traceback: [C]: in function 'receive' <More...> (Another way is to run "nmap -sV -d2" to get a list of scripts, then try running each of them individually with --script. But I think the "press Enter" method will be faster.) David Fifield
"Is this on a 32-bit machine?" -- The hardware is 64-bit capable, but the old version of linux on it is compiled in 32-bit, I think. The OS can address more than 4GB RAM, but the stuff running on it seems to act very 32-bit. Python's maxint is 32-bit, anyway: $ python Python 2.5.2 (r252:60911, Oct 30 2008, 18:03:18) [GCC 4.1.2 (Gentoo 4.1.2 p1.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import sys sys.maxint
2147483647 $ uname -a Linux scan5 2.6.23-gentoo-r9 #4 SMP Tue Oct 12 16:02:12 MDT 2010 i686 Intel(R) Xeon(R) CPU E5507 @ 2.27GHz GenuineIntel GNU/Linux I attached a file with the bottom ~9000 lines of the stack trace before I hit Ctrl-C. If you'd like be to do it again and try to get the top of the stack trace, just let me know. It seemed to be spitting out a stack trace without end, which is why I eventually killed it. Here's a small snippet of the stack trace. The entire ~9000 lines I got seemed to just alternate unpredictably between skypev2-version and jdwp-version. NSE: Running: 'skypev2-version' (thread: 0x114e4950) stack traceback: [C]: in function 'socket_lock' [string "local connect, socket_lock = ...;..."]:3: in function 'connect' /usr/share/nmap/nselib/comm.lua:59: in function 'setup_connect' /usr/share/nmap/nselib/comm.lua:122: in function 'exchange' /usr/share/nmap/scripts/skypev2-version.nse:27: in function </usr/share/nmap/scripts/skypev2-version.nse:26> (tail call): ? NSE: Running: 'jdwp-version' (thread: 0x22ff37a0) stack traceback: [C]: in function 'socket_lock' [string "local connect, socket_lock = ...;..."]:3: in function 'connect' /usr/share/nmap/nselib/comm.lua:59: in function 'setup_connect' /usr/share/nmap/nselib/comm.lua:122: in function 'exchange' /usr/share/nmap/scripts/jdwp-version.nse:34: in function </usr/share/nmap/scripts/jdwp-version.nse:30> (tail call): ? NSE: Running: 'jdwp-version' (thread: 0x15204580) stack traceback: [C]: in function 'socket_lock' [string "local connect, socket_lock = ...;..."]:3: in function 'connect' /usr/share/nmap/nselib/comm.lua:59: in function 'setup_connect' /usr/share/nmap/nselib/comm.lua:122: in function 'exchange' /usr/share/nmap/scripts/jdwp-version.nse:34: in function </usr/share/nmap/scripts/jdwp-version.nse:30> (tail call): ? NSE: Running: 'skypev2-version' (thread: 0xf014ba0) stack traceback: [C]: in function 'socket_lock' [string "local connect, socket_lock = ...;..."]:3: in function 'connect' /usr/share/nmap/nselib/comm.lua:59: in function 'setup_connect' /usr/share/nmap/nselib/comm.lua:122: in function 'exchange' /usr/share/nmap/scripts/skypev2-version.nse:27: in function </usr/share/nmap/scripts/skypev2-version.nse:26> (tail call): ? Does this mean anything to anyone? ~ Nathan
Attachment:
waiting_to_running_stack_trace.txt.bz2
Description:
_______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/
Current thread:
- Weird Crash - "WAITING_TO_RUNNING" Nathan (Nov 01)
- Re: Weird Crash - "WAITING_TO_RUNNING" Patrick Donnelly (Nov 02)
- Re: Weird Crash - "WAITING_TO_RUNNING" Nathan (Nov 02)
- Re: Weird Crash - "WAITING_TO_RUNNING" David Fifield (Nov 02)
- Re: Weird Crash - "WAITING_TO_RUNNING" Nathan (Nov 03)
- Re: Weird Crash - "WAITING_TO_RUNNING" Patrick Donnelly (Nov 03)
- Re: Weird Crash - "WAITING_TO_RUNNING" Nathan (Nov 03)
- Re: Weird Crash - "WAITING_TO_RUNNING" Nathan (Nov 03)
- Re: Weird Crash - "WAITING_TO_RUNNING" David Fifield (Nov 03)
- Re: Weird Crash - "WAITING_TO_RUNNING" David Fifield (Nov 03)
- Re: Weird Crash - "WAITING_TO_RUNNING" Patrick Donnelly (Nov 03)
- Re: Weird Crash - "WAITING_TO_RUNNING" Nathan (Nov 08)
- Re: Weird Crash - "WAITING_TO_RUNNING" Nathan (Nov 08)
- Re: Weird Crash - "WAITING_TO_RUNNING" Rob Nicholls (Nov 08)
- Re: Weird Crash - "WAITING_TO_RUNNING" Nathan (Nov 02)
- Re: Weird Crash - "WAITING_TO_RUNNING" Patrick Donnelly (Nov 02)
- Re: Weird Crash - "WAITING_TO_RUNNING" David Fifield (Nov 05)