Nmap Development mailing list archives
Re: [Nsock] Scalable event handling
From: David Fifield <david () bamsoftware com>
Date: Thu, 25 Jul 2013 11:24:16 -0700
On Sat, Jun 15, 2013 at 09:13:46AM -0400, Henri Doreau wrote:
Longer version: I'm working on cleaning and optimizing nsock, for the fun of it and because of the plans we have to add features (like proper ssl proxy support) and to eventually use nsock for connect scan. I'd eventually like to introduce a hooking framework into nsock, so that both proxy handlers and SSL code could transparently intercept and overload connect/read/write operations. Overloading would consist in adding events internally. The problem is that adding many events is currently quite expensive. You can easily observe it by stressing nsock a bit, via nping[1] for instance. The reason why it's that slow is because each iteration of the runloop goes through _all_ events and check for completion (I/O or timeout). This big patch (it'd be complicated to split it...) I'm proposing here makes nsock iterate over the active events only. IOW only IODs that either have I/Os to handle or expired events are processed at each iteration. I had to change the internal data structures for that: * Introduced a gh_heap data structure, to efficiently make priority queues. All "expirables" (timers and timeouts) are now handled via such a tree. So that we don't need to iterate over _all_ elements at each nsock loop, only the active I/O events and the expired timers and I/O events are processed. Needless to say that the difference is noticeable when you have tens of thousands of events queued! * Replaced gh_lists containers by embedded objects (intrusive data structure). All operations are inline, and adding an item to a list doesn't allocate extra memory. It's needed so that we can manipulate the events lists simply from events that we extract from the gh_heap.
Thanks Henri. This looks like a high-quality, worthwhile change as always. I tried it and the TCP flood against localhost goes faster as you say. gprof shows that epoll_loop stops dominating the profile after the change. + #if !defined(container_of) + #define container_of(ptr, type, member) \ + ((type *)((char *)(ptr)-(char *)(&((type *)0)->member))) + #endif Perhaps this should use offsetof? offsetof appears to already be used in the code of a few dependencies. + gh_lnode_t nodeq; + gh_lnode_t nodeq2; /* used for pcap events */ Can you elaborate on how nodeq and nodeq2 work? From what I understand, nodeq2 is used for pcap-related events and nodeq is used for other kinds of events. Why do these need to be separate? I wonder if there is a more orthogonal way to do it. first_ev_next taking a nodeq2 parameter to choose which list to work on is a little strange. run_tests.sh is failing for me: started UDP listener on port 55234 (pid 5694) started TCP listener on port 55235 (pid 5696) started TCP SSL listener on port 55236 (pid 5697) nsock pool user data [OK] test timer operations [OK] set standard log levels [OK] check error log levels [OK] simple tcp connection [FAILED] (Connection refused)
Patch generated via git, let me know if you prefer another format or if you want me to open a dedicated branch on nmap-exp.
It looks like a "git show" patch; "git format-patch HEAD^" is better because it's easier to apply with "git am". David Fifield _______________________________________________ Sent through the dev mailing list http://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/
Current thread:
- Re: [Nsock] Scalable event handling Henri Doreau (Jul 24)
- <Possible follow-ups>
- Re: [Nsock] Scalable event handling David Fifield (Jul 25)
- Re: [Nsock] Scalable event handling Henri Doreau (Jul 25)
- Re: [Nsock] Scalable event handling Henri Doreau (Jul 26)
- Re: [Nsock] Scalable event handling David Fifield (Aug 06)
- Re: [Nsock] Scalable event handling Henri Doreau (Aug 10)
- Re: [Nsock] Scalable event handling David Fifield (Aug 10)
- Re: [Nsock] Scalable event handling Henri Doreau (Aug 11)
- Re: [Nsock] Scalable event handling Henri Doreau (Jul 25)
- Re: [Nsock] Scalable event handling David Fifield (Aug 06)
- Re: [Nsock] Scalable event handling Henri Doreau (Aug 07)
- Re: [Nsock] Scalable event handling Henri Doreau (Aug 10)