Wireshark mailing list archives
Re: RFD: The Future of Memory Management in Wireshark
From: Evan Huus <eapache () gmail com>
Date: Sun, 28 Oct 2012 10:12:41 -0400
On Sat, Oct 27, 2012 at 3:09 PM, Evan Huus <eapache () gmail com> wrote:
On Sat, Oct 27, 2012 at 2:17 PM, Dirk Jagdmann <doj () cubic org> wrote:General thoughts from the list on whether or not this would be a good idea?some general comments on the whole wmem idea: memory allocation is done almost everywhere in Wireshark, also in somewhat hidden places. For example all the proto_tree_add_* functions will need to allocate memory. It will be a real pain to pass down the current allocator object into each of those function invocations, you're looking at changing a considerable amount of code.Yes, it wouldn't be a small job.Also when thinking about writing dissector code, it's probably not useful to mix different allocators while dissecting a protocol.Not sure what you mean here. Mixing ep_ and se_ memory? Or mixing g_alloc and mmap memory? The separation of the allocators (glib, mmap, etc.) from the front-end isn't driven by a desire to mix and match allocators during a specific dissection (I agree that would be odd). It was more driven by the fact that the current allocators are all jumbled together and thus hard to maintain. It also makes it easy to add new scopes where necessary without writing hundreds of new wrapper functions.An alternative idea (although a little more ugly) is to use a thread local global variable which contains pointers to the current ep and se allocators. The se_* and ep_* functions retrieve the allocator object from that thread local global variable and do their allocations. If a function wants to change the allocators for its own function calls, it can make a copy of the global variable pointers to local variables, set it's desired new allocators and restore the global pointers at the end, like: __thread void* se_allocator; __thread void* ep_allocator; void my_special_func() { void* old_se = se_allocator; void* old_ep = ep_allocator; se_allocator = create_special_se(); ep_allocator = create_special_ep(); dissect_packet(); garbage_collect(se_allocator); garbage_collect(ep_allocator); se_allocator = old_se; pe_allocator = old_ep; }This would certainly require fewer changes to existing code. It is, however, complicated by the lack of nice cross-platform API for thread-local storage. I just took a quick look at glib's gestures in this direction and they seem rather limited (a max of 128 thread-local objects, and they can't be destroyed). One of the things I was hoping to do by removing the allocators from the global scope was make it very obvious what the exact scopes of the various allocators are. Right now ep_ and se_ memory is used in a lot of places that aren't correct because the code has simply been able to grab the global. We really don't want anyone using ep_ memory unless there is actually a packet being dissected, and scoping the ep_ pool would enforce that nicely.
We might be able to fake the proper scoping using thread-local globals if we wrap everything in functions that assert the state of a dissection. Something like: __thread wmem_allocator_t *packet_scope; __thread gboolean packet_in_scope = FALSE; wmem_allocator_t * wmem_packet_scope(void) { g_assert(packet_in_scope); return packet_scope; } void wmem_start_packet_scope(void) { g_assert(!packet_in_scope); packet_in_scope = TRUE; } void wmem_stop_packet_scope(void) { g_assert(packet_in_scope); packet_in_scope = FALSE; wmem_free_all(packet_scope); } Invalid accesses would still compile, but at least they would throw an assertion as soon as the code path was hit. Thoughts on something like this? ___________________________________________________________________________ Sent via: Wireshark-dev mailing list <wireshark-dev () wireshark org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-request () wireshark org?subject=unsubscribe
Current thread:
- Re: RFD: The Future of Memory Management in Wireshark, (continued)
- Re: RFD: The Future of Memory Management in Wireshark Sébastien Tandel (Oct 26)
- Re: RFD: The Future of Memory Management in Wireshark Evan Huus (Oct 26)
- Re: RFD: The Future of Memory Management in Wireshark Sébastien Tandel (Oct 26)
- Re: RFD: The Future of Memory Management in Wireshark Evan Huus (Oct 25)
- Re: RFD: The Future of Memory Management in Wireshark Jeff Morriss (Oct 25)
- Re: RFD: The Future of Memory Management in Wireshark Evan Huus (Oct 25)
- Re: RFD: The Future of Memory Management in Wireshark Pascal Quantin (Oct 25)
- Re: RFD: The Future of Memory Management in Wireshark Dirk Jagdmann (Oct 27)
- Re: RFD: The Future of Memory Management in Wireshark Evan Huus (Oct 27)
- Re: RFD: The Future of Memory Management in Wireshark Evan Huus (Oct 28)
- Re: RFD: The Future of Memory Management in Wireshark Jakub Zawadzki (Oct 28)
- Re: RFD: The Future of Memory Management in Wireshark Evan Huus (Oct 28)
- Re: RFD: The Future of Memory Management in Wireshark Jakub Zawadzki (Oct 29)
- Re: RFD: The Future of Memory Management in Wireshark Evan Huus (Oct 29)