Wireshark mailing list archives
Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.)
From: Evan Huus <eapache () gmail com>
Date: Fri, 11 Jul 2014 20:23:01 -0400
On Fri, Jul 11, 2014 at 6:07 PM, Anders Broman <a.broman58 () gmail com> wrote:
Den 11 jul 2014 23:13 skrev "Bálint Réczey" <balint () balintreczey hu>:Hi All, Please provide the input data for letting others reproduce the results or perform the performance tests on pcap files already available to the publicOk I'll see if we can use something from the wiki instead.I'm not a fan of implementing custom memory management methods because partly because I highly doubt we can beat jemalloc easily on performance and custom allocation methods can also have nasty bugs like the one observed in OpenSSL: http://www.tedunangst.com/flak/post/analysis-of-openssl-freelist-reuseWe have gone through a set of memory allocation schemes already to try to improve performance (g_slice, emen and now wmem)
Technically, emem was originally added to fix memory leaks caused by exceptions. Wmem (as just a better-architected replacement for emem) is similar. The block allocator was added later as an optimization because it provided a *significant* improvement in performance over a simple linked-list of OS-alloced blocks (5-10x faster in my benchmarks).
are you saying that you are opposed to that? As wmem seems to be the faster scheme for packet scope memory allocation /free, why not use it in all possible places where the scope is "packet"?Please don't sacrifice protection for 2% speedup. Please keep wmem usage for cases where it is used for garbage collecting (free() after end of frame/capture file) not when the allocation and deallocation are already done properly.? A slow scheme might be working well but that in it self does not warrant to not replace it with a faster one. With this reasoning we shouldn't have introduced ep memory in the first place. What percentage if improvement do you think makes a change worthwhile? The set of improvements Jacub and I have done lately has given a reduction of 40-50 percent compared to 1.10 measuring with the sample file. The problem is that each improvement only yeald a percent or 2. Agreed that we shouldn't complicate the code for a small speed gain. In your blog you say that people would accept double the execution time with increased security - I'm not so sure. If say the reformatting of a video takes one hour instead of 30 minutes. Just my 2 cents AndersThanks, Balint 2014-07-11 8:58 GMT+02:00 Jakub Zawadzki <darkjames-ws () darkjames pl>:Hi, On Sat, Jun 21, 2014 at 10:12:48PM +0200, Jakub Zawadzki wrote:If we're in topic of optimizing 'slower' [de]allocations in commonfunctions:- tvb allocation/deallocation (2.5%, or 3.4% when no filtering) 243,931,671 * ???:tvb_new[/tmp/wireshark/epan/.libs/libwireshark.so.0.0.0]202,052,290 > ???:g_slice_alloc (2463493x)[/usr/lib64/libglib-2.0.so.0.3600.4]291,765,126 * ???:tvb_free_chain[/tmp/wireshark/epan/.libs/libwireshark.so.0.0.0]256,390,635 > ???:g_slice_free1 (2435843x)[/usr/lib64/libglib-2.0.so.0.3600.4]This, or next week I'll try to do tvb.... or maybe this week: ver0 | 18,055,719,820 (-----------) | Base version96f0585268f1cc4e820767c4038c10ed4915c12aver1 | 18,185,185,838 (0.6% slower) | Change tvb allocator g_slice_*to wmem with file scopever2 | 17,809,433,204 (1.4% faster) | Change tvb allocator g_slice_*to wmem with file/packet scopever3 | 17,812,128,887 (1.3% faster) | Change tvb allocator g_slice_*to simple object allocator with epan scopever4 | 17,704,132,561 (2.0% faster) | Change tvb allocator g_slice_*to simple object allocator with file scopeI have uploaded patches & profiler outputs to:http://www.wireshark.org/~darkjames/tvb-opt-allocator/Please review, and check what version is OK to be applied. P.S: I'll might find some time to do ver5 with slab allocator, butit'll look like object allocator API with epan scope.Cheers, Jakub.___________________________________________________________________________Sent via: Wireshark-dev mailing list <wireshark-dev () wireshark org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-request () wireshark org?subject=unsubscribe___________________________________________________________________________Sent via: Wireshark-dev mailing list <wireshark-dev () wireshark org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-request () wireshark org?subject=unsubscribe ___________________________________________________________________________ Sent via: Wireshark-dev mailing list <wireshark-dev () wireshark org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-request () wireshark org ?subject=unsubscribe
___________________________________________________________________________ Sent via: Wireshark-dev mailing list <wireshark-dev () wireshark org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-request () wireshark org?subject=unsubscribe
Current thread:
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.), (continued)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Bálint Réczey (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Bálint Réczey (Jul 11)
- Message not available
- Fwd: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) darkjames-ws (Jul 21)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Bálint Réczey (Jul 24)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 24)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Bálint Réczey (Jul 24)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Bálint Réczey (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Bálint Réczey (Jul 11)