Re: New packet list - out of memory?

["Anders Broman" <a.broman () telia com>]

-----Ursprungligt meddelande-----
Från: wireshark-dev-bounces () wireshark org
[mailto:wireshark-dev-bounces () wireshark org] För Jeff Morriss
Skickat: den 6 oktober 2009 18:21
Till: Developer support list for Wireshark
Ämne: Re: [Wireshark-dev] New packet list - out of memory?

Anders Broman wrote:
> > Hi,
> > Experimenting loading the file there is 163 calls to VirtualAlloc()
> > ~1.630 GB memory used
> > Changing 
> > #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 512)
> > to:
> > #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64)
> >
> > Gives only 26 VirtualAlloc() so it seems like we waist a huge amount of
> > memory...
> > 15 VirtualAlloc() with 32 bytes.
> >
> > Is there a reason not to change to:
> > #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 32) ?
>
> Well, the disadvantage is that the overhead associated with the chunk 
> headers grows (when using canaries).  For example, with the current /512 
>  number sizeof(emem_canary_t) is 184,328 which means 1.47% of the total 
> allocation for the chunk goes to overhead.  Changing that to /64 means 
> the canary header grows to 1,474,568 and total overhead grows to 14.1%. 
>  Those numbers are not counting the actual canaries at the end of each 
> allocation.
>
> BUT, the waste that goes into those headers is vastly overshadowed by 
> the waste in unused buffer space...
>
> > Loading is actually 2s slower with se_alloc for col_text...
> > Are we better off using glibs memory functions?
>
> Another question: do we really need to use se_ allocations here?  Do we 
> need the canaries or are we just using se_ allocations because they get 
> cleaned up for us "automatically?
I think the rationale for using se_alloc was a belief that it would be
Faster and of course the ease of use (no free).

> "
>
> Should we have a new allocator without the canaries?
I think either the current se_ allocations implementation need to be changed
Or the function removed. The result where WIRESHARK_DEBUG_SE_NO_CHUNKS=T
And loading is faster is puzzling.
Regards
Anders
___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request () wireshark org?subject=unsubscribe

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request () wireshark org?subject=unsubscribe