Re: pcap_next_ex() and pcap_dump() performance decreases over time...

[barcaroller <barcaroller () gmail com>]

On 2015-08-23 03:13:50 +0000, Guy Harris said:

> > It has puzzled me too.  Keep in mind that I am making millions of these 
> > pcap calls; only one of them could be taking 2 seconds.
>
> Then the average might be more interesting than the maximum.

Yes, but the average was only about 20 times higher, which corresponds 
to the data rate dropping from 8GBits/sec to 200 MBits/sec.


> > I actually read the packets from a pcap file (not part of the test), 
> > store the packets in memory (e.g. 50,000 packets), then write them in a 
> > loop to disk (e.g. 50,000 x 1,000) using multiple pcap files (none 
> > greater than 100MB); this is phase 1.
>
> So does "50,000 x 1,000" mean that it writes out each packet 1,000 times?

Yes, but to different pcap files.  At any given time, I have only one 
pcap file open for writing.


> So the reading of the file in phase 1 isn't being tested, so the 
> pcap_next_ex() calls being performance-tested are in phase 2?

That's correct.


> > I then read all packets (e.g. 50,000 x 1,000) in chunks and destroy 
> > each chunk before reading the next; this is phase 2.
>
> "In chunks" meaning that you have a loop that reads a chunk worth of 
> packets, with each packet read with pcap_next_ex(), and then free all 
> the packets in the chunk before reading the next chunk?

That's correct.


> > Right.  But something in the process is making pcap_next_ex() and 
> > pcap_dump() slower with time.
>
> Something *somewhere* is.

Are there any numbers that you think would be useful in analyzing this problem?


_______________________________________________
tcpdump-workers mailing list
tcpdump-workers () lists tcpdump org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers