Re: speed optimizations

["Didier" <dgautheron () magic fr>]

Hi,
On Thu, 31 Oct 2013 11:57:35 +0000, Anders Broman wrote
> > Are wireshark's developers interested in reviewing and committing these
patches?
> reviewing ,yes certainly . Committing, depends I suspect :-)
>
> > -1 Base on Yami's work "snapshot feature: apply filter upon previous
displayed packets only" (without the SAT, only using a true table). help a lot
when >users are building filters incrementally or move a lot between them.
> > 2 When filtering don't dissect packet if not necessary. Help a lot, can
speed filtering up to 2 orders of magnitude (twice to three time as fast common).
> Could you give a brief description on how it works or is it simpler 
> to just submit a patch? Seems like a waste to do a patch and having 
> it rejected for some reason. I have some ideas on 1 and 2 too and it 
> would be interesting to see if they are similar.

for 1) It add a new op CHECK_HISTORY in filter virtual machine, keep a cache
with the last 64 filter expressions and a per packet 64 bits filters result in
frame_data structure.

When using a new filter it try first to factorize from the cache history up to
8 sub expressions, ex:
filter history:
tcp.stream eq 6

second filter
!(tcp.stream eq 6) 
is substituted with
!(CHECK_HISTORY 1)

then it build a three-states true table for the filter expression, states are
TRUE (passed), FALSE (not passed), UNKNOWN (must dissect packet)
for the above there's one history variable and the table is:
V1  R
0   TRUE  passed
1   FALSE not passed

If there's no taps, post-dissectors or column-info there's no need to dissect
packets (minus packet which passed and have dependent frames) the filter
result is known only by looking at the packet bitfield in rescan_packet.

Another filter:
tcp.stream eq 6 and http ==> (CHECK_HISTORY 1) and http

the true table is:
V1  R
0   FALSE   known to be false without decoding packets
1   UNKNOWN must decode packets.

and it only dissect (tcp.stream eq 6) packets

I have a very rough first patch from our version working with current svn.


==========
2) Is doing the same but with protocols ie: during the first pass (when
visited is false) it save in a per packet bitfield its protocols.

When compiling wireshark it's also extract which dissectors don't call
subdissectors or only call them via wireshark API 

Then when filtering it can:

- from the packet protocols bitfield bypass packet decoding (there's no point
to decode and filter on a http field if you already know there's no http in
the packet).

during dissect:
- bypass some 'leaf' dissectors, if a dissector don't call subdissectors and
has no interesting fields for the filter evaluation.
    
- set tree to NULL when calling dissectors with no 'interesting fields'. I'm
not sure this one is worth it without the AOT transform which convert a
dissector to a bunch of get_tvb..., offset += .., new_tvb_subset ...,
call_dissector(next_dissector) 

Regards
Didier


___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request () wireshark org?subject=unsubscribe