Re: hf_http_response_code in packet-http.c

["Sultan, Hassan via Wireshark-dev" <wireshark-dev () wireshark org>]

> -----Original Message-----
> From: Wireshark-dev [mailto:wireshark-dev-bounces () wireshark org] On Behalf
> Of Erik de Jong
> Sent: Thursday, July 13, 2017 11:58 AM
> To: Developer support list for Wireshark <wireshark-dev () wireshark org>
> Subject: Re: [Wireshark-dev] hf_http_response_code in packet-http.c
>
>
>
> On Thu, Jul 13, 2017 at 8:47 PM, Sultan, Hassan via Wireshark-dev <wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> > wrote:
>
>
>
>
>       > -----Original Message-----
>       > From: Pascal Quantin [mailto:pascal.quantin () gmail com
> <mailto:pascal.quantin () gmail com> ]
>       > Sent: Thursday, July 13, 2017 11:26 AM
>       > To: Sultan, Hassan <sultah () amazon com
> <mailto:sultah () amazon com> >
>       > Cc: Developer support list for Wireshark <wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> >
>       > Subject: RE: [Wireshark-dev] hf_http_response_code in packet-http.c
>       >
>       >
>       >
>       > Le 13 juil. 2017 19:54, "Sultan, Hassan" <sultah () amazon com
> <mailto:sultah () amazon com>
>       > <mailto:sultah () amazon com <mailto:sultah () amazon com> > > a
> écrit :
>       >
>       >
>       >
>       >
>       >       > -----Original Message-----
>       >       > From: Pascal Quantin [mailto:pascal.quantin () gmail com
> <mailto:pascal.quantin () gmail com>
>       > <mailto:pascal.quantin () gmail com
> <mailto:pascal.quantin () gmail com> > ]
>       >       > Sent: Thursday, July 13, 2017 10:39 AM
>       >       > To: Developer support list for Wireshark <wireshark-
>       > dev () wireshark org <mailto:dev () wireshark org>  <mailto:wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> > >
>       >
>       >       > Cc: Sultan, Hassan <sultah () amazon com
> <mailto:sultah () amazon com>
>       > <mailto:sultah () amazon com <mailto:sultah () amazon com> > >
>       >       > Subject: Re: [Wireshark-dev] hf_http_response_code in packet-
> http.c
>       >       >
>       >
>       >       > Hi Hassan,
>       >       >
>       >       >
>       >       > 2017-07-13 19:25 GMT+02:00 Sultan, Hassan via Wireshark-dev
>       > <wireshark-
>       >
>       >       > dev () wireshark org <mailto:dev () wireshark org>
> <mailto:dev () wireshark org <mailto:dev () wireshark org> >  <mailto:wireshark-
> <mailto:wireshark->
>       > dev () wireshark org <mailto:dev () wireshark org>  <mailto:wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> > > >:
>       >
>       >       >
>       >       >
>       >       >
>       >       >
>       >       >       > -----Original Message-----
>       >       >       > From: Wireshark-dev [mailto:wireshark-dev-
> <mailto:wireshark-dev->
>       > bounces () wireshark org <mailto:bounces () wireshark org>
> <mailto:wireshark-dev-bounces () wireshark org <mailto:wireshark-dev-
> bounces () wireshark org> >
>       >       > <mailto:wireshark-dev-bounces () wireshark org
> <mailto:wireshark-dev-bounces () wireshark org>  <mailto:wireshark-
> <mailto:wireshark->
>       > dev-bounces () wireshark org <mailto:dev-bounces () wireshark org> >
> > ] On Behalf
>       >       >       > Of Erik de Jong
>       >       >       > Sent: Wednesday, July 12, 2017 10:12 PM
>       >       >       > To: Developer support list for Wireshark <wireshark-
>       >
>       >       > dev () wireshark org <mailto:dev () wireshark org>
> <mailto:dev () wireshark org <mailto:dev () wireshark org> >  <mailto:wireshark-
> <mailto:wireshark->
>       > dev () wireshark org <mailto:dev () wireshark org>  <mailto:wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> > > >
>       >       >       > Subject: Re: [Wireshark-dev] hf_http_response_code in
> packet-
>       > http.c
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       > On Thu, Jul 13, 2017 at 1:12 AM, Sultan, Hassan via
> Wireshark-
>       > dev
>       >       > <wireshark-
>       >
>       >       >       > dev () wireshark org <mailto:dev () wireshark org>
> <mailto:dev () wireshark org <mailto:dev () wireshark org> >
>       > <mailto:dev () wireshark org <mailto:dev () wireshark org>
> <mailto:dev () wireshark org <mailto:dev () wireshark org> > >
> <mailto:wireshark- <mailto:wireshark->
>       > <mailto:wireshark- <mailto:wireshark-> >
>
>       >
>       >       > dev () wireshark org <mailto:dev () wireshark org>
> <mailto:dev () wireshark org <mailto:dev () wireshark org> >  <mailto:wireshark-
> <mailto:wireshark->
>       > dev () wireshark org <mailto:dev () wireshark org>  <mailto:wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> > > > > wrote:
>       >       >       >
>       >       >       >
>       >       >       >       Hi,
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       >       I am starting to learn the Wireshark code base, and one
> thing
>       >       > puzzles
>       >       >       > me…
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       >       Why is hf_http_response_code defined as a FT_UINT16
> with
>       >       > BASE_DEC
>       >       >       > rather than an FT_STRING ?
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       >       It’s a text field… not an integer.
>       >       >       >
>       >       >       >
>       >       >       > Presenting it as a number allows for filtering like:
>       >       >       > http.response.code > 200
>       >       >       >
>       >       >       > Which would not be possible when presented as a string.
>       >       >
>       >       >       Thanks for the info, but in that case would it not be more
>       > appropriate to
>       >       > have the normal field as an FT_STRING and add a generated field
> as
>       > FT_UINT16
>       >       > ? My understanding of generated fields is that this is their
> purpose :
>       > represent
>       >       > data that doesn't exactly correspond to the packet data.
>       >       >       We could still keep the field named as is today (hence
> ensuring all
>       >       > existing filters still work), but simply make it a generated field,
> and add
>       > an
>       >       > FT_STRING to represent the actual data as it is in the packet.
>       >       >
>       >       >       Thoughts ?
>       >       >
>       >       >
>       >       >
>       >       >
>       >       > Does having the field as it is designed today generating you any
> issue?
>       > For fields
>       >       > having numerical values only, it makes much more sense to have
> the
>       > digits
>       >       > directly instead of the string and that's already what we are doing
> in
>       > numerous
>       >       > places. And personally I do not see any valid reason to change
> that.
>       > Your
>       >       > suggestion implies that many fields would start to be duplicated
>       > without any
>       >       > added value.
>       >
>       >
>       >       My reason is that anyone wanting to build automation based on
>       > Wireshark's parsing will not be able to rely on Wireshark's reporting of
> field type
>       > & format to interpret the packet data.
>       >       The automation would walk through parsed fields and be told that
> the
>       > http response code is a 3-bytes UINT16 with value say 200, so bytes 00
> 00 c8,
>       > which is wrong. Right now the way these fields are setup works for
> human
>       > interpretation, by people looking at the rendered value on a screen,
> but it is far
>       > from ideal for automation.
>       >
>       >
>       > Thanks fact that HTTP is text based does not change the fact that the
> code is
>       > supposed to be a number. So I'm not sure to get your point.
>       > Moreover tons of people use tshark for automation and I do not
> remember
>       > someone complaining about the current output. So maybe I simply did
> not
>       > understand your use case.
>       >
>       >
>       >       One could argue it's the main purpose of Wireshark, but I think it
>       > needlessly prevent a whole class of use-cases from being achieved
> with
>       > Wireshark's technology.
>       >
>       >
>       >
>       > Could you elaborate? What use case do you have in mind exactly?
>
>
>       Imagine automation that attempts to perform modification on packets
> using the parsed information provided by Wireshark, or somehow goes and
> directly reads fields in the packet data based on what Wireshark's parsed
> information says (type/offset/length/...). Wireshark reporting the field as
> FT_UINT16 instead of say a UTF-8 FT_STRING throws off the automation as it
> would either attempt to write an FT_UINT16 at the offset of the field, writing
> the wrong format, or be confused by seeing a 3 bytes FT_UINT16 and give up.
>
>
>
>
> But in that case it's more like you're doing a part of the parsing in another
> application, why not implement a full parser in that application? The whole point
> of an analyzer is to make sense of the data passed into it for the user. And what
> about protocols that could be send with different encodings (BER, XER, etc)
> there a field type should still be the same - eg UINT16 - but the representation
> on the wire is totally different.
 
The beauty of this is specifically that the other app does not need to do any parsing at all. It fully relies on 
Wireshark's parsing engine to get the tree of fields, the offset/length/type/name/encoding/... of each field and 
instead of having to write custom code for each protocol (you know, a parser), the app simply has to implement generic 
handling for FT_STRING, FT_UINT16, etc... and the various encodings. So for your BER encoded integer, having an 
FT_UINT16 with an ENC_BER encoding flag for example and a size of 4 (type+length+2 bytes for most short ints) would 
convey all the info needed to handle the field.

Will such a thing work for absolutely every single protocol out there ? Maybe not, but it works for the vast majority 
of them. I did a very similar thing on top of Network Monitor in a previous life at Microsoft (I had the code available 
being internal to the company) and it was a huge success, helping many people in various ways.

>       How about having an #ifdef for such things so that Wireshark's use-case
> today does not have to suffer the addition of generated fields, and such
> automation cases could still get full fidelity parsing by building Wireshark's
> engine with the specific #define ? Today they have to build directly against
> Wireshark's code anyway so that shouldn't be a problem.
>
>
>       __________________________________________________________
> _________________
>       Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org
> <mailto:wireshark-dev () wireshark org> >
>       Archives:    https://www.wireshark.org/lists/wireshark-dev
> <https://www.wireshark.org/lists/wireshark-dev>
>       Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-
> dev <https://www.wireshark.org/mailman/options/wireshark-dev>
>                    mailto:wireshark-dev-request () wireshark org <mailto:wireshark-
> dev-request () wireshark org> ?subject=unsubscribe

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
Archives:    https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request () wireshark org?subject=unsubscribe