Re: [NSE] http.lua and delimiters

[Sven Klemm <sven () c3d2 de>]

Hi jah,


jah wrote:
> On 02/10/2008 05:00, David Fifield wrote:
> > I have to say, those are compelling numbers. I may have been too quick
> > to judge your patch earlier. I'm looking forward to seeing your updated
> > patch.
> >   
> Useful input - as usual, David.
> >   
> > > I'm pretty much ready to submit an updated patch as used for this test,
> > > but there's just one thing I'm wondering about adding.  The header value
> > > containing the status code (Status-Line) is currently discarded after
> > > the code itself is captured, but I'm tending toward keeping it to be
> > > more complete.  Also, sometimes they're almost interesting:
> > > HTTP/1.1 403 Forbidden ( The server denied the specified Uniform
> > > Resource Locator (URL). Contact the server administrator.  )
> > >     
> > That's fine by me. I don't it should be part of the header, rather a
> > separate table entry, because it appears it's not really considered part
> > of the header:
> >
> >         generic-message = start-line
> >                           *(message-header CRLF)
> >                           CRLF
> >                           [ message-body ]
> >         start-line      = Request-Line | Status-Line
> >
> >         message-header = field-name ":" [ field-value ]
> >   
> The attached patch adds to the previous one with the following:
>
> The library returns status-line: {status, status-line, header, body} if
> the status code is successfully matched - otherwise it's nil, the same
> as status.  I've updated the scripts nsedoc with this info and attached
> a possible patch for docs/scripting.xml.
>
> The test for the shortest header when the response matches both \r\n\r\n
> and \n\n.  This was present during the testing I did yesterday.
>
> A new pattern to split the header from the body: \n\r\n.  I managed to
> hit (once) the code which catches responses where there is no clear
> separation between header and body and this pattern was introduced in
> reaction to that one response caught by this code  (The same code I was
> thinking of removing).  The new pattern wasn't present during yesterdays
> testing, but more testing has been done since then to ensure that this
> addition doesn't cause side-effects (and so far, it hasn't).  I've
> decided to leave the catching code in too as there's the possibility
> we'll discover other ways to separate header from body (a possibility
> might be \r\n\r).
>
> Comments and debug statements are removed or tidied as applicable.

thanks for your patch. Had never thought so many HTTP server wouldn't 
speak proper HTTP. I've attached a patch with a few minor modifications.

It is unnecessary to prefer targetname over the ip in the request() 
function since host is only passed to socket:connect() which would 
have to resolve the targetname back to the ip.
I've rewritten the header/body separation to use a single regex which 
should do the same as your code and since the first match is 
non-greedy should always prefer the shorter version.
I've done something similar for the header splitting.
I've removed the type checks for header and body since they are always 
strings.


Cheers,
Sven





--
Sven Klemm
http://cthulhu.c3d2.de/~sven/

Index: nselib/http.lua
===================================================================
--- nselib/http.lua     (revision 10474)
+++ nselib/http.lua     (working copy)
@@ -164,36 +164,14 @@
   response = table.concat( response )
 
   -- try and separate the head from the body
-  local header, body, h1, h2, b1, b2
-  if response:match( "\r\n\r\n" ) and response:match( "\n\n" ) then
-    h1, b1 = response:match( "^(.-)\r\n\r\n(.*)$" )
-    h2, b2 = response:match( "^(.-)\n\n(.*)$" )
-    if h1 and h2 and h1:len() <= h2:len() then
-      header, body = h1, b1
-    else
-      header, body = h2, b2
-    end
-  elseif response:match( "\r\n\r\n" ) then
-    header, body = response:match( "^(.-)\r\n\r\n(.*)$" )
-  elseif response:match( "\n\r\n" ) then
-    header, body = response:match( "^(.-)\n\r\n(.*)$" )
-  elseif response:match( "\n\n" ) then
-    header, body = response:match( "^(.-)\n\n(.*)$" )
+  local header, body
+  if response:match( "\r?\n\r?\n" ) then
+    header, body = response:match( "^(.-)\r?\n\r?\n(.*)$" )
   else
-    body = response
-  end
-
-  local head_delim, body_delim
-  if type( header ) == "string" then
-    head_delim = ( header:match( "\r\n" ) and "\r\n" )  or
-                 ( header:match( "\n" )   and "\n" ) or nil
-    header = ( head_delim and stdnse.strsplit( head_delim, header ) ) or { header }
+    header, body = "", response
   end
 
-  if type( body ) == "string" then
-    body_delim = ( body:match( "\r\n" ) and "\r\n" )  or
-                 ( body:match( "\n" )   and "\n" ) or nil
-  end
+  header = stdnse.strsplit( "\r?\n", header )
 
   local line, _
 
@@ -224,8 +202,11 @@
     end
   end
 
+  body_delim = ( body:match( "\r\n" ) and "\r\n" )  or
+               ( body:match( "\n" )   and "\n" ) or nil
+
   -- handle chunked encoding
-  if type( result.header ) == "table" and result.header['transfer-encoding'] == 'chunked' and type( body_delim ) == 
"string" then
+  if result.header['transfer-encoding'] == 'chunked' and type( body_delim ) == "string" then
     body = body_delim .. body
     local b = {}
     local start, ptr = 1, 1


_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org