Re: Ncat and windows line endings

[David Fifield <david () bamsoftware com>]

On Wed, Jun 03, 2009 at 06:11:07PM -0600, David Fifield wrote:
> On Fri, May 08, 2009 at 08:42:25PM -0700, Fyodor wrote:
> > On Thu, May 07, 2009 at 10:48:46PM +0100, jah wrote:
> > > I felt sure that something like this has been raised before, but haven't
> > > found anything pertinent.  Is this a known issue and, more to the point,
> > > can we afford to modify ncat to do something about it given that \r\n
> > > might have been sent as part of a HTTP GET, for example?
> >
> > Hi Jah.  Excellent point.  I wish we there was a way we could
> > automatically detect and do "the right thing".  But as you noted,
> > passing \r\n straight through may be desirable in some cases
> > (e.g. writing HTTP requests) while changing \r\n to \n is desirable in
> > others (talking to a UNIX shell).  We should probably at least provide
> > options for both approaches.  We currently offer:
> >
> > First of all, maybe -C should format all bare \n in the buffer to
> > \r\n.  Is there a good reason (other than ease of implementation) for
> > only handling bare \n which happens to appear at the end of the
> > buffer?  Of course it needs to make sure that the \n chars in the
> > buffer are bare (not preceeded by a \r already).  We don't want to end
> > up with \r\r\n.
> >
> > Second, we should probably have an option which does the opposite and
> > converts \r\n in input to plain \n.  Maybe it could be -N/--newline or
> > -L/--lf or something.  It should check the whole buffer for \r\n too,
> > IMHO.
> >
> > Does anyone have a different idea or should we just implement these
> > two changes?  Anyone want to write a patch?
>
> I found a way to retain the _O_TEXT setting after duplicating the file
> descriptor. This is like a permanent -L setting, and is the way Ncat
> used to work. There's no extra work on Ncat's part; as far as it can
> tell, the bare \n it sees comes straight from the terminal. -C works to
> convert the \n that Ncat sees into \r\n, which is nice because the
> command line usage will be exactly the same on Windows and Unix; on
> Windows we can simply pretend that the terminal sends \n. Windows users
> don't have to remember to use -C to talk to a web server when they're on
> Unix, and Unix users don't have to remember to use -L to talk to a shell
> when they're on Windows. You always use -C when talking to protocols
> that require it, and otherwise your newlines are \n on any platform.
>
> A problem with this is that the newline translation applies to all forms
> of input: pipes and files, not just the console. That will break binary
> file transfers, for example. So this is what I propose: Set _O_TEXT on
> Windows when stdin is a terminal, and set _O_BINARY otherwise. Set
> _O_BINARY on stdout unconditionally.
>
> With this, all terminals would appear the same, so there would be no -L
> option. We would apply half of jah's patch, the part that fixes -C to
> replace \n with \r\n anywhere in an input buffer. I have a patch for
> _O_TEXT that I'll submit in a little bit.

Here is the patch. It sets _O_TEXT mode on stdin when the input is from
the console (checked using the _isatty function), and _O_BINARY
otherwise. It sets _O_BINARY on stdout. So the terminal sends \n for
line endings, and no newline translation is done anywhere else, just
like on Unix.

The only problem I can envision is if it's somehow possible to fool
_isatty. I can confirm that it returns true for
        ncat target
and false for
        ncat target < infile
        cat infile | ncat target
but there might be other cases. Documentation is here:
http://msdn.microsoft.com/en-us/library/f4s0ddew(VS.80).aspx

Setting _O_BINARY on stdout fixes another bug that was not discovered
until now. Ncat on Windows was doing newline translation on output as
well, breaking binary transfers.
        unix$ md5sum /etc/services
        aca587f0966a6f14536dae8fa15f095a  /etc/services
        C:\> ncat.exe -l > services
        unix$ ncat win --send-only < /etc/services
        C:\> C:\cygwin\bin\md5sum.exe services
        4458bb84b47c93cfa7f5e2432f21ca68 *services

What do you think?

David Fifield

Attachment: ncat-lf-mode.diff
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org