Jacek's status report - #12 of 16

[Jacek Wielemborek <wielemborekj1 () gmail com>]

Hi guys,

This is the report 12/16 for the Google Summer of Code project
“Bringing Lua to Ncat”.

It was a really pleasant week. No long hours spent debugging, loads of
features added (so, probably, loads of bugs as well). For a quick demo
you might want to try the following with the code from my
d33tah/ncat-lua-callbacks branch:

ncat -l &
ncat -L scripts/filters/rot13.lua --proxy-type http -l 8013 &
ncat -L scripts/filters/rot13.lua --proxy-type --proxy localhost:8013 localhost

And see that you just spawned a HTTP proxy that talks ROT13 ;) Of
course, you could use any other kinds of filters and even stack your
own ones. I know that the example is rather silly, but the point there
is to show how easy to achieve such things - now that I added the
infrastructure

Accomplishments:

* Used my userdata trick to solve the buffer overflow problem that
happened when Lua code returned more data that would fit in the buffer
provided by fdinfo_recv. The temporary solution, definitely not
satisfactory, was to drop the excessive data, now I set *pending to 1
and pass data in chunks.

* Added connect() and close() methods. During my last meeting with
David, when we played with my chat server, I noticed how many times
they'd prove useful to see if somebody new joined the conversation or
left it. To test if the close() actually closes the connection, I
added /kick command to chat.lua; this is only available for the server
administrator. responder.lua also disconnects, when you say “bye” to
him.

* Moved my Lua hooks from ncat_recv to fdinfo_recv to enable them in
proxy connections. This turned out to be not enough yet for connect
mode yet, because there were some recv()/send() calls that didn't yet
go through the fdinfo_recv and fdinfo_send functions (but this showed
up later in testing and I'll get back to that).

* Introduced experimental connect-mode hooks. They work well enough to
demonstrate the Base64+ROT13 demo, but hadn't been tested extensively
yet. Currently there's connect(), close(), recv() and send().
Actually, recv() is a bit tricky - the listen-mode non-blocking recv()
implementation discarded the buffer and I needed a way to emulate this
behavior in Nsock. The implementation might change later on.

* Since even I started to be getting lost in my code, I did a major
refactoring of ncat_lua.c. This involved splitting the file into five
modules: generic Lua code, --lua-exec code, generic filters code,
listen-mode filters code and connect-mode filters code. Hopefully
there will soon be a file for handling connections spawned by filters,
too.

* Solved the bug related to initializing socket abstractions before a
complete parsing of command line. The cause of the problem was that we
couldn't know if we're spawning listen-mode or connect-mode sockets
until we were sure that the --listen switch is there or not.

* Wrote a report that wraps up quite a lot of information on the
current socket abstractions interface and implementation. My message,
along with a link to secwiki page, can be found here:
http://seclists.org/nmap-dev/2013/q3/431

* Did some changes to the interface - for example, make_socket
function is no longer visible to the user. Also, connection roots are
stored in a separate table for convenience and to speed up lookups.
Moreover, undefined methods are now by default set to transparently
call super, so adding a new event to the standard interface will not
break old scripts.

* Made proxy code use fdinfo_send and fdinfo_recv wrappers. This
exposed a bug that happens when you use these functions in connect
mode, which is initialized to call lua_nsock_read. I solved it with
code that swaps the default function for the given file descriptor,
which made the proxy example from the beginning of this e-mail work.


* Added early support for UDP. Not tested much yet, though recv and
send seem to be working fine.

* Now that Fyodor published Ncat Users' Guide XML file, I added some
Ncat tricks to the document. The related mailing list thread can be
found there: http://seclists.org/nmap-dev/2013/q3/379

* Updated my scripts to conform to my “forward compatibility” concept.
Re-tested them to make sure they work again (which turned out to be a
good idea because they needed some changes then) and found two bugs in
the process.

* Added eval.lua, a remote Lua shell that can be used to debug
filters. Already reminds the real Lua interpreter a lot.

Priorities:

* Talk with David about --lua-exec documentation Fyodor asked for. I
was supposed to do it last week, but I forgot during the meeting.

* Find a good model for error handling in my socket abstractions code.
There's so many things that can fail along the way that even detecting
them would be a lot of work. I need to discuss that with David to find
a good strategy for that.

* Wrap up what I already did with socket abstractions? I'm currently
waiting for David's review of my code and then I expect to have loads
of work to improve the design, probably also write some docs and -
which will probably take most time - the tests. I was reluctant to do
it so far because I didn't know what the more final shape of the
feature would be.

* Add a way to spawn additional connections from inside the filters,
perhaps also to change the destination address. I have a lot of ideas
for the features, need to sort out with David which are worth
implementing and in what order.

Yours,
Jacek Wielemborek
_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/