Andrew's GSoC Summary

[Andrew Jason Farabee <afarabee () uci edu>]

Hi everyone,

It's been a fun summer of code! Now that it's the firm "pencil's
down" date, I'd like to go over the status of my project.


== PROOF OF CONCEPT ===============================================

I started my GSoC by working on a couple of "proof of concept"s
(detail on the last one: http://seclists.org/nmap-dev/2015/q2/317).
This branch has quite a few issues. It relies on experimental NSE
scanning code that isn't as efficient for port scanning.  It is
also necessary to specify -Pn in order to prevent nmap's host
discovery, which operates outside of a proxy.

The code also uses socks4a instead of socks5 for script scanning,
so without some elaborate timeout magic nmap will be unable to
determine whether the proxy server won't allow the connection due
to a timeout or due to a reset packet.  This means all ports that
are filtered will be mislabeled as closed.  Since socks5 specifies
a dedicated response for a timeout, I should be able to determine
if a port is filtered.

There is also an issue when scanning through tor.  Many malicious
exit node operators will spoof responses to port 25 (smtp) requests
in order to get credentials for sending spam.  Unfortunately, this
means that about 25% of the time smtp is marked as open even though
it is closed on the target.

https://svn.nmap.org/nmap-exp/pasca1/nmap-nseportscan-socks4a


== SOCKS4A ========================================================

The socks4a code that was added to nsock works by calling a
separate connection handler from the other proxy types.  This uses
the targetname when creating the proxy connection instead of an IP.
This means that as it is socks4a is unable to be used in a chain of
proxies unless it is the last node.

Merging the socks4a code won't make sense until after a new address
structure is created that is able to contain hostnames or IPs.
This will also make it easier to implement Socks5 connections by
hostname later.

It might make more sense to not explicitly support socks4a at all.
Instead, we could use socks4a by default and only use socks4 when
the user only provides target IP addresses, which would be
reflected by the new address structure.

https://svn.nmap.org/nmap-exp/pasca1/nmap-nsock-socks4a


== NCAT PROXYCHAINS ===============================================

I got pretty distracted by this part of the project.  In order to
test and demonstrate nsock's proxy functionality, we decided to
refactor ncat client proxy functionality into nsock's proxychains.
This turned out to be a larger issue because ncat already supported
proxy authentication and socks5 while nsock did not.

Merging this branch alone would be a sizable regression in
functionality.  The current tests which can be seen in
https://svn.nmap.org/nmap-exp/pascal/tests show that this branch
currently fails most HTTP tests and all SOCKS5 tests.

Even with the socks5 code included, ncat-test.pl fails for
unauthenticated socks5 connections.  This is due to ncat-test.pl
requesting a proxy connection from localhost instead of ::1 or
127.0.0.1.  It might just be my machine, but I don't know enough
about IPv6 to figure this out right away.

There is still an ugly piece of code here to check if the argument
to --proxies is empty when the string is non-NULL.  This is
necessary at the moment because the calling code in nsock does not
check if the string is empty before accessing it.  This could be a
small fix.

https://svn.nmap.org/nmap-exp/pasca1/nmap-ncat-proxychains


== PROXY AUTHENTICATION ===========================================

Nsock proxy authentication should be able to be merged on its own
once a few things are complete:

The http digest code included is temporary.  A better solution to
what is currently in this branch can be found in nmap-ncat-socks4a.
This temporary http digest authentication code fails 2 tests in
ncat-test.pl when merged with nmap-ncat-proxychains (although this
would probably be an easy fix).  The current http digest doesn't
support as many possible challenge scenarios as the ncat http
digest code does.

Socks5 authentication is also missing.  This is only an issue if
the socks5 branch is merged as well.  Adding this might need a
couple extra states to nsock/src/proxy_socks4.c but it shouldn't
take too much work.

Finally, any usernames or passwords included to the --proxies
option are displayed in cleartext to other users on that machine
when using commands like 'ps -aux' or 'history'.

https://svn.nmap.org/nmap-exp/pasca1/nmap-nsock-proxyauth


== SOCKS5 =========================================================

Socks5 code is working well right now.  As with the other branches,
all the relevant test results can be found in
nmap-exp/pasca1/tests/nmap-nsock-socks5.

If nmap-nsock-proxyauth and nmap-nsock-socks5 are merged together,
there will be an issue with socks5 authentication missing.  Again,
this shouldn't take long to add.

Also, the socks5 code supports IPv6 scanning.  I haven't tested
this thoroughly because (embarrassingly) I don't know much about
IPv6.

Requesting socks5 tunnels by domain name should also be added, but
this doesn't make much sense until a new structure is created to
pass different target specification types (IPv4,IPv6, domain name).

https://svn.nmap.org/nmap-exp/pasca1/nmap-nsock-socks5


== LIBHTTP ========================================================

This branch is a combination of nmap-nsock-socks4a, nmap-nsock-
socks5, nmap-ncat-proxychains, and nmap-nsock-proxyauth.  The only
difference is that instead of the temporary http digest code, this
branch contains the start of a separate library "libhttp" (name in
progress) which attempts to move the existing http digest code to
be used by both nsock and ncat.

This branch currently does not compile due to issues with libhttp.
https://svn.nmap.org/nmap-exp/pasca1/temporary-nsock-http-digest
is a copy of this branch that using the temporary http digest code
is nsock instead.

https://svn.nmap.org/nmap-exp/pasca1/nmap-ncat-socks4


== PORT VULN SCRIPTS TO VULNS LIBRARY =============================

This folder was created to solve "Port 5 *-vuln NSE scripts to the
`vulns` library" (https://github.com/nmap/nmap/issues/147).  I
believe it is ready to be merged.

I created vulnerable virtual machines to test http-iis-webdav-vuln
and smtp-vuln-cve-2010-4344.  I tested http-vmware-path-vuln and
http-vuln-cve2010-0738 by using fake functions.  Some of the test
results are available here:

https://gist.github.com/andrewfarabee/dffc9e8c245d29271db4

smb-check-vulns was not included because it will have to be split
up in order to address https://github.com/nmap/nmap/issues/171 ,
which is "Split smb-check-vulns into separate smb-vuln-msXX-XXX.nse
scripts".

https://svn.nmap.org/nmap-exp/pasca1/issue147


== HOSTNAME PROXY SCANNING OUTLOOK ================================

When I started the summer, my main goal was to get scanning working
by name through proxies.  Unfortunately this goal is still very far
away.

The next step is to create a structure that will allow for
specifying a domain name without an IP (or along with an IP,
depending on the situation).  This struct may just contain a struct
sockaddr_storage ss (which would allow for an IPv4 or IPv6 address)
and a char* targetname.  The difficult part would be deciding where
it is implemented and moving from using sockaddr_storage to this
new struct throughout nsock and other parts of nmap.

There are a lot of open questions on what to do next, though there
are a lot of good ideas here:

http://seclists.org/nmap-dev/2015/q3/att-66/meeting.txt


== TL;DR ==========================================================

So what will it take to merge this mess of intertwined branches?
Here is a breakdown by functionality:

Socks5:
        Nothing!
        Okay, IPv6 testing.

Proxy Authentication:
        Better http digest challenge handling (or libhttp).
        Masking passwords.

Ncat Proxychains: (if you want to reduce regression in ncat)
        Merging Proxy Authentication.
        Merging Socks5.
        Adding Socks5 Authentication.

Socks4a:
        You probably shouldn't.
        Documentation.

All of them together:
        Socks5 authentication.
        Better http digest handling.
        Socks4a authentication (can be found in nmap-ncat-socks4a).
        Masking passwords.
_______________________________________________
Sent through the dev mailing list
https://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/