Re: portspoof IDS trouble

[Daniel Miller <bonsaiviking () gmail com>]

List,

There has been considerable off-list discussion of the mechanics of this
option, so I will summarize the current design goals here. Suffice it to
say that Jay's initial patch will undergo some serious revision before
being re-submitted.

First, we agreed the name should change to use "ignore" instead of "forget"
as the verb. This means we are currently calling it --ignore-after.

Next, we will use Henri and Fyodor's suggestion of a percentage/number
combination, where the user can supply one or both. Here is a pseudocode
breakdown:

ign_ratio = (user_given_percent / 100) or 0.00
ign_number = user_given_number or ign_ratio * total_requested_ports
ports_scanned = 0
open_ports = 0
while port_state = scanner_gets_response() do
    ports_scanned += 1
    if port_state == open then
        open_ports += 1
    if open_ports > ign_number AND (open_ports / ports_scanned) >=
ign_ratio then
        ignore host
        break
done

Some examples:

"--ignore-after 10" ignores hosts as soon as they reach 11 open ports.
"--ignore-after 100%" ignores hosts with 100% of scanned ports open after
the port scan phase completes.
"--ignore-after 50%" must not ignore hosts before 50% of requested ports
have been scanned (e.g. must not ignore a host whose first port is found
open (100% of attempted ports, but not yet 50% of requested ports))
"--ignore-after 100%,30" will ignore hosts if and only if the first 30
ports tried are found open.
"--ignore-after 50%,30" will ignore hosts with 50% of attempted ports open,
but only if the number of open ports is more than 30.

Lastly, it will be clear in all output formats that the host was ignored
because of having too many ports open. Right now, this looks like it will
require a new host state "ignored" besides "up" and "down". This may also
be used for hosts that exceed the host timeout in the higher timing
templates (e.g. -T5).

Your input is requested on whether this option should be applied
automatically in some way under -T5.

Dan


On Mon, Jul 7, 2014 at 11:23 PM, Fyodor <fyodor () nmap org> wrote:

> On Tue, Jun 24, 2014 at 6:53 AM, Jay Bosamiya <jaybosamiya () gmail com>
> wrote:
> > I wrote a patch (attached) that does this. Using --forget-after X makes
> > Nmap forget any host that has more than X open ports.
> > This patch may need improvement and testing before it can be considered
> > for inclusion and so I have marked the option as experimental.
> > However, I am posting this since Andrew (and others who come across
> > hosts with portspoof) might find this useful.
>
> Thanks Jay.  It's hard to win an arms race with programs like Portspoof
> that try to fool Nmap, but hardly anyone ever uses them anyway.  However, I
> see all-open behavior like this from many of my large scale scans due to
> SSL accelerators, load balancers, various proxies, and other network
> equipment.  It can be annoying when you scan millions of addresses and then
> look to find hosts with an obscure port open, only to find the results
> dominated by these false positives.
>
> One problem with adding yet another obscure Nmap performance option is that
> hardly anyone will ever use it.  It is usually better to focus on
> improvements which affect more users.  However, if done properly, this
> feature might be worthwhile as the default for the more aggressive scans
> like -T4 and -T5.  After all, it's not much different than including a host
> timeout in those scans (which we do).
>
> So let's think about what we might want if we were to make this feature a
> -T4 default.  Well, we can't just go with a total number of open ports,
> since that's not fair when comparing a five-port scan (where no more than
> five can obviously be open) and a 65,535 port scan of a busy server.  But
> we can't just rely on a percentage either.  Even if we say "only forget
> when 100% of the ports are open", that's not appropriate if you only
> scanned 5 common ports.  -T5 would use either that or a more restrictive
> value.  If a percentage is given, we would need to continue scanning until
> we have enough open ports that reaching the specified limit is guaranteed
> even if all the rest of the ports to be scanned prove closed.
>
> Rather, we should probably be able to specify both.  So we would only
> forget the host if at least X% of the ports in a protocol are open, and
> that constitutes at least Y total.  So for -T4, maybe we'd say at least 50%
> ports open AND that it is at least 100 ports total.  Maybe the syntax could
> be like "--forget-after 50%,100" and you could specify either a percentage
> or a total number or both.  I feel like maybe we could think of a better
> name than --forget-after, but none come to mind immediately.
>
> Even if we make this a -T4/-T5 default, we would want to provide the option
> so people can override it if desired.  If you are only scanning like 5
> ports and want to be sure to block these all-open machines, you'd probably
> pick a random port like 22794 to add to your scan list and then specify
> "--forget-after 6" or (same effect) "--forget-after 100%" to exclude the
> machines with all 5 real ports, plus the highly unlikely port open.  There
> would need to be an option to tell Nmap not to ever skip hosts based on
> open port count.  Maybe "--forget-hosts -" or --dont-forget-hosts or
> something.
>
> As Henri notes, the output should tell you which hosts were skipped for
> this reason.  That way you can still find and probe those hosts, but you
> don't have a giant 65K-line entry in your log file and your searches for
> individual open ports are not polluted with these.
>
> Also, the feature would have to be documented.
>
> Cheers,
> Fyodor
> _______________________________________________
> Sent through the dev mailing list
> http://nmap.org/mailman/listinfo/dev
> Archived at http://seclists.org/nmap-dev/
_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/