Re: Forwared to me

[mhw () ALCOVE WITTSEND COM (Michael H. Warfield)]

Hello,

Raymond Medeiros enscribed thusly:
> I would have to only completely agree with you.  This fix which was
> contained in the ISS security announcement was indeed very weak.  My
> suggestion was to at the very least deny access to finger requests from
> the outside.  This attack really isn't that bad however I have been able

        Huh?  Did you read the entire advisory or just Solar Designer's
excerpt.  The above recommendations were all in the original advisory.  I
agree with Solar Designer's opinion that the shell script idea was rather
weak, but it was the last of several suggestions listed.

        Here are the two relavent paragraphs taken fron the original
advisory:

]       Disable finger service on any systems connected to any NIS based
] network.  Disable access to internal finger services at all perimeter
] defenses (firewalls, gateways, filtering routers, etc.).  If the finger
] service is required for some specific purpose, limit it to the minimum
] number of restricted hosts or to hosts which are NOT participating
] in NIS.

        The above suggestions were the ones recommended first.

]       For those who wish to continue to run the finger service on their
] systems, there are other possible actions that could be taken, like using
] a public domain fingerd that doesn't do ambiguous lookups.   The finger
] service can also be protected by even something as simple as wrapping
] finger to not do ambiguous lookups like as follows:

        The questioned script then followed, last.

        More on the recommendations below...

> to take out a machine on my own subnet using a simple perl script.  In
> reality it doesn't appear to be more of a threat than a ping flood.  I
> have also looked into using it as part of the beginning to a spoofing
> attack (under controlled conditions of course) and it has no apparent
> value.  Never the less it should be brought to everyones attention as it
> is such a simple implementation and just one more reason to be suspicious
> of the use of yp.

        Sigh...  I was afraid of this...

        Keep trying!

        My contacts at Sun were concerned that I was revealing too much
information in this advisory (the complete text of which is attached below).
My concern was that I was revealing too little and would fail to convey
the magnitude of the problem or run into people who would be unable to
reproduce the problem and discount it.  That appears to be the case here
so I will provide a few more hints and guidance to help reproduce this.
(Gee I feel like that faint voice in the game "Adventure" that asks you
if you would like a hint. :-) )

        Just as a side note...  This was not the result of an academic
exercise.  This was the result of the analysis of several independent
instances of serious network disruption on large, independent, networks.
The aftermath left numerous systems disabled and hundreds of finger
processes in evidence on those systems which were left viable enough
to respond to interactive input.

        Obviously, I can not go into any more detail about the networks
originally affected by this problem.

        To reproduce this problem, you are going to have to do some tricky
juggling.

        1) Make sure you have a thousand or more account names in your
NIS password map (this is very IMPORTANT)  Obviously this affects only
large networks in practice.  You need to add that many fake names if you
are attempting to reproduce this in a test environment on a small network.

        2) Start a few process, for each address to attack, to attack each
of a large number of hosts.

                - In each process -

        3) Open up a finger connection and feed down the request.

        4) If the request returns data within a certain timeout, sleep
for a period of time before launching the next request.

        5) If the request does not return data within the timeout, close
the connection and start another connection immediately (this is the first
indication that you are suceeding in congesting the network).

        6) Cycle through this process, steps 3-5, for the specified number
of names in your attack list or for the attack period of time.

        Now...  This is something like "tickling the dragon's tail", an
old, very dangerous, no longer practiced, experiment in nuclear physics
where one would determine the critical mass of a radioactive element by
piling it up and shaping the pile until it went critical (several scientists
died doing this).  Everything behaves resonably quiet until you hit that
right combination and then it goes through the roof.  In what I refer
to as "tuning for maximum smoke", you play with the number of processes
attacking each host, the number of hosts, the timeout before abandoning
the connection, and (important) the sleep time between successful requests.
When you find that right combination, the results are pretty impressive.
It's also likely to piss off anyone you failed to warn when starting a run.
It may even piss off the ones you warned.  I heard a few "WHAT DID YOU DO?"
and "YOU DIDN'T TELL US IT WOULD DO THIS!" even from people I warned.
After a few runs it became standard for people to stop working for a while
("ducking behind barracades" was the term used more than once) when I
announced that I was testing an nis-nuke run.

        The reason I am not providing these numbers is that it's not trivial
to determine them if you do not have a large enough network to test on.  I
would prefer not to release that much information at this time (I suspect
I would be scraping my buddies at Sun off of the ceiling if I did :-) ).
That is also one reason why I am not releasing the exploit script...  Sorry...

        A typical test run would take less than one minute against 10
attacked hosts.  The network wide disruptions would last anywhere from a
half an hour to over an hour and some systems had not recovered for
over two hours.  Several systems, AIX and IRIX boxes in particular,
never seemed to recover and required hard power cycles to recover.  The
Linux boxes seemed to recover fastest while the Solaris boxes seemed to
be able to clean themselves up eventually.  Until each and every attacked
system had either recovered or was rebooted, the network disruption, due
to the ongoing NIS traffic, would continue to interfere with the entire
network and bog down the NIS servers.

        I would also like to remark about one thing.  Solar Designer
quoted one possible action from the advisory.  That one point was a
suggestion made by my Sun contacts.  It was NOT our recommendation as
the action to be taken.  My PERSONAL recommendation is to disable finger
if at all possible.  It provides way too much information about accounts and
logins on your system and only provides information to potential attackers
that you really don't need them to have.  If you DON'T NEED IT - DON'T
USE IT!  Barring that extreme action, limit finger to the minimum number
of systems on which it is required AND limit external access to finger
service to only well control service points (i.e. a few very limited systems).
For some people who want to publish plans and other dynamic information
(Linus Torvald's varying ascii-gram signature was cute), this may still be
too restrictive.  The suggestions from Sun included the script suggestion
quoted and/or using a public domain version of finger which does not support
the "wild card" matching feature.

        Someone else mentioned an NIS caching facility in another message.
That sounded to me like yet another possibility for dealing with this nearer
to the root cause and could avoid this problem if yet another service is
uncovered which is capable of committing this kind of mayhem...

        Considering that this is proven capable of "taking out" a
network for hours or even days beyond the end of the attack itself,
I would consider it to be a little more serious than a "ping flood"
or even a "Smurf" attack.  :-)

        In a couple of ways, this is related to the same class of attacks
that the "Smurf" attack belongs to.  They both involve an issue of
"resource demand amplification".  In other words, a limited resource demand
(packets and bandwidth) going into a network, from the attacker, results
in an amplification of that demand within the network.  "Smurf" does it
by spoofing the return address of ICMP packets to be the network broadcast
address.  Nisnuke does it by generating prodigious levels of NIS traffic in
response to relatively limited levels of finger requests.

> -------------------------------------------------------------------------------
> Raymond R Medeiros II                   email: medeiros () eng usf edu
> Junior Systems Administrator            www: http://www.eng.usf.edu/~medeiros
> Engineering Computing
> University of South Florida

> On Fri, 10 Jul 1998, Solar Designer wrote:

> > Hello,
> >
> > > # mv /usr/bin/finger /usr/bin/finger.exe
> > > # cat > /usr/bin/finger
> > > #!/bin/sh
> > > exec /usr/bin/finger.exe -m $*
> > > ^D
> > > # chmod +x /usr/bin/finger

> > Hmm, weird, this doesn't look safe to me. Why trust the extra parsing done
> > by the shell? Look at this:

> > sunny:~$ finger "a -b"
> > finger: a -b: no such user.

> > sunny:~$ finger a -b
> > finger: illegal option -- b
> > usage: finger [-lmps] [login ...]

        Excellent points.  More the reason to either just limit access to
finger or to use one of the public domain implimentations that doesn't
support the wild carding.  The script listed in the advisory was the last
of several suggestions for use only if nothing else was practical...

> > Now, many implementations of fingerd just run finger on data received from
> > the remote, doing some sanity checks first, and splitting the arguments
> > for execv(). These checks often include denying passing of some or all
> > options to finger. If fingerd knows about less word separators than the
> > shell does, then an attacker might be able to pass a forbidden option to
> > finger. For example, if our fingerd didn't know about tabs (which isn't a
> > security hole yet: our fingerd uses execv(), remember?), a remote attacker
> > could send us "user\t-option".

> > I admit that the problem isn't serious: not all fingerd's are done this
> > way, forbidden finger options are likely to violate someone's privacy
> > only, etc. Still, it's not a good idea to trust the shell, in general.

> > Signed,
> > Solar Designer

        Mike
--
 Michael H. Warfield    |  (770) 985-6132   |  mhw () WittsEnd com
  (The Mad Wizard)      |  (770) 925-8248   |  http://www.wittsend.com/mhw/
  NIC whois:  MHW9      |  An optimist believes we live in the best of all
 PGP Key: 0xDF1DD471    |  possible worlds.  A pessimist is sure of it!

===== Begin Attachment ===== nisnuke.asc =====

-----BEGIN PGP SIGNED MESSAGE-----

                     ISS Security Alert Advisory
                            June 29, 1998


        Distributed DoS attack against NIS/NIS+ based networks.

        For purposes of this report, NIS refers to both NIS and NIS+
as this problem has been observed and reproduced on both services.


Synopsis:

        It is possible, through a well orchestrated attack using the finger
service against multiple NIS clients, to disrupt an entire NIS based
network and/or starve the NIS servers for resources.  The problem is
in the finger service but the attack causes long duration, network-wide,
congestion and resource exhaustion on NIS servers.


Recommended action:

        Disable finger service on any systems connected to any NIS based
network.  Disable access to internal finger services at all perimeter
defenses (firewalls, gateways, filtering routers, etc.).  If the finger
service is required for some specific purpose, limit it to the minimum
number of restricted hosts or to hosts which are NOT participating
in NIS.

        For those who wish to continue to run the finger service on their
systems, there are other possible actions that could be taken, like using
a public domain fingerd that doesn't do ambiguous lookups.   The finger
service can also be protected by even something as simple as wrapping
finger to not do ambiguous lookups like as follows:

# mv /usr/bin/finger /usr/bin/finger.exe
# cat > /usr/bin/finger
#!/bin/sh
exec /usr/bin/finger.exe -m $*
^D
# chmod +x /usr/bin/finger

        These recommendations would permit the continued safe use of finger
(or as safe as finger ever gets).


Description:

        A finger request results in multiple NIS requests as the responding
daemon attempts to locate all account records matching the finger request.
A request for finger foo () bar com will result in one finger daemon searching
incrementally through all of the entries in the passwd map to locate any
accounts with foo in the name.  As a consequence, a single finger request
can result in a significantly larger amount of traffic between the NIS
client and the NIS server than the originating traffic to and from the
finger service.  The amount of NIS traffic is dependent on the size of
the NIS passwd map.  With a passwd map of 10,000 entries, a single finger
request would result in roughly 10,000 NIS requests and 10,000 NIS responses.
This does NOT count retries from packet loss or other failures (a highly
significant factor in this attack).

        By sending a large number of overlapping finger requests to a single
host, it is possible to load that host down with a very significant amount of
traffic just processing the NIS requests.  If this is done to multiple hosts,
the network traffic rises dramatically.

        Eventually, a condition arises in which congestion and/or resource
exhaustion on the NIS server begin to cause a significant rise in lost
packets and failed requests.  This results in retry attempts from the
NIS clients, adding to the already overloaded network traffic.  The
failure / retry / failure cycle becomes an NIS traffic "storm" in which the
retry traffic dominates and little other traffic can squeeze through.

        Network congestion combined with NIS server resource exhaustion
work together to not only deny service to the requesting clients but also
to rapidly clog the network bandwidth and render the network unusable by
anything on the network.


Analysis and details:

        In analyzing this attack, a perl script was used to generate finger
traffic attacking a dozen hosts with four finger requests for each of
approximately 100 names (~400 finger requests per host).  A demonstration
NIS map of approximately 1000 accounts was used.  At an issue rate of
approximately 4 finger requests every two seconds against a given host,
10's to 100's of lingering finger requests would build up even as some
finger requests would be fufilled.  These lingering finger processes
would be attempting to paw their way through the entire NIS password map.
A typical test run attack lasted approximately 30-50 seconds in duration.

        During analysis of this attack, network traffic from even a short
~30 seconds blast from the perl test script resulted in traffic levels that
caused network disruptions extending for as much as 45 minutes to an hour
after cessation of the attack.  During this time, some systems were impacted
to the extent that screen savers froze and systems were unresponsive to the
keyboards.  Many systems were left with seemingly hung finger processes.
These stayed on the system for a half an hour or more while the network
congestion cleared.  Some systems ran out of swap space because of the
resource demands of the finger processes.  On a few of the test runs the
network traffic was observed to have risen to a level which caused a
switched ethernet hub to disable ports due to excesive collisions.

        Finger requests to perform this action have to be distributed and
timed properly.  Too many requests, too quickly, seem to result in inetd
disabling the finger service.  Too slowly, and the network traffic rises too
slowly and fails to reach the catastrophic level where packet loss and retries
become the dominant traffic input to the network.

        Because the finger requests are TCP based and not dependent on
preauthentication, finger requests can still be delivered by the attacker
to the systems under attack even in the face of increasing network congestion.
By the time the attacking connections are significantly impacted by the
network congestion, the network has been rendered unusable by systems
requiring NIS or other services.

        Timed correctly, an attack of only a few seconds, targeting as few
as a dozen NIS clients on a network with a moderate NIS passwd map can render
even a small network unusable for as long as a half an hour to an hour or
more.  Increasing the size of the NIS passwd map, the number of attacked
clients, or the number of requests sent to any given client causes the
recovery time to extend out dramatically and disproportionately to any
particular increases in any particular factor.

        If the NIS server is also one of the attacked systems, it can
rapidly run out of system resources, causing NIS request failures and
accelerating the resulting NIS traffic "storm".  When the NIS server
was one of the systems attacked by finger requests, it was not unusual
to see warnings about unable to grow stack, exhausted virtual memory, or
other resource related errors.

        MOST client systems seem to clean themselves up EVENTUALLY.  This
can take anywhere from a few moments for some Linux boxes, to a significant
fraction of an hour for some SUN boxes.  It was observed that some IRIX
boxes and AIX boxes would become unreachable from the network and
unresponsive to the keyboard, requiring a power cycle to recover.
These last systems may have recovered on their own eventually, but that
time frame appears to be geological.  Recovery time seems to also be
dependent on the recovery time of the NIS server for those clients which
were observed to recover.  Resetting the targeted systems permits the
network to recover.  All tested systems were affected to some extent.

        Because the resulting traffic and congestion is proportional to the
size of the NIS passwd map times the number of attacked hosts times the
number of requests in the attack, large networks are disproportionately
vulnerable to this attack.  Even small networks of a few dozen systems
can be disabled by a determined attacker if they have a sufficiently
large NIS passwd map.


Conclusion:

        The finger service permits a condition where a limited number
of requests can result in a vastly larger number of internal requests
against a central naming service such as NIS.  This permits an attacker
to mount a distributed attack by launching smaller attacks against numerous
hosts.  These combine to form a disasterous level of congestion on the
internal systems, disrupting an internal network for an extended period
of time.


Afterword:

        It is unknown, at this time, if any other services exhibit similar
characteristics with regard to NIS traffic as does finger.  Disabling finger
prevents it from being exploited against a network.  It obviously does
not guarantee that some other service might be similarly exploitable.


Credits:

        We would like to extend our appreciation to Sun Microsystems, Inc.
for their assistance and consultation with regard to the vulnerability.


Michael H. Warfield
Senior Researcher
ISS X-Force
Internet Security Systems, Inc.

________

Copyright (c) 1998 by Internet Security Systems, Inc.

Permission is hereby granted for the redistribution of this alert
electronically.  It is not to be edited in any way without express
consent of X-Force.  If you wish to reprint the whole or any part
of this alert in any other medium excluding electronic medium, please
email xforce () iss net for permission.

Disclaimer

The information within this paper may change without notice. Use of this
information constitutes acceptance for use in an AS IS condition. There
are NO warranties with regard to this information. In no event shall the
author be liable for any damages whatsoever arising out of or in connection
with the use or spread of this information. Any use of this information is
at the user's own risk.

X-Force PGP Key available at:   http://www.iss.net/xforce/sensitive.html
as well as on MIT's PGP key server and PGP.com's key server.

X-Force Vulnerability and Threat Database: http://www.iss.net/xforce

Please send suggestions, updates, and comments to:
X-Force <xforce () iss net> of Internet Security Systems, Inc.

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBNZfaJjRfJiV99eG9AQEFDAP/Y5k+8q5lmasrZotzcVw5z43LBUy9AL6l
By8onp07PmIdkQD7vCyiSIHqEvWiGigvhl76gdnkZxUowSYjdI+PfZBbB9S1HoZB
eMRNjd0m5X7aEimIH208TwdegxwLu+A7/FKnl3EImd182Nzma1PhAqXpclPEmJ38
mUbk7Lm6Yls=
=HXJi
-----END PGP SIGNATURE-----