Re: Chasing bugs / vulnerabilties

[Crispin Cowan <crispin () WIREX COM>]

Theo de Raadt wrote:

> > Try the UNIX Fuzz experiment, first conducted at the University of Wisconsin
> > on multiple UNIX operating systems and when tried again several years later
> ...
> > http://www.cerias.purdue.edu/coast/ms_penetration_testing/v11.html) tried
> > the same experiment on WinNT with 'interesting' results.
> After Michael mentioned this, I decided to download this and see if it
> found any bugs in OpenBSD.  I was pretty sure I'd find at least a few.
> I hacked it up a bit, and then ran it against every single binary in a
> "default install" openbsd system.
>
> [long list of bugs found]
>
> However, as I said, it helps us understand the mistakes people make
> when they write programs.  It helps us understand that many of those
> mistakes are pretty basic C errors.  It helps prove the case for
> getting a second or third eye to look at the source code.  Too often,
> I look at code from 1985 and discover bugs which lead me to believe I
> am the second, third, or perhaps fourth eye.

It helps illustrate the limitations of auditing:

   * you can't audit everything; there's too much of it
   * even that which you do audit may contain vulnerabilities
   * trying to limit auditing to "security critical" software doesn't work, because
     you can't tell what root will choose to run

Make no mistake:  I *fully* support source code uditing as a primary approach to
securing systems.  However, auditing is no more a complete solution than tools is a
complete solution.


> I still consider fuzz to be somewhat of a crutch.  For about half of
> these fixes, inspection found other things we could improve.  Now that
> we've run this, we will probably discard the tool

That sounds like a crucial mistake.  The tool just demonstrated its value by revealing
a large number of bugs.  Why would you now choose to discard it?  Those same factors
that allowed the bugs to creep in as the software developed will allow them to creep
in *again* as the software is maintained and ported.  The fuzz tool (like other
security tools) will continue to provide value as code is maintaned just as they do in
initial application.

Auditors are human, and they *will* make mistakes.  These "crutches" are more like a
safety net:  you may not fall off the high wire very often, but it's sure nice to have
a net on occasion when you do slip up.


> However, I think that once a person learns to audit code by just
> reading it, that skill is transferable and reusable on a daily basis.

I totally agree.

Crispin

--
Crispin Cowan, Chief Scientist, WireX Communications, Inc. http://wirex.com
Free Hardened Linux Distribution:                          http://immunix.org