Bugtraq mailing list archives
Re: Paper announcement: Is finding security holes a good idea?
From: Benjamin Franz <snowhare () nihongo org>
Date: Wed, 21 Jan 2004 18:26:11 -0800 (PST)
On Wed, 21 Jan 2004, Eric Rescorla wrote:
Bugtraq readers might be interested in this paper: Is finding security holes a good idea? Eric Rescorla RTFM, Inc. <http://www.rtfm.com/> A large amount of effort is expended every year on finding and patching security holes. The underlying rationale for this activity is that it increases welfare by decreasing the number of bugs available for discovery and exploitation by bad guys, thus reducing the total cost of intrusions. Given the amount of effort expended, we would expect to see noticeable results in terms of improved software quality. However, our investigation does not support a substantial quality improvement--the data does not allow us to exclude the possibility that the rate of bug finding in any given piece of software is constant over long periods of time. If there is little or no quality improvement, then we have no reason to believe that that the disclosure of bugs reduces the overall cost of intrusions.
It is a very weakly justified assumption that the number of Black Hat exploitations following Black Hat Disclosure is 'almost certainly less' than the 'peak rate after (Public) disclosure'. The rest of your paper rests heavily on that assumption - and it in fact nearly pre-determines your conclusion. It only takes _ONE_ more Warhol Worm from non-public security bugs than publically disclosed ones to invalidate your argument (this is obviously true in the _converse_ as well). This is because the _COST_ of such a worm is magnified beyond reason by the network. Each individual worm represents a large fraction of _all_ such intrusions and their cost. This is NOT a large statistical universe. Individual events skew the whole dataset. One major exploit may cost as much as thousands or even millions of smaller exploits. A bug allowing complete remote compromise of a box that is merely _connected_ to the net is of a completely different magnitude than one that causes Mozilla to mis-display a URL. Treating the two bugs as comparable is a significant error. It is not justified to believe that they are _generated_, _discovered_, _exploited_, or _fixed_ at the same rates or that they have even order of magnitude similiar costs. There remain too many large unknowns to justify _any_ good conclusions re bug hunting and disclosure vs the cost of intrusions. -- Benjamin Franz On that of which one cannot speak, one must remain silent. ---Wittgenstein
Current thread:
- Paper announcement: Is finding security holes a good idea? Eric Rescorla (Jan 21)
- Re: Paper announcement: Is finding security holes a good idea? Oliver Friedrichs (Jan 22)
- Re: Paper announcement: Is finding security holes a good idea? Benjamin Franz (Jan 22)
- Re: Paper announcement: Is finding security holes a good idea? Kurt Seifried (Jan 22)
- Re: Paper announcement: Is finding security holes a good idea? Robert Lemos (Jan 22)
- Re: Paper announcement: Is finding security holes a good idea? Christopher E. Cramer (Jan 22)
- <Possible follow-ups>
- RE: Paper announcement: Is finding security holes a good idea? Daniel Whelan (Jan 22)