Re: On product vulnerability history and vulnerability complexity

[Gadi Evron <ge () linuxbox org>]

Crispin Cowan wrote:
> Kind of: absence of evidence is not evidence of absence, but that
> applies both ways. The absence of a vulnerability history does not
> indicate that the product is secure or insecure, it indicates that no
> one has looked, or at least no one has reported looking.

Like you stated earlier, our history and statistics gathering in the 
industry are very lacking and self-biased at best. Still, this really is 
the case of "the truth is probably somewhere in the middle".

Looking at how many past vulnerabilities were found, and of what types, 
does tell you something. Looking at what the code looks like tells you a 
bunch. As an example, if a product has 5 basic buffer overflows every 
year, obviously something is wrong there. If the vulnerabilities are 
obscure at best rather than basic buffer overflows - we can at least 
tell it wasn't the coder being bad but rather something that can happen 
to anyone.

Looking even at web applications and their history one can easily tell if:
1. They are professionally written.
2. The vulnerabilities seen before and the ones we could find are not 
trivial or really say anything about the coder.

That's how we chose WordPress for blogging.

I don't see why closed-source software should be any different.

I recently had a chat with a friend and we talked exactly on this issue:

Although there is some un-touched code-base around (Excel being a recent 
example)...
Looking at Microsoft's software of today, it is extremely well-written 
and professional. Far beyond that of most others. Finding 
vulnerabilities in them is extremely difficult. Most vulnerabilities you 
will find will be logical in nature and not easy.

That does not come to speak to my (bad to worse) opinion of their 
disclosure handling process, etc., but rather to show that they indeed 
seriously changed in that regard.

> Consider Postfix and Qmail again; neither has any substantive history of
> vulnerabilities, but both have a substantive history of fussing over
> whether some arcane anomaly is a vulnerability or not. This indicates
> that people were looking really hard at them. This is a very good thing.
> We need some way to capture that.

That is key, as today's data is very lacking to base much on. But we use 
what we have, right?

        Gadi.