top 10 software security surprises

[ken at krvw.com (Kenneth Van Wyk)]

On Dec 16, 2008, at 1:25 PM, Gary McGraw wrote:
> Using the software security framework introduced in October (A  
> Software Security Framework: Working Towards a Realistic Maturity  
> Model <http://www.informit.com/articles/article.aspx?p=1271382>), we  
> interviewed nine executives running top software security programs  
> in order to gather real data from real programs.

Wow, this is great stuff.  Kudos to Gary, Sammy, and Brian.

I have a couple comments/observations on some of your conclusions:

- You obviously wrote the top-10 list in C, since it went from 9 to  
0.  :-)

- "Not only are there are no magic software security metrics, bad  
metrics actually hurt."  This is an excellent point.  I think it's  
also worth noting that it's important to carefully consider what  
metrics make sense for an organization _as early as possible_ in the  
life of their software security efforts.  Trying to retro-engineer  
some metrics into a program after the fact is not a fun thing.

- "Secure-by-default frameworks can be very helpful, especially if  
they are presented as middleware classes (but watch out for an over  
focus on security "stuff"). "  Yes yes yes!  I've found significantly  
more "traction" to prescriptive guidance vs. a "don't do this" list of  
bad practices.  Plus, it inherently supports a mindset of positive  
validation instead of negative.  It's important to look for common  
mistakes, but if you really want your devs to follow, give them clear  
coding guidelines with annotated descriptions of how to follow them.   
Efforts like OWASP's ESAPI are indeed a great starting point here for  
plugging in things like strong positive input validation and such.

- "Web application firewalls are not in wide use, especially not as  
Web application firewalls. "  I can't say I'm much surprised by this  
one.  Even with PCI-DSS driving people to WAFs (or do external  
independent code reviews), I just don't often see them often.  But you  
go on to say, "But even these two didn't use them to block application  
attacks; they used them to monitor Web applications and gather data  
about attacks."--but you don't come back to this point.  One serious  
benefit to WAFs can be enhancing the ability to do monitoring,  
especially of legacy apps.  Adding one network choke point WAF can  
quickly add an app-level monitoring capability that few organizations  
considered when rolling the apps out in the first place.

- "Though software security often seems to fit an audit role rather  
naturally, many successful programs evangelize (and provide software  
security resources) rather than audit even in regulated industries"   
This one too is very encouraging to see.

- "Architecture analysis is just as hard as we thought, and maybe  
harder." And this one is very discouraging.  I've seen good results in  
doing architectural risk analyses, but the ones that produce useful  
results tend to be the more ad hoc ones -- and NOT the ones that  
follow rigorous processes.

- "All nine programs we talked to have in-house training curricula,  
and training is considered the most important software security  
practice in the two most mature software security initiatives we  
interviewed. "  That explains the quarter-million miles in my United  
account this year alone.  :-) Ugh.

- "Though all of the organizations we talked to do some kind of  
penetration testing, the role of penetration testing in all nine  
practices is diminishing over time. "  Hallelujah!


Cheers,

Ken

-----
Kenneth R. van Wyk
SC-L Moderator
KRvW Associates, LLC
http://www.KRvW.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://krvw.com/pipermail/sc-l/attachments/20081217/d45996ab/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2252 bytes
Desc: not available
Url : http://krvw.com/pipermail/sc-l/attachments/20081217/d45996ab/attachment.bin