Secure Coding mailing list archives

Security in QA is more than exploits


From: bugtraq at cgisecurity.net (bugtraq at cgisecurity.net)
Date: Wed, 4 Feb 2009 21:17:39 -0500 (EST)

For starters I believe you misinterpreted my comments on QA. I was in no way slamming
their abilities. With this in mind comments below. 

Before anyone talks about vulnerabilities to test for, we have to figure ou=
t what the business cares about and why. What could go wrong? Who cares? Wh=
at would the impact be? Answers to those questions drive our testing strate=
gy, and ultimately our test plans and test cases.

We absolutely agree here. At the same time an externally exploitable sql injection needs to get 
fixed. The way qa/development is informed to its impact is through education likely via training. 
Not a single company with average hiring/skill requirements will have everybody (who needs to) know 
what sql injection is, and why it is bad. 

Bias #3 is that idea that a bunch of web vulnerabilities are equivalent in =
impact to the business. That is, you just toss as many as you can into your=
 test plan and test for as much as you can. This isn't how testing is prior=
itized.

I said "A better approach in my opinion is to identify the top 10/25/x attacks/weaknesses/vulnerabilities 
that are likely to affect your own organization"

These would be associated to customer or business impacts likely to affect you. 
Perhaps this could have been articulated better.
 
As someone who has been training in risk-based security testing for several=
 years now, I totally agree with some points, but very much disagree with o=
thers. I agree that the "bug parade" (as we call it) of top X vulnerabiliti=
es to find is the wrong way to teach security testing. Risk management, tho=
ugh, has been a fundamental part of mainstream QA for a very long time. Lik=
ewise, risk management is the same technique that good "security people" us=
e to prioritize their results. Risk management is certainly how the busines=
s is going to make decisions about which issues to remediate and when. Risk=
 management is what ties this all together.

We agree.

If there's something that QA needs to learn that they're not already learni=
ng, it's the weaving of "security" into the risk management techniques they=
 already know how to do. If testers fall short in their ability to apply ri=

In your experience do you find average QA people doing risk management?

In my experiences a Sr QA person/Team lead identifies what is going to be tested
for a given release, and usually are the ones writing/tracking the test plans. 

So, in some ways we agree: speak the lingo of QA. But in other ways we disa=
gree. I think the original article fails to give credit to the decades of s=
ubstantial research and practice in QA. In other words, it's a lot more tha=
n speaking the language. It is standing on the shoulders of giants, not the=
ir toes.

Actually the main goal of the article is that information security people need to set appropriate expectations 
as to what QA cares about as their primary business function. They need to factor in that the majority of QA 
people don't care about security as a primary job function, and that if infosec wants them to care they had 
better be prepared 

- to speak their language and understand their needs
- to customize and prioritize the security testing they may be doing instead of solely using generic top x lists 

Paco

Have a fantastic day Paco!

Regards,
- Robert



Current thread: