PaulDotCom mailing list archives
Re: Test Effort Estimation
From: Josh More <jmore () starmind org>
Date: Mon, 29 Oct 2012 12:02:11 -0500
This same issue has been cycling around in the Agile community for years. I do believe that anything can be measured... but I'm not sure that everything can be predicted. There are a lot of variables in the mix: skill of tester, motivation of tester, time available, tools available, position on the attack/defense cycle, rules of engagement, etc. As a result, after arguing back and forth with people for the last five years or so, I have completely abandoned the idea of metrics with a goal of full coverage. Instead, I frame the discussion around the value of the data and the value of continued service and compare that to the cost of an assessment. We figure out how much they want to spend and I convert that into hours. They then get that much analysis. Not every client goes for it, but really, since most engagements I've done turn up more findings than any organization can truly address before it's time for the next assessment, narrowing the coverage makes a lot more sense (assuming that known exploration gaps are documented so they may be hit in the next cycle). -Josh More On Sun, Oct 28, 2012 at 6:45 PM, Ryan Dewhurst <ryandewhurst () gmail com> wrote:
Hi, I was wondering how to make Test Effort Estimation more efficient on my black box web app tests. I think this is easier to do when doing white box tests because you have a good metric, Lines of Coce (LOC), but in black box testing a metric might be less easy to find. What I normally do and I expect most other people do is give an estimation based on past experiences, but in my opinion this can be time consuming and sometimes inaccurate. Time consuming because you have to manually view each application to be tested to mentally compare it. Inaccurate because I'm human and on that particular day I might be feeling *really* motivated and under-estimate the amount of time (effort) needed or vise-versa. This I feel can lead to inaccuracy and wasted time. Another approach is to try and find a metric to use, that metric could then be quantified into man hours. A reasonable metric (by far not perfect) I can think of when doing a typical black box web app test (using automated tools and manual interaction) is the amount of unique dynamic pages the application has. This can normally and quite easily be obtained. Let's say it takes 1 man hour to test 10 pages. (plucking these numbers out of the air) If an app has 100 unique pages, the Test Effort Estimation would be 10 man hours. So my questions are: Do you think there are better metrics to use other than number of unique pages? Do you think there are better ways to do Test Effort Estimation on black box web application tests? How many man hours do you think it should typically take to test 1 unique page? I think it is an interesting topic which hasn't been discussed much as far as I could tell. Ryan _______________________________________________ Pauldotcom mailing list Pauldotcom () mail pauldotcom com http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom Main Web Site: http://pauldotcom.com
_______________________________________________ Pauldotcom mailing list Pauldotcom () mail pauldotcom com http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom Main Web Site: http://pauldotcom.com
Current thread:
- Test Effort Estimation Ryan Dewhurst (Oct 28)
- Re: Test Effort Estimation gold flake (Oct 29)
- Re: Test Effort Estimation gold flake (Oct 29)
- Re: Test Effort Estimation Josh More (Oct 29)
- Re: Test Effort Estimation Ryan Dewhurst (Nov 16)
- Re: Test Effort Estimation gold flake (Oct 29)