Re: Measuring security

[Chris Green <cmgreen () UAB EDU>]

I highly recommend
http://www.amazon.com/Security-Metrics-Replacing-Uncertainty-Doubt/dp/03
21349989 for assistance on what makes a good and bad security metric and
lots of insight into great metrics.   Prioritizing its information is
one of my ongoing goals.

 

One of the few comparisons metrics I've gotten implemented is related to
a subset of nessus and WSUS information.   By department, I keep tabs on
"number of days a system has a patch failing" and number of days that a
groups system has on average remained exploitable per remote
vulnerability scanners.    The unfairness category I've got to normalize
is accounting for the number of assets a single group runs.  Having 9
systems failing a patch doesn't seem so bad if you run 5000 systems but
if you run 10 systems, it's pretty bad.

 

All over different time periods:

 

Applications in Use

Applications Reviewed for Security Issues

Applications with stale users

Notebooks in Use

Notebooks Encrypted

 

I try very hard to downplay the metric of "number of incidents
discovered or reported".  Getting people to be more open about virus
infections and not treating them like break/fix increases your number
but is actually a better awareness metric. 

 

From: The EDUCAUSE Security Constituent Group Listserv
[mailto:SECURITY () LISTSERV EDUCAUSE EDU] On Behalf Of Heather Flanagan
Sent: Wednesday, November 05, 2008 4:06 PM
To: SECURITY () LISTSERV EDUCAUSE EDU
Subject: [SECURITY] Measuring security

 

Hi all -

 

I've been asked to create some measurable target goals for data
security.  This is proving to be a tricky set of metrics to define!
What I've realized so far is:

 

1 - trying to go by how many holes or warnings are found by nessus won't
work; way to many false positives

2 - trying to go by what a third-party penetration test might find won't
work; what they are measuring varies too much and there have so far been
way too many false positives or things we considered completely
acceptable (yes, a domain controller is going to act as a time server to
anyone who checks)

3 - trying to go by "well, doesn't look like we've been hacked
recently"...  not quite the business metric I'm looking for

 

Is anyone out there finding any particular set of metrics working for
you and your campus leadership?  

Heather Flanagan

Director, System Administration

heatherf () stanford edu