Dailydave mailing list archives
Re: Octave
From: George Capehart <capegeo () opengroup org>
Date: Tue, 16 May 2006 16:53:45 -0400
Dave Aitel wrote: <snip> Apologies in advance for pulling the topic away from OCTAVE specifically to problems with risk assessments and the risk management process in general . . .
""" There are many approaches for evaluating information security risk. At the heart of any approach is an assessment, or evaluation. This slide defines two common approaches: a tool-based analysis and workshop-based analysis. The tool-based analysis normally requires someone to input information about the organization?s assets, threats, and infrastructure characteristics into a software-based analysis tool. The tool takes the information and performs a risk analysis, often based on proprietary mathematical algorithms. There are usually no restrictions on who enters the information into the process (often it is a small group of people) or on how they collect the required information. The interaction and number of people required by this type of analysis is small.This approach can be quick (after the initial information is entered into the tool), but it relies on only a few perspectives. The organization is also placing trust in proprietary analysis algorithms that might not be well understood by the organization?s personnel.
Nor is it very likely that the tool even uses the appropriate metrics or is even sensitive to the appropriate dimensions. The first phase of a true risk assessment should be to identify the aspects of the entity that need to be protected, and then understand the threats to those aspects and the vulnerabilities to those threats. Any third-party cookbook tool will cover the "common" cases, but will miss the idiosyncratic cases . . . which are frequently aspects that are strategic differentiators . . . and therefore the ones that need protecting the most.
A workshop-based analysis requires the participation of many people to build an understanding of assets, threats, and characteristics of the infrastructure. A small group of people (an analysis team) leads the process and gathers information using interviews or workshops. The analysis team reviews and analyzes the information that has been gathered and creates mitigation plans. Decision-support tools can be used to assist the analysis team, but the analysis team is responsible for making all decisions. This approach involves many staff members in the organization and can be time intensive. However, the people in the organization make the decisions and understand why the decisions have been made. OCTAVE is a workshop-based approach. """ We did a number of these at @stake, and I personally didn't find them to be of value. Workshops have a number of built in problems: o People lie to you. Often, people won't know the answers at all, but will still pretend to to look good. In many cases you will get conflicting information simply because people don't really know what they're talking about. You can spend forever tracking down the truth here. What this means is that at the end of the process you don't have hard evidence and you don't know how reliable your results are. o Workshops are hugely expensive for what they produce. You're trying to get a meeting with the CSO, CISO, CEO, various levels of management, and the actual technical staff. This involves a huge amount of effort even for a small organization, and is typically going to be not worth it. The loss of productivity is mind boggling when you add it up. o Workshops draw weak conclusions. I'm not sure why this is, but my experience with them tells me that overall, we didn't end up telling people anything they didn't know. A good process will, sometimes at least, produce results that surprise you. Workshops never will. Perhaps consensus based brainstorming is not a replacement for leadership or individual knowledge.
In other words, workshops rarely involve the individuals in the organization whose job it is to manage risk. And it's been my experience that outside the financial services industry, there are few organizations which have a formal risk management process, and even in financial services, the formal risk management process rarely includes information security risk.
So to sum up: I feel that OCTAVE and things like it are a huge waste of time. This might not be the answer you were hoping for, but it's my opinion based on having done things like it and having read the materials presented on the website.
Much like the Certification and Accreditation Process. The idea is great: theoretically, it forces management to understand the risks and formally (in writing) sign off the controls being implemented and accept the residual risk. In practice it's turning out to be a waste of time and money because it's frequently implemented by people who don't understand the risk management process, but who are very good at creating punchlists . . .
Current thread:
- Octave dan (May 16)
- Re: Octave Dave Aitel (May 16)
- Re: Octave George Capehart (May 16)
- Re: Octave m3c (May 17)
- Re: Octave George Capehart (May 16)
- RE: Octave Edward Ray (May 16)
- Re: Octave Dave Aitel (May 16)