Re: OSCP

["Andre Gironda" <andreg () gmail com>]

On Fri, Dec 12, 2008 at 4:32 AM,  <christopher.riley () r-it at> wrote:
> > If you actually do real security, OSCP is a bullshit cert just like
> > CEH, CNOP, SCNA, GSE, et al.  Anything SANS or ISC2 is crap.  All of
> > these certs mean absolutely nothing if you dig deep into the actual
> > meaning behind them.
>
> Just out of interest, have you actually done any of the training that you
> think is "crap". Having been through some of the training you mention
> (some of it good, some of it bad) I would disagree with your broad
> assumption. Have you ever actually met somebody with a GSE for example.
> It's not an easy qualification to achieve. The courses you mentioned are
> designed to educate in a specific area and not be a wide ranging (but thin
> on information) course like the CISSP (and other) training appears to be.

Individuals are individuals.  Some are better than others, but nobody
is cookie cutter in our industry - let alone most other industries.

Speaking to the material itself, I have read and maintain current
copies of all of this training material and I think I'm fairly
familiar with most of the people who write the training material.
CNOP is only a certification criteria, but they set out a pretty
specific goal and timeline.  Even GSE is fairly well known, and the
training is just a collection of the total sum of SANS training.

> The question I have is what do you class as "real security". I work as a
> penetration tester and can say from my side that CEH is not a good course
> (in regards to preparing you for working in the ethical hacking field),
> however the SANS classes (SEC:560, SEC:504, SEC:709, SEC:542, etc...) do
> provide a great deal of information. No training course or certificate is
> going to guarantee a person is fit for the job, but dismissing them all as
> "crap" is in my opinion unfounded.

I suggest that you read the full OSSTMM 3.0 for "real" aka
"operational security".  Also worth checking out would be NIST
SP800-30, NSA IAM/IEM/RTM, DOD DIACAP, and Andrew Jaquith's
SecurityMetrics book/blog/mailing-list.  There have been interesting
threads on the scadasec mailing-list lately as well.

I have read/viewed/listened-to SANS 502, 503, 504, 505, 508, 517, and
617 training material and know some that have attended those classes.
There are descriptions and outlines (more detailed than what is
available from SANS) for 560 on some wikis and blogs in various
places.

I am positive that 542 is a joke/crap because I am a regular web
application blogger and guru.

Here is a good summary of 709:
http://c22blog.wordpress.com/2008/12/10/sans-sec709-developing-exploits-for-penetration-testers-day-2/
The outline for the 4-day course is up
http://www.sans.org/sans2009/description.php?tid=2717

I think all of these courses are interesting to some degree, but how
do they help with operational security?  What does the certification
say about the person?  Why not just list the training classes you have
attended on your resume, instead of purporting to be capable of
"securing" something instead of being only knowledgeable about
"demonstrating use of the knowledge demonstrated in 1-N class(es) or
1-N exam(s)"?

Finally, what is the correlation between offensive security
skill/research and operational security?  Just because a company gets
hit with one zero-day that owns a partial part of the infrastructure
doesn't indicate that anything serious (i.e. PII or confidential data)
has been breached.  It doesn't talk about access controls,
auditability, incident response, et al.

The OSSTMM 3.0 lists only 10 controls (5 interactive and 5 process)
for a security posture, but the RAV and STAR calculations are worth a
serious look.  The NSA IEM has IPP (rating system) and DOD DIACAP has
a scorecard.  These say something about the security of something
(network/infrastructure, or even a particular system in the case of
DIACAP).

Over the many years I've seen people talk about certifications -- most
admit that they really like CCIE because you have to demonstrate
something and "It's not an easy qualification to achieve" (like some
say about GSE).  While there are even fewer GSEs than CCIEs (even when
CCIE was as mature -- as many years young -- as GSE), I always found
CCIEs to be completely clueless.  It was common unspoken practice that
Cisco-employee CCIE examiners/trainers would have to be certified
(Train the Trainer, or TTT).  This material leaked inside Cisco (as
well as outside the organization) and you had tons of paper-CCIEs.

How does leaked information lead to success with a lab based exam, you
ask?  Because a lab is a sample set of processes that are repeated
over and over.  It's common practice today to download a pass4sure or
actualtests PDF guide that contains all of the possible answers for
any IT exam (including CISSP, CISA, and many many others).
Statistically, if you know the exam's potential maximum amount of
questions and have all of the potential answers, you can find a happy
medium of how many you have to memorize at minimum in order to meet
the pass rate for that exam.  This works equally in a lab environment
as it does on a computer-based test package.

I have found that the best way to fix this error is not only to have
multiple exams, but to also have very long exams, with an extremely
large amount of potential questions that are chosen at random.

However, at this point, a prospecting certification organization
should simply open up their questions AND answers.  i.e. "In order to
certify for the OPCP, you must pass 10 exams, each consisting of 2000
in-exam questions, taken from a total of 12,000 potential
question-answer combinations".  This prevents actualtests/pass4sure,
and it also evens the playing field.  Another problem is the design of
the questions and answers.  You'd have to look at the structure of a
medical or law degree program to come to something close to what we'd
need for our field (i.e. "operational security").

This is actually the intent of the OWASP OPCP project.  It is not a
certification, nor does it provide any classes or material for
training.  It is simply an open set of questions and answers that can
be used.

If ISC2 would like to steal these ideas (which aren't even really
mine), that's fine.  They already started to snake in the appsec
industry with CSSLP_WhitePaper_3.pdf (hrmn... is that a leaked file?).
 Will SANS move to this sort of model?  I don't know the answers to
these questions.  However, there comes a time when this madness must
cease to continue and we should all work together to stop the hamster
wheel of pain.

Cheers,
Andre

------------------------------------------------------------------------
This list is sponsored by: Cenzic

Security Trends Report from Cenzic
Stay Ahead of the Hacker Curve!
Get the latest Q2 2008 Trends Report now

www.cenzic.com/landing/trends-report
------------------------------------------------------------------------