Re: climate gate and programming bugs

[Dan Kaminsky <dan () doxpara com>]

On Wed, Dec 9, 2009 at 12:25 AM, Robert Graham <
robert_david_graham () yahoo com> wrote:

> > From: Dan Kaminsky <dan () doxpara com>
> > Took a look. There are mild issues
> > but nothing I'm seeing yet that
> > causes clear error. Maybe the <1% error from the
> > nonspherical nature
> > of the planet could yield something interesting, but thus
> > far I'm not
> > impressed that a statistically significant fault has been
> > found.
>
> Nor would you find anything like that.
>
> The situation is like security vulnerabilities in code. Those who write the
> code are motivated not to see the bugs because they want to believe there
> are none. At the same time, vuln researchers are motivated to figure out how
> to make any minor bug into something major they can exploit.
>
> The same is true of this code. I see lots of problems, such as failure to
> sanitize inputs, failure to sanity check results, and table of arbitrary
> values that adjust the final result with no documentation as to why they are
> there. For example, look at line 47 of "cru-code/linux/mod/homogeneity.f90".
>
> In any case, the issue isn't "accidental" bugs so much as "intentional"
> ones.
I agree with the assertion that the bugs described thus far are *precisely*
like security vulnerabilities in code.  Specifically, most
"vulnerabilities"...aren't.

A long time ago, I knew nothing of attacking integer overflows.  I asked a
friend of mine, "So the integer wraps.  How could that be exploited?"  And I
got a good lesson in how (for example) the following construct:

char *foo = malloc(count * sizeof(bar));

...would lead to pain, since an attacker controlled count would cause malloc
to wrap around to zero, while the system still assumed *foo pointed to some
ungodly amount of RAM.

So, first thing I did was search everything I could find for mallocs that
included a multiply within their arguments.  I was so excited!  Look at all
these bugs!

Then I started realizing, heh.  Wait.  Can an attacker actually set count?
Is count bounded by, for example, it being a char or a short, or being read
in from a 32 bit field in the original file format?  Do I ever get
sufficient control of how *foo is used, to be able to corrupt much of RAM
interestingly?

Do I already have to have code execution as root, in order to alter this
input file format in the first place?

Grep is not exactly a wonderful static analysis engine, it turns out.

Where we are now is this exact sort of fairly naive analysis of the Climate
code.  There's no findings yet -- that sort of rigor hasn't shown up yet,
and who knows if it ever will -- but oh, how people are grepping for badness
that could, maybe possibly cause issues.

The irony that people are complaining about lack of rigor, while having none
themselves, should not be lost on anyone.

Look, the code could have issues.  Both the 1% error in the spherical nature
of the earth, and the failure to correctly account for the wrapping nature
of the globe, could cause problems in the data.  But, you know, do they?

Or, are they like most things in both statistical analysis and security
auditing -- interesting in isolation, but swept aside by greater forces in
the deployed system?

It's not enough for there to be constants and correction tables.  These are
normal, though the tables need to be documented.  It's not enough for there
to be insufficient comments.  Comments are very rarely sufficient, and
what's there is almost always the grumbling of an angry programmer.  It's
not enough for the code to be ugly.

The world runs on ugly code.  See www.thedailywtf.com .

And frankly, of course the code has miserable sanity checking.  Only secure
code sanity checks, and the climate modeling code is not expected to parse
untrusted input!  The environment isn't going to raise some sector of the
ocean to 2^32-1 degrees Celsius just to overflow the climate modeler.

Finally, "intent" is a loaded word.  Certainly we know from security that
backdoors (even including vendor maintenance passwords) are far rarer than
unintended vulnerabilities.  Consequences exist for the former, not for the
latter.  I really don't see some climate scientist cackling as he fails to
account for the slightly nonspherical nature of the earth.

To be clear, I'm not saying the code is perfect.  It could very well have
bugs.  But after hearing about how uncasted transforms between reals and
integers in Fortran are a very effective random number generator, only to
find out they aren't, after seeing two vaguely promising statistical errors
get publicized without testing, and after an ungodly amount of whining that
the code was not in fact passed down from the heavens, pristine, well
commented, and utterly bug free, I gotta say to the CRU code deniers, just
like developers say to me:

Show me the inputs that cause this code to return statistically significant
error.

_______________________________________________
Fun and Misc security discussion for OT posts.
https://linuxbox.org/cgi-bin/mailman/listinfo/funsec
Note: funsec is a public and open mailing list.