Re: Buffer Overruns

[Joseph S D Yao <jsdy () cospo osis gov>]

On Sat, Dec 18, 1999 at 05:45:13PM -0500, Vin McLellan wrote:
>         It there something in the emergence of a popular Internet, or some
> other timely aspect in the industry's evolution, that has brought to light
> the vulnerabilities associated with buffer overruns in recent years? 
>
>         Maybe some shift in program design or programming engineering
> practice?  What left so many of these vulnerabilities unexposed and their
> risks unappreciated for so many years?
>
>         Sometimes even in <ahem> widely distributed source code.
>
>         _Vin

Vin, pardon me while I rant a bit.

What "recent years"???  This problem, this "vulnerability", has been
known since the days of FORTRAN.  All beginning programming students
have been warned about putting something too big into an array that was
not sized to handle it!  But still, it's been done since arrays were
created.

Many programmers these days are self-taught.  This is great!  But I
could wish that they hadn't been hired before they had been taught
something about software engineering methodologies.  True software
engineers, or even programmers who use some software engineering
methodologies, are extremely rare!  And I'm not even talking about
choosing one of the churches of the holy Methodologies, capital-M, and
sticking to them as if they were in fact a religion.  I'm talking about
simple, trivial, easy-but-boring engineering skills, like checking your
input for validity, checking in each function/subroutine/method, and
checking your output.

Why has it been seen a lot lately?  I'd guess two reasons.  One is, it's
a popular "fad".  The guys who get their kicks out of such things have
suddenly come to the realization that there are such things, and are
getting their jollies pointing them out wherever possible.  And why is
this so possible?  That's the second thing.  As you mention, now there
IS the Web on top of the Internet, that makes so much accessible to so
many.  I was very glad when the first ISPs made the Internet available
to me at home.  I was less glad when the next wave arrived ...  but
that's me being provincial.  Web apps are rushed to market on "Internet
time".  Applications are not checked for any degree of safeness, if
they ever were.  Many are written by or for that pre-Internet company
that thought putting kernel networking code into their word processor,
so that it wouldn't be bound by having to go through the kernel's
network security code, was such a hot idea.  ;-\

So.  The problems were always there.  They are not checked for,
especially when people want apps out yesterday.  Novices are writing
critical sections of code.  [Novices may have 40 years in the business,
OBTW; but if they haven't learned anything in 40 years, they're
novices.]  And it's all out there for everyone to see on the Web.

The Open Source model, if I may be so heretical, has helped some but
nowhere near enough.  If you believe, with Ted Sturgeon, that 90% of
everything written is c***, then at least 90% of everyone out there is a
c***-writer.  No, that wasn't "code".  I won't do the statistics to bear
this out, but my gut feel is that if you have 3 or 4 people review each
piece of code THOROUGHLY, then 65+% of the code will still not be
reviewed by one of the 10%.  And I feel that I am being generous in my
assumptions.  And it totally does not help with proprietary
applications.

In my younger days, when people were getting excited about software
engineering and data-oriented applications, it was felt that if you had
a number of bright SE's doing code walkthroughs and other such stuff,
you could eliminate a lot of the common problems.  Nice model.  Most
shops don't consist of a number of bright SE's.  Those that do, the
time-to-market overrules.

It's always been there.  Surely somewhere the RISKS listings are
archived.  It's just that all the dreamed-of advancements in "computer
science" and "software engineering" never came to pass; and some that
did have fallen like the seed that was trampled in the field.

OK, I sound pessimistic.  I am somewhat aware of the human condition,
and I don't expect this condition to be eliminated in my lifetime.
There are some tools that help people write code without these flaws,
but some of them introduce other flaws, and most are not as general-
purpose as we who hack at lower levels would like.  It takes discipline
to "do it right."  People would rather do an easy hack than a
disciplined, engineered one.  And public discussion of this does not
seem to have reached everyone on the planet who codes, just yet.

-- 
Joe Yao                         jsdy () cospo osis gov - Joseph S. D. Yao
COSPO/OSIS Computer Support                                     EMT-B
-----------------------------------------------------------------------
This message is not an official statement of COSPO policies.