Re: [PEN-TEST] Network Attack Trend Analysis

[Matt Dickerson <matt () STOMMEL TAMU EDU>]

  As the maintainer and author of the Attrition.org stat and
graph pages, I appreciate the opportunity to reply to Mr.
Carvey.
 It's great that we have the sort of authority in Mr. Carvey to
explain this all for us,  having "taken graduate courses in
statistics and statistical analysis" -- something I would never
have guessed if he had not volunteered this information.

On Wed, Sep 06, 2000 at 05:37:36PM -0000, H Carvey wrote:
> > Just curious why you would consider the
> attrition.org stats "not factual"?
> >
>
> I'd have to agree that perhaps "not factual" is an
> incorrect phrase...how about "hardly substantial"?
>
> Here's my reasoning...
>
> How does Attrition become aware become aware of
> web page defacements?  Is the predominant method
> that someone informs them?  Who does this?  The
> person who defaces the page, or someone who
> notices the defacement?  If the former, it is therefore
> a logical argument that sites like Attrition lead to more
> web page defacements.  If the latter, then what is to
> say that the statistics are representative...if someone
> just happens to notice by accident that a web page is
> defaced?
>
> I have just spent some time reviewing several
> (though admittedly not all) of the graphs available on
> the Attrition site.  While I applaud the efforts of the
> Attrition staff, I have to ask...of what use are the
> graphs?  I have taken graduate courses in statistics
> and statistical analysis...yet it isn't clear at all what
> the graphs are intended to represent.
>
> Take for example:
>
> http://www.attrition.org/mirror/attrition/defacements-
> graphs.html#HIST
>
> What does the Y-axis represent?  Fraction of what?
> And the X-axis is labeled "Defacements per day,
> simple"...what constitutes a "simple" defacement?

  Anyone that knows the definition of histogram knows that
histograms represent frequency or proportions of frequency of the
intervals or classes on the x-axis.  I'll leave it to the
graduate students among us to infer fraction from proportion.
Mr. Carvey here demonstrates a complete lack of very basic
statistical concepts and diagnostics.
  He baffles himself with my use of the word "simple." I meant
"simple" in the sense of untreated, or unadjusted by
proportion.  The word could be left out, but was meant to
distinguish the variable from other "Defacement Per Day" (dpd)
variables, which were sometimes moving averages of dpd of
differing composition, proportions of dpd, and so on.

> This one:
>
> http://www.attrition.org/mirror/attrition/graphs/bar_osto
> tals.gif
>
> is entitled "OS totals by month"...but what do the
> various colors on the bars indicate?

  It is reading this that leads me to believe that perhaps our
graduate student is subjecting Attrition to gratuitous abuse.
Until a couple of weeks ago, this graph was part of
http://www.attrition.org/mirror/attrition/os-graphs.html where
the color of the bars were clearly labeled.  The most recent
version of this graph is now on that page, where it is now named
"bar_ostotals_stacked.gif", where it is likewise labeled.  None
of the graphs are erased month-to-month, but are typically
renamed.  They can be found in the browseable
http://www.attrition.org/mirror/attrition/graphs/, and often you
can find my tar-balls of the graphs there as well.  Yes, gifs,
sans HTML legends or headings.  A casual perusal of our graph
pages would have discovered the labeled HTML page.

> I guess the point is this...if you have nothing better to
> do and want to waste someone's time...sure, show
> these graphs to your boss.  They are meaningless,
> though colorful and probably quite enjoyable to look at
> when printed on a color printer.

  Mr. Carvey's conclusions are as out of proportion as his
authoritative observations.  And we are meant to take these
seriously?

> Not only are the graphs meaningless, but the very
> data that the graphs are based on is suspect.  How is
> the data collected?
>
> To be fair, though...I have to say the same thing about
> the CSI/FBI survey...the statistics that are generated
> as a result of the survey are largely misunderstood
> (and very often misquoted), but the very method used
> to collect the data is suspect, as well.
[snip]

  "Meaningless.... suspect, but hey, to be fair...." is like
saying, "With all due respect, [insert gratuitous insult here]".

Matt Dickerson ("munge")