Re: Feedback requested - XML XSL transform changes

[Fyodor <fyodor () insecure org>]

On Sun, Nov 21, 2010 at 06:58:22PM -0600, Tom Sellers wrote:
>     I have been working on adding the XSL changes that you requested.
> I expect to finish the work in about 2 weeks, but the current version
> should be safe to commit.  Please look at the progress so far and
> let me know if I am on the right track.

Hi Tom.  Sorry it took me so long to respond, but I like your new
changes!  Feel free to check them in whenever you are ready.  So
that others can more easily review the generated HTML and comment on it, I
posted a couple files here:

My test scan: http://insecure.org/tmp/nmap-newxsl.html
Your test file: http://insecure.org/tmp/nmap-newxsl-tom.html

Here are some notes I wrote down while going through the file (some
are very minor):

o The way names are presented in the scan summary is a big improvement

o I think closed ports in the port table look much better in the new
  light gray.  But now the orange for "filtered" ports seems overly
  dramatic.  After all, filtered ports are usually even less
  interesting (certainly less useful) than closed ones.  So I'd argue
  for making filtered ones the same color as closed ports.  People can
  still distinguish them based on the state column.

o In the same way you turned the closed port backgrounds from red to
  gray, I might also change down hosts from red to gray in the scan
  summary and their individual title bars.

o The new expanders work great!

o Regarding the very top header, which looks like "nmap scan
  report - scan @ Sun Nov 14 14:22:06 2010":
  o I would probably rewrite it like "Nmap Scan Report — Scanned
    At Sun Nov 14 14:22:06 2010"
  o Maybe this title bar would be better off as white text on "Nmap
    purple" background?  The color for that is #2A0D45.

o A controls section up near the top might be nice.  I can think of at
least two buttons or checkboxes which would be useful:
 o Only show hosts with open ports - This would act like --open and
 cause all down hosts (and up ones without any open ports) to
 disappear from the report.  Only the open ports would be shown in
 port tables.
 o Expand all - would expand all the expanders, you could also uncheck
 (or click again or whatever) to collapse them all.

o In the scan summary, I'd change "Debugging was disabled, the
  verbosity level was 1" to "Verbosity: 1; Debug level: 0".

o Someday we might put Nmap's "normal" output in the XML.  At that
  point it might be nice to be able to hit a button to see plain Nmap
  output in some cases.

o Regarding OS detection:
  o I might add its own section for each host (if OS detection was
    enabled)

  o In addition to showing the OS matches (like you already do),
    you might want to show the OS classification data there too.
    This is the part of Nmap output which lookes like:
      Device type: WAP
      Running: Linux 2.4.X
    Or:
       Device type: general purpose|webcam|WAP|PBX|broadband router
       Running (JUST GUESSING) : Linux 2.6.X|2.4.X (96%), AXIS Linux
       2.6.X (91%), Linksys embedded (90%), Sphairon embedded (90%), AXIS
       embedded (89%) 

  o It would be great to describe the OS detection results better.
    For example, if there are no exact matches, normal Nmap says "No
    exact OS matches for host ", followed up with "(test conditions
    non-ideal)" if that is the case.  I think we should give a warning
    like this.  Also, in the case that there are too many matches,
    normal Nmap says "Too many fingerprints match this host to give
    specific OS details"

  o The OS match listings should have a space between the OS name and
    the accuracy percentage.  Right now they look like "Linux
    2.4.27(92%)"

  o If there are no exact matches, and Nmap feels that the quality is
    high enough for a submission, it would be great if the OS
    detection section would encourage the user to submit, just like
    normal Nmap does.

o It would be great to show version detection fingerprints along with
  a submission link too.

And regarding the other points in your email:

> The host index has now been changed to the format of  hostname (IP address).
> As far as the hostname portion preference is given to the user supplied
> hostname.

Looks good!

> The color for downed ports has been changed to light gray.  I am going
> to change how down hosts are shown altogether.

I'm looking forward to that (some thoughts above).

> Traceroute data is current green because each of the hops is up.  The
> XSL has code to deal with down and unresponsive hops.  The hops that
> don't respond are not output to the XML file even though they are
> displayed on the console.
>
> Test case:
>       sudo nmap -sP --traceroute -oX test.traceroute.xml  www.cnn.com
>
> Is this as expected?

Here is an excerpt from that example:

<hop ttl="12" ipaddr="4.68.103.46" rtt="116.39" host="ae-2-52.edge4.Atlanta2.Level3.net"/>
<hop ttl="13" ipaddr="4.59.12.2" rtt="122.97" host="CNN-AMERICA.edge4.Atlanta2.Level3.net"/>
<hop ttl="18" ipaddr="157.166.226.25" rtt="122.90" host="www.cnn.com"/>

I think the idea is that it might be a waste of space to include
entries like:

<hop ttl="14"/>
<hop ttl="15"/>

After all, they don't convey any extra information that you can't get
from programatically noting the break in sequence between hop 13 and
18.  Hopefully XSL is powerful enough to note the break in sequence
and do what you want with it?

> This data has been moved to a new section named Misc Metrics.  This
> is a click to expand div element that is collapsed by default.

Looks good.  I find this new version much easier to read because all
the obscure details are hidden away in expanders.

> > o Related to the idea above: you could consider omitting the closed
> >  ports and down hosts by default, unless the user clicked a button to
> >  add them (there could be a controls/customization section near the
> >  top, I suppose).  I'm starting to wonder if Nmap should even include
> >  closed and filtered ports in the host table by default--maybe they
> >  should just be listed.
>
> I am going to work on summarizing this in the next revision.

Great.

> > o There is a section titled "remote operating system guess" which
> >  should probably be "guesses" since it usually contains several
> >  values.  It might be nice if it used text more like Nmap, noting
> >  that there are no exact matches but here is a list of the closest
> >  ones.  And I don't think you need to give things like "reference
> >  fingerprint line number: 9336" for each entry.  Although it looks
> >  like this section is used if a result is exact too.
>
> Done, guesses fixed, fingerprint line removed.  The OS output has been
> modified to be more like the nmap output   OS match (accuracy) which is
> more compact.

That does look better.  And I made even more suggestions above :).

> > o It would be nice to see OS detection (operating system and/or system
> >  type) icons for introducing hosts.  But a challenge is that we'd
> >  prefer not to load them off a 3rd party site like nmap.org because I
> >  suppose that could be a privacy risk.  It would tell that a user
> >  from the IP was reading an Nmap scan report and it contains at least
> >  one of the device type or operating system identified by the logo.
> >  Isn't there a way to include small images inline by including the
> >  hex data?
>
> I looked into this (using img src="data: ... ) and could not get
> Firefox to display the icon.  Internet Explorer 8 (!) would display
> it just fine.  According to Wikipedia most of IE's support for this
> tag is limited and started with 7, but it seems to work the best.
> Most other browsers appear to support it though.  I will work some
> more on this.

OK.  Wikipedia does think it is supported by Firefox
(http://en.wikipedia.org/wiki/Data_URI_scheme).  Also, the inline
picture on this page works in my Firefox 3.6.12 (Linux x86-64):

http://www.sveinbjorn.org/news/2005-11-28-02-39-23

Thanks for your work in improving this XSL.  It is already looking far
better than before!

Cheers,
Fyodor
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/