Re: NSE Infrastructure: Eliminate script.db

[Brandon Enright <bmenrigh () ucsd edu>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

<pure_opinion tone="friendly">

Hi Kris,

I'm actually glad that NSE uses a DB index file rather than dynamically
building the list.  The build may seem instantaneous now, but on a busy
file system with lots of scripts the cost of dynamically loading the
list every time Nmap runs could end up being very high.

A perfect example of this problem is Nessus.  If you are running just
one check oftentimes the script enumeration process takes longer than
the scan itself.

I have a system setup where when someone comes online unregistered, a
very small targetted Nmap and Nessus scan are run against them.  At the
start of the school year this can mean thousands of individual
Nmap/Nessus invocations an hour during a time when the disk is
basically pegged.  The _only_ way I was able to get Nessus to respond
fast enough was to removal all unused scripts in the directory and
solve all of the script dependencies by hand.  I'd hate to have to do
the same with Nmap.

I know when you profile code you typically focus on repeated routines
rather than startup costs but I'm very glad Nmap uses the DB approach
over beating the filesystem to death every time it starts.

I think it is quality design decisions like this and good code that
keep people like you and I still interested in developing on Nmap.  The
same could not be said for Nessus or many other open source projects
that have trouble getting developers or fighting off the rumblings of
"fork it".

Brandon
</pure opinion>


On Mon, 24 Mar 2008 15:27:46 -0500
Kris Katterjohn <katterjohn () gmail com> wrote:

> Hey everyone,
>
> I tend to write quite a few pretty simple NSE scripts to interact with
> something, or to test something, or whatever it may call for.
>
> Unfortunately, everytime I write a new script I have to set NMAPDIR
> and use -script-updatedb to update the script database in my nmap
> directory--for a lot of simple scripts this gets old :).  The
> Metasploit Framework seems to build their "database" of exploits,
> payloads, etc. straight from the directory structure rather than from
> a database file.
>
> I was thinking that Nmap could use the steps it takes to build the
> database file, but instead of writing to the script.db it could just
> build the internal list of scripts (or however it works--I'm not
> well-versed in the innards of NSE).  This would eliminate the
> - -script-updatedb option and the script.db.
>
> This way, all that's needed for a script to be available to Nmap is
> for it to the placed in the directory.  This makes for a lot simpler
> use of new scripts.
>
> Are there details I am missing that makes this infeasible for NSE?
> The NSE usage [1] says: "For efficiency reasons, NSE generates a
> script.db file which maps categories to the scripts they contain," so
> is the script.db for efficiency in this respect?  Running
> -script-updatedb seems instantaneous, so perhaps I'm mistaken by what
> the usage refers to (although it could mean in the future when there
> are many more scripts)?
>
> I figure this could make a good NSE Infrastructure SoC project if it's
> any good.
>
> Thanks,
> Kris Katterjohn
>
> [1] http://nmap.org/nse/nse-usage.html

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)

iD8DBQFH6BN0qaGPzAsl94IRAtIBAJ4s+qKRK69qbuVNq7iN1TlSz4pDBACgqzom
VZvLoKZW7h4gA3scx7rdJqM=
=p9K8
-----END PGP SIGNATURE-----

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org