RE: webapp dependencies

["Matt Fisher" <mfisher () spidynamics com>]

Hi Ory, 

I don't think the issue here is about "quality" items such as page load
time, broken links, etc. I think he's talking about having irrelevant
files junking up his webroot to avoid things like  all sorts of source
code, config files, stack traces and other such items from being browsed
or picked up in a scan. 

This is a really interesting subject to me: I've always recommend
keeping your webroot clean, and I suggest that SA's list the dir's and
check with the Dev group on any unusual files ("Hey Bob, is this
login.bob page really supposed to be there ? " ) .   

First of all I don't think crawlers would cut it ( or other client side
products). You'd have to look at the source code to gather the
dependencies, because you can have obvious client side stuff like href's
/ scripts /images, but you can have plenty of server side dependencies
that won't be visible to the client, like SSIs, class libs, modules,
HTML files read from a filestream (but no href), pages referenced
through parameter ( showpage.jsp?pageid=123) etc.  A quick example of
some page1.html being dependent, but not visible to the client:

<a href=page.asp?pageid=1>Read Report</a><br>

<%

Set fso = server.createobject("Scripting.FileSystemObject")
Select case request.querystring("pageid")
        case 1 ' Read a text report imported from a nightly batch
transaction
                Set file = fso.OpenTextFile(Server.MapPath("\") &
"/page1.html") 
                do until file.atendofstream
                        response.write file.readline 
                ...
End select 

%>


Unless I did something lame in this sample, I don't see how a crawler
would know where the content is actually coming from page1.html and not
page.asp.  

Given the server side dependencies issue,  I see two approaches short of
finding an actual tool for this (which, btw, is more likely to be found
in a Dev group than an AppSec on IMHO)

 1. Grep the code for various file patterns and then list those as
"needed." The problem is you'd have to anticipate the patterns 

2.  List the existing files, then grep for THOSE.  If the file exists
but doesn't match a grep, then carefully remove it.  

By carefully remove it I mean wait until a config window, take that box
out of service on any load balancers, rename it, test as much of the app
as possible and then finally delete it or move it to a 'deleteme'
folder, or whatever other processes you take to ensure career longevity
when playing with production systems. 

This is an area that I think is pretty darned important (having done
really keen stuff against multiple sites based on fluff files). I would
imagine the tool would end up being a development tool (non security
related).  There are products that will perform code-to-UML mapping
(vice versa) that may help with this. May.

  I'd really be interested in hearing about it if anyone finds a good
tool / technique but at this point I really don't see how it could be
sufficiently performed from any client sided product such as crawlers,
scanners, accessibility testers etc.  

- Matt @ SPI

http://www.spidynamics.com
WebInspect, DevInspect, 
QAInspect, SecureObjects, AMP

Start Secure. Stay Secure.


> -----Original Message-----
> From: Ory Segal [mailto:osegal () watchfire com] 
> Sent: Thursday, April 14, 2005 4:27 AM
> To: Jarmon, Don R; webappsec () securityfocus com
> Subject: RE: webapp dependencies
>
> Hello,
>
> Watchfire ( http://www.watchfire.com ) has a platform for 
> automating scanning, analysis and reporting of online 
> businesses. The platform is called WebXM, and it includes 
> many modules, some of which you mentioned.
> For example:
>
> 1) Scanning for web application security
> 2) Scanning for site accessibility problems
> 3) Scanning for web site quality issues (what you have mentioned)
> 4) Manage web site privacy
>
> And many more.
>
> The platform is installed as a server, with a web interface 
> acting as a reports dashboard, which will also present 
> regression testing.
>
> Thank you,
>
> Ory Segal,
> Watchfire 
>
> -----Original Message-----
> From: Jarmon, Don R [mailto:Don.Jarmon () Intergraph com]
> Sent: Thursday, April 14, 2005 2:23 AM
> To: webappsec () securityfocus com
> Subject: webapp dependencies
>
> I looking for a tool that will analysis content hosted on a 
> web site, identify all the webapp dependencies, and report on 
> any non-essential content.  The tool would run from the 
> server.  Does such a tool exist?