Nmap Development mailing list archives
Re: Nmap Website Directory Tree Help
From: Dave Horsfall <dave () horsfall org>
Date: Mon, 27 Jul 2015 06:30:32 +1000 (EST)
On Sun, 26 Jul 2015, Hur wrote:
Hi all, I would really appreciate if you can help me with something.How do I use nmap to view website directory tree? So that I can see all possible links on that server.
Any web spider/crawler (that is what the tool is called) needs to honour any "NOFOLLOW" tags that may be present. At least one web script, Wpoison, generates random pages of random gibberish with random email addresses to defeat just such things (I use it myself). At least one crawler, MJ12bot, happily ignores these tags, and can get trapped for hours... Hmmm; Bingbot seems to be another (I can see it as I type). In short, if you ignore the warning signs about entering a minefield, well... -- Dave Horsfall DTM (VK2KFU) "Those who don't understand security will suffer" Watson never said "I think there is a world market for maybe five computers." _______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/
Current thread:
- Nmap Website Directory Tree Help Hur (Jul 26)
- Re: Nmap Website Directory Tree Help Dave Horsfall (Jul 27)