Nmap Development mailing list archives
Re: Tudor's Status Report - #15 of #17
From: Daniel Miller <bonsaiviking () gmail com>
Date: Tue, 9 Aug 2016 21:43:17 -0500
Tudor, I have been putting some thought into how you could best accomplish the scanning. Since a big part of the hassle of Internet-wide scanning is managing exclusion lists, I suggest you take one address at random from each /24 in your "alive hosts" list and do a scan against each. This will get complaints from lots of folks without being as resource-intensive as a full scan. At this point, you could reduce your target list by removing any addresses from /24 blocks that resulted in complaints. Also, I would suggest removing any addresses from /24 blocks that resulted in all ports open (or 900/1000 open or similar). These are likely not real targets. You can also split up the ports to get a statistically-significant sample instead of scanning all 65535 ports on every target. For each port, sampling 5000 targets will give an accurate frequency +/-2 at 99% confidence, so with 100M potential targets, you could break them into 20K groups and scan a different 4 ports on each one. That seems a bit strange, so maybe do 64 groups of 1024 ports against 5000-ish hosts each. Significantly less workload, but decent results. Dan On Tue, Aug 9, 2016 at 12:38 PM, Tudor-Emil COMAN < tudor_emil.coman () cti pub ro> wrote:
@Daniel <bonsaiviking () gmail com> That was a mistake indeed, I was scanning the first 1000 ports. I recalculated how long it should take to scan for 65536 ports and it looks like it would take a lot of time, I'm trying to see if I can do some significant improvements before starting again. I'm probably going to end up scanning a random subset of those 115 million IP's. @d33tah I used research.nmap.org. It has: - Intel Xeon E3-1230 v1/2 (4-core) - 32GB RAM - CentOS 7 OS - 2TB SATA drive - 1Gbps ethernet Bandwidth utilization would fluctuate a lot but I've seen it go as high as 79 Mbps. CPU would be at about 100% Memory at 0.2% so about 640 Mbytes. The scan was: ./nmap 0.0.0.0/0 --min-rate 140000 --min-hostgroup 8192 -T5 -n -Pn -p 80 --max-retries 0 &> /dev/null -oG mass.log -sS --excludefile /etc/zmap/blacklist.conf ------------------------------ *From:* Jacek Wielemborek <d33tah () gmail com> *Sent:* Tuesday, August 9, 2016 7:08:26 PM *To:* Tudor-Emil COMAN; dev () nmap org *Subject:* Re: Tudor's Status Report - #15 of #17 W dniu 09.08.2016 o 08:46, Tudor-Emil COMAN pisze:Hello folks, Scanning the entire internet on port 80 finished and took 353716.09seconds. Hi Tudor, Four days, nice! What was the command you used? What were the specs of the server you used and how much resources (bandwidth, transfer, memory) did it use? I'd love to hear more about this. Cheers, d33tah
_______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/
Current thread:
- Tudor's Status Report - #15 of #17 Tudor-Emil COMAN (Aug 08)
- Re: Tudor's Status Report - #15 of #17 Jacek Wielemborek (Aug 09)
- Re: Tudor's Status Report - #15 of #17 Tudor-Emil COMAN (Aug 09)
- Re: Tudor's Status Report - #15 of #17 Tudor-Emil COMAN (Aug 09)
- Re: Tudor's Status Report - #15 of #17 Daniel Miller (Aug 09)
- Re: Tudor's Status Report - #15 of #17 Tudor-Emil COMAN (Aug 09)
- Re: Tudor's Status Report - #15 of #17 Jacek Wielemborek (Aug 09)