Re: Using nmap to detect country-wide Internet shutdowns

[David Fifield <david () bamsoftware com>]

On Fri, Jul 01, 2011 at 11:23:25AM -0400, David Larochelle wrote:
> Hi,
>
> I'm at the Berkman Center for Internet & Society at Harvard
> (http://cyber.law.harvard.edu). We're interested in developing a method
> to determine if a country has shutdown its Internet. The typical use
> case for this would be that we hear scattered reports that a country
> such as Syria or Egypt is not longer Internet accessible and we would
> like a way to verify these reports.
>
> Our initial approach has been to use nmap to perform a ping scan on all
> IPs in all Autonomous Systems that are based in the country. We have
> used data from CAIDA and the Routeviews project to associate CIDR blocks
> with Autonomous Systems. We then map Autonomous Systems to countries by
> querying the Cymru service.
>
> We are then using commands like the following to determine the number of
> IP addresses accessible within a given country:
>
>     nmap --host-timeout 5m -sP --randomize-hosts -iL
>     syrian_cidr_blocks.txt > syrian_results.txt
>
>
> Obviously this will not detect hosts that are behind a firewall or that
> are ignoring pings but we're hoping that it will be good enough to
> detect and verify events such as the recent attempts by governments
> during the Arab Spring to shut down the Internet within their borders.
> It would also be nice to be able to determine which ISPs have remained
> on-line when much of the country is inaccessible. For example, Noor
> Group remained on-line even when the rest of the Internet in Egypt was
> shutdown.
>
> One of the questions that we have is what length of timeout is
> appropriate for this type of scan. We initially tried not giving any
> timeout but found that the scans were taking weeks to finish. We noticed
> that significantly most hosts were detected using a 5 minute timeout
> than a 2 minute timeout but we're unsure how long of a timeout is necessary.

The host discovery happens in blocks of 4096 at a time. But with
--randomize-hosts, it's 16384 at a time. The default ping scan is 4
probes, each of which can be retransmitted once.

So that means you should budget for 16384 * 4 * 2 = 131072 probes to be
sent. 2m gives only 900 μs per probe, too short. Only the hosts that are
scheduled near the beginning of the list will get any responses.

Figure out what your expected RTT is and base your timeout off of that.
Or take off --randomize-hosts and you can multiply your timeout by 0.25.
You might also look at the --min-rate and --max-retries options.

(Also, the scan you posted above will do reverse DNS resolution; be sure
you account for that if name resolution is slow.)

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/