Re: SMB parallelization

[David Fifield <david () bamsoftware com>]

On Thu, Sep 23, 2010 at 11:02:30PM -0500, Ron wrote:
> I'm finishing off a 15-hour day of SMB work, but it's been incredibly productive! This is my final submission for 
> now, I hope I can still type coherently. :)
>
> Anyway, I was asked to look into how to parallelize the SMB protocol today. I started by doing a quick and dirty 
> prototype in Nmap for testing. Once I was off the clock, I went back and refined my code to make it work nicely. 
>
> Basically, this removed all need for mutexes. Most (consumer) versions of Windows can handle roughly 11 SMB 
> connections at once (globally). Once that's exceeded, an identifiable error message comes back. In my patch, if we 
> see that error message, we back off for rand(1..10) seconds then try again. The 10th time we get it, we give up with 
> the message, "Too many active connections" (the same error Windows itself uses). 
>
> The second issue is that opening the pipe would fail at random. To solve that, I a) put a mutex around the 
> create_file call (which seemed to have a race condition of some sort), and b) the rare occasions we DO get the error, 
> we back off for 100ms (which is more than enough) and try again - up to 10 times again. I only got the error about 
> 1/50 of the time, and never twice in 100ms. 
>
> Me and Brandon did some largescale testing against a /16. Without the patch, it took about 4000s. After applying the 
> patch, it's down to about 1000s. That's a 4x speedup! We compared the outputs, and the patched one was about 98% the 
> size of the unpatched. The difference there could simply be explained by the number of hosts found - the unpatched 
> scan found 28 more hosts. I suggested doing a Levenshtein distance calculation on the two files, but Brandon didn't 
> want to spend the rest of the universe's lifespan on the calculation so we decided to say "the files were the same 
> within experimental error". 

I think I understand the technque you are using. I would be concerned
that it could fail just by chance with large numbers of connections, but
your testing with Brandon shows that it's not much of a worry. Feel free
to commit it.

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/