Re: Bittorrent and DHT library and scripts

[Toni Ruottu <toni.ruottu () iki fi>]

Would it make sense to have a bittorrent-trackers script that could be used
to extract trackers from a torrent file (or a magnet link) and to add those
trackers as scan targets?
On 4 Aug 2011 03:08, "Gorjan Petrovski" <mogi57 () gmail com> wrote:
> Hi folks,
>
> I've updated the bittorrent-discovery script as agreed in the NSE
> meeting. The nodes are not shown by default and you need to use the
> <bittorrent-discovery.include-nodes> script argument if you want to
> see them. You can check the new version out in the nmap-exp/gorjan/
> repository.
>
> I also promised to explain the workings of the bittorrent library in
short:
> The usage of the library would be first to initialize a new Torrent
> object. This initialization includes setting values for several
> variables.
> Next, a the torrent information needs to be loaded from a torrent file
> or a magnet link. The information in question would be a list of
> trackers, and the info_hash variable which is a 20 bytes length SHA1
> hash of the info field in the torrent file. The torrent file includes
> the field itself, but the magnet link only includes the info_hash
> value.
> After the basic info for the torrent is set, next the peers from the
> trackers need to be downloaded (torrent:trackers_peers()). There are
> http and udp trackers which use different protocols implemented in the
> Torrent:http_tracker_peers() and Torrent:udp_tracker_peers(). The
> communication is done serially and could be improved by using threads.
> After a few peers have been discovered we can continue in using the
> DHT protocol to discover more. We MUST have several peers in order to
> use the DHT protocol, and what's more at least one of the peers must
> have that protocol implemented. A peer which implements the DHT
> protocol is called a node. What that protocol allows is actually to
> find more peers for the torrent we are downloading/interested in, and
> it also allows us to find more nodes (hosts which implement the DHT
> protocol). Please notice that a DHT node does not necessarily have to
> be a peer sharing the torrent we need. So, in fact we have two
> networks, the network of peers (hosts sharing the torrent we need) and
> the DHT network (network of nodes which allow us to find more peers
> and nodes.
> There are three kinds of commands we need to do DHT discovery:
> - dht_ping, which is sent to a peer to test if the peer is a DHT node
> - find_node, which is sent to a DHT node to discover more DHT nodes
> - get_peers, which is sent to a DHT node to discover peers sharing a
> specific torrent; If the node that we send the get_peers command
> doesn't have a record of peers sharing that torrent, it returns more
> nodes.
> So in the bittorrent library I implemented every command in functions
> which are run as separate threads. They synchronize their work using
> the pnt condvar table. This is the map of pnt (peer node table):
> pnt = { peers_dht_ping, peers, nodes_find_node, nodes_get_peers, nodes }
> The dht_ping thread pings every peer in peers_dht_ping and then
> inserts it into peers. It does this for batches of a 100 peers. If the
> peer responds it adds it to the nodes_find_node list.
> The find_node thread sends find_node queries to the nodes in
> nodes_find_node, after which it puts them in nodes_get_peers. The
> nodes included in the response are added to the nodes_find_node list
> if they are not present in any of the nodes' lists.
> The nodes_get_peers sends a get_peers query to every node in the list
> after which they are added to the nodes list. If undiscovered peers
> are returned they are inserted into peers_dht_ping. If undiscovered
> nodes are found they are inserted into nodes_find_node.
> All of these threads run for a specified timeout whose default value
> is ~ 30 seconds.
> As you can see all newly discovered nodes are added to the
> nodes_find_node, and are processed first by the find_node thread, and
> then by the get_peers thread. All newly discovered peers are added to
> the peers_dht_ping to be processed by the dht_ping thread and so on.
> That enables the three threads to cooperate and pass on peers and
> nodes between each other.
>
> There is also a bdecode function which decodes Bittorrent encoded
> buffers and organizes them into a structure I deemed fit for use.
> There are two known bittorrent structures: the list and the
> dictionary. One problem I encountered was that the bittorrent
> dictionary can have multiple entries with same-name keys. This kind of
> structure is not supported by Lua, so I had to use lists to represent
> the dictionaries as well which made accessing the keys a bit quirky
> :-S
>
> Any questions or comments, let me know ;-)
>
> --
> Gorjan
> _______________________________________________
> Sent through the nmap-dev mailing list
> http://cgi.insecure.org/mailman/listinfo/nmap-dev
> Archived at http://seclists.org/nmap-dev/
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/