Wireshark mailing list archives
extraction of files from SSL and TCP streams automatically
From: Miroslav Rovis <miro.rovis () croatiafidelis hr>
Date: Sat, 5 May 2018 18:17:42 +0000
Hi! How do users climbing the steep path of deep packet inspection extract files, in HTTP/HTTPS protocols, i.e. the streams in SSL (and plain TCP) conversations? Is there a program that can extract files from SSL- or plain- TCP streams automatically? I've long wished to gain that knowledge/that knowhow, since it's absolutely likely the big surveillors have their tools to do that, and do that very comfortably... Just, the public at large are not allowed nor access to nor the knowledge about those programs... No proofs that it is so, but it's so stinking likely that it is so. And asked about extr here on this list: https://www.wireshark.org/lists/wireshark-users/201604/msg00002.html E.g.:
Are there such scripts that could take a stream, and extract all the files from it? [...]
And I was recommended Chaosreader. But Chaosreader does not, well at least currently (who knows about the future), decrypt SSL and so can't extract from SSL streams, only from plain HTTP. And so I've decided to spend some serious time debugging Chaosreader my own way to figure out how it works, and learned a lot of Perl as I went. And I've managed to put together a script that uses a few modified subroutines from Chaosreader on already decrypted SSL TCP streams and extracts files from them. Judge for yourself how successful it is. Here's the script, or the primitive program if I should call it: https://github.com/miroR/stream-cont In the script chread_tcp.pl in the program I explained how i debugged Chaosreader, the commits can be followed from Chaosreader itself which is the initial commit, to the final chread_tcp.pl version and creation of stream-cont.pl itself. And here's a hands on tutorial on the whole process needed to work out a PCAP (or many PCAPs in the same session): https://www.croatiafidelis.hr/foss/cap/cap-180505-schmoog-referendum/ I was able to extract all files automatically, so no right-mouse clicking, chosing, selecting et cetera, no GUI stuff, repeated each time say, for every single stream, or... Or firing hexedit and manually navigating through the GET and HTTP headers, and cutting and pasting and stuff... And that is not the case just on those samples from that tutorial, but on most other samples from a range of internet connections to different places. stream-cont only rarely doesn't accomplish extractions properly, and I'm yet to (some day, not immediately in the world, see below) figure out the details and reasons of failures. And it does all its extractions automatically, no clicking, no cutting and pasting... It's not completed. I want it to extract POST as well, and do other things... But I just dropped dead tired once I started getting first great results that I could only dream of till then... And still wouldn't be able to work on it not even now and not for at least a few more days... I hope other users struggling with similar issues will find my stream-cont program useful. So I decided to present my stream-cont to this list. Feedback welcome! Regards! -- Miroslav Rovis Zagreb, Croatia https://www.CroatiaFidelis.hr
Attachment:
signature.asc
Description:
___________________________________________________________________________ Sent via: Wireshark-users mailing list <wireshark-users () wireshark org> Archives: https://www.wireshark.org/lists/wireshark-users Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-users mailto:wireshark-users-request () wireshark org?subject=unsubscribe
Current thread:
- extraction of files from SSL and TCP streams automatically Miroslav Rovis (May 05)
- Re: extraction of files from SSL and TCP streams automatically Peter Wu (May 07)
- Re: extraction of files from SSL and TCP streams automatically Miroslav Rovis (May 08)
- Re: extraction of files from SSL and TCP streams automatically Peter Wu (May 09)
- Re: extraction of files from SSL and TCP streams automatically Miroslav Rovis (May 09)
- Re: extraction of files from SSL and TCP streams automatically Gedropi (May 09)
- Re: extraction of files from SSL and TCP streams automatically Miroslav Rovis (May 12)
- Re: extraction of files from SSL and TCP streams automatically Peter Wu (May 13)
- Re: extraction of files from SSL and TCP streams automatically Miroslav Rovis (May 08)
- Re: extraction of files from SSL and TCP streams automatically Peter Wu (May 07)