Snort mailing list archives
Re: File extraction during http/ftp transaction
From: Y M <snort () outlook com>
Date: Wed, 11 Mar 2015 20:30:43 +0000
See this post as well: http://seclists.org/snort/2014/q1/489 Basically, in order for files to be captured their sha256 must be specified in the file lists. Date: Wed, 11 Mar 2015 22:51:39 +0530 From: rishabh420 () gmail com To: huica () cisco com; snort-users () lists sourceforge net Subject: Re: [Snort-users] File extraction during http/ftp transaction Hi Hui, I missed creating the directory(assumed that snort would create one). It is working now. Thanks a ton Hui. One minor query regarding the new files:-rw------- 1 root root 7091 Mar 11 22:48 9D29C44863C6A27D45F8621E6A636DF0746245C5F436DB9CA488252A7FF76579-rw------- 1 root root 22016 Mar 11 22:49 67792ACE824606664CE51973800D6B952CA4733CAF6F03CCF5F636768EFB39B1 Can it not retain the name/extension of the file? Thanks,Rishabh. On Wed, Mar 11, 2015 at 10:12 PM, Hui cao <huica () cisco com> wrote: Sorry. Don't change the conf, but check whether you have permission "write" on the folder /home/file_capture/tmp/ Best, Hui. On 03/11/2015 12:37 PM, Rishabh Shah wrote: Hi Hui, I removed signature and transferred two pcap files, but no luck: File Preprocessor Statistics Total file type callbacks: 2 Total file signature callbacks: 2 Total files would saved to disk: 2 Total files saved to disk: 0 Total file data saved to disk: 0 bytes Total files duplicated: 0 Total files reserving failed: 0 Total file capture min: 0 Total file capture max: 0 Total file capture memcap: 0 Total files reading failed: 0 Total file agent memcap failures: 0 Total files sent: 0 Total file data sent: 0 Total file transfer failures: 0 =============================================================================== File type stats: Type Download (Bytes) Upload (Bytes) PCAP(145) 2 3870 0 0 Total 2 3870 0 0 File signature stats: Type Download Upload PCAP(145) 2 0 Total 2 0 File type verdicts: UNKNOWN: 2 LOG: 0 STOP: 0 BLOCK: 0 REJECT: 0 PENDING: 0 STOP CAPTURE: 0 Total: 2 File signature verdicts: UNKNOWN: 2 LOG: 0 STOP: 0 BLOCK: 0 REJECT: 0 PENDING: 0 STOP CAPTURE: 0 Total: 2 Total files processed: 2 Total files data processed: 3870 bytes Total files buffered: 2 Total files released: 2 Total files freed: 0 Total files captured: 2 Total files within one packet: 2 Total buffers allocated: 2 Total buffers freed: 0 Total buffers released: 2 Maximum file buffers used: 1 Total buffers free errors: 0 Total buffers release errors: 0 Total memcap failures: 0 Total memcap failures at reserve: 0 Total reserve failures: 0 Total file capture size min: 0 Total file capture size max: 0 Total capture max before reserve: 0 Total file signature max: 0 Maximum buffers can allocate: 3196 Number of buffers in use: 0 Number of buffers in free list: 3194 Number of buffers in release list: 2 On Wed, Mar 11, 2015 at 10:02 PM, Hui cao <huica () cisco com> wrote: Can you remove signature? If this is enabled, it only captures file that matches to a signature list. preprocessor file_inspect: type_id, capture_disk /home/file_capture/tmp/, capture_queue_size 5000 Best, Hui. On 03/11/2015 12:24 PM, Rishabh Shah wrote: Hi Hui, I included file_magic.conf in my snort configuration file. After starting the snort process, I transferred 3 files and this is the output after stopping snort: File Preprocessor Statistics Total file type callbacks: 1 Total file signature callbacks: 1 Total files would saved to disk: 1 Total files saved to disk: 0 Total file data saved to disk: 0 bytes Total files duplicated: 0 Total files reserving failed: 0 Total file capture min: 0 Total file capture max: 0 Total file capture memcap: 0 Total files reading failed: 0 Total file agent memcap failures: 0 Total files sent: 0 Total file data sent: 0 Total file transfer failures: 0 =============================================================================== File type stats: Type Download (Bytes) Upload (Bytes) PCAP(145) 1 1935 0 0 Total 1 1935 0 0 File signature stats: Type Download Upload PCAP(145) 1 0 Total 1 0 File type verdicts: UNKNOWN: 1 LOG: 0 STOP: 0 BLOCK: 0 REJECT: 0 PENDING: 0 STOP CAPTURE: 0 Total: 1 File signature verdicts: UNKNOWN: 1 LOG: 0 STOP: 0 BLOCK: 0 REJECT: 0 PENDING: 0 STOP CAPTURE: 0 Total: 1 Total files processed: 3 Total files data processed: 8124 bytes Total files buffered: 1 Total files released: 1 Total files freed: 0 Total files captured: 1 Total files within one packet: 1 Total buffers allocated: 1 Total buffers freed: 0 Total buffers released: 1 Maximum file buffers used: 1 Total buffers free errors: 0 Total buffers release errors: 0 Total memcap failures: 0 Total memcap failures at reserve: 0 Total reserve failures: 0 Total file capture size min: 0 Total file capture size max: 0 Total capture max before reserve: 0 Total file signature max: 0 Maximum buffers can allocate: 3196 Number of buffers in use: 0 Number of buffers in free list: 3195 Number of buffers in release list: 1 =============================================================================== On Wed, Mar 11, 2015 at 9:34 PM, Hui cao <huica () cisco com> wrote: In READMe.file: Pre-packaged file magic rules: A set of file magic rules is packaged with Snort. They can be located at "etc/file_magic.conf". To use this feature, it is recommended that the these pre-packaged rules are used; doing so requires that you include the file in your Snort configuration as such: include etc/filemagic.conf On 03/11/2015 12:01 PM, Hui cao wrote: Have you added file magic into your configuration. What's the snort output? Best, Hui. On 03/11/2015 11:56 AM, Rishabh Shah wrote: Thanks Hui. That worked for me! Now I started snort after adding file_inspect preprocessor. preprocessor file_inspect: type_id, signature, capture_disk /home/file_capture/tmp/, capture_queue_size 5000 (Got the following console logs to confirm that file_inspect has started) File config: file type: ENABLED file signature: ENABLED file capture: ENABLED file capture directory: /home/file_capture/tmp/ file capture disk size: 300 (Default) megabytes file sent to host: DISABLED (Default), port number: 0 File service: file type enabled. File service: file signature enabled. File service: file capture enabled. File capture thread started tid=0x7f0aaa783700 (pid=19354) I initiated file transfer via HTTP/FTP as shown below: rishab%ftp 192.168.2.200 Connected to 192.168.2.200:21. 220 (vsFTPd 2.0.5) Name (192.168.2.200:21:fwdevtest1): fwuser 331 Please specify the password. Password: 230 Login successful. Remote system type is UNIX. Using binary mode to transfer files. ftp> get new.pcap 200 PORT command successful. Consider using PASV. 150 Opening BINARY mode data connection for new.pcap (1555 bytes). 226 File send OK. 1555 bytes received in 0.4 seconds (3887 bytes/s) ftp> ftp> quit 221 Goodbye. rishab%wget 192.168.2.200/dns.pcap --2015-03-11 21:23:16-- http://192.168.2.200/dns.pcap Connecting to 192.168.2.200:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1935 (1.9K) [text/plain] Saving to: ?dns.pcap? 100%[======================================================================================================================================================================================>] 1,935 9.39KB/s in 0.2s 2015-03-11 21:23:19 (9.39 KB/s) - ?dns.pcap? saved [1935/1935] After killing the snort process, I do not see any file created in that location: root@fwuser-virtual-machine:/home# ls fwuser Am I missing anything? On Wed, Mar 11, 2015 at 9:09 PM, Hui cao <huica () cisco com> wrote: Have you done make clean before you do a make? Best, Hui. On 03/11/2015 11:38 AM, Rishabh Shah wrote: Hi Hui, I am hitting the same issue while executing make. These are the commands that I issued: root@fwuser-virtual-machine:~/snort_src/snort-2.9.7.0# ./configure --enable-file-inspect --enable-open-appid --enable-sourcefire root@fwuser-virtual-machine:~/snort_src/snort-2.9.7.0# make /root/snort_src/snort-2.9.7.0/src/plugbase.c:216: undefined reference to `SetupAppId' detection-plugins/libspd.a(detection_options.o): In function `detection_hash_free_func': /root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:553: undefined reference to `optionAppIdFree' detection-plugins/libspd.a(detection_options.o): In function `detection_option_hash_func': /root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:252: undefined reference to `optionAppIdHash' detection-plugins/libspd.a(detection_options.o): In function `detection_option_key_compare_func': /root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:409: undefined reference to `optionAppIdCompare' collect2: error: ld returned 1 exit status make[3]: *** [snort] Error 1 make[3]: Leaving directory `/root/snort_src/snort-2.9.7.0/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/root/snort_src/snort-2.9.7.0/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/snort_src/snort-2.9.7.0' make: *** [all] Error 2 On Wed, Mar 11, 2015 at 8:40 PM, Hui cao <huica () cisco com> wrote: Hi Rishabh, You need to add —enable-open-appid to you ./configure. ./configure --enable-file-inspect —enable-open-appid Best, Hui. On 03/11/2015 10:33 AM, Rishabh Shah wrote: Hi Joel, Thanks for your prompt reply. I did a ./configure --enable-file-inspect and while executing make, I saw the following error messages: /root/snort_src/snort-2.9.7.0/src/plugbase.c:216: undefined reference to `SetupAppId' detection-plugins/libspd.a(detection_options.o): In function `detection_hash_free_func': /root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:553: undefined reference to `optionAppIdFree' detection-plugins/libspd.a(detection_options.o): In function `detection_option_hash_func': /root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:252: undefined reference to `optionAppIdHash' detection-plugins/libspd.a(detection_options.o): In function `detection_option_key_compare_func': /root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:409: undefined reference to `optionAppIdCompare' collect2: error: ld returned 1 exit status make[3]: *** [snort] Error 1 make[3]: Leaving directory `/root/snort_src/snort-2.9.7.0/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/root/snort_src/snort-2.9.7.0/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/snort_src/snort-2.9.7.0' make: *** [all] Error 2 I am not sure why am I seeing those messages as I see a reference to the above errors: root@fwuser-virtual-machine:~/snort_src/snort-2.9.7.0/src# grep -r "optionAppIdFree" . Binary file ./detection-plugins/detection_options.o matches Binary file ./detection-plugins/sp_appid.o matches ./detection-plugins/sp_appid.c:void optionAppIdFree(AppIdOptionData *optData) ./detection-plugins/sp_appid.c: optionAppIdFree(optData); Binary file ./detection-plugins/libspd.a matches ./detection-plugins/detection_options.c: optionAppIdFree(key->option_data); ./detection-plugins/sp_appid.h:void optionAppIdFree(AppIdOptionData *optData); I appended the following line in snort.conf: preprocessor file_inspect: type_id, signature, capture_disk /home/file_capture/tmp/, capture_queue_size 5000 While executing snort process, I got a core file with the following message: File config: file type: ENABLED file signature: ENABLED file capture: ENABLED file capture directory: /home/file_capture/tmp/ file capture disk size: 300 (Default) megabytes file sent to host: DISABLED (Default), port number: 0 Segmentation fault (core dumped) The traceback of the core file points to: root@fwuser-virtual-machine:~/snort_src# gdb snort -c core GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from snort...done. warning: exec file is newer than core file. [New LWP 10904] warning: .dynamic section for "/usr/local/lib/snort_dynamicengine/libsf_engine.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/usr/local/lib/snort_dynamicpreprocessor//libsf_reputation_preproc.so" is not at the expected address (wrong library or version mismatch?) [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/bin/snort -c /etc/snort/snort.conf -Q -i eth1:eth2 -l /var/log/snort'. Program terminated with signal SIGSEGV, Segmentation fault. #0 strlen () at ../sysdeps/x86_64/strlen.S:106 106 ../sysdeps/x86_64/strlen.S: No such file or directory. (gdb) bt #0 strlen () at ../sysdeps/x86_64/strlen.S:106 #1 0x00007f6ab63050a6 in appIdStatsInit (appFileName=0x7f6ab6628170 <config+16> "appstats-unified.log", statsPeriod=10, rolloverSize=20971520, rolloverPeriod=86400) at appIdStats.c:264 #2 0x00007f6ab62fa2d0 in AppIdCommonInit (memcap=268435456) at commonAppMatcher.c:297 #3 0x00007f6ab6303798 in AppIdInit (sc=0x1eb9770, args=0x1f516e0 "app_stats_filename appstats-unified.log, app_stats_period 10, app_detector_dir /usr/local/lib/openappid") at spp_appid.c:157 #4 0x000000000042048e in InitVarTables (p=0x1eb9770) at parser.c:5728 #5 0x000000000046c3d0 in CheckAppId (option_data=0x0, p=0x0) at sp_appid.c:342 #6 0x0000000000000000 in ?? () (gdb) Quit I had installed openappid as well. On Wed, Mar 11, 2015 at 7:00 PM, Joel Esler (jesler) <jesler () cisco com> wrote: On Mar 11, 2015, at 9:23 AM, Rishabh Shah <rishabh420 () gmail com> wrote: Hi Snort Team, Is it possible to extract any file during http/ftp transactions? The HTTP preprocessor makes it possible to read the HTTP URI/content. Does snort have the intelligence to extract the file during any transfer? Beginning with 2.9.6.0, Snort has had the ability to extract files from streams and write them to disk. Check out the README: https://www.snort.org/faq/readme-file -- Joel Esler Open Source Manager Threat Intelligence Team Lead Talos Group -- Regards, Rishabh Shah. ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news! ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news! -- Regards, Rishabh Shah. -- Regards, Rishabh Shah. ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news! ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news! -- Regards, Rishabh Shah. -- Regards, Rishabh Shah. -- Regards,Rishabh Shah. ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!
------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!
Current thread:
- Re: File extraction during http/ftp transaction, (continued)
- Re: File extraction during http/ftp transaction Joel Esler (jesler) (Mar 11)
- Re: File extraction during http/ftp transaction Rishabh Shah (Mar 11)
- Re: File extraction during http/ftp transaction Hui cao (Mar 11)
- Re: File extraction during http/ftp transaction Rishabh Shah (Mar 11)
- Re: File extraction during http/ftp transaction Hui cao (Mar 11)
- Re: File extraction during http/ftp transaction Rishabh Shah (Mar 11)
- Re: File extraction during http/ftp transaction Hui cao (Mar 11)
- Re: File extraction during http/ftp transaction Hui cao (Mar 11)
- Message not available
- Message not available
- Message not available
- Message not available
- Re: File extraction during http/ftp transaction Rishabh Shah (Mar 11)
- Re: File extraction during http/ftp transaction Joel Esler (jesler) (Mar 11)
- Re: File extraction during http/ftp transaction Y M (Mar 11)
- Re: File extraction during http/ftp transaction Rishabh Shah (Mar 11)
- Re: File extraction during http/ftp transaction Joel Esler (jesler) (Mar 11)