Educause Security Discussion mailing list archives
Re: Open source SIEM
From: Kevin Wilcox <wilcoxkm () APPSTATE EDU>
Date: Tue, 11 Feb 2020 15:56:53 -0500
On Tue, 11 Feb 2020 at 13:49, Zepu Chen <zepu.chen () denison edu> wrote: We are researching the possibility to implement an open-source SIEM
solution at our University. The project we are currently reviewing is MozDef from Mozilla. Does anyone currently have MozDef or other open-source SIEM implemented in your environment? How are the implementation and operations experience so far? We are interested in seeing what other schools are doing. We would greatly appreciate it if you would be kind enough to share any pitfalls, constraints and roadblocks as well as implementation recommendations.
We have been big fans of Elastic (formerly "ELK", aka "Elastic Stack"). We're still on the smaller side, yesterday we consumed about 425 GB of data (about 525 million logs). Our outlay has been in server hardware but we use "free" Elastic. I'm happy to talk deployment specs if you decide you're leaning that way. You're going to run into some myths commonly spouted by "Big SIEM" and those who have succumbed to their propaganda =), regardless of what you decide to do, so let's put some of those to rest. 1) Going with an open source SIEM is a time sink The reality: SIEM is a time sink, regardless of the product. Log aggregation is a time sink, regardless of the product, if you want it to be effective and efficient. If you go with Splunk + ES, you're going to have to deploy Splunk and learn how to manage it - and then you're going to have time on top of that learning ES and helping it normalise your custom logs and working on translation tables and... There is no useful commercial vs OSS comparison here because you're going to need to invest human time in learning how to run whatever you get. 2) It's unsupported The reality: most major OSS SIEM vendors have paid support options - Elastic offer it, GrayLog offer it, AlienVault offer it. You want it? You got it. 3) Nothing integrates with it The reality: most things speak syslog/event channel or can output json, and that's really all you need. Working in a Windows shop and things log to event channels? No problem. A Linux or Unix shop with syslog/rsyslog/syslog-ng? Fine, you're covered. Custom application writing a text file or to an event channel? No big deal. Don't like the agent your OSS or proprietary SIEM uses? Fine, use NXLog (and you can pay them for support) - it speaks syslog and reads from event channels, can encapsulate in json, lets you filter noise on the endpoint. Integration isn't the problem, folks who "broken record" about no integration without testing the waters in the last decade are the problem. 4) It can't handle the load/it's slow The reality: see my 425 GB of data, 525 million events yesterday comment. We're a small shop. You should see the numbers for some other Elastic schools...or for companies like Mandiant, for that matter. 5) The system administration costs are too high The reality: unless I'm patching the OS or doing an Elastic version upgrade, I don't do sys-admin work - and most update processes should be automated anyway. OS updates + reboots across the entire environment take about an hour total - the bulk of that is waiting for systems to finish booting. We could cut that down to fifteen minutes but stretching it over an hour+ lets us do reboots without affecting usage. Yes, I was a Unix/Linux admin in a previous life so I make sure that's a skill our SOC keeps. 6) It's too much effort to tune The reality: see 1 and 5. Tuning of _any_ SIEM should be an ongoing process. The biggest reason SIEMs fail is lack of executive support (read: they don't devote the necessary resources), the second biggest reason is that people think they're "set and forget" (arguably related to reason number one...). They're not. There's a large oil company with over 500 people devoted to maintaining and tuning their proprietary SIEM. Expect to have someone devoted to SIEM "stuff", regardless of what you do. =============== All of that said, there _ARE_ legit criticisms of Open Source SIEMs. Most of them have horrible, or no, out of the box dashboards or meaningful alerts. Take a look at things like DSIEM, HELK, SecurityOnion and the Sigma project (for alert normalisation and sharing in a common syntax). There are options missing in the "free" versions - Elastic, for example, don't give you access to their granular permission system, SAML or their "machine learning" options unless you give them money, but you can build a 500-node cluster consuming petabytes per day if you can get the hardware for it. Okiedoke, that's me off my soapbox 8^) Good luck in your search!! kmw ********** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community
Current thread:
- Open source SIEM Zepu Chen (Feb 11)
- Re: Open source SIEM Cleary, Kevin (Feb 11)
- Re: Open source SIEM Rogers, Zach (Feb 11)
- Re: Open source SIEM Max McGrath (Feb 11)
- Re: Open source SIEM Kevin Wilcox (Feb 11)
- Re: Open source SIEM Kimmitt, Jonathan (Feb 11)
- Re: Open source SIEM David Eilken (Feb 12)
- Re: Open source SIEM Powell, Andy (Feb 12)
- Re: Open source SIEM Nevin, Dave (Feb 12)
- Re: Open source SIEM David Eilken (Feb 12)