Hi Experts,
I'm a DevOps Engineer for a Tech Services company with 10,000+ employees.
I'm comparing ELK and Splunk. We're looking to use one solution to process logs for our IBM CLM application and for application server log analysis. Which of these two solutions would you recommend and why?
Also, what are the main differences between the ELK open source and Enterprise versions?
Thanks! I appreciate the help.
First of all, we need to understand what those two softwares are; Splunk is a finished SIEM that is mainly used to analyze data, such as logs, net flows, etc. Splunk comes in different flavors, below I will include a link of all the products they have.
www.splunk.com
Some of them can be even downloaded or you can try them in the cloud, below I will give you a link of Splunk enterprise, in the link you can see that you can download it, as a trial.
www.splunk.com
ELK can be used for the requirements that you included, such as log analysis, the difference is that you will have to write the normalizers (this is a configuration file based on regex that reads the raw log and devices the log in small pieces), you will have to write the configuration file of the different widgets in the dashboard, alerts will have to be also written, etc.
Elastic.co has already made an app that works as a SIEM, from all the products I think this will be the one that will make the most sense, as a log storage/analyzer, below is the link and you can try it as a cloud deployment.
www.elastic.co
Also, this is a more complete list of all the features that are included in the enterprise version, here you can check them out and decide if this is something that will work for you.
www.elastic.co
Those two softwares are very good, but it will be better if you give them a try by yourself and try to compare them to see which one is the best for your network environment.
Generally Elastic is very strong in datasearch, and Splunk has a strong security solution. However Elastic has partnered with SIEM provider empow and together this integration provides a very strong platform both in datasearch and next generation SIEM.
Here's a thorough article on ELK vs. Splunk and here's a description from Elastic on what's included in the different versions.
Splunk: hard to use, expensive with predatory pricing, few OOTB rules, SOAR is a premium, good luck training analyst on their platform in under six months. SPLUNK SEARCH.
ELK Stack: easy to use, open-source, no predatory pricing, more robust use cases OOTB, loved and used by millions all over the globe, open ecosystem that can integrate with almost any major IT stack out of the box. LUCENE.
We use ELK or other freeware stacks in isolated small scenarios.
Think of a small or medium company with a „midsized“ webshop. You can easily do your Log management with an ELK-Stack, let's say size 5 up to 10 GB, no Problem. Please keep in mind to order Hardware. The best thing on ELK is that you can start immediately you don't have to wait for licensing and it's easy to build the first small things.
Another Example:
Your Marketing Dep. wants to do some singular evaluations and very specialized marketing stuff. It is temporary and they don't have the budget for licensing. The results are not for permanent use. Just use ELK.
In my opinion, ELK is only cost-effective if you don't need to buy their professional service. You must leave the cases small.
If you are looking for bigger scenarios or you want to build-up a SIEM, SOC or even doing elevated things like SOAR it is a very different kind of thing.
There can be account issues that a developer usually won't mind at the first glance but a Controller will.
You have to look at the Total Cost of Ownership, Scalability, Time to Market, Secureness of future development, maintenance e.g.
If you want to build up a complex scenario with the secureness of scalability you should go with SPLUNK. If tomorrow there is a better tool with lower costs and less need for input of manpower I will refer to this.