Recently, our user activity has shown that Splunk is the most commonly searched solution on our site.
3,643 of our community members follow Splunk, and it's listed in five of our product categories: Log Management, Data Visualization, IT Operations Analytics, and Security Information and Event Management (SIEM).
What are some of the best features and use-cases of Splunk, and why are people explicitly searching for it to learn more?
I agree with Aaron & Tom on their points. Along their use cases, I have been able to show more than Log data in Splunk views. We tested several plug-ins during a small pilot, and we were able to bring O/S (Win/Unix/Linux) & APM data metrics into the same views as Logged data. I've seen others use it to visualize a wider range of data types, too.
That said, Tom's point resonates with me. Their are better tools for visualization (ZoomData & Kibana come to mind), but as an aggregator Splunk has the most plug-in types out there. IF (big if) you have the $$ to support ingesting everything, you could theoretically pull data that lives in 40 or 400 source tools and thousands of hosts/systems into a single set of enterprise views. I am not fortunate enough to have that kind of budget though... After proving the concept in pilot, we had to dismantle our 'unified views' due to lack of funding.
First, I agree with the comment that any solution is only as good as the people who use it. If you don't have the right people managing and using it, then it will be essentially useless. If you assign people with no interest in Splunk, to lead the effort, then you will probably fail. As I say for projects, you need the right people in the right roles.
Splunk, like many other software packages, is a tool. A tool must be in the right hands to be effective. You need to understand data at a low level to truly extract the power of Splunk, especially with pattern matching (Regular Expressions) which is an extremely important part of data extraction and transformation in Splunk.
Most of all, you need a list of use cases, before you get started with Splunk. Why do you want to use it? What use case do you have? What problem are you trying to solve? Just saying you want to "manage log files" is not a use case.
Here are 5 basic ways to use Splunk (just remember, a "search" is equivalent to a SQL statement, but in Splunk terms).
1. Manual search - simplest case. This is what people mean when they say "analyze log files".
2. Scheduled search (Example: if 1 or more results of a saved search, send an email to XYZ)
3. Single report - this is a saved search + visualization. Can be a table or a chart.
4. Dashboard - a collection of saved reports
5. Dashboard - automated PDF attachment email (snapshot of Dashboard in PDF for, sent at scheduled times). For example, a beginning of day or end of day report could be useful case.
It can be a daunting task to understand your data (not just logfiles) across the enterprise. Splunk can be a part of the solution, but you must have the right people to lead the effort.
The flexibility that it offers, One of the most powerful features of Splunk is its ability to extract fields from events when you search, creating structure out of unstructured data. It takes a small amount of “learning time” to start creating or getting searches that are meaningful to you. You can start “splunking” for free, which allows you to see the benefit. There is a ton of resources on the web, uses cases, and step by step instructions.
Good log management solution you can use if you know what you ae looking for. Not a SIEM solution though even though customer should be aiming for solutions that go beyond what a SIEM does, that is, a Security Intelligence platform.
Splunk,is a great log management tool and has various builtin features that ,a user can use in order to present in a varied ways that make sense to
a) Business
b) Techno Functional community
Competitors (you have lots in the market, naming a few I am aware of):
a) IBM Smart Cloud Analytics - Log Analytics
b) QRadar
The pros of Splunk are :
a) multiple log format categories supported ( from top of my head, around 7 categories).
b) Lots of search commands and customization
c) Visualizations (Maps, Charts,time charts et al)
d) Ease of use (less learning curve)
e) Realtime searching
f) Custom dashboards are easy to make
g) Lots & lots of documentation .
h) Great community that comes handy for troubleshooting(answers.splunk.com)
Cons :
a) License & IP is tricky
b) License is based on the volume of data indexed/day.(So if you have 10 events in a 10 GB log file,you still have index all the 10GB ,to filter out the 10 events)
c) Authorization is poorly mapped against roles. Very less control over what the users can use and cannot
Use cases our team implemented as a PoC :
a) Used perfmon data to map cpu usage & mem usage ,to show the performance of web services during Peak seasons
b) Parsed IIS logs to map the usage of a specific web based tool ( hits,http status codes,uri against time)
Splunk originated as a centralized log index platform ingesting machine data from a variety of sources. They now have a hosted cloud option called Splunk Cloud, but the main use case in via an on-premises installation.
Splunk indexes on pretty much any field in the unstructured text and can give it structure. For example, for a log message with "error occurred: key1=value1, key2=value2", the key/values can be extracted out so that a user can perform complex searches, aggregations, etc. You can even use regex.
Now, Splunk can ingest from a variety of sources from log data, to time series metrics data, to sensor data like in Iot, etc.
A good overview of various logging options:
www.moesif.com
easy to install and easy to use. you can implement many of the use-cases to comply the company to the known standards.
Any hardware or software solution is only as good as the people who use it. The short version is that Splunk improves the effectiveness and efficiency of skilled cyber defenders in developing actionable intelligence from their log data.
In extremely general terms, log data is machine readable and human incomprehensible. Splunk allows the user to create searches to find and extract the meaningful data, providing feedback to improve and refine the searches. In other words, locate the meaningful 1s and Os and separate them from the vast majority of Xs ("don't care" data).
Splunk is not an SEIM solution. While it can improve visualization of log data, it is not really a visualization solution, it is just an improvement over much of what is available.
Bottom line: Splunk, as any other solution, is only as good as the skilled and experienced personnel who use it.