Splunk Enterprise Security Reviews

Security. We have built SIEM solutions three times from the ground up (not ES) using Splunk for some of the largest companies in the world.

Out clients went from unhappy using inflexible, poorly-supported products (in some cases barely functionally) to confident and excited when using Splunk. Not only are they able to do their security jobs and investigations, but they are also easily able to modify and evolve their implementations themselves to keep up with the shifting sands, which is the SecOps landscape.

• Core Splunk
• Saved searches
• Dashboards (SimpleXML) 

With good domain knowledge, one can build almost anything. If you throw in Alert Manager or an integration with ServiceNow. Then, you have your own SIEM.

• It needs integration with a configuration management solution. 
• It could use better password management for forwarders. 
• It needs a better way to export dynamic views without requiring a ton of code and user/pw.

Almost 10 years.

Unfortunately, lately every release has a new memory leak.  Be SURE to upgrade late and READ THE RELEASE NOTES, especially the "Known Issues" section.

We only ever have issues when deployed on VMs and the VM admins do not do what we tell them to do which is EXCLUSIVELY RESERVE OUR RESOURCES.

It used to be great (but perhaps that was because my employer at the time was a key prospect in a vertical where Splunk had no customers) but Splunk support is definitely a victim of Splunk's explosive growth.  The first tier support is as bad as it is most places and getting worse all the time.  If you KNOW your problem is not run of the mill, ask for escalation immediately.  Also the clock on the case does not start until somebody adds a note to the case so always call in and ask if they got your diag file (always attach a diag) and the person who answers will have to add a note to the case which will start the clock.

I have dabbled with LogRythm and ArcSight and they are both OK, but Time-To-Value is WAY shorter with Splunk, IMHO.

Use bare metal severs on Linux and you will be fine.  Use Windows and you will have much trouble.  Use VMs and your admins will cheat you and you will have much trouble.  Do not use NAS!!!!

In-house.  We at Splunxter are Splunk experts.  We can do anything with Splunk.  We always hit homeruns.

We usually get multi X-factor within a quarter.

Get free PS if you can (ask) or USE THE DOCS.  The documentation will get you to success.  If you are not getting more value out of Splunk than the license you are paying, then you are doing something wrong and should spend a tiny bit more to get a consultant like Splunxter.com to help you.

No,we went with the free trial and got so much value so quickly we bought in.

You can also get GREAT help at answers.splunk.com.

The primary use case of this solution is to monitor Cyber Mission databases.

I create the diagrams to create an architecture that is then implemented. However, creating these diagrams are for my own learnings since these implementations are usually already available in the cloud office logs.

The features I have found most valuable are the dashboards. 

I monitor the complete capacity that users are using in the company.

An area of improvement would be the licensing of the solution. They need a free license, which would allow faster lead times.

They also need to update their documentation.

The solution is stable.

The scalability of the solution is amazing because it can collect a lot of data and you can have your own structure to monitor this data.

The customer service/technical support was helpful and they answered my questions as best they could.

The setup was easy, but you have to have a VPN connection depending on the security protocols in place.

The deployment was in-house and took about two days with the correct licenses and permissions.

It is important to define different guidelines to integrate Splunk in development, QA, and production deployments. Additionally, define the applications that will be used and the configuration of the databases to collect the data. If this is not done, there will be a lot of issues due to, for example, master access or permissions to use the database collector and blocks.

Hybrid Cloud

Amazon Web Services (AWS)

We use Splunk on-premise. We mostly use it for log analysis and fraud detection. We are also testing using it in machine learning and other solutions. We have 10 people managing Splunk and we have approximately 150 people using the product in total.

With Splunk, we got more insights out of our data as it includes machine and secure data. It also has a logging attendance system and this helps to protect our resources from any  attackers hacking system information at a granular level

The logging features are useful as are the dashboards and alerts in addition to the organization of data. It has options for creating dashboards and alerts. You can also create queries in the SQL language. Splunk is a user-friendly solution.

Index performance is a bit slow but this is partly due to the huge volumes of data for our industry within our environment This makes the index very large and inefficient in terms of performance. Performance could be improved to cater to this, however. We have also had problems with the compatibility between Splunk and other systems. We have previously been on 5.3 and migrated to 5.5. We are now planning to migrate to version 7.7. It has been difficult to find documentation about the compatibility with Linux. In terms of the interface, it could include some improvements for the look and feel.

We have been using Splunk for one year in our infrastructure environment.

The users access the native cloud solution. So we are taking advantage of the native cloud solution provided, and by using the gentle scaling approach this has helped stability.

We scaled up gradually from three processes up to five, and the performance is okay. So we used gentle scaling  but this also helped stability.

We have used Splunk tech support often. If we have a critical issue such as server down or frequently occurring issues they are always reliable and provide us with solutions to our problems. Technical support for Splunk is good.

Setup is complex. We tried to cluster five indexes. This helped us migrate our data into the Splunk environment. We are using 20 applications which make use of this indexed data. The actual deployment took us about two to three weeks because of some problems getting the data into the system.

We worked with a Splunk consultant who shadowed us to help ensure we performed the process correctly. 

Licencing occurs yearly. We now have a three-yearly support contract as of now.  Licensing is a yearly, one-time cost.

We considered a few alternative products because the logging was faster. In the end, we decided to go to Splunk.

I would definitely recommend Splunk. We will review performance within two years of our three-year contract and then decide at that point what other aspects we need to consider. I would rate Splunk 8 out of 10.

On-premises

It's the primary place where I'd go to do an investigation if I want to see what's going on within an endpoint, or on a network, or with a user.

The flexibility of the solution is quite good.

The product is stable.

It offers good scalability if you are willing to pay.

The technical support on offer is responsive.

The solution has a high learning curve for users. It's a little complicated when you're trying to figure out all the features and what they do.

The solution needs a bit more functionality. For example, being able to save a search and select it when you're doing an investigation. I know you can create dashboards and things like that, however, sometimes being able to have a pre-saved search and just fill in whatever value you need would make everything so much easier.

I've been using Splunk for four years so far. It's been a while.

I haven't had any stability issues with it. It's pretty stable. There aren't bugs or glitches. It doesn't crash or feeze.

You can scale the solution, however, users need to be aware of the product increasing in cost as well.

The technical support is very good. We're quite satisfied with the level of service provided. They are knowledgeable and responsive.

When I came to the company, they were already using Splunk. It's only now that we're looking to possibly move to another vendor. The cost of Splunk is much too high.

I wasn't here when this solution was put into place, however, from looking at the documentation and things like that, the setup is pretty involved. I'd say it's a bit more complex than straightforward.

We find the solution to be quite expensive. Therefore, we're looking for other options.

I don't know of the exact costs, as licensing is handled by another department.

We're just users. We don't have a business relationship with Splunk.

We're on a variation of version seven. I'm not sure of the exact one. It's not quite the latest.

I'd advise new users, if they have the budget for it, to go and take the training that they offer. Or, for casual users, you just want to spend as much time watching YouTube videos as you can. It will help lessen the learning curve.

As a solution, it's still pretty much industry standard. I would give it a nine out of ten overall, even though I have my gripes with it.

We need something to collect all our logs in a centralized solution. We have several servers but we don't have any log collection system.

Without Splunk or a similar product, if I want to check the log files every day, I have to log in to the individual hardware components in our system. I have to log in to the firewall, I have to log in to Windows. There are so many devices I would have to manually log into, one-by-one. It would take a very long time for me. 

Also, we don't have a dashboard so we don't know which issues are critical. When we use a centralized log monitoring system we can see things on the dashboard and it is easier for the IT manager or an IT engineer to take corrective action in the system.

The most valuable feature of Splunk is the log monitoring.

If possible, we would like to have not only a log monitoring system but a network monitoring feature in this solution as well.

It's very stable.

Up until we trialed Splunk we did not have any solution. We used Splunk because we don't have anything to monitor our system. I contacted our local vendor in Vietnam, and they suggest using the trial version of Splunk to see how it works in our environment. This is the main reason I trialed Splunk. We just used the trial version in our office and, since it expired, we haven't used it.

For me, the initial setup was not too complex. For an IT person like me, it was okay.

Our local vendor knows Splunk very well. He had already implemented Splunk for another customer. I called him to our office to have him install the Splunk. It took a couple of hours for him to finish.

We used a consultant for the deployment, from KDDI Vietnam. Our experience with him was good.

Because it was a trial version, I was the only one who used it in our company.

I kept some snapshots from our trial with the Splunk system and we are preparing a proposal to submit to our manager in Vietnam. If in the near future we have enough money to purchase the system, we will invest in this system for our company.

We use it for searching logs in a production environment.

It has reduced the time to resolution, time to investigate, and time to troubleshoot for debugging issues. 

Being able to search across all the different production environments at the same time, then being able to do search queries to scope out specific environments, specific components, or specific logs from different languages, such as Java or C++. Thus, being able to have really fine grain control on log searching is really good.

Out-of-the-box, it seems very powerful.

The search query seems slow, but I am not sure if that is just because it is searching millions upon millions of lines of text. Also, I just started using it, so I might have no idea what I am doing. I could probably speed up the queries by improving my search skills.

My company could benefit from doing more Splunk training with Splunk consultants teaching us how to use it. It is possible that we have already done this and I haven't participate, but this type of training would be helpful.

Less than one year.

It is always up when I need to search. I am probably not using it that much. I will maybe search a couple times a day for something specific, so I am not using it too much. I know plenty of the people who are doing a lot more for debugging, and who use it a lot all day.

It seems like it scales well. We have hundreds of production and development environments, and we are searching on all of them. Therefore, it seems like the scale is good. 

We have hundreds of production environments, and each production environment has ten to 20 host machines. Each production environment can manage tens of thousands of customers.

Maybe going to AWS and scaling it better would be more cost-effective for our company. However, I am not involved in those decisions.

I have not used technical support.

We have other log searching tools, but we have standardized on Splunk. 

It is a great product. We have a lot of different tools to do this type of debugging. Yet, it is one of the first ones that I will reach for, and I think that is a good sign.

It works well and is the industry standard for log searching. It probably has other features too. Therefore, if you use it, I would recommend the training, so you know what you are doing. 

I am using the on-premise version.

The primary use case is for log analytics. Although, we have been using it as a hammer which hits all the nails. We have sort of overused it in some areas where it doesn't need to be used.

We have a one stop dashboard for health of some of our services where you can click in and it takes you to other dashboards that have custom near real-time metrics that show the application's health. From there, you can drill in to see the real deep dive example of what is happening in your environment. It has reduced our time to resolve incidents. 

The most valuable feature is its centralized log analytics.

The historical data extraction needs improvement. I would like the capability of taking data and having it trend longer. Splunk is good about viewing data within the last seven or 14 days, but if you want to see a year-over-year trend, you have to do a lot of work to get to that point. If there was a better way to extract the data point and put it into a long-term viewing ability for a year-over-year analysis, then compare that to your other business metrics. That is what I am looking for, as an example, for a call center you want to see the time it takes for your customer to be handled on their need comparatively to the system performance that is happening, then overlay that data. 

Three to five years.

We put a lot of trust in it. It has been pretty rock-solid outside of a couple of changes that we made. Upgrades sometimes don't always go smoothly, but otherwise the system performs, and operates. 

When we were trying to implement an enterprise solution on-premise, we had scaling issues. It was very difficult to search the data retention beyond a few days. A lot of talent was given to the ability to go into AWS and scale with our need. We still had to do some administrative things to prevent consumers from trying to search all records for all time in very inefficient searches. This could sometimes bring our core system functionality to a halt, so we had to do some user administration in it.

I don't engage with the support directly. Another member of my team does. Any time that we have needed support, he hasn't had an issue opening a ticket and receiving the help that he needs.

The integration and configuration in the AWS environment was pretty good. They have a consumption method for pretty much every service. They might be able to do a little better at advertising different patterns for best practices for different service, but overall there's a method to get everything.

We have had a reduction in the time it takes to resolve issues and correlate what has failed. This has significantly helped.

We looked at the Elk Stack, Kibana, and Sumo Logic.

We chose Splunk because their cost is better, the maintenance factor is a little higher, and the core functionality is higher than what other products provide. The core functionality is out-of-the-box. E.g., with a Toyota Scion, you can customize the parts to make it whatever you want, but it's a lot of work to get there. Where if you buy a Cadillac, you pay the Cadillac's price, but it's a Cadillac. It will work right out-of-the-box.

It works well when searching logs. If you looked to try to do things beyond this, the problem that we ran into is that we treated it as the hammer which hits all nails. That is not really feasible, and there are other tools out there that can do more specialized things.

User administration is key. Trying to prevent users from being able search records all the time is a huge problem. You need a tight approval process on dashboards, making sure the dashboards are queried in the most efficient way possible. 

The on-premise version that we had was not scalable at all. It was very difficult to use. We have EC2 instances in the cloud with Splunk installed, which is more scalable and easier to use. It now works much better.

Splunk – ease of searching large amounts of data. 

Splunk – real time alerts on critical indicators, compliance reports, troubleshooting and predictive abilities using trends. 

Splunk – 3 years 

Splunk – Had one issue requiring a support call regarding the configuration of the automated configuration deployment package. Quickly resolved. 

Splunk – None. 

Splunk – Not needed yet. 

Splunk – Splunk has a very knowledgeable support staff and the Splunk support website is outstanding. The message boards are very active and often using them will often prevent having to call support. 

Splunk – Easy, but can get very complex depending on the type of logs to ingest. While Splunk, out of the box, handles most common types. The extraction of data from custom logs can be problematic. Although Splunk does provide tools for accomplishing this. 

Both Splunk and LogLogic excel at their intended purpose. If you are looking for an appliance that you can stick in the rack, minimally configure and then forget about, you will like the LogLogic solution. If you need to regularly search different logs for different data you will like Splunk better.