Splunk ITSI (IT Service Intelligence) Reviews

We use Splunk ITSI to monitor the different stages, spaces, and processes of payment operation.

Splunk helps us improve our incident response time. We have a dedicated observability monitoring team that continuously monitors our systems for failures or delays in payments, 24/7. This monitoring generates alerts that we use to identify potential issues. We have established SLAs for all of these issues. Splunk allows us to alert the appropriate people well in advance of a potential breach, so they can resolve the issue faster and minimize downtime.

I would rate Splunk's predictive analytics for preventing incidents an 8 out of 10.

Splunk ITSI has helped reduce our mean time to resolve.

We use ITSI in the health industry. In the UK, the NHS currently uses ITSI as one of its monitoring sources of information. In ITSI, service components are based around each area of the NHS. For any solutions that have been digitally transformed and require monitoring related to our vaccination campaigns, the logs are ingested through Splunk and monitored through ITSI.

We realized ITSI's benefits immediately after it was deployed. When the COVID pandemic broke out, it kicked off a lot of crazy stuff within the UK. Having a powerful tool to aggregate data and allow real-time monitoring helped our campaign.

ITSI can help us right-size resources, but it depends on how you do things. We have a culture, and Splunk told us not to do this because they have different methods and stuff. In ITSM, you skim what you need at the source and then push that into Splunk. Having that as the centralized logging analytic is great for that, especially when so many things are tied to ingestion, storage, etc. However, for what we do, it leaves much to be desired. You're talking about an enterprise solution on the scale of the NHS with multiple people, contractors, and all these moving parts. Some services do it well where they only send in what you need. Some services just dump everything. You've got a load of load of logs. We can right size appropriately, but it's just yeah. For us, it's it's not really done now as well, I think.

ITSI has helped us streamline our incident management. We have a 24/7 service team working around the clock, responding to alerts that Splunk produces. It's linked to ServiceNow, our service management tool. When the team inputs all the information from Splunk into these tickets, they're raised in ServiceNow. Previously, we used software called Cherwell that looked horrendous. This helps bring the package together.

We've reduced our alerts, but it requires a conscious effort to configure them. That depends on how you use the platform. It goes back to getting the right metrics out of the logs that you're producing. The tool itself is powerful, but if you don't use it properly, things can be a bit noisy, and this is quite noisy, whereas that's down to our configuration sometimes.

Reducing alert noise also takes some tweaking. You've got KPIs and correlation searches that are great for real-time monitoring, but if you set them up immediately, you will get a lot of noise anyway. It depends on how you configure it. They have a couple of tools in the forwarders to say you're only ingesting alert logs or error logs, so you pick up on whatever those error logs would trigger.

It would help to give you accuracy in your ITSI alert noise. However, it might get a bit noisy if you've got more than that and they're not configured into the perfect use case you need. Overall, it's been a conscious effort to ensure we've got our stuff configured right.

It has reduced our mean detection time. For Microsoft/CloudStrike stuff, we can have an SLA as short as three minutes. The feeds are coming in quickly, so our detection time is between three and 10 minutes. For major outages, an SLA of a few minutes is good, especially when it's not a cyber-level threat. 

The resolution time is determined by how quickly we can pass the detection along to the IT team and triage the logs to determine the issue. We've had quite quick resolutions because everything's partitioned in a way where it is specifically service-bound. You can look through the data and specific areas. You can optimize these things.  The search system in Splunk is powerful and helps speed up resolutions. 

ITSI helps to automate routine tasks. That's what the safe searches are for. It's a complete package with Splunk Cloud and ITSI for deeper drill-downs, but not everyone can access the ITSI dashboard all day. Automation helps us get these alert structures, especially at night. When you've got a file that's meant to come in at 3 a.m., you don't need someone waiting around to look at that. 

This is what those alerts and automation are for. You can put custom wrappers around stuff. It's a custom output. However, Splunk is trying to make something more standardized at the moment. It saves our IT services multiple hours a week because you don't have to do tasks or sit and look through dashboards to ensure everything is all right. These constant checks every five minutes add up over the week, so that equals tens of hours a week for a lot of different services.

I have experience utilizing Splunk ITSI in financial institutions and federal government settings. As a Splunk administrator at a bank, I focus on the platform's administration and development aspects. We are migrating from an on-premises environment to the cloud, leveraging Splunk ITSI to provide a unified view of the client's infrastructure. Through ITSI-generated reports, we are developing a strategic roadmap to guide our clients' IT journey.

End-to-end visibility simplifies our configurations by allowing us to index or search at the cluster level. We can utilize multiple indexers or split the workload as needed. For instance, long-running queries exceeding 15 minutes can be removed from the main list, improving efficiency for other users.

Splunk ITSI is a powerful tool for predictive data analysis. When we create and test KPIs within ITSI, it becomes significantly easier to set targets. For instance, if a system's memory capacity is 100 GB and usage consistently approaches or exceeds 80 to 90 percent, ITSI can generate alerts, visualize in a dashboard, and send notifications to the team. This proactive monitoring prevents potential issues. Similarly, ITSI can identify performance bottlenecks in search queries, allowing workload distribution to optimize system efficiency. The entire environment becomes transparent, simplifying tasks for developers and users alike. Regarding user criteria, ITSI offers a tree diagram visualization to easily understand data distribution across indexes, source types, business units, states, and communities with a single click.

Splunk ITSI enables us to allocate resources more precisely to meet demand. Its unified view provides full information in one location, allowing me to monitor index CPU and memory usage, injection rates, and individual user data. While gathering this information might take around ten minutes, the streamlined process significantly simplifies my work.

Splunk has significantly streamlined our incident management process. Its ability to analyze usage, memory consumption, and other environmental factors makes it superior to other tools, allowing us to delve deeper into complex issues. Regardless of length, we can effortlessly examine any log and pinpoint the exact cause of problems, such as UI errors or system failures. We can quickly identify code changes, root causes, and error origins by simply writing a query, providing invaluable insights that accelerate problem resolution and enhance overall system reliability.

It has been instrumental in reducing the overall volume of incidents by automatically triggering alerts when potential issues are detected before they escalate into full-blown incidents. This proactive approach simplifies data analysis and enables us to identify and rectify errors before they impact our systems. Consequently, we can more confidently implement changes or updates without fear of unforeseen complications, as ITSI helps us prevent errors from occurring in the first place.

Splunk ITSI has helped reduce our alert noise by ten percent and improved the mean time to detect down to ten minutes.

Our mean time to remediate is less than one hour when using Splunk ITSI.

We've implemented automation using Splunk, replacing multiple tools previously used for backend testing. We integrated Splunk with ServiceNow to automatically send alerts to the team whenever issues arise. This eliminates the need for manual ticket creation and assignment, streamlines the process, and ensures timely responses, saving us around ten hours weekly.

Splunk has helped us significantly reduce downtime, manpower costs, and the penalties for missing service level agreements. Previously, we relied on two to three people, primarily from the testing team, to manage these issues. By implementing Splunk, we've decreased staffing needs while improving workflow efficiency and reducing overall costs.

We get our customers' requirements and onboard their logs into the SIEM tool using agent-based integration or some DB Connect method. After the integration, we write the use cases. There are two types of data: fault monitoring and performance monitoring. In fault monitoring, the customer typically wants every event as an alert, so we'll do a correlation search for that alert. 

We'll add fields to the alerts, such as summaries and descriptions, and write the regular expression from the raw event to extract and display it on a table. After writing the correlation search, we will enable the policy that we'll use to trigger an incident in the ITSI tool. In our Splunk tool, there is a technical add-on called Remedy that we use to create a ticket for a correlation search and alert. 

After writing the NEAP policy, we'll display the number of tickets and all that information in a single dashboard. In the first panel, we'll display a summary of all the applications and the number of tickets divided according to the severity. The second panel displays the alert information, such as the ID, reported date, and the host. 

We have a team of four people. Two integrate the log sources into Splunk, while two write correlation searches, enable the new policies, and generate tickets in incident service with the ITSI tool. They also work on the dashboards, tables, and service analyzer.

With Splunk ITSI, we don't need to manually raise tickets for analysis. For example, it will not trigger a ticket if we receive an alert about a suspicious event when a set of conditions are met, but it's an invalid alert. Based on a NEAP policy, an incident will be created for each valid alert with the help of our ITSM tool. Each NEAP policy has two components: filtering criteria and action rules. The filtering criteria include sections. If the alert source equals the application log monitoring, it will group that particular event. 

For each event, it groups by incidents based on the job ID, and we write conditions for the second-action rules. The incident ticket remains in progress if the event exceeds one and the status is not closed. It shouldn't create a second incident for the same job name.

Splunk ITSI helps customers to reduce their resources. For example, they don't need extra resources to raise manual incidents for each alert. This solution enables us to raise incidents for only valid alerts, and it displays them all in a single dashboard.

It doesn't affect the effectiveness of the application monitoring, but it decreases the resources and associated costs. It will improve the performance compared to raising incidents manually and reduce human error. 

ITSI reduced the time needed to create a ticket. Instead of raising a manual ticket, we can automatically create one after an alert is triggered based on our policy. We can see all the incidents and alerts on the dashboard. 

It has also reduced the volume of incident alerts. We sometimes raise a manual ticket for the same alert triggered yesterday or a few days before. If it is not closed, and we raise another incident by human error. We can write a new condition so that an alert name by the same name will produce no new tickets. It will update the ticket as "in progress" or change the severity from minor to major.

We can also reduce our alert noise using ITSI by writing a complex set of specific correlations. We'll write the exact conditions based on customer requirements. For example, we'll use Windows event ID 4625 for a failed login attempt. If a user wants, we can add the search criteria so only this event ID will be triggered. 

When integrating our customer logs sources, we directly integrate the real-time events into Splunk. There is no time difference from the customer side. It goes directly into Splunk ITSI. Previously, we used some integration method so that when an alert triggers in EMS, it will reach out to Splunk to create an incident within a minute.

We send the artifacts, logs, and analysis to an incident response team to resolve an incident. The response time depends on the team. They receive all the evidence about an alert. 

ITSI helps automate some reports and dashboard features. When we want to run some individual searches, it takes some time to run each search to generate a report and share it with the customer. We can add all these reports into a single dashboard, and we have a query for each report. We add all these queries into a dashboard and schedule the reports, so it generates a report daily showing all the graphs. We can download that report and share it with the customers.

When we automatically generate a ticket based on the alert, it reduces the detection time and makes the ticket-raising time nearly instantaneous. The time difference between the alert trigger and ticket creation time will be minimal as the machine is generating the ticket. The customer response time is five to 10 minutes. 

Splunk ITSI has been a great aggregator for creating dashboards for all our app teams when we ingest logs.

Splunk ITSI has been the central location for log aggregation and information via dashboards.

Splunk ITSI is used to analyze data and create alerts. This helps us to maintain our security best practices.

Our organization was looking for a security monitoring tool. I use Splunk ITSI as a monitoring and security tool. It helps me to protect data and prevent malware and hackers from accessing my environment. Splunk ITSI can be used to protect our role and infrastructure. It can also provide insights into how and what is helpful within our infrastructure.

Splunk ITSI provides great end-to-end visibility into our network environment. It can identify the exact root cause of an issue without any additional troubleshooting on my part.

Predictive analytics is valuable for preventing incidents before they occur because it allows me to see when the data stopped being indexed, which saves me time from having to investigate.

Splunk ITSI makes it easier to secure our entire infrastructure. Before Splunk ITSI, our environment was chaotic.

Splunk ITSI streamlines our incident management by providing a financial report of all applications in our environment.

Splunk ITSI has helped us reduce alert noise. After configuring ITSI, we set certain parameters based on our alerts. These alerts are the conditions that ITSI uses to automatically reduce noise.

Splunk ITSI helps to reduce our mean time to detect by monitoring key performance indicators such as CPU overload and the percentage of use revenue trend. On average the automation has reduced our mean time to detect by five minutes.

Splunk ITSI significantly reduces our mean time to resolve because most of our time was previously spent troubleshooting. With ITSI, we don't have to troubleshoot at all.

Splunk ITSI can help reduce downtime, but the extent of its effectiveness depends on how it is implemented.

In my recent projects, we have used ITSI to monitor the entire infrastructure using multiple features, such as service KPIs, aggregation policies, base searches, correlation searches, notable events, dashboards, blast tables, service analyzers, and drill-downs.

It helps in every respect, including performance, monitoring, or visualization of the important indicators. It improves the quality of service to the clients. It is crucial that the clients have no website failures because that means the loss of business. ITSI helps us track those issues. We've seen fewer environmental failures since we started using ITSI.

We saw immediate benefits from Splunk ITSI. For example, let's say you have a project for monitoring hybrid Linux servers running JBoss, SAP, and any server containing a client's critical data. It isn't easy to monitor each of these through the back end. 

Splunk ITSI shows you all the data on the screen and lets you visualize the data from various applications. We can see all the applications running on the server and issues with CPU or memory utilization. We have that data in Splunk and can immediately see the alerts triggered. If there are any failures in the environment, we can fix them in seconds. 

The solution has helped us streamline our incident management. We can monitor server KPIs, which trigger an alert if the server is impacted. We can track all the notable events and integrate ServiceNow with Splunk. ITSI is integrated with the ticketing tool, so when an alert triggers, it automatically creates a ticket on ServiceNow. 

ITSI has also reduced the alert volume. Before ITSI, we were unsure why an issue happened. We would see the alerts triggered in bulk and log them one by one for every server. ITSI gives you a feature that lets you drill down to find the precise issues on the server. 

It has a service KPI feature that allows you to monitor exceptions that may lead to server failure. For example, we might be in trouble if the value exceeds 10. We put five or eight values in the threshold field with a high criticality, so it triggers an alert whenever the count is breached. 

ITSI reduced our alert noise because it was very hard to monitor every aspect when we used search and reporting. After running the query, we needed more insights, and ITSI gave us a clearer picture of the incident. That helps you reduce issues.

Many use cases can be automated through ITSI because we previously built our reports manually.  After introducing ITSI, we sent all the data via the forwarders to Splunk. Once we have the data, we create and schedule all those queries and reports so that the management can see them without any IT involvement. It previously took us two or three hours daily to create all those reports, so automating reports saves almost 60 hours each month. We're automating 10 to 15 daily.

Splunk ITSI is a service intelligence platform that monitors services, availability, endpoints, and interactions within an environment. My experience with ITSI focuses on web application APIs. I installed and configured it for a telecommunications company to monitor web application API services, troubleshoot downtimes, and mitigate failures. ITSI offers a comprehensive view of the environment, enabling top-to-bottom visibility into services, endpoints, and performance. It provides correlation analysis, deep dives, and episode reviews, leveraging AI and machine learning algorithms to detect signals, predict issues, and prepare engineers for potential problems.

Splunk ITSI's dynamic and highly beneficial end-to-end visibility allows us to gain comprehensive and clear visibility once we configure our settings, services, and entities.

Splunk ITSI's machine learning and AI capabilities are powerful tools that help prevent incidents before they occur. As an engineer, I appreciate the ability to visualize potential future scenarios within my environment. This predictive forecasting feature provides valuable insights into our environment and services.

Due to its complex functionalities, Splunk ITSI requires significant learning. Proper training is essential to understand how these features operate effectively. While the benefits were not immediate, they became apparent over time as we configured, implemented, and utilized the various functionalities. It took several months before the full value of Splunk ITSI was realized.

For incident management and incident response, ITSI assists us by enabling us to create numerous knowledge objects as Splunk users. Whenever an issue arises, these objects can be centered around our services or entities, such as reminders, emails, or notables. Consequently, ITSI significantly aids our management and incident response efforts.

Splunk ITSI effectively reduces the volume of incidents by providing predictive capabilities, enhancing environmental visibility, and facilitating efficient troubleshooting. This deep-dive approach minimizes the occurrence of noisy alerts and consequently lowers the overall incident rate.

It helps reduce alert noise by allowing users to review and group notables. Through the episode review functionality, analysts can examine fired alerts, assign them to specific investigators or analysts, and group them to minimize the occurrence of noisy alerts.

Splunk ITSI has been instrumental in reducing the mean time to detect. While I have other tools as an engineer, ITSI, in conjunction with Splunk SOAR, offers preconfigured automation and quick responses that can further enhance our MTTD. ITSI provides the necessary visibility, and when integrated with SOAR, it aids in detecting and resolving issues more efficiently. These tools work seamlessly together, streamlining our incident response process and improving operational efficiency. Combined, our MTTD is under 30 seconds.

Splunk ITSI has helped reduce the mean time to resolve the issue because we can detect the incidents faster.

It is a valuable tool for cost savings. In a recent project involving web application APIs, ITSI's top-to-bottom visibility and machine learning capabilities enabled us to predict and prevent downtime, reducing losses significantly. By integrating ITSI with an automated tool like SOAR, we implemented automated responses that quickly resolved issues and minimized disruptions. This resulted in substantial savings, estimated to be between five and ten million dollars. Before ITSI, downtime in the web payment application APIs was frequent, leading to significant financial losses. ITSI's implementation has eliminated this issue and provided substantial cost benefits between five and ten million dollars.