Splunk On-Call Reviews

I have been using Splunk On-Call for nearly about two years. Our main use is incident alerting and on-call scheduling for our engineering and DevOps team. Basically, whenever something goes down, a service outage occurs, error rates spike, or deployments fail, Splunk On-Call routes the alerts to the right person at the right time. We also use it heavily for escalation policies. If the first person does not respond within a few minutes, then it is automatically paged to the next engineer in the chain.

The first feature I like in Splunk On-Call is the smart alerting route. Basically, it automatically sends an alert to the right person based on who is on call, not just blasting the whole team. The second feature is the escalation policy. If nobody responds in five to ten minutes, then it automatically pages the next person in line, so no alert ever dies silently. After that, there is one feature that I appreciate most: the multi-channel patching. In the future, Splunk On-Call can page you via phone call, SMS, email, Slack, or push notification however you want to use it.

It saves a lot of time overall. Before Splunk On-Call, we did manual things managing the call routing and the alert routing. After Splunk On-Call, we automatically send the incident to the respective team, which saves a lot of time for us. It is very impactful for our whole team. We save nearly about twenty to twenty-five percent of our time.

A few things I would improve in Splunk On-Call. Honestly, the reporting and analysis side feels a bit weak. If I want to deep dive into the on-call load, including MTTD and MTTR rates, I kind of have to export data and build my own dashboard, which is not as out-of-the-box as I would prefer. The UI has improved, but it still feels a little chunky in certain areas, particularly when managing schedules. When I have a lot of teams, it can get messy. Additionally, the pricing model is per user, which starts to get expensive as the team scales.

I have been using Splunk On-Call for about the last two years. We started with a pilot in one team and then rolled it over across the whole engineering organization within the first six months, so it is fairly well embedded at this point.

I find it stable overall. There are no issues, and in the last two years, we had one or two incidents where the alerts were delayed, but Splunk was upfront about it on its status page. For a tool that essentially acts as my safety net, I would say the uptime has been reliable enough to trust it, so no major complaints here.

It scales well from a technical standpoint. We went from fifteen users to over eighty and did not feel any friction on the platform side. Where it gets tricky is on the management side. Keeping the escalation policies and schedules organized as teams grow takes discipline. The tool supports scalability, but you need a process around it.

My experience with tech support for Splunk On-Call is a mixed bag. The standard support ticket response times are decent, nothing exceptional. Where it gets better is if I am on a higher-tier plan, then I get a dedicated CSM, and the experience improves noticeably. I had one billing issue and one SSO configuration problem, and both got resolved within one or two days. I would not say the support is a standout feature, but it is not a pain point either.

For the decision to switch from PagerDuty to Splunk On-Call, pricing was a significant factor. Splunk On-Call is generally more affordable, especially for smaller teams, while PagerDuty gets expensive fast as you scale up with user features. Alert routing is a strength for both, but PagerDuty has slightly more advanced routing rules out of the box. Splunk On-Call keeps it simpler and is easier to set up with no time to set up. Regarding integration, PagerDuty wins on the sheer number of integrations, connecting with almost everything, but Splunk On-Call is focused on the Splunk ecosystem for seamless use. Finally, on the analysis aspect, if you are already on Splunk, the reporting and visibility are way better because everything feeds into one platform, whereas PagerDuty's analytics are good but siloed.

The initial setup of Splunk On-Call was smoother than I expected. When I had the setup done within a week, it included the integration, user accounts, and basic escalation policy. The documentation is very good. The part that took more time was designing our escalation policy and on-call rotations, but that is more an initial process problem than a tool problem. If I already know my on-call structure, then the process runs fast without any issues during the initial setup.

We purchased Splunk On-Call through the AWS Marketplace.

I have not done a formula ROI calculation, but the impact is clear. Before this, on-call was managed through a mix of spreadsheets and direct Slack pings. Engineers were getting woken up for unnecessary alerts or missed incidents, which dragged on longer. Since implementing Splunk On-Call, our mean time to acknowledge has dropped significantly, and we have fewer escalations to senior engineering for things that could have been caught earlier. From a cost-of-downtime perspective, that is real money saved.

The pricing aspect is per-user licensing, which is standard for this category, making it competitive compared to PagerDuty. In the past, we used PagerDuty, but it is not cheap once you start adding users across multiple teams. The setup cost was minimal; no professional service is needed for our use cases. If I had a more complex environment, I might need some implementation help, but for a mid-sized team, I can self-serve. An annual contract gave us a better rate, so I would recommend going that route instead of month-to-month.

The benefits my organization has gained from Splunk On-Call's reporting and analytics are significant. The routing rules and the escalation policies show their value. The flexibility there is really strong, allowing you to build a very complex escalation chain without it feeling complicated. The mobile app is also solid, and engineers actually use it without complaining, which says a lot. The timeline view during an incident is super useful. Everyone can see what is happening in real time, which cuts down on the chaos of uncertainty about whether anyone acknowledged the alert. The integrations are also a big win for us. We have hooked into the PagerDuty-style monitoring, Datadog, and Slack, and it just works. We do not have monitoring tools for this integration part of Splunk On-Call. I give this product a rating of nine out of ten.

I have been using Splunk On-Call for the last three years. My main use case for Splunk On-Call is incident alerting and real-time on-call management. It helps me to route critical alerts to the right teams, for example, the Telemetry team or backend DevOps team, and reduce the response times and ensure issues are acknowledged quickly. I also use it to automate on-call rotations and centralize incident communication.

One notable scenario was when a production API started failing during peak hours. Splunk On-Call immediately routed the alert to the particular engineer, the DevOps engineer, triggered our escalation policy, and pulled the backend team to the response channel within minutes. This fast coordination helped me to restore the service much quicker than before.

Some of the features of Splunk On-Call include automated alert routing and escalation that will ensure the right people get notified immediately. Also, on-call scheduling and rotation management. This feature makes managing shifts and coverage very easy. Another feature is real-time incident collaboration and communication. This will help bring the right team together fast. Another feature is alert de-duplication and suppression. This will reduce the noise, so only meaningful alerts reach the on-call staff.

The feature I rely on the most in my daily work is automated alert routing and escalation. It ensures that critical issues go directly to the right engineer without delay, which saves time and prevents the incident from being missed. This has been the biggest contributor to improving our response times.

The noise reduction feature is also very helpful. By filtering out non-critical or duplicate alerts, Splunk On-Call keeps our team focused on what actually matters and prevents alert fatigue.

Splunk On-Call has helped my organization to improve response times, reduce missed alerts, and coordinate teams more efficiently during incidents. It also reduced alert fatigue and made our on-call process more reliable and predictable. Our average incident response time dropped by around 20% to 25%, mainly because alerts reached the right people faster. We also noticed fewer unnecessary escalations, and missed alerts almost completely stopped after we implemented automated routing and escalation policies.

One area for improvement would be enhancing the mobile app experience. While it works well, smoother navigation and faster load times would make on-call handling even easier. More advanced analytics and a reporting dashboard would also be helpful.

It would also be useful to have deeper integration with more monitoring tools and better customization options for alert rules. These improvements would make it even more flexible for different teams and use cases.

I have been working in my current field for the last three years.

Splunk On-Call is fairly stable and reliable. It has a good reputation. Many users find it very useful. It is very balanced.

Splunk On-Call scales very well. As our team has grown, we have been able to easily add more users and on-call schedules without extra overhead. The platform handles increased alert volumes and more complex rotations without degrading the performance.

My experience with Splunk On-Call customer support has been largely positive. The support team is reachable via live chat, portal tickets, or phone. Many users report helpful and responsive support when an issue arises.

Negative

We previously used a generic on-call scheduling tool. We switched to Splunk On-Call because it offers better automated alert routing, real-time incident collaboration, and also integrates seamlessly with our monitoring systems, which significantly improved our response time.

My experience with Splunk On-Call pricing and licensing has been straightforward. The setup cost was minimal since it's a SaaS solution, and the subscription-based licensing model makes it easy to scale users and on-call schedules as our team grows.

We have seen a clear return on investment. By automating alert routing and reducing missed incidents, our average incident response time decreased by around 20% to 25%. This has saved significant engineer hours each month and reduced costly downtime, improving overall operational efficiency.

My experience with Splunk On-Call pricing and licensing has been straightforward. The setup cost was minimal since it's a SaaS solution, and the subscription-based licensing model makes it easy to scale users and on-call schedules as our team grows.

We evaluated a few other options, including PagerDuty. We chose Splunk On-Call because it offered the best combination of alert routing, on-call scheduling, and integration with our existing monitoring tool.

I would advise others to fully leverage its automated alert routing and escalation features from day one. Also, invest time in setting up clear on-call schedules and integration with your monitoring tools. This will ensure faster response times and smoother incident management.

Splunk On-Call has greatly improved our incident management and on-call process. It's reliable, scalable, and helps our team to respond faster. With minor improvements in the mobile app and reporting, it could be even more effective.

Splunk On-Call is deployed in our public cloud environment. Since it's SaaS-based, it integrates easily with our cloud workloads and monitoring tools without requiring any on-premise setup. We run Splunk On-Call on AWS as our primary cloud provider. It integrates well with our monitoring and alerting stack, so AWS is the main platform we use.

I would rate this product a 9 out of 10.

We used it for on-call rotations. We used it to send alerts for monitoring. We also used it for escalation, so when we actually had an issue, it would find out who to call and call that person.

It's deployed on-premises.

About 200 people were using this solution in my organization.

The flexible schedule is the most valuable feature. It was very easy to set out a rotation. If somebody went on a holiday, it was easy to take over for those particular dates. In many other systems, you either have to take over for good and then remember when the person is able to take it back. But here, you just say that on a particular day, somebody else is covering for you. It was very flexible. The feature was called Override.

For alerts, we could choose to get a text message, app notification, or a phone call. The phone calls were very unusable, because it just read a bunch of numbers, like an ID of the alert. If there was a way to customize the phone call message, that would be great.

Later, we would try to read the message, but it wasn't great at reading that.

They had some sort of internal chat functionality where if we got an alert, we could write to somebody else and ask them for help, but that was super cumbersome. There could be improvements with communicating an incident or alert.

Imagine you call the help desk and you say that your computer is broken and then they say, "Actually, the internet is broken, so let us forward your alerts to the network people." And the network people say, "Actually, the electricity is the problem, so let us forward it to the electricity people." Basically, you could send the alert between the support teams inside the company.

The solution is stable.

It's scalable.

The guys who deployed the solution on-premises had some questions for the API, and they were able to reach the help desk quite quickly.

Setup wasn't complicated.

At some point, someone in our company was trying to move to a different tool  but it crashed because you can only set daily, on-call duty. There was a problem with flexible scheduling.

I would rate this solution as nine out of ten. 

The solution can be very spammy at times. We have received the same alert  multiple times, despite thinking that we have already accepted it. If we're working on something and get an alert, it will keep reminding us of it, which is a pain.

My advice is don't over-complicate your schedule. I spent a lot of time trying to make a special schedule for 20 people.

I would recommend this solution to those who are interested in using it.

Our company used the solution to provide infrastructure monitoring, IT alerts, and incident management for one of our customers. The project ended about six months ago. 

The alert calling feature is the best because notifications are delivered via phone calls or text messages. We have 150 users who receive mobile alerts with no issues. We like the solution's calling feature better than competitive products. 

The dashboards are easy to use. 

The third-party configuration tool could be easier to use. The solution is not compatible with most of the tools in the monitoring world so a third-party tool is needed in between.

The solution could move to the next level by providing a ticketing tool similar to ServiceNow. There should be no need to integrate or work with multiple tools. Administrators and users would prefer using one tool that is a mix of alert and ticket abilities with a dual dashboard view. 

I have been using the solution for three years. 

The solution is stable with no issues across three years of use. 

The solution is scalable. We have not needed much scaling because our team has remained the same size. 

I contacted technical support a few times for verification and validation issues. The issues were not too technical but the response time was good. 

I rate technical support an eight out of ten. 

Positive

We also use ServiceNow. 

The setup is easy so I rate it a six out of ten. 

We implemented the solution for our customer.

We have not experienced any downtime and have only needed one update in three years. 

The solution is used for alert calling. 

ServiceNow is used for next-level ticketing that can be customized. 

If you want to notify your users via mobile calls or text messages, then I recommend the solution for alert notification management.

I rate the solution an eight out of ten. 

The solution is used to track, communicate, and escalate issues.

The most valuable feature of the solution is helpdesk escalation. VictorOps has a feature that acknowledges the incident and escalates to the next level with a simple click of a button. Each level can continue to escalate the issue by clicking on the button.

The solution can be improved by including a wider list of permissions.

I have been using the solution for two years.

The solution is stable.

The solution is more flexible than scalable.

I give this solution ten out of ten.

Registering your cell phone using the mobile application for the solution is very important if you want to receive your notifications in real time. 

VictorOps is our alerting system for the on-call process. Our developers use this solution to be alerted when something goes wrong with our services. The solution is connected with our monitoring system.

VictorOps has decreased the meantime to acknowledge an incident management process. Our developers can be on-call faster when we are using this solution. We can fix our incident much quicker.

VictorOps has been good enough for us and it's effective for our needs in case of an on-call escalation process.

At that stage, all our needs are fulfilled, but at the beginning, we had some feature requests and they were deployed during their roadmap.

I have been using VictorOps for approximately four years.

In the beginning, we had some minor issues with stability, but we do not face any issues anymore.

The solution is scalable.

We have approximately 300 developers using this solution. We have no plans to increase usage because it is working very well at the moment.

Many years ago, when VictorOps was a standalone company, when it was not part of Splunk, their support was more dedicated for us. We had a dedicated customer success team, but nowadays when they were acquired by Splunk, we are not at that tier level of dedicated support or dedicated customer success team. This is not good for us.

The implementation was straightforward. We had some minor issues and we discussed is with our customer success team and their support. It was quite smooth overall.

The whole process took us one or two quarters because we changed our internal processes at the same time. It was connected and we had that strategy that we started from one team, and then we roll out our idea to all teams in our company.

We did the implementation of the VictorOps with the help of VictorOps support.

This solution is part of our incident management process tool, and we only need a few people to manage it. These people are responsible for processes and responsible for this solution. We do not have dedicated people to maintain VictorOps.

The price of the solution could be less expensive.

We have evaluated other vendors before choosing VictorOps.

At that point in time when we were evaluating other solutions, VictorOps was a good fit for our needs. For example, we had a dedicated support customer support team. Individual treatment was good for us, a great price point. They are very flexible in pricing. In the end, the price to quality ratio was the best among the other vendors.

My advice to others is you should start small and then roll out to other teams.

Use a proof of concept, at the beginning. Find the first teams which can use the solution. Then you can roll out based on that POC experience.

The biggest lesson learned from using VictorOps was we change our approach to internet management after implementation. It was a part of a big change in our culture of modern incident management. 

I rate VictorOps an eight out of ten.

Transmogrifier and automatic solution report gives me a report with the solution and the way to solve issues when an error occurred. This reduces communication and error time significantly. It sends a report with the solution for you to perform.

It reduces the communication around CI/CD and production errors in about 90% of the cases and made our support tasks much easier.

Should have more YouTube webinars.

About three months.

Not at all.

Not at all.

In the initial implementation, we used their guys. It was very quick and responsive.

Straightforward, the system was up and running in a day.

I did. I have evaluate PagerDuty and OPSGenine. I end up choosing VictorOps because their solution gave me much better capabilities, and their overall communication during the PoC stage was much better and understandable.

Go for it, it truly is a game changer.

The most valuable aspects of this product are the integration pieces. It is pretty much limitless, and I also enjoy the reporting aspects as I use this product to assist in RCA’s.

The most valuable feature of the product is how they allow you to do the scheduling compared to the other companies. I switched from PagerDuty to VictorOps because the time that it would take for me to create a full-on schedule for a single item would be anywhere from 45 minutes to an hour and a half in PagerDuty. That same schedule takes me no more than ten minutes in VictorOps.

That’s because with VictorOps, you’re in a single-pane window to create that calendar or that schedule, compared to PagerDuty where, once you set up a single session, you have to go back and set up another session, and that's your escalation policy. The escalation policies in VictorOps are: you have a drop-down menu where you add a task in that same window where you see all of your people and the times you're going to roll over the on-call; you're in the same screen the entire time. That's not the same as it is in PagerDuty, where you have to go to three separate screens to do the same function; it just takes so much longer.

Also, the transmogrification feature was really awesome. We use that quite heavily, so we can make sure the messages coming in are getting properly formatted. We're able to add whatever little customization that we want to for that type of message, so that the drops can accept it, and then give us valuable feedback based upon that.

Then, another great feature, because we were part of their beta system, is being able to do the calls and start a call within the timeline. That was awesome. Once you start a conference bridge, it would show the users and you notified the users within VictorOps, which was awesome. You could see in the timeline, for RCA purposes, who joined the call, when they joined the call, and if they typed out anything into the timeline concerning that incident that took place. We use it, I currently use it, for RCA tracking purposes. Any incident that takes place, we tag an incident number to it, and then we tell everybody that's associated with that incident to make sure that they put all of their findings related to that incident number and then we print out a report at the end as our RCA.

We are using this product to streamline RCA processes. We make all our engineers enter in notes to a specific monitor we have designated as the issue and then do cross-team collaboration to add notes to that problem. Later we can run reports on the entire timeline of the event so we aren’t having to do it twice. I can upload the logs to management for them to review what took place and who were involved in troubleshooting.

Obviously Net TGA and Net TGR are huge in the DevOps community, so one thing that we like to do is see what our NTGA and NTGR are per route, and try to figure out where we have gaps and why we have those types of gaps, whether we have an employee that simply just doesn't answer his on-call, which goes against your NTPA, and then ultimately against your NTPR. Then the reports, obviously for the RCA, our upper management expects to see those media files when issues arise.

Because we're in the beta program, we've submitted quite a bit of changes that we'd like for them to do. They actually included us in that process. The only feature that we are currently waiting on right now is really kind of an annoyance factor: when we get on the timeline on the main dashboard, there’s an ACK All button. It should not have the ability to ACK others' alerts, but only your own. I have sent several emails to their DevOps group and support about it.

It's something we've brought up twice with them now. After the second time, they really took it seriously. From what I heard from the product team, we weren't the only ones to complain about that. Now that other customers are saying something about it, they've really said, "Okay yeah, we need to do something about this." And they are, and I believe that they will.

Everything they've ever said that they're going to do, they've done it.

I believe it has been a year now.

There were no issues with the deployment.

There were some issues with stability that were quickly addressed and haven’t arisen since. It's been stable as long as we've used it. I think they've only had one time that they said, "Hey, we need to do a maintenance," and it lasted maybe five minutes. It wasn't even enough for us to even notice.

I have not encountered any scalability issues, because, really, that's on their side not ours, but we need to scale as users add answers, all I have to do is just add users; they just bill me appropriately. It's done. They've made it entirely too easy.

We have separate call centers; we kept 41 on our side and I believe there's a little over 100 on the corporate IT side of the organization. So I would say about 150 users total.

Technical support is an A+ or a 10/10. If I could give an 11/10, I would. It is what I love about them. I got a call from their support team out of the blue on a Saturday afternoon, and they basically said "Hey, we've been seeing some strange activity on your account, some strange stuff is happening. Wanted to bring that to your attention, is that expected behavior?" It was something we were doing, we were working on something internally, and they caught it. That tells me that they were actively monitoring those systems. They called us up to say, "Hey, is everything okay? Anything we can help with to help you all do this?" I was like, holy crap, this is good customer service.

It probably took me 30-45 minutes to set up all aspects of the alerting. Not complex at all, which is why we chose VictorOps. It was straightforward; it's literally point-and-click. Set up the users, make sure that they get the notification, make sure that they log in and set up their individual profiles. Then from there, it was just building the schedules. Like mentioned elsewhere, I built out the team's schedules that had been in PagerDuty, and it took me 30 minutes. It would've taken me almost all day in PagerDuty to do that.

We used an in-house team. Take your time and think of all possible avenues of how can get alerts. Literally anything that can send an email can be alerted on and “transmogrified” to fit your needs.

We were a customer of PagerDuty and we chose VictorOps because of the ease of administration. It takes me five minutes to perform an analysis.

Before choosing VictrOps, we also looked at another company, but the name evades me. Because we were already on PagerDuty, we looked at VictorOps, did an in-depth POC for about 30 days, and fell in love with it and pretty much was done.

Play with it, break it down, try to break it as much as you can. Actually do limited production systems, have production systems alerting to it. Play around with the scheduling pieces as compared to the competitors. It's a no-brainer. Everybody that I've ever talked to, the company that has offered me a job, they're actually using VictorOps. I asked, "Why? Why are you using VictorOps?" He said, "Because it's just so damn easy."

I haven’t given it five stars because nobody's perfect. It better be laying golden eggs if it's going to get five stars.

It captures the incidents from New Relic and sends out notification when there is a critical alert.

Integration with Amazon CloudWatch and New Relic. Most of the AWS applications are monitored using CloudWatch but it cannot notify using sms/call. Integrating with VictorOps has helped the notification across global telephone numbers.

User management has to be improved and more user types need to be added as there is currently only Admin or User.

The timeline display should be improved as currently you need to scroll down the incident list to find a specific incident. As of now, there is no dashboard which shows you the numbers of incidents and their state - Open, Resolved, Acknowledged, etc.

I have been using it for almost 10 months. I am using VictorOps as an alerting tool for all the AWS based applications.

I am not managing the deployment of this application. It’s a web-based application and not under our control.

I have had no issues with its stability.

As it's a web-based application, and not managed by us, we have not scaled it ourselves.

Customer service is good. I have always received prompt responses for the queries and concerns raised.

The configurations is straightforward and the Knowledge Base of this application is good.

I am not implementer -  I am just user of this applications to configure alerts for the applications that I manage.

As I am a technical person, I do not have any information about this.

I was not involved in an evaluation and so I don't know.

I would recommend VictorOps for global distributed support teams. The on-call and escalation notifications can be easily managed as it includes SMS, push notifications, and call out features which we are using. There are also integrations with tools like Amazon CloudWatch, New Relic, and Slack which are also currently used in the company.

Overall, we're pretty happy with the SMS, push and call notification features. The timeline is not very user friendly, and a real-time dashboard feature which can show incident numbers, status of the incidents, who is working on them, etc. is missing.

We are very satisfied with the tool. We were using a similar tool, but the features were not enough for us. I did some research and found VictorOps, which we then tested. It was perfect, so we decided to go for it.

My team as well as other teams are now using the tool. As for me and my team, we use it mainly for our phone alerting. If there are a few non-critical incidents, a notification is sent to VictorOps which converts it to an email notification and then to a phone alert.

One thing which we would like is an improvement to the history log on the main page. There is some history, and I think you can filter it, but the filtering options are not sufficient for us. We would like to have more options. You can find some history records for the previous two or three days, but you cannot dig into them. For example, you cannot find logs that are a week old. Another example is that you cannot check the incidents created in VictorOps from the previous Friday. This is one limitation that they need to fix.

When I plan according to the schedule in the platform, for some reason I'm sometimes not able to change the on-call duty person. For example, if I want to remove a person who is currently on-call and assign the on-call duty to someone else from my team, it's not possible. I need to log out and log back in first. Maybe it is browser-related, I'm not sure, but this is something that causes me difficulties from time to time.

Eight months.

I appreciate that whenever we have an issue, we have very good support. If there is something wrong, I always have somebody from VictorOps support to step in and help us out.

It was email2form, which was not a professional tool. From time to time the tool didn't work and when it was down it was a big issue for us. If tool wasn't working, we didn't actually have a phone alert, so we didn't know that there was a critical situation. With VictorOps, whenever there is a critical incident we have a phone alert.

I was in touch with the support center of the previous tool and they were terrible. Now with VictorOps, we create support tickets (via VictorOps as well), and the response time from support is always very good. I usually have updates within 24 hours of logging a ticket, which is something that I appreciate, and need.

It was not very difficult, and user account enrollment is very easy. What I like, is the feature where you have an account created in the platform, then you assign a phone number to the user, and then you can very easily test it. You send a test SMS and/or phone call to make sure that the person who will be on-call will receive the call. This is a very nice feature in the platform with the initial setup, because we use this tool to monitor other tools.

It depends on your requirements and expectations. You need to be sure of what you need and what you would like to use the tool for. Eventually you can find other tools on the market which might be a better fit for you, but for our purposes which I described, VictorOps is very good and even for the price we paid, it's perfect.

What was very good is that from the beginning, our VictorOps an account manager, has been working with me almost every day. Whenever I ask for support, he organizes a call and gives me access to a Sandbox, or a testing environment. I would advise any potential user who would like to investigate VictorOps to start using that.