We receive alerts all day long - alerts about emails, incoming Whatsapps and SMSes, posts on social media, etc. At some point we become desensitized to these alerts and stop noticing them anymore - a phenomenon known as “alert fatigue.” Seventy percent of a SOC analyst’s workday is spent dealing with alerts, so SOC analysts are more at risk for alert fatigue than pretty much anyone else.
SOC analyst and IT Central Station user Geofrey M. says that he receives more than 20,000 alerts a week - and 60% of these are deemed critical. With numbers like these, alerts can easily start piling up and don’t always get dealt with in a timely manner - or sometimes at all - leaving what may be important issues to fall through the cracks.
Alert fatigue can be harmful to your business for a number of reasons. These include:
- Ignored alerts - Obviously, when alerts get missed due to alert fatigue, this can lead to damaged customer relationships and overall devastation to your business.
- Wasted time - The more time your team spends responding to alerts that are not necessarily critical, the less time they spend doing the other critical tasks they are being paid to do.
- Employee burnout - Your staff may, in fact, manage to resolve most of the significant alerts and therefore your customers may not be directly impacted. But the fact remains that the more alerts your employees receive and have to deal with, the less productive they will be.
- Psychological effects - The more alerts SOC analysts receive, the more reason they have for concern. Fear that they may have missed something can slow down releases, ultimately impacting customers.
Some of these factors, like the number of alerts that get missed or the amount of wasted time, can be measured. In his article, “Alert Fatigue – A Practical Guide to Managing Alerts,” Itiel Schwartz writes that psychological ramifications, such as how burned out your staff gets, cannot. But in response to his article, Reddit user SU1PHR disagrees, stating that just because there is no quantitative way to measure employee burnout does not mean it can’t be measured. He suggests monitoring employee retention rates and one-to-one meetings at which managers can routinely receive feedback on how their employees are managing. He warns that not doing so can cause “a deadly spiral that will lead to more fatigue, more errors and more missed alerts.”
Several other Reddit users on the same thread mentioned that at their places of employment, alerts get saved and fed into business analytics tools but that, other than being able to say “we are collecting the data,” nothing else ever gets done with them.
So what can be done to minimize alert fatigue and help SOC analysts stay on top of everything that needs to get done?
First of all, it’s important to minimize human error whenever possible. Sometimes engineers inadvertently create a code malfunction or an alert isn’t calibrated properly. Putting better organizational processes in place can help ensure that the people involved in setting the alerts do so appropriately.
But it’s best, when feasible, to remove the human element altogether. This is why IT Central Station user Tshepiso M. points out that it is important to automate wherever possible. Using technology to sort alerts by importance can help ensure peace of mind in your staff by taking some of the burden off of them. The less your employees feel responsible for keeping track of and dealing with alerts, the less you’ll have to worry about psychological effects such as burnout or fear of failure.
One way to increase efficiency, as Geofrey M. points out, is by implementing a SIEM solution. Security information and event management (SIEM) solutions can help prevent alert fatigue by streamlining security. Part of the problem with alerts is the amount of sources from which they originate. Organizations are constantly adding more tools, which makes IT environments increasingly complex. A SIEM solution can become your primary security monitoring tool by consolidating the data streams and integrating unique data sources. It can also take security data from a variety of systems and analyze them, putting them all into context and gleaning new insights from one centralized location.
Every organization is different, and a SIEM allows you to adapt and build your own nuances into the security alert process for your business. When you are considering deploying a SIEM solution, keep the following things in mind in order to ensure that you are only receiving the notifications you actually need.
- Consider context - Rather than setting the same alerts for each new asset, take the time to think through each asset’s function and role within the wider context of the environment and adjust the defaults and settings accordingly. This allows for proper prioritization and allows the number of notifications to be reduced significantly.
- Limit who will receive what alerts - Without a SIEM, every single alert may be sent out to every single admin. But this is rarely necessary. A SIEM will allow you to have different staff members alerted depending on the event or the operating system affected. This reduces redundancy and prevents a buildup of excess alerts over time.
- Revisit and readjust - Make changes as you go. If your initial configuration leaves you getting alerts you don’t need, you can always lower the priority or filter it out altogether. SIEMs allow you the flexibility to change your settings as needed, maximizing the capabilities of your security tools and freeing up your security team to be available when they are really needed.
Conclusion
Putting better organizational processes in place in order to minimize human error can help reduce alert fatigue for SOC analysts. But an even better strategy is to try to automate and remove the human element altogether. One great way to do this is to implement a SIEM solution.
Learn about SOC Analyst Appreciation Day here.
