Choosing What to Log and What Alerts to Trigger

In cyber security, your detection effectiveness is measured in terms of your organizations’ ability to log and alert on attacks. To be successful, you need to be able to do both. First, you need to log data from their security, IT and business applications to get a full picture of the activity taking place in your organization. Second, you need to set up rules and/or anomalous behavior models to trigger alerts for attacker behavior. All of this takes time, something that you probably have in limited quantity.

In the Blue Report 2023, our analysis of 14 million attack simulations found that organizations are making four “impossible” tradeoffs when it comes to managing their threat exposure. One trade-off is choosing whether to invest resources in better logging or better alerts. 

The report found that there is a significant gap between log scores and alert scores, with alert scores being lower. Faced with a trade-off in time and resources, organizations are prioritizing logging over alerting. We also found that this gap held true across all the industries and all the regions we studied. 

Unfortunately, our analysis also uncovered that the average security organization is not doing either one very well. On average, they only log 37% of attacks and only trigger alerts for 16% of attacks.

Performance by Industry

Across all industries, there is a significant gap between log scores and alert scores. This discrepancy is concerning. Fewer than half of the number of attacks being logged and recorded by security systems are being alerted and flagged for action.

Log and alert scores by industry

This gap gives adversaries an opportunity to exploit these non-alerted security breaches, potentially leading to significant data loss or business disruption.

Performance by Region

Alert scores are also much lower than logging scores regardless of region. The alert scores for all regions fall on the lower end of the scale, ranging from 13% to 37%.

Log and alert scores by region

This suggests that detection security controls, such as SIEM systems, are not optimized to generate alerts for a significant number of attacks. This might be due to an overwhelming number of false positives, improper tuning of the alerting mechanisms, or an inability to effectively correlate and prioritize security events.

Overcoming the Trade-off

To address the gap between logging and alerting, organizations need to enhance their alert mechanisms. Options for doing so could include improving automated alerting systems or incorporating more sophisticated threat detection algorithms. The objective should be to ensure that every logged attack generates an appropriate alert and subsequent action, minimizing the opportunities that cybercriminals have to exploit unnoticed vulnerabilities.

Picus Security can provide you with a better way to manage your log sources and detection rules. Picus provides two solutions as part of The Picus Complete Security Validation Platform that help you and your security team improve your detection capabilities:

Picus Detection Rule Validation (DRV): This AI-powered tool statically analyzes your detection rule base and suggests remediations to common issues in log source ingestion and rule syntax. It also uses artificial intelligence (AI) to map your existing rules to the MITRE ATT&CK framework to highlight existing detection coverage and surface critical gaps.

Picus Detection Analytics (DA): Detection Analytics, a component of Picus Security Control Validation (SCV), is like a dynamic stress test of your detection effectiveness with real-world threats. It analyzes the impact of simulated attacks in a customer environment:

• Do the security controls trigger logs? 
• Are they ingested into the SIEM in a timely manner? 
• Are the detection rules triggering alerts consistently?

To learn more about other trade-offs organizations face in managing their threat exposure, download the Blue Report 2023, or read our other blogs in this series.