Why Detection Rules Fail: Causes, Effects, and Corrective Actions

In this blog post, we aim to raise awareness about the risks posed by broken detection rules within the cybersecurity landscape. These risks can be minimized by identifying and improving the effectiveness of underperforming detection rules. To achieve this, we will explore the reasons why a rule might fail, detailing the potential causes. Additionally, we will discuss methodologies to detect and address these failures. 

By providing these insights, this post aims to equip detection engineers with valuable knowledge. Ultimately, it serves as a foundation for further studies and contributions in this field.

The Essential Role of Detection Rules 

Detection rules are essential for identifying threats in an organization's environment and enabling quick responses. These rules are foundational elements of Security Operations Centers (SOCs), where much of the work revolves around creating and analyzing them. By acting as the "eyes and ears" of cybersecurity, detection rules provide critical information to experts monitoring potential risks.

This critical role is supported by cybersecurity frameworks, which offer guidelines for developing robust cybersecurity policies and practices. Detection rules are integral to meeting these guidelines, helping organizations quickly and effectively spot threats. By distinguishing between regular activity and signs of malicious behavior, these rules ensure organizations stay ahead in maintaining security. This importance is further emphasized in frameworks like NIST 2.0, where "detection" is a crucial component [1].

Figure  1: NIST CSF 2.0

Due to their importance, detection rules are expected to work properly. However, as we will discuss further, there are instances where detection rules do not operate as expected, and identifying such cases is often challenging. This challenge arises because SIEMs lack sufficiently advanced mechanisms to determine whether a rule is functioning correctly, and creating such advanced mechanisms is a difficult and costly task for users.

As a result, this situation can lead to the failure to detect relevant threats, leaving organizations unaware of them. Even worse, users may fall into the misconception that they have effective detection rules in place for these threats, leading to unnoticed risks.

Moreover, preventing every threat is not always possible. According to the Picus Blue Report, security organizations prevent just over half of attacks (59%) using their existing security controls, such as IPS, NGFW, or WAF solutions [2]. This highlights the increased importance of detection rules, as non-functioning detection rules exacerbate the risk further.

Consider your own SIEM. With hundreds of detection rules present and this number increasing daily, can we be sure that all these rules are working as expected? How much effort and time do we devote to ensuring this? Are we following systematic methods to be aware of these situations?

In the next sections, we'll seek answers to these questions and explore why detection rules may not work as expected.

Why Do Your Detection Rules Sometimes Fail?

In modern SOCs, new rules are continually added to SIEMs, and existing rules are updated as needed. Additionally, a rule may have multiple dependencies, such as other rules, functions, and lists, referenced within it. This complexity increases as new log sources are added to enhance visibility. However, the dynamic and constantly changing infrastructure can lead to the emergence of broken detection rules.

Consequently, this can result in a lack of awareness of relevant threats due to the failure of these rules to function properly. This underscores the importance of maintaining and monitoring detection rules to ensure they are effective and up-to-date.

Figure  2: Picus Blue Report - Detection Effectiveness 

According to another statistic from the Picus Blue Report, it has been observed that the alert scores of organizations often remain below a certain threshold [2]. One of the primary reasons for this issue is that the detection rules within their security systems are not working as expected. 

We can roughly classify the five main reasons why a rule may not work as expected as follows:

1. Challenges with Data Sources

Issues related to the data (logs) and data sources (log sources) that come into the SIEM will be addressed. Any problems that arise here will directly affect the rules working as expected.

1.1 Accuracy and Completeness: When the data sent to the SIEM isn't accurate or complete, it can mess up the detection rules. This might happen if the logging on the systems sending the data isn't set up right, or if important data sources aren't connected to the SIEM. Plus, SIEM systems usually need data to be in a certain way. If the incoming data doesn't match, the SIEM might not understand it right, so it won't detect stuff properly.

1.2 Volume: Too much or too little data can cause problems. Too much data can slow things down or make it hard to spot real problems, while not enough data can mean we miss things we should be watching out for. For example, when the volume of log data exceeds a SIEM system's license capacity, it can lead to logs being dropped or processed with delays. This situation can result in missed detections of security incidents. 

1.3 Context: Detecting threats often depends on understanding the context surrounding an event. If the SIEM doesn't have all the necessary information from logs, it might struggle to accurately assess the severity or relevance of an event. Additionally, for effective detection, SIEM systems rely on timely data. Delays in data delivery can result in missed detections or alerts on issues that have already been resolved. For instance, if a security incident occurs but the SIEM doesn't receive the relevant log data until hours later, it might miss the opportunity to respond promptly, allowing the threat to escalate. As an example, if the configuration for collecting Windows Event ID 4688 logs is not properly set up, the Process Command Line information may not be captured. This detail is crucial for understanding how processes are initiated, including any command-line arguments used, which is vital for security analysis and detecting potentially malicious activity. Without this information, any rules designed to detect suspicious or anomalous activity based on command line parameters will fail, leading to missed detections. 

1.4 Correlation Challenges: The effectiveness of a SIEM relies on its ability to correlate data from different sources. However, if the data can't be properly correlated due to issues like differing time formats, missing identifiers, or lack of synchronization, detection rules may not work as expected. For example, if one system logs events in a different time zone than another system, the SIEM might struggle to match up events accurately, leading to missed detections or false alarms.

1.5 Resource Limits:  System resources can prevent data from being processed properly. Limited processing power, storage, or network bandwidth can impact the SIEM's ability to handle and analyze data effectively, leading to missed detections.

2. Rule Dependency Complexities

Issues arising in the components (dependencies) on which the rules rely for their operation will be discussed. The problems that occur here will directly affect the rules' ability to work as expected.

2.1 Inconsistencies in Custom Fields: Custom fields that are used in rules might fail to produce expected values. This could be due to mismatches between the anticipated data format and the actual data, leading to missed detections or false alarms. Such as, in a SIEM system, logs are parsed and assigned to custom fields, but incorrectly applied parsing or changes in log formats can lead to errors in data extraction. This can affect the detection and response to security incidents. 

2.2 Outdated or Empty Lists/Tables: Lists/tables that are essential for the operation of rules may not be regularly updated or could be completely empty. Such outdated or missing data in lists/tables can significantly impair the rule's ability to detect anomalies accurately.

3. Structural Complexities

Detection rules are comprised of various components, each imparting different capabilities to the rules and enhancing their modularity. Issues within these components may cause the rules not working correctly and result in the absence of these capabilities.

3.1 Rule Components: Building blocks, functions, macro searches, or data models within the rules might not be designed or implemented optimally. This can result in the generation of false positives or negatives, compromising the reliability of the rule.

3.2 Rule Interactions: Within a SIEM system, rules often have complicated relationships with one another. When one rule fails or triggers incorrectly, it can set off a chain reaction of failures or false positives/negatives in other linked rules. This interdependence makes troubleshooting and fine-tuning the rules challenging. Additionally, maintaining consistency across the rule set becomes difficult. A change in one rule may necessitate adjustments in multiple others, leading to a complex maintenance process. Furthermore, identifying the root cause of a detection failure becomes complex. An issue in one rule can obscure the actual source of the problem, making it harder to diagnose and rectify.

3.3 Irrelevant Objects: Rules might include network objects or domain structures that are either default,irrelevant or lack necessary data. This can render the rules ineffective, as they are processing data that does not contribute to accurate threat detection. For instance, SIEM systems often come with default rules that include generic or placeholder values, which may not be effective in real-world scenarios. For example, a rule might use a network object set to 127.0.0.0/32, a localhost address, which is not practical for monitoring external network traffic. It's essential for users to customize these default settings to align with their specific network environments. 

4. Configuration and Setup Issues

Misconfigurations in the rules or issues arising from their configuration will be mentioned. These problems can prevent the rules from triggering correctly or result in inappropriate actions once they are triggered.

4.1 Rule Configurations: Rules require precise configurations to work effectively. If these configurations are incorrect or incomplete, it can prevent the rule from triggering necessary alerts or lead to erroneous alerts.

4.2 Time-Range Accuracy: In scheduled rules, incorrect settings for time-range can lead to missed incidents or irrelevant alerts. Accurate time-range settings are crucial for the timely and relevant execution of rules. For example, in a scenario where a scheduled rule in a SIEM system is set to run every 10 minutes but only analyzes logs from the last 5 minutes, logs from the first 5 minutes of each cycle are not reviewed. This oversight can lead to missed detections of potential security threats.

4.3 Customization in Content Packs: Rules installed from content packs often have default settings that need customization. Failing to modify these settings to fit the specific environment can reduce the rule's effectiveness.

5. Rule Performance Issues

The performance of the rules impacts the overall effectiveness of the SIEM. Rules that operate inefficiently or fail to adhere to the system's best practices can lead to numerous issues and false detections. These problems will be explored under this heading.

5.1 Query Best Practices: When rules do not align with the best practices for the specific SIEM product's query structure, it can lead to inefficient processing, slower response times, and increased load on system resources.

5.2 Resource-Heavy Filters: Using filters that require significant system resources can slow down the SIEM system. This affects the overall performance of the rules and can delay critical threat detection. As an example, when a rule utilizes filters that search through the entire log payload rather than focusing on indexed or extracted fields, it can lead to significant performance issues. This approach is much less efficient because it requires the system to process each log in its entirety for every query, which is far more resource-intensive than querying against fields that are specifically prepared and optimized for quick access.

5.3 Filter Sequencing: Incorrect sequencing of filters can lead to inefficient rule execution. The order in which filters are applied can significantly impact the speed and accuracy of the rule processing. In SIEM systems, it's important to prioritize indexed custom fields in rule queries as the first conditions. These fields are pre-indexed, making them faster and more efficient for searches. Using indexed fields as the primary conditions enhances the performance of the SIEM system by allowing it to quickly narrow down data, thereby reducing processing times and improving the speed of threat detection and response. 

5.4 Data Parsing at Runtime: If a rule is designed to parse a large amount of data in real-time, it can lead to performance bottlenecks. Efficient rule design should aim to minimize runtime parsing to ensure quick and accurate threat detection.

5.5 Rule Scopes: Rule that processes too many logs or checks against overly large lists, it can become less effective at identifying specific, critical security incidents. Narrowing the scope can enhance the precision and effectiveness of the rules.

Identifying, Prioritization and Resolving Challenges

As we have mentioned before, the mechanisms in SIEM systems that check whether the rules are working as expected are not sufficiently advanced. A systematic approach and dedicated effort are necessary to identify, prioritize and resolve the problems we have discussed.

We can divide the relevant process into three parts: 

• identifying, 

• prioritization and 

• resolving. 

As you can appreciate, in order to solve a problem, it first needs to be identified. The actions that can be taken during the identification and resolving processes can be summarized as follows:

Firstly, regular monitoring and checks are vital. By consistently monitoring SIEM detection rules, you can quickly identify when a rule is not performing as expected. At this stage, the components that trigger the problems mentioned above should be monitored with dashboards, saved searches, or periodic manual checks. Also the SIEM itself can provide clues about broken rules. Checking product warnings and error logs can offer valuable insights. These logs and warnings should also be monitored.

Once issues are identified, the next step is prioritization. Not all detected issues may impact the system equally; thus, it is important to prioritize them based on their potential impact on security posture. This prioritization ensures that resources and efforts are allocated effectively to address the most critical issues first.

During the resolving phase, it is crucial to understand the root causes behind the rules not performing as expected. Mastery of the relevant SIEM product is very important both for understanding the root cause and for taking the correct action afterward.

A rule can become broken at any given moment. Checks conducted at long intervals cannot fully mitigate the risk that arises. Therefore, this process must be continuously operated in a cyclical manner. For this reason, the process should be automated as much as possible.

Conclusion

In this blog, we explored numerous factors that could lead to the failure of a rule within a SIEM system. To effectively pinpoint the root causes behind these failures, automated detection mechanisms can be established. These mechanisms, utilizing saved searches or reports, should align with the capabilities of the specific SIEM product being used.

The data obtained from SIEM user interfaces may not always be sufficient to identify some of the root causes behind rule failures. To develop more advanced logic and effectively address complex issues, accessing more detailed data through APIs can be pivotal. By retrieving this deeper level of information, users can write scripts that process this data, enabling the automated detection of complicated problems.

It's important to note that SIEM systems are complex and constantly changing. As a result, developing and maintaining automation to address potential issues will require significant time and effort.

The Picus Detection Rule Validation (DRV) module identifies broken and inefficient detection rules within your SIEM. It has a continuously updated checklist for identifying these detection rules and uncovers the root causes behind rules not performing as expected. By doing so, it saves significant time and effort compared to manual operations while also enhancing the effectiveness and efficiency of security controls. To use DRV, simply complete the integration process with your SIEM, which takes only a few minutes.

You can request a demo to identify issues related to SIEM rules and gain insights to enhance your rules here.

References

[1] “The NIST Cybersecurity Framework (CSF) 2.0.” Available: https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.29.pdf. [Accessed: May 21, 2024]

[2] “The Blue Report 2023.” Available: https://www.picussecurity.com/resource/report/blue-report-2023. [Accessed: May 21, 2024]