Ten Key Criteria for Choosing the Right Breach and Attack Simulation Solution

The cyber threat landscape grows as threat actors develop new attack techniques and as digital transformation introduces new technologies that increase the attack surface. Consequently, organizations allocate more resources to enterprise cybersecurity solutions. However, increasing spending on cybersecurity technologies does not guarantee that security defenses can prevent and detect cyberattacks. Only organizations that continuously test the efficacy of their security controls to identify and address gaps quickly will be able to maintain a strong security posture. 

Consequently, Breach and Attack Simulation (BAS) has become the most effective approach for security control assessment organizations by providing real-time visibility, automated gap analysis, and actionable mitigation insights in a cost-effective manner. According to Gartner, end-user clients' interest in BAS grew more than 90% in 2021 [1]. However, BAS solutions have developed significantly over the past few years, and choosing a suitable BAS tool has become challenging. Moreover, not every BAS tool has the essential features to assess enterprise security infrastructure entirely. In this blog, we explained the ten key criteria that can help you evaluate BAS solutions and answer the challenging question, "How effective are our security technologies against cyber attacks?"

2X Prevention with BAS

How to leverage Breach and Attack Simulation to optimize your security
controls and block twice as many threats in 90 days.

DOWNLOAD THE E-BOOK

What is Breach and Attack Simulation (BAS)?

Breach and Attack Simulation (BAS) is a technology that continually and consistently simulates the full attack lifecycle against enterprise infrastructure, as defined by Gartner [2]. BAS is the newest tool in organizations' toolset for cybersecurity assessment and complements traditional assessment methods such as vulnerability scanning, penetration testing, and red teaming. Unlike penetration testing, which typically focuses on one-time evaluations, BAS provides continuous testing, offering real-time visibility and actionable insights into security gaps. The choice between breach and attack simulation vs penetration testing depends on the organization's need for continuous security control assessment or specific testing during a limited time frame."

10 Criteria to Evaluate Breach and Attack Simulation Tools

This section provides ten essential criteria to consider when choosing a BAS tool, which will assist you in identifying the solution that best meets your organization's cybersecurity requirements. These criteria cover Gartner's BAS definition and organizations' regulatory and operational requirements.

1. Threat Simulation Across the Full Attack Lifecycle

Cyber threat actors use adversary techniques individually or in combination, and these malicious activities make up the cyber threat landscape. Understanding and simulating adversary techniques and attack campaigns are great ways to assess security controls against the threat landscape. Many vendors maintain a threat library to represent and simulate real-world adversary activities. 

BAS must have a comprehensive threat library that includes attacks for all stages of the attack lifecycle. The attack lifecycle can be divided into pre-compromise attacks and post-compromise attacks. The threat library must also have predefined and customizable attack scenarios that emulate adversary activity.

• Pre-compromise attacks
• Email attacks
• Malware download attacks
• Vulnerability exploitation attacks
• Web application attacks
• Post-compromise attacks
• Atomic endpoint attacks
• Data exfiltration attacks
• Lateral movement attacks
• Attack campaigns
• Malware attack scenarios (e.g., LockBit ransomware campaign)
• Threat Group attack campaigns (e.g., MuddyWater threat group campaign)

Another requirement for threat simulation is to be consistent and safe. Security teams must be sure that their assessments do not disrupt daily operations. Consistency in threat simulations allows security teams to objectively evaluate their security controls and configuration changes.

2. Up-to-date Against Current and Emerging Threats

The cyber threat landscape expands with new adversary techniques, vulnerabilities, and attack campaigns. Since the cyber threat landscape is not static, neither should the threat library be used by BAS. The threat library needs to be updated swiftly so that organizations' defenses can simulate and keep up with new threats. These updates should include and prioritize emerging threats that pose a significant risk to organizations. Note that some BAS vendors might charge a premium for early access to newly added simulation content.

3. Validation of Enterprise Security Controls

Organizations deploy 80 security controls with various capabilities, and these security controls are used in different networks and locations. BAS should be able to assess the entire security infrastructure and provide seamless integration with a wide range of prevention and detection technologies.

• Network Security Controls
• Email Security Controls
• Data Loss Prevention (DLP)
• Intrusion Prevention System (IPS)
• Next-Generation Firewall (NGFW)
• Secure Email Gateway (SEG)
• Secure Web Gateway (SWG)
• Web Application Firewall (WAF)
• Detection Controls
• Endpoint Detection and Response (EDR)
• Intrusion Detection System (IDS)
• Security Information and Event Management (SIEM)

4. Continuous and Automated Simulation

Unlike traditional security assessment methods such as penetration testing and red teaming, BAS can run continuous and automated attack simulations. Continuous testing allows BAS to identify weak points in the security controls against the ever-changing threat landscape and configuration changes in security controls. In addition to continuous attack simulations, BAS is expected to provide automated simulations and not require an operator. 

Continuous and automated attack simulations allow BAS to assess and detect security controls that are newly added, removed, disabled, and in by-pass mode.

5. Detection Rule Validation

One of the leading causes of inefficiency in SOC operations is false positive detection alerts. Efficient and effective detection rules are vital for reducing false positive detection alerts. However, designing and testing detection rules is highly technical and time-consuming. BAS should be able to validate detection rules and help SOC teams evaluate their detection rules. Also, SOC teams should be able to use BAS to verify alerts and fine-tune their detection rules.

6. Threat Customization 

The cyber threat landscape of each organization is unique and depends on its industry, location, infrastructure, and many other factors. As a result, organizations should do threat prioritization to minimize their cyber risks effectively. BAS should provide threat profiling for organizations and help SOC teams prioritize threats.

After threat profiling and prioritization, security teams may want to run custom attack simulations and test their attack payloads and malware samples to simulate their threat landscape. BAS should allow SOC teams to create custom threat simulations and attack campaigns to assess their security posture. 

7. Direct and Actionable Mitigation Insights

Threat simulations are great for assessing security controls and identifying security gaps. The next step for SOC teams is to mitigate the identified gaps as swiftly as possible. Since organizations utilize various security controls from different vendors, coming up with mitigation for cyber threats may take a while, especially for emerging threats and zero-day vulnerabilities.

A BAS solution should provide direct and actionable mitigation content for gaps identified in threat simulations. SOC teams can also use these mitigation insights to create custom mitigation strategies.

8. Real-Time and Customized Reporting

A comprehensive security assessment produces lots of data and must be reported to many stakeholders in the organization. As a result, BAS must present its findings in assessment reports that can be used by executives, SOC teams, and auditors.

An assessment report should include real-time metrics such as:

• overall security score
• detection rate
• mean time to detect (MTTD)
• trend statistics
• log collection
• detection
• prevention
• compliance-related data

9. Mapping to MITRE ATT&CK and Other Frameworks

The MITRE ATT&CK framework is the common language among security professionals worldwide to describe adversary activities in cyberattack campaigns. Many organizations and regulatory bodies use heatmaps mapped to the MITRE ATT&CK framework and use these heatmaps as a visual representation of their security posture against adversary techniques.

BAS should provide automated mapping to the MITRE ATT&CK framework for simulated threats, adversary actions in a simulated attack scenario, and identified security gaps. 

10. Ease of Use and Ease of Deployment

Enterprise cybersecurity infrastructure uses a multitude of security technologies. These technologies might be deployed on the cloud or on-premises; their management requires SOC teams' time and effort.

BAS should

• have a simple and easy-to-understand interface
• avoid adding complexity and workload
• make optimizing security controls easier
• empower security teams to achieve greater impact with less effort
• be easy to deploy on the organization's existing infrastructure
• support cloud and on-premises deployment

References

[1]       "2022 Gartner® Report: Prepare for New and Unpredictable Cyberthreats," Optiv. [Online]. Available: https://www.optiv.com/insights/discover/downloads/2022-gartnerr-report-prepare-new-and-unpredictable-cyberthreats

[2]      "Gartner Forecasts Worldwide Security and Risk Management Spending to Exceed $150 Billion in 2021," Gartner. [Online]. Available: https://www.gartner.com/en/newsroom/press-releases/2021-05-17-gartner-forecasts-worldwide-security-and-risk-managem