A Complete Guide to Getting the Best Out of BAS Assessments

What Is a BAS Assessment?

Breach and Attack Simulation (BAS) assessments are proactive cybersecurity evaluations designed to simulate real-world attack scenarios and test an organization's defenses. Recognized by Gartner as a key component of the adversarial exposure validation process, BAS testing focuses on identifying and validating exposures across security controls while assessing their potential impact. BAS platforms streamline this process by continuously testing defenses across various attack vectors, such as email gateways, web gateways, web application firewalls, and endpoints. This approach enables organizations to proactively uncover vulnerabilities, evaluate their resilience against evolving threats, and prioritize remediation efforts effectively.

How Do BAS Assessments Work?

Agent-based and Agentless BAS Assessments Explained

fAgent-based and agent-less BAS (Breach and Attack Simulation) testing refer to two different approaches for implementing and executing BAS solutions in an organization's environment: 

Agent-based BAS assessments rely on lightweight software components (agents) installed directly on endpoints, servers, or within the target environment to simulate attacks and evaluate security measures. Agent-less BAS, often browser-based, does not require installation of software on the target environment. Instead, it uses web browsers or external systems to simulate specific attack scenarios.

For example, Picus Security Control Validation (SCV) uses lightweight agents to emulate threats effectively. To conduct simulations, you need to deploy these agents within the environments you wish to test. Currently, there are four types of simulation agents available:

1. Windows Agent
2. Linux Agent
3. MacOS Agent
4. Browser Agent (Agentless - does not require installation)

The first three agents—Windows, Linux, and MacOS—require agent installation (agent-based) in the target environment. The Browser Agent, on the other hand, can be used without installation (as agentless) but is limited to simulating network infiltration attacks, and not be able to perform endpoint, web application, e-mail infiltration, and data exfiltration attacks. This is not related to the technical capabilities of the Picus Platform but rather an educated and experienced decision based on understanding the true nature of attacks and how they can be crafted to mimic real-life adversarial behaviors.

Figure 1. Attack Modules from the Threat Library of Picus Security Control Validation

How Does Agent-based BAS Assessment Work: Stress Testing Your Preventative Solutions

In this section, we will demonstrate how to run an agent-based simulation to stress test your organization’s implemented preventive security measures, such as a Next-Generation Firewall (NGFW) and Intrusion Prevention Systems (IPS). Here is a three-step overview of an agent-based simulation: installation of the agents, simulation execution, and communication.

• Installation: Agents are deployed on target systems, such as Windows, Linux, or MacOS machines.

• Simulation: These agents simulate various stages of an attack, including malware execution, lateral movement, privilege escalation, or data exfiltration.

• Communication: During simulations, these agents interact with the Picus Manager by requesting attack scenarios, reporting results, and updating the manager about their operational state. This communication ensures accurate and efficient threat emulation and result analysis.

Here, the figure below showcases how Picus BAS platform integrates with an organization's network to evaluate the effectiveness of its security measures across different layers. At the core of the simulation process is the Threat Library, which provides a comprehensive set of attack scenarios mimicking real-world cyber threats. These simulated attacks are executed along the red paths shown in the figure, representing how threats propagate through the network infrastructure.

Figure 2. Agent-based BAS Assessments to Stress Testing Preventative Security Solutions

The simulation begins by targeting critical security components. The Next-Generation Firewall (NGFW) and Intrusion Prevention System (IPS) are tested to ensure they can identify and block threats at the network perimeter. Further along the simulation path, the Web Application Firewall (WAF) is evaluated for its ability to protect web applications from attacks like SQL injection and cross-site scripting. The Mail Security layer is also tested to determine its effectiveness against email-based threats, including phishing and malware.

Additionally, the figure shows simulations extending to Proxy Servers, HQ Endpoints, and the Data Center, ensuring that endpoint security, data protection, and network filtering mechanisms are adequately assessed. Throughout this process, the BAS platform communicates with these systems to analyze and report on the success of the simulations, providing actionable insights to strengthen the organization’s overall cybersecurity posture.

How Realistic BAS Assessments Are Compared to Real Adversarial Behaviors

BAS assessments achieve high realism by leveraging continuous Cyber Threat Intelligence research. 

The dedicated lab teams (red & blue) actively monitor the evolving threat landscape to identify and analyze the latest malware campaigns, ransomware attacks, zero-day exploitations, advanced persistent threat (APT) group operations, and other adversarial behaviors.By incorporating these findings, BAS assessments are designed to replicate the tactics, techniques, and procedures (TTPs) used by real-world attackers. This ensures that simulations remain up-to-date and accurately reflect the methods employed by threat actors. 

As a result, organizations can test their defenses against realistic attack scenarios, gaining valuable insights to enhance their security posture and prepare for actual adversarial events.

Top Three Benefits of Running a BAS Assessment

BAS testing hold paramount importance in the cybersecurity domain for several reasons:

Benefit 1:

Proactive Security Posture Improvement

BAS assessments allow organizations to identify security vulnerabilities and misconfigurations within implemented security measures before attackers can exploit them. It’s important to note that investing thousands of dollars into cutting-edge security solutions does not guarantee they will function as intended 24/7. 

Gaps, often caused by human errors such as misconfigurations, are inevitable and can leave your organization vulnerable. BAS helps catch these weaknesses by simulating real-world attack scenarios, providing critical insights into your defenses. This enables organizations to address issues proactively, minimize risks, and build a more resilient security posture before it’s too late.

Statistics Highlighting Gaps in Security Controls

According to the Picus Blue Report 2024, which analyzed anonymized simulation results from customers:

• 41% of cyberattacks bypass network security controls, such as IPS, NGFW, or WAF.

• 45% of web application attacks cannot be prevented by current security solutions.

• Only 18% of data exfiltration attacks are successfully prevented.

• 44% of multi-stage attacks evade security controls entirely.

These statistics underscore the critical need for BAS assessments to validate security measures, identify gaps, and address them proactively before it’s too late.

Benefit 2:

Continuous Validation of Multi-Layered Defense Strategy 

BAS assessments play a critical role in continuously validating the effectiveness of an organization's multi-layered defense strategy. A multi-layered defense strategy involves deploying multiple, complementary security controls across different layers of an organization's infrastructure, such as the network, endpoints, applications, and user behaviors. 

Testing Prevention Layer Solutions with BAS

Prevention mechanisms are designed to block threats before they can penetrate the network. BAS assessments play a vital role in evaluating the effectiveness of various prevention-focused security controls. These include, but are not limited to:

• Next-Generation Firewalls (NGFW)

• Web Application Firewalls (WAF) 

• Intrusion Prevention Systems (IPS)

• Endpoint Protection Platforms (EPP)

• Secure Email and Web Gateways (SEG)

• Data Leakage Protection (DLP)

By testing these solutions, BAS ensures that preventive security measures are properly configured and capable of stopping malicious activities before they escalate.

Testing Detection Layer Solutions with BAS

Detection mechanisms are critical for identifying and responding to malicious activities that have already breached the network. BAS assessments help validate the effectiveness of various detection-focused security solutions, including:

• Security Incident and Event Management (SIEM)

• Intrusion Detection Systems (IDS)

• Endpoint Detection and Response (EDR)

• Extended Detection and Response (XDR)

Through rigorous testing, BAS ensures these detection solutions are accurately configured, capable of identifying malicious behaviors, and able to generate timely alerts for effective incident response.

Benefit 3:

Data-driven Results for Better Communication with Decision Makers

BAS assessments deliver objective and quantifiable insights into an organization's security posture by testing the effectiveness of controls against simulated cyberattacks. This tangible data enables decision-makers to make strategic, informed decisions to strengthen security, even without deep technical expertise. 

When the true risks of security gaps are clearly communicated—such as the potential for reputational damage, revenue loss, ransom payments, and regulatory penalties—leaders can better understand the urgency of addressing vulnerabilities. By simulating both known and emerging threats, BAS assessments provide a comprehensive perspective on security readiness, helping organizations prioritize improvements and align cybersecurity strategies with business goals.

For instance, a random BAS assessment of 24 different threats shows that one of the threats was able to infiltrate the implemented prevention layer solutions (which had an 89% effectiveness score). The BAS assessment also indicates whether the unblocked threat was logged and alerted on the detection layer solutions. However, for the sake of simplicity, we will focus on the prevention layer in this example.

Figure 3. Effectiveness Score Output by a BAS Assessment

If we were to look at the threat that was achieved, we are seeing that it was a malware variant used by the BianLian ransomware. 

Figure 4. BianLian Ransomware Variant Infiltrates the Implemented Prevention Measure

When we investigate a little bit further, we can actually see the variant, its corresponding hashes, which can be searched on different platforms such as VirusTotal, etc.

Figure 5. Attack Timeline and Ransomware Variant Used in the BAS Assessment with Picus SCV

Top 6 Vendors Providing BAS Assessments Reviewed by Gartner

Gartner Peer Insights highlights the top BAS assessments, showcasing their effectiveness in strengthening cybersecurity defenses. BAS assessments are designed to simulate real-world attack scenarios, evaluate the performance of security controls, and identify gaps in an organization’s defenses.

The leading platforms enabling effective BAS assessments include:

• Picus Security

• Cymulate

• AttackIQ

• SafeBreach

• XM Cyber

• Pentera

These BAS assessment solutions offer advanced features, such as comprehensive threat libraries and detailed reporting, to help organizations test and improve their security posture. For more insights into how these platforms deliver BAS assessments, users can explore detailed feedback on Gartner Peer Insights.

How Does BAS Testing Differ from Traditional Vulnerability Assessments and Penetration Testing?

BAS assessments differ significantly from traditional vulnerability assessments, penetration testing, and red teaming in their approach, scope, and outcomes.

Automation and Continuous Assessment: Unlike traditional methods, BAS assessments are automated and capable of running continuously. Penetration testing and red teaming, on the other hand, rely on manual processes and are performed periodically. Vulnerability assessments are automated but lack the breadth of BAS in simulating real-world adversarial behaviors.

Assessing Security Controls: BAS assessments focus on testing the effectiveness of implemented security controls, covering the entire kill chain. Traditional approaches, such as penetration testing and red teaming, are limited to predefined objectives and provide only a partial view of the security posture. Vulnerability assessments, while comprehensive in identifying weaknesses, do not simulate adversarial tactics to assess security control effectiveness.

Actionable Mitigation: BAS assessments provide ready-to-use mitigation content, offering specific, actionable steps to address identified gaps. In contrast, penetration testing and red teaming often provide limited suggestions, while vulnerability assessments typically recommend software patches without deeper insights into threat mitigation.

Scope and Responsiveness: BAS assessments cover the entire kill chain and can respond to new threats within 24 hours, ensuring relevance in a dynamic threat landscape. Traditional methods lack this adaptability, as they often require new engagements or pentesting cycles to evaluate evolving threats. Vulnerability assessments update plugins within 3–5 days but do not dynamically simulate adversarial behaviors.

Risk-Free Environment: BAS assessments are risk-free, as simulations are conducted in controlled environments without impacting operations. In contrast, penetration testing and red teaming involve real-world testing, which carries inherent risks.

Comparison Table for BAS Assessments vs. Traditional Security Assessment Methods

Below, you will see a table that compares BAS solutions to traditional assessment methods, such as red teaming, penetration testing and assessment.

Figure 6. BAS vs. Traditional Security Assessment Methods Comparison Table

How Often Should Organizations Conduct BAS Assessments?

Organizations should conduct BAS assessments continuously to maintain a proactive security posture. This allows them to validate security controls, address misconfigurations, and respond promptly to emerging threats. 

Continuous assessments are especially crucial during publicized malware or exploitation campaigns targeting their sector or region, as they enable organizations to assess preparedness in real-time. Designed to be non-intrusive, BAS assessments impose minimal pressure on network resources, ensuring they can run safely without disrupting operations while providing valuable insights to improve defenses.

Step-by-Step Case Study:

Running a BAS Assessment Picus Security Control Validation (SCV)

Picus Security Control Validation (SCV) is a component of the Picus Security Validation platform, utilizing award-winning Breach and Attack Simulation (BAS) technology to automatically and continuously assess the effectiveness of your security tools, thereby enhancing cyber resilience.

In this section, we will demonstrate how Picus' Security Control Validation platform performs a BAS assessment.

Step 1:

Selecting Threats and Templates for Your Simulation

At the time of writing this blog, Chinese state-sponsored APT groups such as Volt Typhoon and Salt Typhoon were actively conducting cyber espionage campaigns targeting both South Asia and prominent telecommunication companies in the United States. 

So, suppose we want to locate a specific chat template that focuses on Chinese-based APT groups and their operations. By simply entering "Chinese state-sponsored APT groups" into the search bar, we would retrieve a curated list of relevant information and resources detailing their campaigns and activities.

Figure 7. Choosing a Threat Template for a BAS Assessment

In the subsequent stages, we proceed with installing our agent, carefully configuring it to meet the necessary requirements, and scheduling it to execute immediately. This streamlined process ensures a smooth setup and allows for prompt initiation of operations, setting the foundation for accurate and efficient outcomes.

Figure 8. BAS Assessment Configuration Settings

Step 2:

Performance Analysis of Prevention and Detection Layer Solutions

The BAS assessment results provide a detailed analysis of both prevention and detection capabilities observed during the evaluation process. While the prevention mechanisms showcase a moderate level of effectiveness, significant gaps in detection require immediate attention. These results are crucial for identifying weaknesses and opportunities to enhance the overall security posture.

To better understand these findings, the Prevention Results Overview focuses on the system’s ability to block threats, while the Detection Results Overview examines its capability to identify (a.k.a log) and alert in real time. Let’s delve into the specifics.

The Prevention Results Overview reveals that out of the 18 tested threats, only 7 were successfully blocked, leaving 11 unblocked. This translates to a 95% prevention rate, but it’s important to note that the 11 unblocked threats pose considerable risks. Furthermore, of the 347 attacker objectives assessed, 18 were achieved while 326 remained unachieved, highlighting the system’s strengths and weaknesses in blocking adversarial attempts.

Figure 9. Prevention Results Overview

In the Detection Results Overview, the performance is more concerning. None of the 3 tested threats were logged, and no alerts were generated. This means that despite having successful threats in the prevention phase, the detection mechanisms failed entirely to log or alert for any of them. This indicates a critical gap in the detection capability, as successful threats went unnoticed, potentially leaving the system vulnerable to undetected exploitation or breaches.

Figure 10. Detection Results Overview

Detailed Analysis:

Understanding Threats That Were Not Prevented

After gaining an overview of the assessment results, the next step is to analyze the specific threats that were not successfully prevented. This detailed examination allows us to identify patterns, weaknesses, and areas requiring immediate improvement to bolster the overall security posture.

The table highlights a range of threats, their severity, and the associated actions taken. For example:

• The CISA Critical Infrastructure Web Application Attack involved 11 actions, of which 2 attacker objectives were achieved, leaving 9 unachieved. However, it failed to be prevented or detected.

• Threats like the Web App Vulnerabilities Header Exploitation and Atlassian Confluence Web Attack Campaign reveal significant prevention gaps, with 18 and 12 actions, respectively, not being adequately blocked or detected.

Figure 11. Threats Run by a BAS Assessment 

After we have a good understanding of where our security solutions are not working as intended, we will look for mitigation suggestions provided by Picus Security Control Validation tool.

Step 3:

Applying the Ready-to-Apply Mitigation Suggestions

After identifying the threats that the compensating controls are unable to block, it is recommended to implement the ready-to-apply mitigation suggestions. The Picus Threat Library provides both vendor-specific and vendor-neutral mitigation suggestions. 

For instance, in the figure above, taken from the CISA Critical Infrastructure Vulnerabilities threat template, the "Cisco Unauthenticated Configuration Exposure Vulnerability" exploitation threats were not prevented. To mitigate this specific threat, the following mitigation suggestions can be implemented across various security vendors.

Figure 12. Cisco Unauthenticated Configuration Export Vulnerability Mitigation Suggestion

Implementing these mitigation suggestions ensures a proactive approach to threat management by addressing vulnerabilities before they can be exploited. This not only enhances an organization’s overall security posture but also reduces the risk of service disruptions, data breaches, and compliance violations.

Pricing for BAS Assessments with Picus Security Control Validation (SCV)

The pricing for BAS assessments is typically adaptable to the specific needs and priorities of an organization. Organizations can select tailored solutions based on their focus areas and desired functionalities. For instance, companies aiming to enhance ransomware preparedness can opt for packages specifically designed to assess defenses against ransomware threats. Similarly, organizations focusing on endpoint security can include options for prevention and mitigation content or expand their scope by adding detection capabilities for SIEM or EDR systems.

This flexibility ensures that organizations only pay for the features most relevant to their security goals, making BAS assessments both scalable and efficient while aligning with their operational needs.

BAS Assessments and Their Role in Adversarial Exposure Validation (AEV)

BAS assessments have long been integral to cybersecurity strategies, enabling organizations to evaluate the effectiveness of their security controls by simulating real-world attacker tactics. Recently, BAS assessments have been incorporated into a broader category known as Adversarial Exposure Validation (AEV) technologies, as recognized by Gartner. This shift reflects their expanded application in identifying and validating exposures across diverse IT environments, underscoring their critical role in proactive cybersecurity programs.

AEV encompasses technologies like 

• BAS Assessments, 

• Automated Penetration Testing, 

• Attack Path Mapping, and 

• Red Teaming solutions. 

While BAS assessments remain vital for emulating and simulating adversarial behaviors, the AEV framework broadens their scope to focus on continuous exposure validation and risk prioritization. This evolution, often referred to as "BAS 2.0," underscores the strategic importance of BAS assessments in modern security practices.

By simulating attacks and analyzing potential risks, BAS assessments within the AEV framework enable organizations to prioritize vulnerabilities effectively, align their defenses with evolving threats, and adopt a more dynamic, holistic approach to exposure management. This integration makes BAS assessments indispensable for managing risks in today’s complex threat landscape.