BAS Tools

The threat landscape is constantly evolving. Adversaries are enhancing their malware to be stealthier and more advanced, typically employing an average of 10 TTPs. This makes it challenging for organizations to stay updated on the latest APT and malware campaigns in the wild. For many organizations, especially those with limited resources, maintaining a dedicated team to monitor threat actors targeting their region or sector is daunting. Even if these organizations subscribe to a threat intelligence service, manually performing adversary emulations and simulations that replicate threat behaviors is not always feasible. This is where automation becomes crucial. The Picus Security Control Validation platform, armed with an ever-updating threat library, boasts Breach and Attack Simulation (BAS) tool capabilities. This combination of advanced BAS security features ensures continuous validation of your defenses against real-world threats.

In this blog, we explore open-source BAS tools and their limitations in terms of required BAS capabilities. We also highlight how the Picus Security Control Validation platform addresses these limitations and differs by offering customers advanced BAS capabilities. Additionally, we provide actionable, vendor-based mitigation signatures to ensure customers are never left pondering, 'What's next?'

What Is a BAS Tool?

A BAS tool is a cybersecurity solution designed to simulate real-world cyberattacks on an organization's infrastructure in a safe and controlled manner. By mimicking the tactics, techniques, and procedures (TTPs) used by potential attackers, it assesses the effectiveness of the organization's security measures, identifies vulnerabilities, and offers recommendations for improvement. This proactive approach helps organizations strengthen their defenses and mitigate risks before actual threats materialize.

All Breach and Attack Simulation (BAS) Tools Are not the Same

All Breach and Attack Simulation (BAS) tools are not designed the same. 

While their overarching aim is to assess and enhance cybersecurity postures, differences arise in terms of their functionalities, threat libraries, and customization capabilities. An effective BAS tool offers comprehensive threat simulations, capturing both pre and post-compromise attack stages. The frequency of library updates, to remain relevant with the ever-evolving cyber threats, is also crucial. Some tools empower users to create customized threat simulations, catering to specific organizational needs. Additionally, integration capabilities, reporting features, and alignment with frameworks like MITRE ATT&CK can vary widely. Therefore, selecting the right BAS tool necessitates a deep understanding of these differentiators to ensure resilient security posture.

In the upcoming section, we are going to list 9 important criterias when you are considering buying a BAS Tool.

Top 9 Criteria to Consider When Selecting a BAS Tool

When deciding on a Breach and Attack Simulation (BAS) tool, several critical factors come into play. Given the continuously evolving nature of the cyber threat landscape and the increasing need for reliable and dynamic cybersecurity solutions, the following are ten essential criteria to consider:

1. Maximizing ROI with BAS Tool: Comprehensive Threat Simulation Throughout the Entire Attack Lifecycle

Your BAS tool should encompass a comprehensive threat library. It should simulate both pre-compromise (like email attacks and vulnerability exploitations) and post-compromise (like lateral movement and data exfiltration) attacks, capturing the entire cyber threat landscape.

Figure 1. Picus Security Control Validation Platform.

A BAS tool can only achieve effective performance and good ROI if it's backed by a threat library that accurately mimics the full attack path, such as malware and ransomware campaigns. An incomplete kill chain means customers can't fully test the effectiveness of their implemented security controls against specific threats, especially those targeting a particular industry in a specific country. In these situations, it's crucial to understand and analyze the threat's malicious actions and safely emulate them through simulations in the desired test environment.

2. Up-to-date Against Current and Emerging Threats with BAS Tool

Ensure that the BAS tool's threat library is regularly updated. With the cyber threat landscape ever-evolving, having a library that quickly incorporates emerging threats is paramount. Otherwise, adversaries can act on the gaps in an organization's security posture before there is a chance to remediate it.

For instance, on July 28th, 2023, the Cybersecurity and Infrastructure Security Agency (CISA) issued a security alert regarding a critical remote command injection vulnerability in the Barracuda Email Security Gateway (ESG) [1]. Identified as CVE-2023-2868, this zero-day vulnerability boasts a CVSS score of 9.8, categorizing it as 'Critical'. Notably, the Chinese cyber threat group UNC4841 has been exploiting this since October 2022. 

How did Picus Labs respond to this threat?

Figure 2. Barracuda CVE-2023-2868 Exploit Attack Threat in Picus Threat Library

In response to the CISA alert, within a mere 4 hours, Picus Labs incorporated this threat into its Threat Library. Accompanying this inclusion are actionable vendor-based mitigations and a detailed blog post elucidating the Proof of Concept (PoC) of the exploitation attack, thereby alerting Barracuda Email Security Gateway (ESG) users.

3. Validation of Enterprise Security Controls with BAS Tool

The BAS tool should seamlessly integrate with a range of both network security controls like NGFW and WAF, and detection controls like EDR and SIEM, ensuring thorough evaluation of your entire security infrastructure.

Below, you'll find a list of security controls whose effectiveness can be tested using the Picus Security Control Validation platform.

Figure 3. Some of the Enterprise Security Solutions You Can Test with Picus Security Control Validation Platform.

Picus Security Control Validation positions our agents in specific network segments to rigorously test your security controls. This approach empowers organizations to assess the efficacy of both their defensive and preventative security layers.

Figure 3. Testing Enterprise Security Solutions with Picus Security Control Validation Platform.

As depicted in the figure, the Picus agent breaches the Next Generation Firewall (NGFW) and the Intrusion Prevention Security (IPS) systems, ultimately accessing the targeted agent situated in the network segment housing the HQ endpoints.

4. Continuous and Automated Simulation with a BAS Tool

With the Picus Security Control Validation platform, you're empowered to go beyond traditional security assessment methods. Unlike conventional approaches like vulnerability scanning, penetration testing and red teaming, our platform is powered with Breach and Attack Simulation functions to conduct continuous and automated attack simulations. 

This continuous testing capability ensures that the system can readily identify vulnerabilities in security controls, especially given the ever-evolving threat landscape and frequent configuration changes. 

5. Creating Customized Threats with a BAS Tool

While the BAS is adept at conducting continuous and fully automated simulations, negating the need for manual intervention or an operator, this shouldn't lead you to believe it's entirely hands-off and unmanageable. 

Figure 4. Picus Security Control Validation Allows Customers to Run Customized Threats

For example, the Picus Security Control Validation platform gives users the flexibility to design their own unique threats, tailored with specific payloads, to test designated assets. As illustrated in Figure 4, there's a range of attack actions available for users to construct a Windows Endpoint Security attack simulation. Beyond this, the platform also accommodates testing for Linux and macOS endpoints, as well as facilitating various other types of attacks.

Figure 5. Creating Customized Threats for Five Different Attack Modules with Picus Security Control Validation Platform

For instance, in the figure given above, we are seeing that platform allows customization for all 5 attack domains;

• Network Infiltration

• Windows Endpoint Scenario

• Web Application

• E-mail Infiltration

• Data Exfiltration

to have a more solid and comprehensive visibility on their assets and security posture.

6. Actionable Vendor-based Mitigation

A BAS tool should not merely leave users with the question, “What is next?” Understandably, they might think, “It's revealing to see that the security controls, into which I've invested a significant amount of money, aren't fully guarding against these costly threats. So, what should I do with this knowledge?” 

Simply knowing that your security posture isn't as robust as you had hoped is just the beginning. Without the right and prompt remediation actions, you aren't maximizing the utility of your BAS tool. After the assessment, the BAS tool should present actionable mitigation strategies for the pinpointed security gaps, making the response process more streamlined.

For example, as of the date of this blog post, the Rhysida ransomware was gaining traction in both Latin America and the Middle East, notably targeting sectors such as education, government, manufacturing, and technology. Recognizing the potential repercussions on assets that hadn't been validated or adequately protected, Picus Labs promptly incorporated the attack simulation into the Picus Threat Library. Subsequently, 17 mitigation signatures from seven distinct vendors were introduced to fend off a potential attack if encountered.

Figure 6. Mitigation Suggestions for the Rhysida Ransomware by Picus Security Control Validation Platform

To see the vendor-based mitigations, click here. 

7. Real-Time and Customized Reporting

An exhaustive security assessment generates a significant amount of information, which has to be shared with various stakeholders in an organization. Therefore, BAS should effectively communicate its results through assessment reports tailored for diverse audiences, from executives to SOC teams and auditors.

Figure 7. Simulation Results Charts with Picus Security Control Validation Platform

These reports need to be detailed, providing a clear picture of the security landscape. They should showcase metrics like the overall security score, offering a general view of the system's security health. 

The detection rate, illustrating how well threats are identified, is crucial. Additionally, the mean time to detect (MTTD) gives an idea of the system's responsiveness. Trend statistics are important for understanding patterns over time. Metrics such as log collection can reveal the system's activity, while detection and prevention stats highlight its protective measures. 

Lastly, the inclusion of compliance-related data ensures that regulatory benchmarks are being met.

8. Mapping to MITRE ATT&CK and Other Frameworks 

The MITRE ATT&CK framework is a widely accepted standard that cybersecurity experts use to outline the strategies and techniques of cyber adversaries. 

It's common for organizations to visualize their defenses against these strategies using heatmaps. A top-tier BAS tool should automatically align its simulated threats and results with the MITRE ATT&CK framework. This ensures that the simulated attack methods and any highlighted security weaknesses are contextualized according to this industry-standard reference, making it easier for security teams to understand and address potential vulnerabilities.

Figure 8. MITRE ATT&CK Mapping of the Simulated Threats by Picus Security Control Validation

For instance, in the figure given above, it can be seen that an arbitrary host has run many simulations so far, which included threats. Each threat contains attack actions, and simulation attack actions are mapped to the MITRE ATT&CK framework. For instance, we see that out of 1562 attack actions run, the host managed to block (prevent) 944 of them, and only 25% of the non-blocked attack actions were logged and alerted.

With this statistical visibility, organizations can discern at which steps of the kill chain of a possible attack they lack resilience and can initiate actionable mitigations and remediation processes for the pinpointed vulnerabilities in both preventative and defensive layer solutions.

9. Ease of Use and Ease of Deployment 

In the intricate web of modern cybersecurity systems, enterprises employ a diverse array of security tools, some stationed on the cloud and others anchored on-premises. Handling these tools can be resource-intensive, drawing heavily on the time and expertise of SOC teams.

A well-designed BAS tool should:

• Boast an intuitive and user-friendly dashboard, making navigation straightforward.

• Minimize additional complications and not exacerbate the team's existing workload.

• Streamline the process of refining security measures.

• Boost the efficiency of security personnel, enabling them to deliver more with minimal strain.

• Seamlessly integrate into an organization's pre-established infrastructure.

• Offer versatility in terms of its deployment, catering to both cloud-based and on-site configurations.

Limitations of Open-Source BAS Tools

Figure 9. Open-Source BAS Tool Comparison

MITRE Caldera

MITRE Caldera offers a sophisticated emulation of cyber threats, giving users the capability to autonomously emulate red team engagements and customize adversary scenarios. What sets Caldera apart is its comprehensive coverage of ATT&CK techniques, making it a preferred choice for organizations leaning heavily on the MITRE framework.

Drawbacks:

• Complexity: Caldera is intricate, requiring operators with a deeper understanding to effectively utilize its capabilities.

• Post-Compromise Emphasis: While it's comprehensive, Caldera's primary focus remains on post-compromise techniques, potentially leaving some pre-compromise vectors less explored.

Atomic Red Team

Atomic Red Team is designed for granularity, allowing security teams to focus on specific ATT&CK techniques and test them individually or in chained sequences. It's a favorite in the community, owing to its comprehensive atomic test library.

Drawbacks:

• Automation Lacking: Its default setup does not automatically run tests, which means manually triggering each test, potentially reducing efficiency.

• Limited Scenario Emulation: Individual tests may not capture the complexity of real-world attack chains unless manually strung together by operators.

Infection Monkey

Guardicore's Infection Monkey is renowned for its aggressive breach simulations, focusing on lateral movement across networks. It operates more like a rampant monkey than a stealthy adversary, which is both its strength and its limitation.

Drawbacks:

• Noise Generation: Infection Monkey is aggressive and can create significant noise during simulations, which isn't representative of sophisticated, stealthy adversaries.

• Unpredictable Emulations: Its modus operandi, while thorough, may not align with specific scenarios organizations want to test, given its rampant approach.

Stratus Red Team

Stratus Red Team fills a niche in the open-source space, offering emulation tools explicitly tailored for cloud environments. Its unique focus on cloud-based threats makes it a go-to for businesses heavily invested in cloud infrastructure.

Drawbacks:

• Limited Scope: Being cloud-specific, it doesn't address threats in non-cloud environments, potentially leaving gaps in holistic security assessments.

• Narrow Emulation Range: It doesn't cover the full spectrum of threats, focusing primarily on cloud-based attacks.

Honorable Mentions:

DumpsterFire:

• Outdated Scenarios: Many of its test scenarios were created years ago and might not fully represent the current threat landscape, potentially missing new and emerging threats.

Metta:

• Limited Updates: With no significant updates since 2018, its library lacks newer adversary techniques, making it less relevant for today's threat landscape.

Red Team Automation (RTA):

• Maintenance Concerns: It hasn't been updated or maintained since 2018, raising concerns about its relevance and effectiveness against modern threats.

Conclusion:

Open-source adversary emulation tools, while valuable, come with their set of challenges. 

Organizations must be aware of these limitations to ensure they get a comprehensive view of their security posture. When relying on these tools, it's crucial to supplement them with up-to-date threat intelligence and possibly consider combining them with other tools or enterprise solutions to fill in the gaps.

In the following section, we are going to examine how Picus’ Security Control Validation platform addresses the limitations and challenges listed to provide a better RIO practice of BAS assessments.

Picus Security Control Validation Platform as a BAS Tool

Picus Security Control Validation (SCV), powered by award-winning Breach and Attack Simulation (BAS) technology, helps you to measure and strengthen cyber resilience by automatically and continuously testing the effectiveness of your security tools.

In order to provide a user-friendly experience, the Security Control Validation platform offers its customers ready-to-run attack simulations. These simulations are equipped with a single threat or multiple threats which are stored in our comprehensive and up-to-date threat library. With the Picus BAS tool, organizations can continuously validate their defenses against an ever-evolving threat landscape, ensuring that their security measures are up to date and effective in mitigating real-world cyberattacks.

Figure 10. Ready-to-Run Attack Templates in the Picus Security Control Validation Platform as s BAS Tool

How does the library get updated? 

Our dedicated Labs engineering teams conduct continuous cyber threat intelligence (CTI) research to keep up with the 

• latest threats,

• emerging threats announced by joint works of CISA, FBI and NSA,

• Advanced Persistent Threats (APTs) campaigns, 

• ransomware campaigns, 

• zero-day exploitation attacks for which there is a publicly available Proof-of-Concept (PoC), and so on.

By meticulously analyzing the samples and behaviors of malware observed in the wild, our engineers continuously update the Picus Threat Library with new threats. Before adding each attack action within a threat, we test it to ensure it doesn't harm the testing environment. As a result, organizations can swiftly assess their readiness against these threats, eliminating assumptions. They can directly test the effectiveness of their implemented security controls.

With every new threat added to our library, we also swiftly provide corresponding vendor-based mitigation suggestions. These patches play a crucial role, especially in cases where threat actors exploit vulnerabilities. For situations where a patch isn't available or feasible for organizations to implement, these mitigation suggestions become even more vital.

Equipped with visibility into their security infrastructure gaps and having actionable, quick-to-deploy vendor-based mitigations, organizations are better armed against both emerging and known threats.

To provide a solid understanding of how Picus Security Control Validation is powered with Breach and Attack Simulation capabilities, we are going to take a look at an example. Let us say that we created a simulation that is going to test our both prevention and detection layer solutions against the latest malware campaigns found in the wild. 

For this simulation, we collected 67 different ransomware threats, which contain various amounts of attack actions as you see in the figure.

Figure 11. Arbitrary Attack Simulation using the Picus Threat Library

After completing the necessary setup steps, which include agent installation, appropriate configurations, and defining the simulation schedule, the simulation process is initiated.

Upon completion of the simulation, users are greeted with an intuitive dashboard that presents the results in a clear manner. Complementing this is an in-depth analysis, encapsulated within a comprehensive report. This ensures that not only are the simulated threats being blocked, but the secondary layers of defense, like SIEM platforms, are also actively detecting, logging, and alerting any threats that might have bypassed the initial preventive measures.

Figure 12. Simulation Results with Picus Security Control Validation

Our detailed reports present metrics for both the detection and prevention layer solutions' performance during an attack scenario. To illustrate, take a look at Figure 12, where out of 61 simulated threats, 41 are blocked. The remaining 20, however, manage to navigate their way to the location of our agent.

The success of an attack is tested by if any of the individual attack actions comprising the threat reach our Picus agent. If even one action gets through, the entire attack is immediately labeled as successful. It's worth noting that in the case of the 20 threats that circumvented the defenses, the detection layer solutions failed to log or generate alerts. This discrepancy indicates a pressing need for refinements in the SIEM configurations.

For instance, take a look at Figure 13. Out of money blocked ransomware threats, a variant of Spacecolon ransomware could manage to breach the defenses. Also, we see that two of the implemented SIEM solutions of an arbitrary host is not logging and alerting the breached threats.

Figure 13. Spacecolon Ransomware Not Being Prevented and Detected

Below, you can see the overview of the attack and the example file that Picus Labs engineers constructed for the threat. The VirusTotal analysis of the malicious file can be reached from here.

Figure 14. Spacecolon Ransomware Threat Card

Having visibility into the security gaps against the Spacecolon ransomware, organizations can apply the vendor-based mitigation suggestions that the Picus Security Control Validation platform provides.

Figure 15. Spacecolon Ransomware Not Being Prevented and Detected

In Figure 15,  you are seeing how Picus Labs incorporated mitigation recommendations from five distinct vendors, enabling customers to promptly enhance their prevention layer solutions against potential Spacecolon ransomware attacks.

In conclusion, the Picus Security Control Validation platform stands as a beacon of cyber resilience in an increasingly volatile digital landscape. Harnessing the power of its state-of-the-art Breach and Attack Simulation (BAS) technology, it offers organizations a distinct advantage in preemptively identifying vulnerabilities and responding effectively. The platform's commitment to real-time updates, rigorous threat intelligence research, and actionable mitigation solutions underpins its high return on investment. By continuously refining and simulating threat scenarios, it allows organizations to stay one step ahead, ensuring their defenses are robust and prepared. The future of cybersecurity demands a proactive approach, and Picus exemplifies this principle, bridging the gap between knowledge and actionable defense.