From pentesting to red teaming: Security testing solutions compared

In our recent blog, What is security testing and why is it important?, we talked about how security testing is one of the single most important jobs an effective security department can do.

Without it, security leaders have no way to make informed and pragmatic decisions about the areas of investment they need to prioritize - and no basis on which to make the argument for a bigger security budget.

However, while it’s uncommon nowadays to find a business without some form of security testing program in place, different organizations tend to be at very different levels of maturity when it comes to testing. This is often reflected in the tools, techniques and processes they use for the purpose. That’s not to say that some security testing solutions are right and some are wrong - they all have their own strengths and weaknesses, and the most sophisticated security teams know how to use them in conjunction to achieve the desired outcome.

Here are the four key security testing solutions and how they compare.

Find out more about improving your security testing in our whitepaper, Breach and Attack Simulation: A Novel Cybersecurity Validation Approach.

Security testing solution 1: Vulnerability management

The role of a vulnerability management solution is to scan your environment for network and application vulnerabilities that haven’t been patched yet, and to help you manage the process of getting them fixed.

It’s one of the oldest and best-known security testing solutions, and - on the surface - has a compelling use case: many successful cyber attacks exploit vulnerabilities that have been known about to the security community for weeks or months, but haven’t been patched by their victims. If only they had been faster to identify and address those vulnerabilities.

In reality, of course, vulnerability management isn’t the silver bullet it may sound like. The challenges of working with this 25-year-old technology are twofold:

• Even with real-time visibility of new vulnerabilities (and, with some modern solutions, predictive prioritization), most security teams oversee such complex IT environments that patching remains an onerous, time-consuming task. If the solution lacks context on other control capabilities, false positives can also muddy the waters.
• More importantly, vulnerability management focuses on vulnerabilities - not the actions of threat actors themselves. So, while they help draw attention to possible points of compromise, they can’t advise on whether there’s a real risk that one of those points of compromise will be targeted. This stops security teams from taking a pragmatic view of each vulnerability, and prioritizing patches based on the value to the business.

Security testing solution 2: Pentesting

Penetration testing, or pentesting, is another common and well-known security testing solution. In a penetration test, an organization hires a trusted third party to attempt to breach their IT environment using the same tools and techniques as a real threat actor.

For obvious reasons, a pentest offers far more insight than a vulnerability scan when it comes to the question of whether a system would really stand up in the event of a cyber attack. The value of a pentest is also easily communicated to and understood by business stakeholders, and many compliance regimes such as PCI DSS specifically state they should be carried out on a regular basis.

So what are the downsides?

Well, from a security perspective, perhaps the biggest issue with pentesting is that it only reflects your defenses at a specific moment in time. Most pentests are conducted within a limited timeframe on a monthly, quarterly or annual basis - enough time for the threat landscape to be almost unrecognizable from one test to the next.

Moreover, pentesters normally look for security gaps within a pre-agreed scope. If you coordinate a pentest to look for security gaps within the parts of your IT infrastructure used to process card payments, it won’t tell you much about your overall security control capabilities.

Finally, while pentesters do normally report back on their findings, it’s not their job to give specific mitigation instructions. Establishing and coordinating the followup actions after a pentest are up to you.

Security testing solution 3: Red teaming

A red team exercise is essentially a much more sophisticated and comprehensive version of a pentest, taken a number of steps further in terms of replicating real-world threat behavior. 

Over the course of the exercise, a multidisciplinary team of ethical hackers will attempt to circumvent your defenses and achieve a specific outcome by any means necessary. Their job isn’t to test for weaknesses in a specific system or bypass a specific defense measure, but to think and act like a real threat actor. A skilled red team will offer a wider and deeper view of your threat readiness than almost any other security testing solution.

Another key part of a red team’s mandate is to work alongside the organization’s internal security team - or “blue team” - and pass on specific mitigation instructions. This helps ensure the same security gaps won’t be exploited by threats in the wild.

However, bear in mind that red team exercises do have a significant downside. Running an exercise like this is time and resource-intensive to plan, coordinate and deliver, so it’s not a technique you can rely on for anywhere near real-time visibility into how well your environment would deal with new threats.

Security testing solution 4: Breach and attack simulation (BAS)

Finally, breach and attack simulation (BAS) is a relative newcomer to the security testing world. 

BAS is a software solution that follows the same threat-centric mindset as a red team exercise, where real and documented threat behavior is used as a starting point to identify and prioritize security gaps. However, the key difference is that BAS automatically simulates this behavior to provide 24-7 insight into your readiness to defend against new and emerging threats.

As the BAS market is new and less mature than some of the other security testing solutions described above, there tend to be a few small differences in the way different vendors define BAS. We believe it should deliver on five key requirements:

• It should keep up with the threat landscape and use the latest threat intelligence as it becomes available.
• It should provide continuous security validation 24 hours a day, seven days a week, 365 days a year.
• It should be able to assess existing control capabilities, ensuring security teams aren’t flooded with false positives.
• It should provide mitigation instructions for each threat sample, linked back to existing detection and prevention technologies in use (such as detection rules for your SIEM system).
• Like red and blue team testing, it should facilitate effective communication and collaboration between stakeholders.

As BAS becomes more common, it should help solve some of the problems we discussed above around vulnerability management, pentesting and red team testing. 

That’s not to say it’s a replacement for them, of course. Effective security testing has always been about using the right tools and techniques in the right context. BAS won’t, for example, offer the same depth of insight (or, say, social engineering capabilities) as a world-class red team.

However, when it comes to balancing speed and coverage against real threat behavior, it makes for an extremely effective foundation to your overall security validation strategy.

Find out more in our whitepaper, Breach and Attack Simulation: A Novel Cybersecurity Validation Approach.