Should I choose a Penetration Test or Vulnerability Scan?

By Tony Wilson

Cyber Scheme Team Leader

Key differences between a Vulnerability Scan and a Penetration Test.

We are often asked why penetration tests are conducted when there are “scanning vendors” claiming they can perform a weekly automated penetration tests at a fraction of the cost.

There is a glaring mistake in this claim, namely, as Penetration Tests are conducted by humans, the term “automated penetration test” has no meaning.

So, we need understand the terminology to appreciate which approach to use in each situation. Here are some useful definitions:

• Vulnerability Scan: an automated approach that probes a given system to help identify vulnerabilities, without actually exploiting the weaknesses found.
• Penetration Test: a detailed check, performed by a human, that probes a system using proven methodologies, skill and experience, that follows avenues that are not open to vulnerability scanners, in an attempt to exploit the weaknesses found.

Both methods have their merits and help an organisation become more informed about the technical risks in their organisation.

There are typically two flavours of technical security assessments:

1. Web application testing
2. Infrastructure testing (including systems exposed to the internet and those within the confines of an internal network)

Vulnerability scanning, Penetration Testing, or both, are valid options that can be considered in either of the above assessments

Understanding which testing approach to take relies on an understanding of the impact a security breach would have on the company.

After all, a hacker defacing the “brochure” website of the local coffee shop may cause some irritation but is nothing compared to the defacement of a website that handles payment card transactions, a breach of such a site could cause distress to customers and bring the company’s reputation into question.

Therefore, the Payment Card Industry insists on quarterly scans of some sites and also mandates Penetration tests for others.

First we must consider the limitations of a vulnerability scan:

• It will not find as many vulnerabilities as a human. It may identify areas of weakness that are worth exploring further, but it cannot demonstrate the full effect of a human exploiting that weakness. A seemingly benign weakness may open-up areas of the system that could lead to further, higher impact vulnerabilities.
• The scanner may not have a definition file capable of discovering that vulnerability as it may not lend itself to automation.
• False positives are often reported and detecting these can be challenging and lead to the company focusing on the wrong area.
• Implications of a vulnerability are often unclear. The reports are generic and not written in the context of the system being tested.
• They can be deceptively complicated to configure properly. Just going with the defaults is usually a bad idea. If vulnerability scanners are used incorrectly, and there are many ways to do this, they can lull you into a false sense of security. For example, a full port scan of your firewall may result in an “all-clear – no unnecessary ports open” finding – when, in fact, the firewall had blocked the probes after the port scan was detected, potentially leaving you with open ports and services that you are unaware of.
• If scanning web applications, the logic in client-side code (such as javascript) is not always understood correctly by the scanner, so key vulnerabilities are often missed.

Compare the above, automated process to the following steps typically found in a penetration test:

• A login box is discovered, the Penetration Tester uses specialist tools to query social media sites for email addresses of employees of the company – these are then used as the usernames. The company website is queried for common words that may be used by that industry, and variations of these are created (swapping letters for numbers and special characters etc). All the usernames found are then tried using various password dictionaries – including those, more relevant words, created from the website.

This is just one of dozens of checks a Pen Tester does that cannot be performed reliably by the single click of a button.

Secondly, we must weigh the advantages of a Vulnerability Scan against a Penetration Test. Here are some benefits:

• Significantly lower cost, so scans can be repeated regularly to check for unexpected changes in externally facing infrastructure.
• Quickly Identify unexpected changes in systems, for example, a maintenance port that has been left open, a service that has become end-of-life, or a known weakness that has been reported for the version of software, or Operating system, hosting the service.
• Can highlight areas that a Penetration Tester did not have time to find (given the budget) – though Penetration Tests often include a Vulnerability Scan to inform the client of key areas to include in the scope of the more detailed tests.
• Some form of basic “penetration testing” methods are attempted by some scanners, such as probing for known default credentials in certain login scenarios, searching for common files that have been backed-up, or renamed, and left in publicly accessible areas.
• Fixing the weaknesses found on a Vulnerability Scan will, without doubt, dramatically improve the security of a system. It also helps to streamline the Penetration Test, as the tester can focus on the things not found by the scanners, rather than spending time attempting to exploit “low hanging fruit”.
• It is estimated that 80% of the most common online attacks can be prevented by employing basic cyber hygiene, and Vulnerability Scanners play a large part in supporting this area. After all, the most common attacks are found, and exploited, by automated means – for example, finding an FTP service with “Anonymous” access and uploading/downloading content.

Conclusion

The sensitivity of the information being protected, the possible impact a breach could cause, together with budget, play a large part in helping a company decide whether to opt for a Vulnerability Scan or to venture into the area of Penetration Testing.

Penetration Test costs can be reduced if the company continues to act upon all the findings of a previous, correctly configured, Vulnerability Scan.

If the company feels they are not likely to be subject of a targeted attack (an attack that looks no further than exploiting the vulnerabilities found in an automated scan), then relying on addressing those known issues may be an acceptable strategy.

Breaches arise, however, when a company underestimates the value of its information to an attacker, and do, in fact, have something of worth that would justify the attacker to manually probe the site to exploit the type of weaknesses Vulnerability Scanners cannot find.