Auditor Insights: Vulnerability Assessments vs. Penetration Testing

by Sean Rosado / April 19th, 2018

Confusion About Vulnerability Assessments and Penetration Testing

In my work as a penetration tester, I work with clients who are attempting to meet security and compliance objectives through penetration tests, vulnerability assessments, and other information security-related exercises. What I’ve seen time and time again is organizations who are confused about the difference between vulnerability assessments and penetration testing. I’m passionate about educating our clients on security exercises and determining what practices they need to implement in their information security program.

The most common source of confusion on the difference between vulnerability assessments and penetration testing rests in the hands of testing firms that, either through ignorance or deceit, mislead their customers by identifying their vulnerability scanning service offerings as penetration testing. Many of these firms submit scan reports to their customers labeled as a penetration test report with little more than an edited title and their firms logo added to the document. Some will attempt to hide this approach by taking the vulnerability scan results and placing them into a custom reporting template without performing any additional testing that would support labeling the service as penetration testing. Despite numerous resources calling out this practice, it continues to be a common source of confusion.

Another very common source of confusion stems from individuals and organizations failing to properly educate themselves on the difference between vulnerability assessments and penetration testing. This is particularly true of those that attempt to manage vulnerability assessment and penetration testing services internally with untrained and/or inexperienced resources. Many times, vulnerability scanning will be erroneously referred to as ‘penetration scanning,’ suggesting that there is some level of penetration testing taking place. Others believe that by purchasing automated tools with added functionality, such as built-in password attacks and exploitation processes, they have covered the testing requirements for penetration testing.

What’s the Difference Between Vulnerability Assessments and Penetration Testing?

A vulnerability assessment is an approach for identifying and rating issues affecting in-scope systems in a given environment. Elements of vulnerability assessments include the following:

Vulnerability assessments are a highly automated process; beyond the initial scan configuration process, a vulnerability assessment does not require a significant amount of human interaction.
Vulnerability assessments are designed to highlight issues on a wide range of systems at regular intervals to allow timely identification and resolution of vulnerabilities and common misconfigurations.
Vulnerability assessments are typically performed on a quarterly basis.
Results from vulnerability assessments can be included within a larger vulnerability management program in order to identify, analyze, risk-rank, and track various metrics over time.
Vulnerability assessment tools work by leveraging a large set of pre-defined checks that are designed to mimic common attacks that might affect the target systems. These checks, which cover a wide variety of ports, protocols, and services, are sent to the target system and the resulting response is analyzed to determine if it matches a known vulnerable state.
The exact mechanism used for each request and the analysis of the response is highly dependent on the nature of the issue and can take many forms.
Although not a requirement during penetration testing, the results returned from vulnerability scanning can be a valuable resource for highlighting a large number of issues in a short period of time. Examples of this include identifying missing patches, vulnerable software packages, and other issues that can be easily identified by automated means. These scan results are particularly useful for identifying softer targets before moving on to more time consuming manual testing processes.

A penetration test is a process for identifying and exploiting vulnerabilities and common misconfigurations affecting targeted systems. Elements of penetration testing include the following:

This process is typically goal focused (e.g. attempting to gain access to an otherwise fully segmented PCI CDE environment).
Penetration testing is designed to demonstrate the risk to the target environment, should discovered vulnerabilities be exploited.
Penetration testing leverages both automated and manual testing techniques.
The frequency of penetration testing depends on the sensitivity of the systems and data in the target environment and can range from a single annual test to quarterly or more frequent execution.
Penetration testing involves a skilled professional or group of professionals analyzing the results of testing activities and using those results to inform future activities.
The penetration testing process allows for a more comprehensive assessment of the overall security of the target environment by including things such as password attacks, phishing, man-in-the-middle attacks, and other testing activities that are not typically performed during a vulnerability assessment.
Penetration testing also includes post-exploitation activity that can drive home the significance of identified and exploited issues by demonstrating ways that an attacker can leverage the level of access gained to obtain sensitive information or further compromise the environment.

Understanding the difference between vulnerability assessments and penetration testing will help you make informed decisions about these important and distinct pieces of your overall information security program, which are required by a number of regulations and compliance frameworks. When properly scoped and executed at regular intervals, these activities provide ongoing feedback that can be used to strengthen system hardening practices, patch management, device configuration management, and more. This results in a more secure environment that is better suited to protecting sensitive data owned by, or entrusted to, your organization.