Penetration Testing

Penetration testing (pentest) receives a disproportionate amount of attention. Penetration testing does not reduce risk or increase security. Penetration testing actually increases the chance of information disclosure and application failure. Penetration testing hits the charts at number 20 (sharing that slot with Red Team Exercises) in the SANS/CIS Top Twenty Critical Security Controls.

Attempting to punch holes in a bucket does not harden the bucket. Failing to punch holes gives confidence about the bucket’s integrity.

Penetration testing provides some confidence that you are effectively implementing the other security controls. A penetration test may reveal gaps in your coverage. Remediating these gaps (working on the other security controls) would reduce risk and make you more secure. A penetration test is not a rigorous examination of the other security controls. You should not expect a penetration test to reveal all weaknesses.

A more rigorous approach to identifying gaps and weaknesses is to review the other security controls to determine their completeness. In other words, check your work. Like other complex issues, break the security problem into its constituent parts and address those parts.

So why do penetration testing at all? Agreements, such as PCI, require penetration tests.  An agreement could not say “implement standard security controls well,” but an agreement could say “implement standard security controls in a industry-approved manner” and here a penetration test is a mechanism to verify an implementation.

Penetration tests provide some confidence that you have implemented other security controls well. Familiarity with penetration testing procedures can drive home the need to focus upon the other critical security controls.

Web application vulnerability testing is often confused with penetration testing. Application software security, including web application security, is Security Control 18. Black box web application testing may be a step in a penetration test. When someone’s job title is “web application penetration tester,” there is confusion about what the job requires but the job has a flashy title.

The more important, the lower numbered, security controls should receive the majority of attention. Vulnerability scanning and remediation (Security Control 4) should occur on the scale of once a month. Vulnerability scanning should consist of configuration verification (Security Control 3) and software version (including patch level) verification. Check your work. If configurations seem weak, examine how the configuration came to exist. If changes failed, investigate why changes failed. Investigate which environments were missed (Security Controls 1 and 2). Alert when administrative privileges are used and follow up on those alerts (Security Control 5). Continue with the remaining of the SANS/CIS Top Twenty Critical Security Controls.

Implement the basics well and check your work. A penetration test will fail to reveal an exposure.

The boundaries of penetration testing must be clearly defined. A penetration test is what the contract says it is.

Case in point: the episode 229 podcast of PaulDotCom, in which the customer acknowledges that their ID card readers have a known vulnerability, tells the pentester “you can’t test that” … which evokes shameless derisive laughter. But why should it? Isn’t the customer trying to find out what they don’t know? Aren’t they trying to identify where their other critical controls are incomplete? Replacing an ID card reader system is an expensive process; knowing that it has a vulnerability doesn’t make a budget appear. Exclude that known vulnerability from the penetration test. A penetration test is what the contract says it is.

See also:


Comments are closed.