In the world of application security testing, the terms “code coverage” and “vulnerability coverage” are frequently used, but what do they really mean? Essentially, code coverage is the amount of the code that is scanned to identify potential vulnerabilities in a software application.  And, vulnerability coverage is the amount of defects or system misconfigurations in the software code that could pose potential threats.

With Static Application Security Testing (SAST) tools, also known as white-box testing tools, 100 percent code coverage is possible since they have access to the internals of an application.  These tools are used early in the software development process to test the application from the inside out and can go line by line to test the source code, byte code or the binaries.

Black-box testing tools, also known as Dynamic Application Security Testing (DAST) tools, often have limited code coverage based on how much of the attack surface they are able to determine and the input they fuzz into the application to cause different types of vulnerabilities.  DAST tools test from the outside looking in.  They test the application when it is running and attempt to penetrate it to find potential vulnerabilities, including those outside the code and in third-party interfaces.

Code Dx eases the pain of leveraging multiple application security testing tools to help increase vulnerability coverage.  Code Dx not only shows the overlap in the tools, it correlates and normalizes the results of all the testing tools into one, easy-to-understand report.

In either case, you may be ecstatic with the results of a test that only identifies a small number of weaknesses; however, if you are only testing a small portion of the code, the results don’t provide a true representation.  Even static analysis tools that have access to all the code, do not provide full vulnerability coverage.

Software developers and penetration testers today leverage sophisticated software assurance testing tools to find vulnerabilities in their software, but the challenge is that no one tool provides adequate coverage of an entire target application.  The application security testing tools on the market today are developed to find certain types of weaknesses in applications, and although they may be excellent in identifying specific vulnerabilities, no one solution can do it all.  Each tool specializes in different languages and different weaknesses classes (e.g. buffer handling, file handling, initialization and shutdown, and number handling).  In fact, experts say that the average tool only covers 14 percent (see Is Your Code Coverage Complete?).  Therefore, it is an industry best practice to leverage multiple tools that complement each other.

Although there are multiple advantages to using more than one application security testing tool, there are also hurdles to overcome.  With each additional tool comes an additional cost, more time to implement and run it and the challenge of comparing the differing sets of results (i.e. naming conventions and severity ratings).

The Code Dx team has also developed a free OWASP solution, called Code Pulse, to provide insight into real-time code coverage analysis testing, and in helping testers see at-a-glance their results while users perform their manual or automated testing and identifies code coverage in real-time as the tests are being performed. For more information on Code Dx, contact us at or at (631) 759-3993.  Download a Free Trial and Start Testing Your Code Today!