An essential element of the application development process is scanning the software to find potential vulnerabilities.  The results of running an application security testing tool can leave a developer feeling devastated that so many weaknesses were found, or possibly gratified that the tool caught so many weaknesses before the application was released to the user community. No matter how they feel about the results, software developers must understand that by running only one application security testing tool, even the best on the market, they are missing most of the weaknesses in their code.  One tool only covers the tip of the iceberg.  There could be thousands and thousands of flaws that the analysis tool is not seeing, which could result in serious weaknesses being missed that could put valuable data at risk of being exploited. According to a study done by the National Security Agency’s (NSA) Center for Assured Software (CAS) the average tool covers eight of the 13 weakness classes (e.g. buffer handling, file handling, initialization and shutdown, and number handling), which is 61.5 percent.  This study also found that the average tool covers only 22 percent of the flaws in each of the 13 weakness classes.  If the percentage of the flaws is multiplied by the percentage of weakness classes covered, the total coverage of the average tool is only 14 percent.  This is eye opening for many software developers who have assumed that their vulnerability scanners cover a much larger area.  Missing more than 80 percent of the weaknesses in the application code should not be acceptable for any organization.

If the percentage of the flaws is multiplied by the percentage of weakness classes covered, the total coverage of the average tool is only 14 percent.

In addition to discovering that each of the analysis tools failed to report a significant portion of the flaws studied, the NSA CAS found that the tools perform differently on different languages and on different weakness classes.  However, they did find that complementary tools could be combined to achieve better results. Not only will two or more tools cover a larger area, the fact that each tool specializes in different weakness classes and different languages eliminates much of the overlap among the tools.  And, when there is an overlap, the developer can be assured that the identified flaws are not false positives and can focus on ensuring those weaknesses are fixed. Leveraging multiple tools does come with its challenges – namely in the additional time required to set up and run the tools and compare the results, as well as in the cost required to add more tools.  Furthermore, comparing the results can be painstaking as each tool produces a set of weaknesses with its own naming conventions and severity ratings. This is where Code Dx comes in to play.  Code Dx shows the overlap in the tools.  Whether commercial scanners, open-source vulnerability tools or a combination of both are being used, Code Dx will show the results of each and identify the vulnerabilities that were found by each tool.  Code Dx correlates and normalizes the results from commercial and open-source tools to deliver a consolidated set of results that provides greater coverage of potential vulnerabilities in the source code, and a better assessment of an organization’s overall enterprise risk.