“I never did anything by accident, nor did any of my inventions come by accident; they came by work.”

Thomas A. Edison

Introduction:

Security has been an afterthought in most product organizations. Security evaluation of a product always happens towards the end of the release cycle. This practice is seen in organizations which follow a waterfall model as well as agile methodologies. This is merely because of a lack of security awareness among the product management as well as the product architects and development teams. As a result, organizations are exposed to the risk of zero-days or massive security exploits post production.

To cope with the fast-growing market needs, product organizations must adopt the latest development methodologies such as Agile and DevOps. Fail fast should be the mantra to prove their existence in the market. Security should become an integral part of their SDLC. DevOps practice emphasizes communication and collaboration among various stake holders and supports this by automating the process of various functions like security, performance, quality, etc.

Conducting security awareness programs and secure coding training programs for product management, product architects and development teams on regular intervals should be in place. Product teams must understand that incorporating security checks early in the secure SDLC will reduce the cost of fixing the security vulnerabilities found at the later point of the life cycle. So, one should incorporate automated security checks as part of continuous integration process which can be scheduled daily or weekly depending on the need.

The importance of Security Static Analysis Tools:

The manual process of reviewing the source code to identify security vulnerabilities without the help of any automated tool is very tedious and time-consuming process. Moreover, this approach will not fit into the latest development practices like Agile and DevOps. Hence, organizations must look for either open source or commercial automated tools to fit into this automation world.

Static Analysis Tools: These are designed to analyze an application’s source, bytecode, or binary code to find security vulnerabilities. These tools find the security flaws in source code automatically.

Open Source Vs Commercial Static Analysis Tools:

Open Source Static Analysis Tools Commercial Static Analysis Tools
There are plenty of open source static analysis tools out there. However, the support they provide is limited to certain programming languages. There are various commercial tools out there in the market which provide greater support for various programming and help to overcome many security challenges.
In-depth coverage of vulnerabilities is a challenge. In-depth coverage of vulnerabilities is considerably good.
We cannot depend on them for any enhancements and support on demand. Might have a better road map to support latest technologies, but choosing the right one, place a very vital role in delivering the secure product.

Table 1 – Open Source Vs Commercial tools

Here is the list widely used open source security static analysis tools.

Open Source Tools Supported Languages
OWASP LAPSE+ Java
Flawfinder C/C++
RIPS PHP
VisualCodeGrepper C/C++, C#, VB, PHP, Java, and PL/SQL
YASCA Java, JavaScript, and C/C++
Bandit Python

Table 2 – prominent open source tools

Details on commercial static analysis tools can be found from various data analysis reports. I am at this moment sharing a Gartner magic quadrant application security testing on security static analysis tools for your reference.

Figure 1- Gartner Magic Quadrant for Application Security Testing

Note: Author of this paper is not recommending any specific tool.

Common mistakes committed by organizations while selecting commercial SAST tools:

  1. Perception: Organizations tend to choose the security static analysis tool based on its position in data analyst’s reports. Alternatively, they tend to choose a tool which is used by a top organization without knowing the reasons behind why they are using a particular top brand commercial tool.
  2. Evaluation: Evaluation is done on a small code base which is not even part of the projects against which they will be using it in future. In the end, after purchasing the tool, when they run the tool on the actual, complex code bases, that is when they realize that, the tool does not serve their purpose. Sometimes, the responsibility of evaluating a tool will be given out to a third-party group who are not going to use the tool extensively. There is a lot of disconnect in this process as the third-party team is not aware of exact needs of product teams.
  3. Missing the big picture: These mistakes include, accepting the tool with Out-of-The Box features provided by the vendor and not investing time and energy to customize the tool to use its capabilities in fullest. Also, the purchaser may not understand the end to end pre-requisites to fulfill the organizational needs and end up in not meeting the ROI.

Framework for selecting the best commercial static analysis tools:

Based on my experience with SAST tools, I have listed down various important aspects to consider before purchasing a commercial tool.

Figure 2 – Framework for commercial tool selection

Accuracy:

  • Does it produce consistent scan results over subsequent runs on same code base?
  • How accurate is it? False Positive rates? False Negative rates?
  • Does it take less time to scan a less #LOC (Lines of Code)?
  • Can we tweak the rule base to optimize the scan results?
  • Can it detect a variety of vulnerabilities in depth apart from OWASP categories?

Integrations:

  • Can it be integrated into Source Control Systems?
  • Can it be integrated into the developer’s IDE?
  • Can it be integrated into Build servers’ tools like Jenkins/Git CI etc.?
  • Can it be used to schedule automated scans independent of build server?
  • Can it be integrated into Defect Tracking Systems?
  • Can it provide automated threshold checking capability?

Ease of Use:

  • Can developers collaborate easily among themselves and work on findings reported by tools?
  • Can we create custom rule base easily to optimize the scan results?
  • Is it easy to setup/use without much struggle?
  • Is it easy to upgrade to latest versions?
  • Can the tool be setup with minimal infrastructure cost?

Support:

  • Does it support programming languages used in your products?
  • Does it support incremental scan?
  • Does it require a fully buildable set of sources/ runs against binaries?
  • Does vendor provide greater support to address tool related issues?
  • Do they have a better road map for improving the product/enhancing the product as per latest technology needs?
  • Does it fit in your budget?

Reporting:

  • Does it support all report formats like “csv,” “pdf,” “xml” & “html” etc.?
  • Does it provide enough remediation details to fix a reported issue?
  • Does it provide dashboards to capture security metrics?

Top 5 Key challenges faced by organizations:

Having attended several conferences/workshops organized by prominent commercial security static tool companies, I have come across consistent challenges faced by many organizations. They have adopted some of the top-notch security static analysis tools but suffering from using these tools effectively. Facing bigger challenges as tools do not cater to their basic fundamental needs resulted in wasting the money spent on tools.

Here are the top 5 challenges, details, and solutions to over these challenges.

Challenge 1: Inconsistent behavior of tool

Details

Scan a source code of a particular change set ID /Commit ID using a standalone tool. Observe the scan results. Scan the same source code again using the same standalone tool. Though there is no change in the code base, one can observe different scan results / new vulnerable categories being reported by tools. Also, there is a change in total #issues.

Now scan the same source code using IDE-scanner plugins. You will see some different set of findings compare to stand alone tool.

Developers get confused which scan results to consider and work on fixes.

Solution:

Look for a proper commercial static analysis tool which provides following features out of the box.

1.    Works on client-server based architecture with centralized engine facility used by standalone tool and IDE plugins.

2.    Produces accurate scan results in any situation.

By this approach, if a project is created in central standalone location, the same is accessible via IDEs. If the user triggers scan using IDE-plugin / Stand Alone, centralized scan engine is used to scan the source code. So, there is not be any difference in scan results.

Table 3 – Challenge 1

Inconsistent behavior of tools brings down the morale of both development and security team. They tend to spend time in analyzing what could be the root cause being inconsistent results there by it reduces their productivity.

Challenge 2: Managing the security vulnerabilities

Details

Challenge areas:

1. From previous release version to the latest release.

2. When there is the latest version of the tool to be deployed.

When the release branch is created in version control system, users have to copy the latest code base to older release source code folder structure manually.

There is no proper automatic way to manage the issues / mark them as false positives and move them automatically to next scan results or next release accurately without manual work.

When there is a new version of the tool, entire scan data is lost. Need to reload their dependent files manually. Some cases, back-up is not maintained by developers. So, it is a complete loss of data. Fresh scan and fresh false positive identification activities need to be performed.

Solution:

Look for a proper commercial static analysis tool which provides following features out of the box.

Integration with version controls systems

Provides a project branching capability

Identifies & remembers the scan results based on the code flows.

Provide lighter version of tool upgrade solution

Such tools provide an option to point to the previous release source folder from version control system while creating a project. When moving to the latest release, you have to branch the existing project from static analysis tool and point to the latest release folder in version control system.

This way, tools will automatically carry forward the previous release scan results with accurate result state as-is, into the newly created release branch.

Also, when there is a light version of tool version upgrade software is in place, no need to un-install the entire solution and reinstall the complete setup from scratch.

Table 4 – Challenge 2

Tool vendors should put themselves into the shoes of development team/security analysis team and develop the tools. They should solve these small fundamental issues and support their customers.

Ethical Hacking Training – Resources (InfoSec)

Challenge 3: High number of False Positives

Details

Most of the static analysis tools are prone to generate a lot of false positives. This kind behavior of tools brings down the morale of both developers and security analysts. Both the parties have to lot of time to identify real issues and false positives from thousands of issues generated by tools.

Some tools do not consider the false positives marked in the previous scan in the current scan. So, the number of issues are high.

Even if security analysts want to tweak the rule base to optimize the scan results, most of the tool vendors do not provide greater visibility towards tweaking rule base to reduce the false positive count. Security analysts are not exposed to see how the rules that are written for existing vulnerable categories in the tool to customize the rule base according to their needs.

It also depends on how vendors are providing their support to organizations.

Solution:

Look for a proper commercial static analysis tool which provides following features out of the box.

Which doesn’t flag already marked false positives as new issues in subsequent scans?

Provides greater flexibility to customize rule base to reduce false positives.

Provides customization to create various result states like “risk mitigated,” “not – exploitable” & “risk-accepted” etc.

Ways to reduce the False Positive count.

In a programming language, if there are default rules for various frameworks related and which are not used in your product turn-off those rules from being executed.

For most of the input validation related categories, if the input is coming from trusted source like config/data source, customize the rule to remove such occurrences from scan results.

Around 40% of the issues marked as false positives fall under the category of error handling. If we clearly observe, most of the issues will be around logging the default exceptions to log file specific to a programming language. There is no sensitive information being logged here. So, you can customize such rules to reduce the false positive count.

Go through the rule base thoroughly and identify the vulnerable categories which may not be possible in your product and un check them from being executed.

Configuration related issues which can be reported by dynamic scan can be grouped and uncheck those rules being executed.

If the tool is by default running rules related to code-quality checks, please uncheck them as those can be covered as part of relevant functions in the organization.

If you do not want to remove the vulnerability category from scan results as you are not completely sure, creating a custom result states and marking the vulnerabilities will help you track the issues in a better way.

If you follow the steps mentioned above, there is a possibility of reducing the false positive count by 50%.

Table 5 – Challenge 3

In the case above, performing threat modeling will help you to identify the vulnerabilities which are possible in a particular product. This also depends on what are the business needs and where the product is going to get deployed etc. This also helps you to define the trust boundaries.

Challenge 4: Lack of collaboration among developers & Tracking

Details

After all Static Analysis Tools are used by developers on a daily basis. If we do not facilitate proper collaboration among developers, we are not doing justice to the development teams.

Take an example of a product team where 10 developers are working on a particular product. It is not going to be productive, if there is no common interface provided to these 10 developers to work on the scan results and update the issue status centrally in the tool itself to avoid duplicate work and provide them an option to filter-out the issues based on the modules/files that they worked on and assigning issues among themselves etc.

It is equally important that security analyst working on the product is provided with a common interface where he/she can go through all the issues marked by the development team.

Without this, tracking the security issues using excel or other formats, manually updating the status of security issues and circulating among 10 developers + one or two security analysts will become tedious and non-productive.

Solution:

Look for a proper commercial static analysis tool which provides following features out of the box.

Common interface to track down issues, assign issues to developer’s/security personnel, the capability to provide comments for false positives in the tool itself for better tracking purposes over the releases.

Results states/scan results should be seamless when working from IDE / standalone software.

Table 6 – Challenge 4

Security Static Analysis tools are mainly used by developers to get the feedback on the security maturity of the source code being written. Tools should be developer friendly so that they can adopt these tools easily.

Challenge 5: Long scan times & automation road blocks

Details

It is observed that most of the commercial tools take a long time (10 hours) to complete scan of small code base 100KLOC for specific programming languages.

If this is the case, automating the static scan as part of build pipeline/DevOps pipeline is impossible.

The same thing holds good if a product team is running daily builds and if there is no support for incremental scans, then this product cannot get into DevOps mode w.r.t security automation. Incremental scans should ideally take less time than the complete code scans.

If the tool does not provide automated threshold check utility to fail a particular build based on the number high, medium, and low severities, we will have to end-up developing a utility for the same.

Solution:

Look for a proper commercial static analysis tool which provides following features out of the box.

Reduced scan time

Better integration with build servers/source control systems for automated scans

Provides support for automated defect raising system with minimal manual efforts

Provide automated threshold checks for decision making

Table 7 – Challenge 5

Key Success Factors:

It is very important during the evaluation process to run the tools against the actual source codes to get rid of last minute surprises after purchasing the tool. Involve a couple of development team security personnel to be part of the evaluation process.

To become successful in identifying the right tool, to use it effectively, to get the accurate feedback on the source code written early in the life cycle, please make sure the following factors can be achieved by tools.

  • Accurate results
  • Reduced scan time
  • Less False Positives
  • Developer friendly
  • Integrations with various third-party tools
  • Customizations
  • Better reporting
  • Support

Conclusion:

Security Static Analysis tools play a very important role in secure software development life cycle. Organizations must understand the importance of static analysis tools and deploy them. Be aware of the key challenges in using these tools and make use of the solutions laid out in this document.