Risk Map & Risk Analysis

TestBrain uses a ML model to determine the risk of a commit containing a defect. #

Background of TestBrain’s Risk Analysis ML Model #

To build a model of the factors contributing to code defects, TestBrain’s machine learning reviews the code development and test history:

  • When you first install TestBrain, it connects to your repository and builds an initial defect model using the git blame feature to find bugs and bug fixes over the entire code development history.
  • If using the optional Jira integration, TestBrain refines this model by importing the database of defects and the commits that caused them.
  • If TestBrain is integrated with your automated testing via the CI/CD, it will continue to learn and refine its risk model from on-going test results.
  • If you manually import results of manual or automated testing, it will learn from this data, too.

TestBrain does not read the code itself, making it independent of the programming language and code structure. Instead, it looks for patterns in the commits which led to defects in the past and uses these patterns to determine the risk profile of new commits. It builds a model based on process factors that have been shown to affect the likelihood of a defect. The model uses approximately 30 factors including:

  • commit details: size of commit, number of files changed, number of files and areas affected
  • area of code changed: some areas of the code area more defect prone than others, or may be stale code
  • purpose of commit: whether the commit is adding new code, updating legacy code, or fixing a bug
  • developer habits: whether the code was submitted at unusual times or before a deadline; how much experience the developer has with this area of the code
  • other activity: how many other developers are working on the same code area
  • For more technical detail, please review the TestBrain risk model white paper.

Risk Map #

TestBrain RiskMap is designed to assist the manual test team to prioritize which areas of the code to test. RiskMap shows all commits for a specified period and their risk of containing at least one defect, and the functional areas of the code affected by each commit.

By showing the number of high-risk commits in each functional code area, RiskMap acts as a guide to prioritize which areas of the code to test first and how much time to spend testing each area.

The outer ring is a list of all commits, coded for risk: red for high risk, orange for medium risk, green for low risk. The risk of defect is determined by TestBrain’s risk model. See this page for a summary of how the risks are determined.

The inner ring maps the commits to the functional areas of the code affected. Commits that affect multiple code areas are listed on the outer ring separately for each code area. In other words, if a commit affects 3 different code areas, the same commit will be shown in 3 separate places on the outer ring. Code areas are colored to match the highest risk of any commit that touches that area. Code areas that are not shown in the inner ring have not been changed.