Anyone using an iterative approach to development knows that as software complexity increases, so does test time. Many organizations find that testing has become an obstacle to rapid software delivery. But could predictive test selection speed up the process?
Companies are decreasing the time between release cycles as they’ve become more proficient in continuous development and integration processes. Lengthy test cycles prevent rapid deployment. In many cases, test times increase as more emphasis is placed on performance, accessibility, and security testing.
Compounding the problem is a lack of resources. Only 18% of software testers officially qualify as trained testers. Yet, over 50% of all companies want trained professionals. Only 35% of businesses are willing to hire non-testers, even though 65% of current testers are self-taught. A shortage of trained professionals means fewer people to run tests. Tech Titan, Meta, uses predictive test selection to ease the test burden while maintaining a robust test environment. According to Meta, they reduced the infrastructure costs of testing code changes by a factor of two. The process delivered 95% of individual test failures and 99.9% of faulty changes back to developers. With such results, why aren’t more organizations using predictive test selection?
What is Predictive Test Selection?
Predictive Test Selection uses artificial intelligence (AI) to identify when to run what tests. It leverages historical test data to make predictions about which tests are most likely to expose errors. This approach reduces the number of tests to run while optimizing the results for a more efficient test cycle.
How Does Predictive Test Selection Work?
It begins with previous test data, including the following information:
- Existing test cases
- Version control logs
- Test results
- Identified defects
The process uses the collected details to identify patterns between tests and defects. From the data, the AI-based solution makes predictions about which tests to use to uncover flaws in the code.
In a continuous integration environment, the predictive model is queried for a subset of tests to run for a given change. The model evaluates the change in terms of existing data to predict the best tests for triggering possible errors. When tests run, the results are incorporated into the existing model. This process ensures that the model is continuously updated.
How is Predictive Test Selection Used?
As more organizations shift testing left, the possible use-cases expand beyond end-to-end and regression testing. Even 20-minute unit testing can benefit from predictive test selection. Manual unit testing can take days to complete if testers are unavailable. When developers receive the results, they may have moved to another task, making it difficult to address any reported errors immediately.
Unit Testing
Reporting possible flaws in the smallest executable code segment in minutes rather than days means developers receive immediate feedback. From the test results, developers can determine how to address any errors and return them for testing as quickly as possible. The earlier in development a fix is made, the less costly it is to make.
Predictive test selection identifies tests that target a specific unit of code changes. Instead of running a standard set of unit tests that takes 30 minutes, a subset that focuses on the changed code runs in five. The results return to the developer for correction.
Integration Testing
Shifting left means integration testing takes place early in the development cycle. If a change breaks another area in the code, it’s less costly to fix than with traditional methods. Waiting for a number of changes to accumulate before testing delays feedback and makes it more difficult to isolate the errors.
Predictive selection means only the tests that are most likely to reveal errors across multiple modules are used. The process relies on test impact analysis to determine which scripts will have the greatest impact on error detection. It’s possible to run a subset of integration tests whenever code is merged into a version control system.
Regression Testing
Ensuring software is backward compatible requires regression testing that can take hours to complete. Many companies automate the process and let it run during off hours. The next morning the test results are available.
Predictive selection models can identify a subset to run whenever code is merged to isolate flaws quickly. When a test is conducted on thousands of lines of code at once, the number of errors can be overwhelming. This often happens when regression testing is performed at the end of the development cycle. Incremental testing can reduce the number of flaws identified when full regression tests are run.
As testing focuses on performance and security, predictive test selection can isolate those tests that minimize the time to execute without reducing the quality of the test. Since both functions can be as time-consuming as end-to-end or regression testing, finding ways to improve operational efficiencies can reduce the impact of software delivery timelines.
What are the Benefits of Predictive Test Selection?
Predictive test selection helps software development teams:
- Receive faster feedback. Shorter test times mean developers can receive faster feedback, allowing them to make informed decisions.
- Optimize testing efficiency. With predictive selection, a subset of tests is chosen based on historical test data, reducing the time it takes to complete automated tests.
- Improve software quality. Prioritizing tests to target those that are most likely to uncover defects increases software quality.
- Enhance understanding. As teams have more time, they can use the insights gained through predictive test selection to improve software systems.
It improves operational efficiency, delivers faster feedback, and increases software quality.
Why Appsurify?
Whether your organization is beginning its shift left or has been using it for years, you need a partner to help optimize the test cycle to ensure that software deployment is not delayed or — even worse — poorly tested. With an AI-powered platform, we offer predictive test selection that optimizes testing, providing insights that strengthen your product as well as your DevOps processes. If slow test cycles are clogging your CI/CD environment, contact us to learn how predictive test selection can help.