FAQ
TestBrain Functionality
What is TestBrain?
How Does TestBrain Speed Up Automated Testing?
TestBrain then analyzes your test results to filter out flaky test failures (tests that fail for reasons unrelated to the code changes) and failures due to already open bugs so they don’t break the builds.
Together, TestBrain allows you to test each commit as it is applied and get useful results back to the developers immediately about whether their specific code changes pass or fail testing.
How Much Faster Will TestBrain Make My Automated Testing?
For typical projects with a substantial code base and large test suite, TestBrain should reduce the number of tests to run by at least 90%, speeding up testing by 10x or more.
The TestBrain configuration allows you to choose between running a wider set of tests and having higher confidence in not missing any failures or running a smaller set and getting quicker results. That trade-off depends on how frequently you run the full test suite and how long the tests take.
What if the Reduced Testing Misses a Defect?
What are Flaky Failures?
What are the Causes of Flaky Failures?
- Device losing connectivity or power
- Browser crashing
- PC or phone downloading updates
- Race conditions (thread.sleep)
- Asynchronous calls
- Stale element exception
- Element not visible exceptions
- Xpath and CSS locators
- Unstable infrastructure
- Unstable application
How Does TestBrain Eliminate Flaky Failures?
How Does TestBrain Help Manual Testing?
TestBrain also provides risk alerts to notify the test team of commits likely to contain defects and any changes to files you’ve marked as sensitive. This allows you to begin checking the riskiest changes immediately instead of waiting until the end of the sprint, or notifying the developers or code reviewers of the risk of these changes so they can be double-checked.
TestBrain Installation and Integration
How Do I Install TestBrain?
- Cloud version: TestBrain runs on Appsurify’s infrastructure on AWS. Just add a webhook from your repo and a script into the Jenkins pipeline to connect to TestBrain in the cloud.
- Self-hosted: Install TestBrain on 3 docker images on VMs on your own servers with no connection to Appsurify.
- Agent: Install a small script on your own VM that connects to the repo and sends only the commit metadata to TestBrain running on AWS. This allows you to take advantage of TestBrain running in the cloud (so we handle all the management and updates) while guaranteeing while limiting our access to the repository to read-only access to metadata.