Search below to view our most pressing questions regarding automation testing tools. Have a question that’s not listed? Impressive. Contact us and we’re happy to provide an answer.
What is TestBrain?
TestBrain is a plug and play software automation testing tool that uses machine learning to make automated and manual QA faster.
Who would benefit from TestBrain?
How does TestBrain speed up automated testing?
TestBrain uses Predictive Test Selection to find and run the subset of tests from your automated test suite that is relevant to the specific code changes.
TestBrain then analyzes your test results to filter out flaky test failures (tests that fail for reasons unrelated to the code changes) and failures due to already open bugs so they don’t break the builds.
Together, TestBrain allows you to test each commit as it is applied and get useful results back to the developers immediately about whether their specific code changes pass or fail testing.
How much faster will TestBrain make my automated testing?
A comprehensive analysis using the open source jsoup project showed that after 500 runs, TestBrain reduced the number of tests by 98.5% without missing a single defect.
For typical projects with a substantial code base and large test suite, TestBrain should reduce the number of tests to run by at least 90%, speeding up testing by 10x or more.
The TestBrain configuration allows you to choose between running a wider set of tests and having higher confidence in not missing any failures or running a smaller set and getting quicker results. That trade-off depends on how frequently you run the full test suite and how long the tests take.
How does TestBrain help manual testing?
Using its automated testing tools, TestBrain generates a heat map to show where to prioritize your testing. TestBrain analyzes each commit for the likelihood of defects then generates a RiskMap showing you which commits have the highest risks and mapping the commits to functional areas of the code. You can use this map to see which areas of the code need the most testing and which haven’t been changed at all.
What are risk alerts?
Does TestBrain reduce local or Cloud infrastructure utilization rates?
What if the reduced testing misses a defect?
What are flaky failures?
What are the causes of flaky failures?
Flaky failures are caused by anything external to the code being tested that can interfere with the test execution. Typical causes include:
- Device losing connectivity or power
- Browser crashing
- PC or phone downloading updates
- Race conditions (thread.sleep)
- Asynchronous calls
- Stale element exception
- Element not visible exceptions
- Xpath and CSS locators
- Unstable infrastructure
- Unstable application
How does TestBrain eliminate flaky failures?
Does TestBrain work if I am running my tests in parallel?
Yes, TestBrain would actually significantly help in this category. TestBrain would push less tests through your parallel threads. So if you are attempting to reduce test execution times by running your tests in parallel, TestBrain will significantly enhance your current practice and see test execution times reduced by 95%+. TestBrain would also give teams the option to reduce the number of parallel threads required to achieve faster test times, thereby, freeing up significant CPU/Memory resources.
Does TestBrain work if I am pushing my tests to the Cloud?
TestBrain Installation and Integration
How do I install TestBrain?
There are two options for installing TestBrain:
SaaS or Cloud Version: TestBrain runs on Appsurify’s infrastructure on AWS. Just add a webhook from your repo and a script into the CI/CD pipeline to connect TestBrain in the cloud.
On-Premise or Self-Hosted: Install TestBrain on a single docker image on a VM on your own servers with no connection to Appsurify.
What repositories does TestBrain work with?
What testing frameworks does TestBrain work with?
TestBrain works with any automated tests that can generate JUnit, XUnit, or XML output and has a way to specify individual tests to run instead of an entire test suite. See full integration list here.
What type of tests does TestBrain work with?
TestBrain works with any automated tests that generate JUnit, XUnit, or XML output. This includes unit tests, integration tests, end-to-end tests, API tests, UI tests, etc. All that is required is a test that returns a pass/fail criteria.
Can I try TestBrain without changing how we work now?
Will TestBrain create tests for me?
Is Jira required?
What is required to be successful with TestBrain?
For prioritized automated testing, two items:
1. Access to the development code repository.
2. Install a small script to the CI/CD pipeline.