Don’t Delete your Flaky Tests, let TestBrain handle them!
Flaky tests are the Darth Vader of testing. The scariest thing in the universe.
A Flaky Test is a script that fails for no apparent reason or change to the code. A lot of companies have major problems with them. Most teams only have two ways of dealing with them though, they either 1) fix the test or 2) delete the test.
Let’s go through why deleting the tests is not a good idea.
Teams have spent a long time writing those tests, and they’ve written them for a reason. It is demoralizing to delete tests after you’ve spent time creating them, especially if you weren’t the reason they became flaky and or they work fine in your test environment.
These tests still have value. They can still find new defects and even the flakiness of the test can be indicative of the overall health of the system. Let’s say you have a test which is flaky one time in a hundred. If that increases to ten in a hundred, that is very important for testers to know.
Additionally, it might not be the test’s fault they are flaky. The flakiness could be caused by the underlying infrastructure, such as your docker container or the mobile device test environment. In this case, deleting the test is unwarranted and the problem may either be patched or solved by changing mobile device. If you were to delete every test that was flaky only to discover it was caused by the mobile phone, that would be a huge loss to the project. And Costly!
There are other solutions to flaky tests. You can deal with them in more elegant ways similar to how the larger corporations like Google and Facebook do. Your team can implement some of these workarounds themselves or at Appsurify this is part of our solution, TestBrain. We handle your Flaky Tests for you, so you don’t have to.
So please don’t delete your flaky tests – let TestBrain handle them!