Installation Instruction #
To use TestBrain to optimize test automation, you will need to add an executable script to the CI to call the TestBrain API with the test conditions.
To manually create using the provided syntax and for up to date versions of the CI script: please go here
Requirements: #
Python 3.7+
Installation Command #
pip install appsurifyci --upgrade
Available Arguments #
| argument | options |
--apikey | API Key from appsurify. Found in top right hand corner drop-down menu. Can be stored as a Secret in CI server secret storage capabilities. Reference the Secret via variable within the Appsurify Script execution in the CI configuration. |
--url | Url of the appsurify instance i.e., https://dummy.appsurify.com/ |
--project | Name of the project |
--testsuite | Name of the testsuite |
--branch | Name of the branch the tests are running against |
--commit | SHA of the commit |
Recommended Arguments #
| argument | options | ||||
--runtemplate | Options – all tests, no tests (used just to push results from prior command), prioritized tests (requires –percentage argument to be added and the percentage of tests to be run. See below for details. | ||||
--testtemplate | Options – mvn, cucumber mvn, sahi testrunner, sahi ant, testim, mocha, rspec, robotframework, cyprus, pytest, mstest, vstest, opentest. For additional integrations, talk to the Appsurify Team. | ||||
--runcommand | Command to execute tests in the target environment, if command is custom to your test suite. Note, runcommand will override generic testtemplate at time of final configuration and prioritized test execution. | ||||
--report | Location of XML reports created by the test run if this is not the default location for the test type |
Parameter Details – Tests Execution #
| Option | Details |
| all tests | Will run all tests. Will fail on any failure. Recommended for nightly or weekly test executions |
| no tests | Will run no tests. Will fail on any failure. Recommended when starting with Appsurify to just upload data |
| prioritized tests | Requires the percentage argument. Will only fail the build on new / re-opened defects. Recommended for your most frequent run type. |
EXAMPLE #
--runtestswithappsurify --url "https://ABC.appsurify.com" --apikey "api key" --project "project" --testsuite "testsuite" --runtemplate "all tests" --runcommand "mvn" --commit "commit id" --branch "branch" --report "report" --pipeoutput "true"Example in CI Pipeline: #
Setup Commands
--runtestswithappsurify --url "https://testbraindemo.appsurify.com" --apikey "NTK6cD1Ua3doV3uNDMdWzz" --project "TestBrainDemoJunit" --testsuite "Selenium" --runtemplate "all tests" --runcommand "mvn" --commit "commit id" --branch "branch" --report "report" --pipeoutput "true"Additional Arguments – For Customization #
| argument | default | options / details |
--fail | newdefects, reopeneddefects | options newdefects, reopeneddefects, flakybrokentests, newflaky, reopenedflaky, failedtests, brokentests |
--rerun | “false” | whether failed tests will be rerun |
--maxrerun | 3 | the number of times failed tests should be rerun |
--failfast | “false” | whether after each set of tests runs to determine if the build has failed (tests will still be rerun if this has been selected) |
--executioncommand | ” “ | Command to be executed following completion of the script [[teststorun]] will be replaced with the formatted list of tests to run |
--percentage | ” “ | Percentage of tests to be run if prioritized tests is selected in the runtemplate, example: percentage “20” |
For CI specific integrations, please contact Appsurify
On Summary Page: Test Bind turns GREEN #
When we start seeing test runs come through successfully, in the Summary Page – the Test Bind button will turn Green!
That’s great – you’re nearly there!
Now that the CI script is successfully installed, test runs should automatically start being pushed and your model will begin to grow in Maturity. You’ll see Test Runs populate below and your graphs being to populate with YOUR data.
Once your Model hit’s 100% or 50 Test runs, you’re Model will be trained and ready for you to begin Prioritize Testing!
Such as here!
Once your Model hit’s “100%” Maturity, your model will be trained and ready to begin optimizing your QA practice!
