'Protractor - run test again if fails

I'm using shell script to run protractor tests. I want to make sure that if the test fails (exit code != 0) then it will run again - three times most. I'm already using Teamcity, but Teamcity sends the 'FAIL' email and only then tries again. I want the test will run three times before sending a message. this is part of my script:

if [ "$#" -eq 0 ];
then
/usr/local/bin/protractor proactor-config.js --suite=sanity

now I want to somehow check whether the Exit Code was 0 and of not - run again. Thanks.

shellprotractorangularjs-e2efalse-positive


Solution 1:[1]

I wrote a small module to do this called protractor flake. It can be used via the cli

# defaults to 3 attempts
protractor-flake -- protractor.conf.js

Or programatically.

One nice thing here is that it will only re-run failed spec files, instead of your test suite.

There is a long standing feature request for this in the protractor issue queue. It probably won't be baked into the core of the framework.

Solution 2:[2]

function to check status

function test {
    "$@"
    local status=$?
    if [ $status -ne 0 ]; then
        echo "error with $1" >&2
    fi
    return $status
}

test command1
test command2

Solution 3:[3]

If you use protractor with cucumber-js, you can choose to give each scenario (or all scenarios tagged as unstable) a number of retries:

./node_modules/cucumber/bin/cucumber-js --help
...
  --retry <NUMBER_OF_RETRIES>     specify the number of times to retry failing test cases (default: 0) (default: 0)
  --retryTagFilter <EXPRESSION>   only retries the features or scenarios with tags matching the expression (repeatable).
          This option requires '--retry' to be specified. (default: "")

Unfortunately if every failed scenario has been successfully retried, Protractor will still return with exit code 1: https://github.com/protractor-cucumber-framework/protractor-cucumber-framework/issues/176

As a workaround, when starting the protractor I append the following to its command line:

const directory = 'build';
ensureDirSync(directory);
const cucumberSummary = join(directory, 'cucumberSummary.log');

protractorCommandLine += ` --cucumberOpts.format=summary:${cucumberSummary} \
      || grep -P "^(\\d*) scenarios? \\(.*?\\1 passed" ${cucumberSummary} \
      && rm ${cucumberSummary}`;

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Nick Tomlin
Solution 2
Solution 3

Related Questions