Silent errors using mocha with delay flag

Question

We are using mocha on our Open Source project https://opentermsarchive.org and ended up in the CI deploying our code despite errors in the tests.

Problem is that those errors failed silently with an exit code of 0, which is really bad.

This means

• tests are launched
• tests fail with no error
• CI considers tests have passed even though they did not run completely

We are using mocha programmatically with the following config

const mocha = new Mocha({
  delay: true, // as the validation script performs an asynchronous load before running the tests, the execution of the tests are delayed until run() is called
  failZero: true, // consider that being called with no service to validate is a failure
});
const VALIDATE_PATH = path.resolve(__dirname, '../scripts/validation/validate.js');

(async () => {
  mocha.addFile(VALIDATE_PATH); // As `delay` has been called, this statement will not load the file directly, `loadFilesAsync` is required.
  await mocha.loadFilesAsync() // Load files previously added to the Mocha cache with `addFile`.
    .catch(error => {
      console.error(error);
      process.exit(2);
    });

  let hasFailedTests = false;

  mocha.run()
    .on('fail', () => { hasFailedTests = true; })
    .on('end', () => {
      if (hasFailedTests) {
        process.exit(1);
      }

      process.exit(0);
    });
})();

Answer 1

Well

this is due to the fact that mocha is silenting unhandledRejection when you are using the delay property

You can see that here https://github.com/mochajs/mocha/blob/master/lib/runner.js#L198

So a way to bypass this is to add

process.on('unhandledRejection', reason => {
  // Mocha catch unhandled rejection from the user code and re-emit them to the process (see https://github.com/mochajs/mocha/blob/master/lib/runner.js#L198)
  // Re-throw them so that the validation command fails in these cases (for example, if there is a syntax error when parsing JSON declaration files)
  console.error(reason); // keep the console error to have the whole stack trace
  throw reason;
});

before the instantiation of mocha