Cypress Mochawesome Report

By Qautomation on August 5, 2020 • ( Leave a comment )

Reports have always been an important aspect of testing and creating useful reports is something of a trend in QA.Everyone would agree that what clients really want to know is the actual state of their system, not how the tests themselves work. With proper reporting, we can tell them if the software functions properly, what errors were discovered and how they affected the system.

When using Cypress for testing applications, there are some reports you get out of the box. Those are video recording of test and if test fails, screenshot in the moment it fails. Quite often, that is not enough. You want some HTML output for report.

What is Mochawesome?

Mochawesome is a custom reporter that you can integrate into your Mocha tests. It provides a custom HTML report so you can easily see the test results and filter which of your tests pass or failed.

There are three libraries that you would need to get started.

mochawesome – this will convert your tests to JSON output

mochawesome-report-generator (marge) – this will take the JSON output provided by mochawesome and transform it into a HTML report

mochawesome-merge – this will merge several mochawesome JSON outputs into one. If you only have 1 test file then you don’t need this. But in reality, you will always have more than 1 test file in your framework .

Integrating with Cypress

Since Cypress is built on top with Mocha, we can use Mochawesome with Cypress. The first thing that you need to do is install the required dependencies.

npm i --D mocha mochawesome mochawesome-merge mochawesome-report-generator

Once the above libraries are installed, we would need to make some modifications to our ‘cypress.json’ file .

Prerequisite

Node js and npm should be installed on your system.

Essential packages

"mocha": "^7.1.2",
"mochawesome": "^6.1.1",
"mochawesome-merge": "^4.0.3"

Report configuration in cypress.json

"reporter": "mochawesome",
  "reporterOptions": {
      "reportDir": "cypress/reports/mochawesome-report",
      "overwrite": false,
      "html": false,
      "json": true,
      "timestamp": "mmddyyyy_HHMMss"
   }

where

reportDir – it’s the directory to which we’re going to output the results of our tests.
The overwrite flag – toggles the rule that allows/disallows overwriting previous reports.
The html flag – generates a report when a test is completed.(set to false because we don’t want the html report to be generated after the test run)
The json flag – generates a json file for each completed test.

Script configuration in package.json

If there was just one file with spec tests in the project, we could simply set the html flag to true and that would be enough for a fully functional reporter. Unfortunately, if we were to run tests on a few files, the HTML report would be generated for each of them separately. Since time and convenience are important considerations for us, we want all the reports to be included in one file.

There is quite a simple way to solve this problem and get a proper, fully compatible reporter.

"scripts": {
    "cleanup": "rm -fr cypress/reports/mochawesome-html-report",
    "merge_reports": "mochawesome-merge cypress/reports/mochawesome-report/*.json > output.json",
    "mochawesome_report": "marge --reportDir cypress/reports/mochawesome-html-report -t Cypress_Test_Report -p Cypress_HMTL_Report -f Cypress_HMTL_Report --timestamp mmddyyyy_HHMMss --charts true output.json",
    "cy:report": "npm run cleanup && cypress run && npm run merge_reports && npm run mochawesome_report"
  }

where

cleanup – removes the directory that contain reports before new tests are run (a script can be added in order to generate timestamps reports. With that, there won’t be any need to remove the directory anymore. Otherwise, the report will include data from previously run tests).
merge_reports – combines all the generated json reports for every single test file in one report.
mochawesome_report – generates a complete HTML report from the merged json file.
cy:report – merge of cleanup,run,merge_reports and mochawesome_report command

Run the test and generate HTML mochawesome report

Under the project directory run the following npm command:

 npm run cy:report

Mochawesome Reports

json reports are created under the path: cypress-mochawesome-report/cypress/reports/mochawesome-report

HTML reports are created under the path: cypress-mochawesome-report/cypress/reports/

The report overview

Now that we have the report, let’s take a closer look at the data it provides.

Right in the top bar, there are some basic information on:

No. of verified test files.
No. of tests included in the files.
No. of tests with a positive result
No. of tests with a negative result.
Duration of all tests.

A top bar of report in Cypress CI configuration

Below, there is detailed information on specific tests. We can check the following:

Duration of each test.
How long it took for a test to conclude in a given describe.
The actual code that was the subject of verification.

In the upper left corner, there is another menu which allows us to find out a couple more things:

Filtering all tests by a result (passed / failed / pending / skipped).
Navigation (useful if the lists include a lot of tests).
The date when each test was run.

An image shows Cypress CI report configuration

Negative tests and filtering

Below, we can see an example of a test which ended with a failed result. Full access to the error’s stacktrace and a cause of the failed result are provided. In this case, the mail was at fault.