Code Coverage Report generation by Functional Testing

Code coverage by functional testing will give a clear matrix for our frequently asked questions. like -

1. is my functional tests are enough to cover all conditional statements of source code.
2. are test cases redundant?
3. did we reach our quality goals or do we need to increase our coverage?

Functional test code coverage can be accumulated during test sessions triggered from the interface and UI. There are multiple ways to achieve the same. For example, We can use Cypress for our automation, but that has its own advantages and disadvantages.

Do you want a solution that is independent of the tool which we are using for automation?

if YES !! The puppeteer can do wonders for you. We can integrate the Puppeteer plugin with our automation tools. Newly added tools, already provide support for puppeteer instance Eg. WebdriverIO. if You are using Selenium, Protractor then we can integrate the puppeteer plugin into our browser.

In this blog, we will implement it with WebdriverIO standalone mode. You might be thinking, when we already have Code Coverage with WebdriverIO, why do we have to do external Integration.

If your current situation includes -

1. preferring chrome driver service instead of DevTool service due to some advantages.
2. Your Test automation code is not present in the same repository of Dev code.
3. instead of using wdio.config to run your test cases, you are preferring mocha, jest, or other test automation frameworks.
4. using the standalone mode to get browser, i.e Standalone Mode with WDIO

Then we have to do external integration to get the code coverage.

I hope we understood Problem statement that we are trying to solve. If Yes !! Let's start our integration.

Pre-requisites :

1. Node installation
2. Initial Setup of the framework with webdriverIO, Jest/Mocha/Jasmine. You can refer to repository Code Coverage by Functional Testing GIT repo
3. Install Packages -

1. npm i -D puppeteer-to-istanbul
2. npm i nyc -g

Let's Start our Spec File : Create a Test File

In BeforeAll, we will get our driver instance and will attach the puppeteer to the browser instance.

import { write as generateReport } from "puppeteer-to-istanbul";
beforeAll(async () => {
   const chromeConfig = {}; // Define Your Configuration
   driver = await remote(chromeConfig as RemoteOptions);
   puppeteer = await driver.getPuppeteer(); // Getting Puppeteer
  [currentPage] = await puppeteer.pages();
  await currentPage.coverage.startJSCoverage(); // Starting Coverage
});

we will write our test cases and perform our functional testing. We can have multiple blocks.

it("Validate URL title", async () => {
   let actualTitle: string = await driver.getTitle();
    expect(actualTitle).toEqual("Cloud Application| Heurko");
});

In AfterAll, we will close our Puppeeter instance and generate the report.

afterAll(async () => {
   const [jsCoverage] = 
         await Promise.all(
                [currentPage.coverage.stopJSCoverage()]);
        //Converting JSCoverage to Istanbul report format
         generateReport(
           [...jsCoverage],{ includeHostname: true,
                              storagePath:"./.nyc_output" });
    await driver.deleteSession();
});

You are all set. Run your Test cases and wait till the time execution is completed.

See your folder, ./.nyc_output folder will be generated in your current directory.

Wow !! You are just one step behind to see your HTML file for the code coverage report.

To Generate an HTML report, Run the command -

nyc report --reporter=html

The coverage folder will be generated in your directory. Open index.html file.

Your Report will look like -

File Level

You can go to subfolders. It will have detailed data, how many lines of code are covered with our functional testing.

For Protractor —

You can integrate Puppeteer plugin by adding below lines in your config.js file .

exports.config = {
  // Your rest of configurations
  plugins: [{
    package: 'protractor-puppeteer-plugin', 
    configOptions?: {
        connectToBrowser: true
        connectOptions?: {
            defaultViewport: null
    },  
  }]
};

attach the puppeteer with your current browser instance -

[currentPage] = await browser.cdp.page;

References :

1. Report Generation in Istanbul Format
2. https://github.com/akagupta9/CodeCoverageByFunctionalTesting for Code reference