Use istanbul to get code coverage of integration tests launched with protractor

Recently, I needed code coverage (generated by istanbul) for integration tests of an AngularJS application. These tests are launched with protractor. Sadly, this is not as easy as it may sound. The idea is to:

• associate a function to the onPrepare key in protractor's configuration. In this function, we define a reporter which will for each test collect the code coverage.
• use a plugin (the idea is from Jeffrey Barrus) in order to wait for the report to be written (with a promise): protractor has a onComplete option (a function called once all the tests are completed). However, this function doesn't support asynchronism: you cannot use callback or promise. So protractor may quit before the report is written or before the reporter has received any data. Hopefully, the tearDown method of a plugin can return a promise and protractor will wait its fulfillment before quitting.

I suppose that you know how to prepare your js files for istanbul (this step is called instrumentation). If you don't, you should read the doc.

Here are the configurations (you can download them at the end of the article):

 1 var istanbul = require('istanbul');
 2 var collector = new istanbul.Collector();
 3 var reporter;
 4 var waitPlugin = require('./waitPlugin');
 5 
 6 function report() {
 7   if (reporter) {
 8     reporter.add('json');
 9     reporter.write(collector, true, function () {
10       console.log('Coverage report successfully written');
11     });
12   }
13 }
14 
15 exports.config = {
16   specs: [
17     '../test/integration/*.spec.js',
18     '../test/selenium/*_test.js'
19   ],
20   seleniumAddress: '${seleniumAddress}',
21   maxSessions: 1,
22   multiCapabilities: [
23     {browserName: 'firefox', shardTestFiles: true, maxInstances: 1}
24   ],
25   framework: 'jasmine2',
26   plugins: [{path: './waitPlugin.js'}],
27   onPrepare: function () {
28     var jasmineEnv = jasmine.getEnv();
29     waitPlugin.setOnComplete(report);
30     browser.driver.manage().window().maximize();
31     browser.get('${testPortalAddress}');
32 
33     jasmineEnv.addReporter(new function () {
34       this.specDone = function (spec) {
35         if (spec.status !== 'failed') {
36           var name = spec.fullName.replace(/ /g, '_');
37           var reportfile = 'coverage/integration/json/' + name;
38           reporter = new istanbul.Reporter(undefined, reportfile);
39           var promise = browser.driver.executeScript('return __coverage__;')
40                   .then(function (coverageResults) {
41                     collector.add(coverageResults);
42                   });
43           waitPlugin.waitList.push(promise);
44         }
45       };
46     });
47   }
48 };

1. We start by charging the modules (istanbul, the collector, the plugin) and we create the reporter variable. It will contain the Reporter object used in various functions.
2. Then we create the report function which will be used to write the report. To do that, we specify the format of the report (json). More formats are supported by istanbul.
3. In protractor's configuration, nothing special until line 27 in the onPrepare function:
   1. Line 29, we define the function that must be called once all the promises of the waitPlugin are fulfilled (here our report function).
   2. Line 33, we add a reporter to jasmine, our test framework. This reporter only has a specDone method which is called each time a spec file is completed. If the tests were successful, we get the code coverage from the __coverage__ global variable and we add the result to the collector. These operation are asynchronous and return a promise. We add this promise to the list of promise to wait for in our plugin.

For the waitPlugin, nothing special, I think the code is obvious enough. Just think to install the q library which is required by the plugin to have promises. You can install it with npm install q or npm install --save q if you want to save this dependency in your package.json. If you have questions, you can leave a comment.

 1 var q = require('q');
 2 
 3 var waitList = [];
 4 var onComplete;
 5 
 6 exports.waitList = waitList;
 7 
 8 exports.setOnComplete = function(cb) {
 9   onComplete = cb;
10 };
11 
12 exports.teardown = function () {
13   return q.all(waitList)
14       .then(function() {
15         if (onComplete) {
16           return onComplete();
17         }
18       });
19 };

If like me, you must launch each spec file in a new browser, in order to have the global coverage, you need to write the result of each spec in a json file and then merge all the results with the following command:

istanbul report --include 'coverage/integration/json/**/*.json' --dir 'coverage/integration'  html

If you can however launch all the tests in a single browser, you can get the report directly in HTML by changing line 8 of protractor's configuration (json should become html).