Covering dynamically loaded (eval) code with Istanbul

Recently I worked on an uploader for some custom functions that could be applied to a SaaS product. They were plain JavaScript functions, maintained in the product through a REST API by uploading the source code as text, and because we maintained multiple environments we had to do some variable insertion into the scripts themselves before uploading. This presented a challenge when testing the scripts because their dynamic nature meant they could only be run using eval(), so they weren’t included in our code coverage report.

We use Istanbul for code coverage on JavaScript projects. Istanbul transforms code before it’s run to instrument it and track lines run.

Let’s start with a file containing a simple function expression.

// code.js
;(function (x, y) {
  return x + y
})

The function defined in code.js can be dynamically loaded using the fs module and eval:

// test.js
const assert = require('assert')
const path = require('path')
const fs = require('fs')

const filename = path.join(__dirname, 'code.js')
const code = fs.readFileSync(filename, 'utf-8')
const func = eval(code) // eslint-disable-line no-eval

it('adds 2 and 2', () => {
  assert.equal(func(2, 2), 4)
})

Try running this with istanbul cover ./node_modules/.bin/_mocha test.js and, while the test will run, you of course won’t get any coverage reported for the code under test.

Fortunately, Istanbul exposes its code instrumentation mechanism as part of its library. Give it some code (as a string) and a filename, and it will return instrumented code that can be run though eval() and will generate coverage information in a global object. The object is named __coverage__. The coverage data does not include code, only line/column information, which means the referenced filename has to actually exist.

// test.js
const assert = require('assert')
const path = require('path')
const fs = require('fs')
const istanbul = require('istanbul')

const filename = path.join(__dirname, 'code.js')
const code = fs.readFileSync(filename, 'utf-8')
const instrumenter = new istanbul.Instrumenter()
const instrumentedCode = instrumenter.instrumentSync(code, filename)
const func = eval(instrumentedCode) // eslint-disable-line no-eval

it('adds 2 and 2', () => {
  assert.equal(func(2, 2), 4)
})

Run this again with istanbul cover ./node_modules/.bin/_mocha test.js and… well, it still doesn’t generate a coverage report. That’s because Istanbul does not always use __coverage__ as its global for coverage data. It uses __coverage__ for the instrumentation generated by our Instrumetor object, but the data generated by the istanbul cover command itself uses a variable named after the pattern '$$cov_' + new Date().getTime() + '$$'.

Fortunately, combining these coverage data objects is easy. Although it’s hacky to rely on implementation specific behavior like this, I haven’t found another way, except for basically reimplementing the cover command itself.

after(() => {
  const coverageVar = Object.keys(global).find((x) => x.match(/^$$cov_/))
  Object.assign(global[coverageVar], global['__coverage__'])
})

Coverage data is keyed by filename, so assigning properties from one object to the other won’t cause any overwrites.

Postscript: If you switch from istanbul to successor nyc, the after clause appears to be unnecessary as the coverage is merged for you.