Code Coverage In Different Tools

Here’s a short rant about code coverage.

I have a project in TypeScript. It’s a tiny tool to calculate code coverage from LCOV files.

I’ve written unit tests to check it’s functionality and handling of various types of data. Now it’s time to execute the tests and maybe get some code coverage as well :)

When I used Node runtime and its test runner with--experimental-test-coverage, I got this result:

Node.js test runner

It’s a similar output that e.g. lcov tools can produce, there’s line, branch, and function coverage.

It also shows coverage of the test files, which is not very useful, I want test coverage reported only for source files.

Anyway, I was also playing around with a different runtime — with Bun which aims at replacing Node.js as a runtime.

Bun also ships with a test runner, so it’s possible to change:

import { test } from "node:test";

to:

import { test } from "bun:test";

So I did it and also used the --coverage option, here’s the result:

Bun’s test runner and coverage

Same project, same tests, but different coverage results. Branch coverage is not reported here at all, this is a bit disappointing, but maybe it’ll be included in later versions. (Or maybe I missed some option, although at this time I’m not aware of it.)

If you pay attention, you’ll notice the results don’t match. I don’t think I’m that much surprised that different tools report different results. But it’s a nice reminder of how tools can mislead us.

I guess I just want to show you here how you can get that test coverage to 100 % as your boss wants ;) Just kidding. But at the same time, I’d not be surprised if this strategy was used at some places.

Seriously, take code coverage with a big bucket of salt.