As developers, we’re given more and more opportunities to “shift left” by finding out as much about the outcomes and quality of our work as possible, as early as possible. I’ve started using a Python unit test setup with live coverage that blows away web-based coverage interfaces. Sure, you can see a color coded HTML tree of your code. Or you can just see coverage while working on your code. Let’s do that.
The VS Code side: Coverage Gutter
Coverage Gutters is a nifty Visual Studio Code extension that allows you to see code coverage in real-time as you write tests. It highlights the lines of code that are covered by your tests, making it easy to identify the untested parts of your code.
To get started, open the Extensions pane to search for and install Coverage Gutters.
Coverage Gutters comes with reasonable configuration out of the box, but you may want to open up its settings to customize things like colors (especially if you need to adjust for red/green colorblindness) and whether the entire line is highlighted or just the gutter. You might choose to use blue and yellow instead of the default red and green. Additionally, you may find that highlighting the entire line is more eye-catching and helpful, or prefer a subtler approach of highlighting only the gutter.
Apart from style, I prefer not to customize the functionality of the extension too much. I would rather set up the project so that anybody with a default installation can use it. This is especially important for usage by a team.
After installing the extension, the next step is to generate a coverage report. You can do this with
coverage, but I’m so used to
pytest at this point that I jump straight to
pip3 install pytest-cov
Let’s set up a simple Python project with a basic structure. To start with, all of these will be empty files:
pyproject.toml livecov/code.py test/__init__.py test/test_code.py
Now we use
pyproject.toml to configure our coverage for our source code:
# pyproject.toml [tool.coverage.run] source = ["livecov"]
Now we can run:
We’ll get a warning about there not being anything to cover. Fine, all the files are empty. Add some basic code:
# livecov/code.py def add(x, y): return x + y
# test/test_code.py from livecov import code
Now you’ll get some sensible output, showing no coverage of your code.
The right filename
But wait… Coverage Gutters complains about our setup. You can try activating the extension by invoking
Coverage Gutters: Display Coverage in the command palette, only to see an error message:
Could not find a Coverage file! Searched for lcov.info, cov.xml, coverage.xml, jacoco.xml, coverage.cobertura.xml
There are several options here. It’s easy to generate an
lcov report like this:
pytest --cov --cov-report=lcov:lcov.info
Now you should be able to see coverage in your
livecov/code.py. If you haven’t changed the settings, it should look like green and red bars in the left gutter of the file.
A better setup is to put default arguments into
pyproject.toml which should be extended like this:
# pyproject.toml [tool.coverage.run] source = ["livecov"] [tool.pytest.ini_options] addopts = "--cov --cov-report=lcov:lcov.info --cov-report=term"
Now you can just type
pytest and will both see the console output and save an
lcov.info file for Coverage Gutters.
Making it update live
But wait… there’s more. We still need to make this live.
First, Coverage Gutters itself has a watch mode. Invoke
Coverage Gutters: Watch in the command palette and you’ll find that you can browse your project and see coverage when you open new files. It’ll also update when you rerun tests.
I combined this with another project,
pytest-watch, a plugin that implements filesystem watch for
pip3 install pytest-watch
Now, run it like this:
Since it runs
pytest behind the scenes, it’ll automatically pick up all of your existing settings and run coverage on every code update.
Better yet, every time
pytest-watch reruns the tests, it outputs a new
lcov.info, which Coverage Gutters picks up and immediately updates the highlighting.
Add another function - it immediately shows as uncovered.
Add a test - the coverage updates.
RIP web interfaces
Although there’s still a reason to have a coverage service, something that hooks into your continuous integration system and tracks coverage over time, the need for standalone web interfaces is diminishing. As we integrate more tools into our development environments, we’re able to access data immediately. This faster feedback and real-time adjustments can lead to better code quality and maintainability.
This then allows the coverage service to be simpler - maybe just a database exposing repo and coverage information to SQL. That’s my next project.