Building a Build Server – Uncovering Test Coverage

In a previous article, titled Building a Build Server – Executing Unit Tests as Part of the Build, we explored the details of setting up automatic unit test execution.

Automatically executing unit tests during the build gives us confidence that our code is correct. Especially if we also:

  • Write good unit tests,
  • Do not ignore when the tests are failing,
  • And our codebase is covered in a relevant percentage.

So, how do we know what is covered and what is not? Well, this article is just about that: getting a report about which lines were executed during the execution of the unit tests. In this article, we are going to add capability to our Jenkins server to assemble report on test coverage. This test coverage report is high in quality because it contains even branch coverage information, which is ignored by some commercial coverage tools too.

Installing the necessary tools

OpenCover

OpenCover is a tool for collecting coverage information during the execution. To collect coverage information from the unit tests, we are going to execute the unit tests through this application. To install OpenCover, execute the following commands from the Cygwin terminal:

mkdir /tmp/OpenCover

cd /tmp/OpenCover

wget https://github.com/OpenCover/opencover/releases/download/4.6.519/opencover.4.6.519.zip

unzip opencover.4.6.519.zip

rm opencover.4.6.519.zip

cd /cygdrive/c

mv /tmp/OpenCover /cygdrive/c

icacls OpenCover /T /Q /C /RESET

To allow OpenCover to monitor other the programs run through it, its two profiler DLL must be registered into the system. To do so, execute the following commands from a Windows command prompt running as an Administrator:

C:

cd C:\OpenCover

regsvr32 /s x86\OpenCover.Profiler.dll

regsvr32 /s x64\OpenCover.Profiler.dll

ReportGenerator

Currently, OpenCover does not have the capability to generate nice looking reports, just a coverage XML. Here comes ReportGenerator into play. ReportGenerator will take the coverage data represented in XML format and transform it into a highly usable documentation composed from HTML pages. Installation of ReportGenerator can be achieved by executing the following commands from the Cygwin terminal:

mkdir /tmp/ReportGenerator

cd /tmp/ReportGenerator

wget https://github.com/danielpalme/ReportGenerator/releases/download/v2.5.6.0/ReportGenerator_2.5.6.0.zip

unzip ReportGenerator_2.5.6.0.zip

rm ReportGenerator_2.5.6.0.zip

cd /cygdrive/c

mv /tmp/ReportGenerator /cygdrive/c

icacls ReportGenerator /T /Q /C /RESET

Integrating code coverage report into Jenkins

The following Jenkins build project will check out the demo repository located at https://github.com/atzimler/CoverageTest.git, build it and create report of the unit test coverage on the codebase:

Source Code Management

Use Git source code management with

Repositories

Repository URL

https://github.com/atzimler/CoverageTest.git

   

Branches to build

Branch specifier (blank for ‘any’)

*/master

Repository browser:

(Auto)

Build

Execute Windows batch command

Command

cd CoverageTests\bin\Debug

C:\OpenCover\OpenCover.Console.exe -target:C:\NUnit.org\nunit3-console.exe -targetargs:CoverageTests.dll -register:user -output:TestCoverage.xml

C:\ReportGenerator\ReportGenerator.exe -reports:TestCoverage.xml -targetdir:..\..\..\CoverageReport

Please note: Each paragraph here is one line of command. In case you are copy-pasting the code it should be ok, but if you manually copying it, please don’t insert spaces where paragraph lines are breaking.

Post-build Actions

Archive the artifacts

Files to archive

CoverageReport/**

After the project has been built, we can find the coverage test under the Last Successful Artifacts sections, CoverageReport/index.htm.

You can connect with me on LinkedIn, Twitter or you can register on the blog. Share if you found this article useful or know somebody who would benefit from reading it.

By subscribing to the email list, you will be notified about new posts.
Loading

One Response to Building a Build Server – Uncovering Test Coverage

  1. MUHAMMAD DENNES February 7, 2020 at 3:44 am #

    thanks your article is really helpful

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Powered by WordPress. Designed by WooThemes