Time Tortoise: Code Coverage for .NET Core Projects

Code Coverage

This is one in a series of articles about Time Tortoise, a Universal Windows Platform app for planning and tracking your work schedule. For more on the development of this app and the ideas behind it, see my Time Tortoise category page.

After I switched my Time Tortoise projects to .NET Standard a few weeks ago, I noticed a problem with code coverage results: most projects didn’t have any. Since I find code coverage measurement to be an important part of unit testing, I spent some time investigating and fixing the problem.

Missing Dependencies

Missing or incorrect dependencies have been a common cause of Time Tortoise unit test problems. Last week, I built a tool to simplify the process of dependency investigation. But even with all assembly binding errors resolved, I still wasn’t getting all of my code coverage results.

CodeCoverage.exe

In my code coverage results, I noticed that coverage was only being measured for my test projects. It can be useful to measure coverage for these projects. For example, it’s one way to detect tests that aren’t being run. However, the main purpose of code coverage measurement is to see what percentage of the product code is being exercised when unit tests are run. But I wasn’t getting any results for product code, despite the fact that the code coverage report told me that 100% of my test code was being run.

In searching the Web for answers about code coverage results only showing up for test projects, I came across a Stack Overflow answer that I wrote a few years ago on this subject. Thanks, Stack Overflow, for reminding me of things I used to know, but have since forgotten.

In my answer, I linked to an MSDN blog post called Troubleshooting missing data in Code Coverage Results. It explains how to use CodeCoverage.exe, a tool that comes with Visual Studio, to analyze code coverage results:

  • Find the tool under the Visual Studio install directory. For example: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Team Tools\Dynamic Code Coverage Tools
  • Find the .coverage file for the code coverage run in question. It should be under a TestResults folder at the top level of your project. For example, I have one under TimeTortoise\TestResults\391bea31-6ea2-4351-b63f-cf30aafbe8e8.
  • Run CodeCoverage.exe analyze /include_skipped_modules my.coverage > analysis.xml.
  • Open analysis.xml and search for the projects that are missing code coverage results.

When I did that, I found these results:

<skipped_module name="timetortoise.client.dll"
path="timetortoise.client.dll" reason="no_symbols" />
<skipped_module name="timetortoise.dal.dll" 
path="timetortoise.dal.dll" reason="no_symbols" />
<skipped_module name="timetortoise.model.dll" 
path="timetortoise.model.dll" reason="no_symbols" />
<skipped_module name="timetortoise.server.dll" 
<skipped_module name="timetortoise.testhelper.dll" 
path="timetortoise.testhelper.dll" reason="no_symbols" />
<skipped_module name="timetortoise.viewmodel.dll" 
path="timetortoise.viewmodel.dll" reason="no_symbols" />

The no_symbols reason means “PDBs (symbol files) are unavailable/missing.” But I checked the output directory, and every Time Tortoise DLL has a corresponding PDB. So something else must be going on.

Portable PDBs

Searching for answers more specifically related to .NET Core/.NET Standard led me to a Visual Studio Developer Community thread with the answer: .NET Core projects generate portable PDBs, but code coverage requires full PDBs. This is issue #800 on the vstest GitHub site.

Fortunately, there’s an easy workaround to use until the issue is resolved: add <DebugType>Full</DebugType> to PropertyGroup in the appropriate csproj files. So my PropertyGroup section looks like this:

<PropertyGroup>
  <TargetFramework>netstandard1.4</TargetFramework>
  <DebugType>Full</DebugType>
</PropertyGroup>

With this simple change, I’m once again getting code coverage results for all of my .NET Core/.NET Standard projects, along with the benefits of that project type.