This is one in a series of articles about Time Tortoise, a Universal Windows Platform app for planning and tracking your work schedule. For more on the development of this app and the ideas behind it, see my Time Tortoise category page.
As I mentioned last week, I’m working on a backlog of unit tests, and have run into a problem involving NuGet package dependencies. Copying DLL dependencies has been an ongoing issue with unit tests. Until now, I have been fixing problems as they come up, sometimes by trial and error. But I decided it’s time to write a more comprehensive explanation.
The Build Output Directory
When Visual Studio compiles a project, the output of the compilation process appears by default in a directory below the one where the project’s .csproj file is located. For example, if the Time Tortoise view model project is in TimeTortoise\TimeTortoise.ViewModel, then TimeTortoise.ViewModel.dll will appear in TimeTortoise\TimeTortoise.ViewModel\bin\Debug during the build process.
If the project being compiled references other projects in the same solution, the DLLs for those other projects also appears in the bin\Debug directory. For example, the View Model depends on the Model, so TimeTortoise.Model.dll also gets copied.
If the project has NuGet dependencies, those get copied as well. So after the work of the past few weeks, the view model Debug directory now also contains Microsoft.AspNet.SignalR.Client.dll.
Finally, some project files have their Copy to Output Directory property set to Copy always or Copy if newer. That means those files also need to be copied to bin\Debug. sqlite3.dll falls into that category, but this property can be used for non-DLL files as well.
NuGet, DLLs, and Unit Tests
For most types of projects, the process described above works fine, and each project’s output directory gets the DLLs it needs. Even references of references are handled correctly. For example, the output directory for TimeTortoise.UWP contains many DLLs for dependencies that TimeTortoise.UWP doesn’t reference directly, like Microsoft.EntityFrameworkCore.
Unfortunately, projects used for xUnit.NET tests don’t work seamlessly when it comes to NuGet dependencies. For example, take TimeTortoise.IntegrationTests. Because this project contains end-to-end tests, it naturally depends on everything else in the solution. For solution projects (e.g., TimeTortoise.ViewModel) that are referenced directly by TimeTortoise.IntegrationTests, this works fine. When Visual Studio builds the project, files like TimeTortoise.ViewModel.dll are correctly copied to bin\Debug. Even sqlite3.dll is copied. But none of the NuGet references used by other projects are copied.
The compiler doesn’t care about this, because the test code isn’t directly using any of those NuGet references. For example, an integration test method might call LoadActivities() on an instance of MainViewModel. That eventually causes a call to a class in Microsoft.EntityFrameworkCore. But because the test code isn’t calling Entity Framework directly, the reference isn’t needed for the code to compile.
At runtime, it’s a different story. The first symptom is an empty Test Explorer list. In other words, no tests are discovered by Visual Studio. The Test Explorer window itself doesn’t provide any information about why no tests are available to run, even though there are clearly test methods in the code. So more investigation is needed.
The simplest method I have found to diagnose this problem is to run the tests from the Package Manager Console (use View – Other Windows to open it if it isn’t already visible). In Time Tortoise, the steps are as follows:
cd TimeTortoise.IntegrationTests\bin\Debug
xunit.console .\TimeTortoise.IntegrationTests.dll
Here’s the output:
xUnit.net Console Runner (64-bit .NET 4.0.30319.42000)
System.InvalidOperationException: Unknown test framework: could not
find xunit.dll (v1) or xunit.execution.*.dll (v2) in
D:\Projects\TimeTortoise\TimeTortoise.IntegrationTests\bin\Debug
That’s simple enough. I need to find a missing xUnit.NET DLL and copy it into that directory. In this case, the file in question is xunit.execution.dotnet.dll.
If that were the end of it, I could simply add xunit.execution.dotnet.dll to the test project as a Copy if newer file. But the problem is that copying this DLL just exposes the next missing DLL. Here’s one that I’ll have more to say about later:
System.IO.FileLoadException : Could not load file or assembly
'System.Diagnostics.DiagnosticSource, Version=4.0.2.0,
Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its
dependencies. The located assembly's manifest definition does
not match the assembly reference.
Since we can’t rely on the Visual Studio build system to copy all of the necessary dependencies to the output directory, we need another approach.
CopyDependencies
Until now, I have been manually copying dependencies into the output directory in order to get unit test discovery to work. Since Visual Studio doesn’t delete manually-copied files from bin\Debug, that was an acceptable solution. I could copy the dependencies once, and they were there for future unit test runs. I only had to copy them again if a version changed (e.g., a NuGet package was upgraded).
But for the past couple of weeks, keeping the dependencies updated has become more complicated (as I’ll explain below) and the manual copy process was no longer realistic. To keep things working, I hacked together a console app called CopyDependencies to replace the manual process with an automated one. (I haven’t pushed it to GitHub for now, since it’s still experimental).
CopyDependencies uses the following steps:
- Read a list of package names from a text file,
packages.txt. This list has to be created manually based on error messages like those shown above. But once I have it, I can use it for the rest of the process whenever I need to copy dependencies. An example of a package name isSystem.Diagnostics.DiagnosticSource. Notice that no version number is required in this step. - For each package name, recursively search folders under
%UserProfile%\.nuget\packages\<packageName>to find DLLs. Write these to another text file,dlls.txt. Note that a package needs to be installed at least once from the NuGet interface before it appears in the.nugetdirectory. So it’s sometimes necessary to open Visual Studio and use NuGet Package Manager to find a missing dependency. - At this point, I go through the text file and mark my best guess as to which ones are needed, based on the previously-mentioned error messages. In my experience, there’s no way for the program to figure these out on its own. Since this is an iterative process, with each missing dependency revealing itself as a previous one is resolved, CopyDependencies needs to be smart enough not to overwrite the manual marks in
dlls.txt. - Copy each file marked in
dlls.txtto the required destination — e.g.,TimeTortoise.IntegrationTests\bin\Debug.
Setting this process up initially takes manual work, but once it’s done, I have a complete list of dependencies. Even if I delete the Debug directory or move to another machine, I can re-create it by running CopyDependencies on dlls.txt.
Troubleshooting
Even with the CopyDependencies tool available, it can be challenging to identify all of the required dependencies. Here are some techniques to find them all.
Assembly Binding Log Viewer
The words “or one of its dependencies” in the error message above aren’t just there for completeness. It’s very possible that the DLL mentioned in the error message isn’t the one you need to look for. Or it could be that you have copied it to what seems like the correct location, but for some reason the .NET Framework still can’t find it. To resolve those problems, there’s the Assembly Binding Log Viewer.
The link explains how to use this tool, but here’s a summary:
- Run it by using the Developer Command Prompt that comes with Visual Studio, or look for it manually somewhere like
C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.2 Tools. - As the documentation says, run it as Administrator.
- For best results, set Log Location to Custom and use the Log bind failures to disk and Enable custom log path settings. Enter a custom log path in the Custom log path textbox. If you get an error message, try it with and without the drive letter. See the image at the top of this post for an example.
- Click Refresh and then Delete All.
- Run a unit test.
- Click Refresh again if necessary, and double-click on one of the results.
- Repeat until you no longer get any results.
Each result will open in a browser window, and will give you a lot of information about how the .NET Framework is trying to load a DLL. Look for lines like this:
LOG: Post-policy reference: System.Reflection, Version=4.0.10.0,
Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
LOG: GAC Lookup was unsuccessful.
LOG: Attempting download of new URL
file:///D:/Projects/TimeTortoise/TimeTortoise.IntegrationTests/bin/Debug/System.Reflection.DLL.
LOG: Attempting download of new URL
file:///D:/Projects/TimeTortoise/TimeTortoise.IntegrationTests/bin/Debug/System.Reflection/System.Reflection.DLL.
LOG: Attempting download of new URL
file:///D:/Projects/TimeTortoise/TimeTortoise.IntegrationTests/bin/Debug/System.Reflection.EXE.
LOG: Attempting download of new URL
file:///D:/Projects/TimeTortoise/TimeTortoise.IntegrationTests/bin/Debug/System.Reflection/System.Reflection.EXE.
LOG: All probing URLs attempted and failed.
This tells you the full name of the DLL that .NET tried to load, and everywhere it looked. Using this information, you can find the correct DLL version and copy it to the correct location.
bindingRedirect
In some cases, two NuGet components request different versions of the same assembly. Since an assembly is identified to the file system only by its name (not a version number), there’s no way to satisfy both requirements at the same time in a single bin\Debug directory. I ran into this problem with System.Diagnostics.DiagnosticSource, for versions 4.0.0.0 and 4.0.1.1. To solve this problem, there’s a feature called binding redirect.
Here’s how it works: In the app.config file for the project in question, add something like this in the <runtime> section:
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="System.Diagnostics.DiagnosticSource"
publicKeyToken="cc7b13ffcd2ddd51" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-4.0.1.1"
newVersion="4.0.1.1" />
</dependentAssembly>
</assemblyBinding>
This instructs .NET to use System.Diagnostics.DiagnosticSource version 4.0.1.1 regardless of which earlier version (e.g., 4.0.0.0) is requested. Then copy version 4.0.1.1 of the DLL to the output directory.
Another option is to use this in the csproj file.
<PropertyGroup>
<AutoGenerateBindingRedirects>
true
</AutoGenerateBindingRedirects>
<GenerateBindingRedirectsOutputType>
true
</GenerateBindingRedirectsOutputType>
</PropertyGroup>
This will generate bindingRedirects automatically for every dependency, whether or not each one is necessary.