Running NUnit Tests in a TFS 2015 Build vNext

Scenario

In TFS2015 CTP you can have a preview of the new build system that will be available with the next version of Team Foundation Server. If you start scheduling a new build that has NUnit test you probably will notice that your unit tests are not executed during the build.

To execute NUnit test in a vNext build you should ensure that appropriate tests adapters are available to agents that will run the build

The easiest way to make NUnit adapters available is downloading them from Visual Studio Gallery, unzip the .vsix extension file, then copy all the extension file in the folder.

C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions

This operation should be done in every machine where you deployed Test Agents that will be used to run build with nunit tests. This can be annoying if you have lots of agents, if you prefer, Test Runner Task has an options to specify the path where the agent can find needed test runners.

The easiest way to automate all the process is adding Nunit Test Adapter nuget package to your test project, adding a standard Nuget Reference.

image

Figure 1: Add Nuget Package for NUnit Test Adapter

Once you’ve added the package, you should see all needed assemblies under the package directory of your solution. Now you can specify the location that contains the Nunit Test Adapter directly from Build Definition using this string

“$(Build.SourcesDirectory)\src\packages\NUnitTestAdapter.1.2\lib”

Please not the use of quotes (“) and the use of the $(Build.SourceDirectory) macro to specify the location where the build is taking place.

image

Figure 2: Specify path of custom Test Adapter inside build definition.

Pro and con of the two approaches:

Copying adapters inside Visual Studio TestWindows folder

Pro: You do not need to specify path of the adapters inside each build definition

Cons: You should do this for every machine where agent is deployed.

Specify Path to Custom Test Adapter with nunit packages

Pro: You can use the version you need referencing different nuget packages. You should not have access to machines where agent is installed.

Cons: You need to remember to use that settings inside each build where you want to run NUnit tests. All Test Adapters should be placed inside the same directory.

Gian Maria.

Published by

Ricci Gian Maria

.Net programmer, User group and community enthusiast, programmer - aspiring architect - and guitar player :). Visual Studio ALM MVP

6 thoughts on “Running NUnit Tests in a TFS 2015 Build vNext”

  1. It depends on the framework used. Microsoft Coded UI Tests requires MSTest, but if you use Selenium you can run UI Test without problem. The only caveat is that you should configure your build to release the software in a test environment and you should use headless browser (unless the build agent is launched interactively or is running as a service and configured to access windows user interface)

  2. Thanks i am able to execute the tests now. However i am facing difficulties with filtering the tests. I want specific tests to be executed. I tried below options but unable to proceed.

    > By using Vstest parameter /UseVsixExtensions:true – Other console options is not available with run functional test tasks. If any way, we can pass this argument with command.
    > By Test case filter. Need to translate mstest filters to nunit test case filter
    > Execute playlists by Vstest. Currently playlist is supported only with mstest command. But not with Vstest. Vstest is using ordered tests. But Ordered tests understands MSTests methods only and not nunit Tests. How to add the nunit tests in ordered tests.

  3. Actually filtering unit test during a build with nunit could pose some difficulties. Actually I tend to run all tests during a build and if I need to run different suite of tests (suppose Unit Tests, Functional Tests, Ui tests) I tend to define them in different assemblies, so it is really easy to choose which of them I want to run.

    How do you want to filter? Traits?

  4. Thanks for the insight; this is helpful.

    Unfortunately, I still don’t have tests running. The log says

    NUnit TestAdapter 3.8.0 executing tests
    NUnit loading path-to-assembly.dll
    NUnit cannot load path-to-assembly.del
    NUnit test execution complete

    I used the Nuget option, and it seems to find the TestAdapter, but it won’t load the tests. Is there a way to get more details on the failure?

  5. With the latest build engine it is enough to use the nuget package for test adapter and the tests are run in the build without any problem.

    This is really better because the build will use the very same adapter that you use in Visual Studio. The concept is, remove the nunit test adapter vsix addin from visual studio and instead add the adapter on all nunit project as nuget package. This should solve everything.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.