Build vNext, support for deploying bits to Windows machines

One of the most interesting trend of DevOps movement is continuous deployment using build machines. Once you get your continuous build up and running, the next step is customizing the build to deploy on one or more test environments. If you do not need to deploy in production, there is no need of a controlled release pipeline (Ex: Release Management) and using a simple build is the most productive choiche. In this scenario one of the biggest pain is moving bits from the build machine to target machines. Once build output is moved to a machine, installing bits is usually only a matter of using some PowerShell script.

In Build, Deploy, Test scenario, quite often copying build output to target machine is the most difficult part

Thanks to Build vNext solving this problem is super easy. If you go to your visualstudio.com account, and choose the TEST hub, you can notice a submenu called machines.

image

Figure 1: Machines functionality in Visualstudio.com

This new menu is related to a new feature, still in preview, used to define groups of machine that can be use for deploy and testing workflows. In Figure 1 you notice a group called Cyberpunk1. Creating a group is super-easy, you just need to give it a name, specify administrative credentials and the list of the machines that will compose the group. You can also use different credentials for each machine, but using machine in Active Directory domain is usually the simplest scenario.

image

Figure 1: Editing of machine groups

Actually this feature still does not support Azure Virtual Machines, but you can easily target machine in your on-premise infrastructure. You just need to be sure that

  • All machines are reachable from the machine where the build agent is running
  • Check your DNS to verify that the names resolution is ok
  • All target machines should have Powershell remoting enabled
  • All target machines should have sharing of file system enabled
  • Firewall port are opened.

I’ve tested with an environment where both machines are running Windows Server 2012 R2, with latest update and file sharing enabled. Once you defined a machine group, you can use it to automatically copy files from build agent to all machines with a simple task of build vNext.

SNAGHTML3b78ea

Figure 3: Windows machine File Copy task

Thanks to this simple task you can simply copy files from build machine to destination machine, without the need to install any agent or other components. All you need to do is choose the machine group, target folder and source folder.

If you get error running the build, a nice new feature of build vNext is the ability to download full log as zip, where all the logs are separated by tasks

image

Figure 4: All build logs are separated for each step, to simplify troubleshooting

Opening the file 4_Copy … you can read logs related to the copy build step, to understand why the step is failing. Here is what I find in one of my build that failed.

Failed to connect to the path \\vsotest.cyberpunk.local with the user administrator@cyberpunk.local for copying.System error 53 has occurred. 
 2015-06-10T08:35:02.5591803Z The network path was not found.

In this specific situation the RoboCopy tool is complaining that the network path was not found, because I forgot to enable file sharing on the target machine. Once I enabled file sharing an running again the build everything was green, and I can verify that all files were correctly copied on target machines.

As a general rule, whenever a build fails, download all log and inspect the specific log for the task that failed.

image

Figure 5: Sample application was correctly copied to target machines.

In my first sample, I’ve used TailspinToys sample application, I’ve configured MsBuild to use StagingDirectory as output folder with parameter OutDir: /p:OutDir=$(build.stagingDirectory) and thanks to Windows Machine File Copy task all build output is automatically copied on target machines.

Once you got your build output copied on target machine, you need only to create a script to install the new bits, and maybe some integration test to verify that the application is in healthy state.

Gian Maria

Quickly run Unit Test With VS 2012/2013/..

With VS 2012 and newer versions we can run Unit Tests from various frameworks directly from Visual Studio IDE, thanks to the concept of Test Adapters. When you are doing Test Driven Development you usually go with Red/Green/Refactor workflow; what you need is a way to quickly run all or part of your unit tests after you modified the code. The quickest solution is using the option to Run Tests After Build but it is available only for Premium and Ultimate edition, but you can also run test with little manual intervention resorting to Keyboard Shortcut.

Simply go to TOOLS->Customize menu, then choose to customize keyboard.

image

Figure 1: Keyboard customization in Visual Studio

Once configuration window is opened, search for TestExplorer.RunAllTests command, place cursor in the “Press shortcut keys:” textbox and press a shortcut, then press “assign” button to assign to this command. In my standard configuration I like to have CTRL+SHIFT+ALT+A shortcut, because it is not assigned to any other command and it is easy to press with left hand.

image

Figure 2: Assign CTRL+SHIFT+ALT+A to TestExplorer.RunAllTests command

Now I can write code, use shortcut and VS will build solution and run test for me automatically, without leaving my hands from the keyboard. Thanks to the various test adapter and the various grouping and filtering possibility offered by Test Explorer, you can do TDD in Visual Studio without the need of third party tools.

Gian Maria.

Nunit test not found for some assemblies with Visual Studio Test Runner

I’ve a project in Visual Studio 2013 where one of the assembly containing Tests refuses to show tests in Test Explorer window. The solution has tests written both in Nunit and in MSpec, and everything is good except for that specific assembly. If you notice that Test Explorer window misses some tests, the first thing you need to check is the output windows, where you can find some output for Test Adapters.

image

Figure 1: Output for test adapters can be found in standard Output Window

In the above situation the assembly that is ignored is Intranet.Tests.dll and the reason is that it is built for Framework45 and x64 platform, while the test runner is executing against x86 platform. Everything seems ok, every project is compiled in .ANY CPU, but looking at the raw project file I can confirm that PlatformTarget is set in x64. Changing to x86 (or removing it completely) solves the problem.

image

Figure 2: Platform target changed from x64 to x86

After I changed the PlatformTarget attribute, all tests belonging to that assembly are now available Test Explorer window.

Gian Maria.

How to deal with Slow Unit Tests with Visual Studio Test Runner

The problem

 

One of the most dreadful problem of Unit Testing is slow testing. If your whole suite of tests runs in 10 minutes, it is normal for developers not to run the whole suite at each build. One of the most common question is

How can I deal with slow Unit Tests?

Here is my actual scenario: in a project I’m working in, we have some multilingual full text search done in Elastic Search and we have a battery of Unit Tests that verify that searches work as expected. Since each test deletes all documents, insert a bunch of new documents and finally commits lucene index, execution times is high compared to the rest of tests. Each test need almost 2 seconds to run on my workstation, where I have really fast SSD and plenty of RAM.

This kind of tests cannot be run in memory or with some fancy trick to make then run quickly. Actually we have about 30 tests that executes in less than one seconds, and another 13 tests that runs in about 23 seconds, this is clearly unacceptable. After few hours of work, we already reached the point where running the whole suite becomes annoying.

The solution

 

This is a real common problem and it is quite simple to fix. First of all Visual Studio Test runner actually tells you execution time for each Unit Test, so you can immediately spot slow tests. When you identify slow tests you can mark them with a specific category, I use slowtest

    [TestFixture]
    [Category("elasticsearch")]
    [Category("slowtest")]
    public class EsSearcherFixture : BaseTestFixtureWithHelpers

Since I know in advance that this test are slow I immediately mark the entire class with the attribute slowtest. If you have no idea what of your tests are slow, I suggest grouping test by Duration in Visual Studio Test Runner.

image

Figure 1: Group tests by Duration

The result is interesting, because Visual Studio consider every test that needs more than one second to be slow. I tend to agree with this distinction.

image

Figure 2: Test are now grouped by duration

This permits you to immediately spot slow tests, so you can add the category slowtest to them. If you keep your Unit Tests organized and with a good usage of categories, you can simply ask VS Test Runner to exclude slow test with filter –Traits:”slowtest”

image

Figure 3: Thanks to filtering I can now execute continuously only test that are not slow.

I suggest you to do a periodic check to verify that every developers is using the slowtest category wisely, just group by duration, filters out the slowtest and you should not have no tests that are marked slow.

image

Figure 4: Removing the slowtest category and grouping by duration should list no slow test.

The nice part is that I’m using NUnit, because Visual Studio Test Runner supports many Unit Tests Frameworks thanks to the concepts of Test Adapters.

If you keep your tests well organized you will gain maximum benefit from them :).

Gian Maria.

Shim constructors to isolate “Future Objects”

I’ve already blogged previously about the new Shim Library of Visual Studio 2012

Now it is time to explore another scenario where shim can save your life. Suppose you are working with Hardware, Es. a barcode reader and you have a class in your system wrote in this way.

public class BarcodeReader
{
    public BarcodeReader() { 
        //connect to the hardware, if the hardware is not there 
        //throw an exception
    }

    public String ReadCode() 
    {
        //dialogate with the hardware and get the Barcode
    }
}

This class manage the connection to a physical BarcodeReader in the constructor and if something went wrong throws an exception and expose a ReadCode method that ask the Physical reader to read a Barcode. This class is used with the following pattern throughout all the software.

public class MyLogic
{
    private BarcodeReader reader;

    public MyLogic() {
        reader = new BarcodeReader();
    }

    public String RetrieveIndustrialCode()
    {
        String rawCode = reader.ReadCode();
        return rawCode.Substring(2, 10);
    }
}

MyLogic is a class with some business logic based on the Barcode returned from the call to ReadCode() method, as you can see it simply instantiate an instance of BarcodeReader in the constructor and read the code in the RetrieveIndustrialCode Method used to extract the industrial code from the raw content of the barcode. In this stupid example the algorithm to extract the Industrial Code from the Raw barcode read from the reader is simply reading chars from 2 to 10, but in real scenario we could have complex string parsing code that needs to be tested, but how?? As you can see from the code you can test only when the real hardware is connected to the machine because if a physical reader is not connected the BarcodeReader will throw exception, but the most annoying part is that it is impossible to simulate a read during a Unit Test.

This is another scenario where shims comes to the rescue, just add a fakes assembly (as described in the previous samples) and you can write this test.

[Fact]
public void verify_retrieval_of_industrialCode()
{
    using (ShimsContext.Create())
    {
        ShimBarcodeReader.Constructor = @this =>  new ShimBarcodeReader(@this) {
            ReadCode = () => "IC1234567890BLABLABLA",
        };
        MyLogic sut = new MyLogic();
        var industrialCode = sut.RetrieveIndustrialCode();
        Assert.Equal("1234567890", industrialCode);
    }
}

Whenever you fake an assembly, each Shim that gets created has one or more static property called ConstructorXXX, where the XXX part is used to distinguish from the overloaded version using paramers type. In this example I have a default constructor so my Shim has a static property called Constructor. This specific property permits you to intercept the future creation of that type and even substitute it with a shim. This specific property needs a lambda where the first parameter is the Instance of the object being created followed by all other parameters of the constructor. In my sample test I pass a lambda that based on the instance created, returns a new ShimBarcodeReader configured to return the code “IC1234567890BLABLABLA” from ReadCode() method.

This technique is really powerful because permits you to Shim every future instance that will be created for a specific object and makes possible to inject Shims almost everywhere. When the test run, the MyLogic constructor creates an instance of BarcodeReader, but thanks to the Shim library I’m able to pass a shim where I’ve already set the return value of ReadCode() method.

Thanks to this capability I can write tests until I eventually found a bug, to show how the Shim Constructor changes when constructor has parameters, I’ve changed the BarcodeReader class to make the constructor accepts a Single String parameter and I want to write a test that verify that when the BarcodeReader returns an Empty String the Industrial code returned is also Empty.

[Fact]
public void verify_retrieval_of_industrialCode_too_short()
{
    using (ShimsContext.Create())
    {
        ShimBarcodeReader.ConstructorString = (@this, value) => new ShimBarcodeReader(@this)
        {
            ReadCode = () => "",
        };
        MyLogic sut = new MyLogic("COM2");
        var industrialCode = sut.RetrieveIndustrialCode();
        Assert.Equal("", industrialCode);
    }
}

To shim the creation of a ShimBarcodeReader I now need to use ConstructorString property, since the constructor has now a parameter of type String. Thanks to Visual Studio 2012 and Shim Library you can now easily test untestable legacy code with minimum effort.

Gian Maria.