Use conditional task in VSTS / TFS Build

When you start using Continuous Integration extensively, some builds become complex and sometimes a simple sequence of task is not what you want. Previous build system, based on XAML and Workflow Foundation allows you to specify really complex path of execution, but the side effect is that builds become really complex, they are painful to edit and also, you need to use Visual Studio to edit. Final drawback is that writing and maintaining custom build activities was not so easy, because you need to keep the compiled version in line with build engine version.

XAML build engine allows for complex workflow, but it was far more complex to manage.

The new build system is really far superior to old XAML build, but having a simple sequence of tasks can be too simplistic for complex scenario. The solution is really really simple, just use conditional execution.

In the control options section of the task configuration you can specify that the task should be run only if a custom condition is met.

Figure 1: Conditional execution condition in action

In the Control Option section of any task configuration, you have various options to decide when the task should run.

List of options that are described in the text of the post.

Figure 2: Options available to decide when the task is run

First four options are straightforward to understand, you can decide to run the task only if all previous task succeeded, or when a previous task failed, but also you can decide to run the task even if the build was cancelled.

The last option, Custom condition is where the real power shows up, with that option you can specify a custom condition, like the one that is shown in Figure 1. The whole syntax is shown in this MSDN Page and it is quite powerful. In Figure 1 I’ve used a simple expression that runs the task only if a variable called SkipTests is not equal to true (ne operator means Not Equal). Thanks to this, I can decide to set this variable to true at queue time, but I can also change a variable from previous task.

Even if custom condition does not gives you the full power of a workflow, you can still decide that some task runs only on certain conditions and it is simple to simulate flowchart if condition or more complex flow. As an example you can decide to run some tasks if not all tests are green and other tasks if some of the tests are failing.

Have a look at the documentation MSDN Page to verify all the conditions that you can use.

Gian Maria

Running powershell before Get Sources in VSTS / TFS Build

I have a VSTS build where I need to run a PowerShell scripts at the very beginning of the build, before the agent starts downloading the sources, and it seems that this cannot be done with a simple task, because there is no way to place a task before the Get Sources default task of VSTS Build.

Luckily enough there is a way to solve this problem, because each task can define a specific script that should be run when the build starts and before each task runs, here is how. First of all create a new task with the usual TFX-Cli interface:

tfx build tasks create

I’ve already explained how to create a build task in a previous post, but it is time to add some more context, because the build system is slightly changed and you need to do some more steps to be able to run the task. First of all you can incurr in the error

##[error]File not found: ‘C:\vso\_work\_tasks\LibreOfficeKiller_9073fee0-4de5-11e7-925d-cb284842266e\0.1.0\ps_modules\VstsTaskSdk\VstsTaskSdk.psd1’

This error is telling you that VstsTasksSdk is missing from the installation folder. New scripts uses a base library to interact with the build system, you can find all the information here, it is just a matter of cloning the VSTS Task Lib in a folder of your disk, then locate  the PowerShell folder, that contains a folder called VstsTaskSdk, that should be copied in your task folder, under a folder ps_modules.

VSTS / TFS Build system is evolving rapidly, new versions have nice new capabilities, but this requires you to always read latest documentation.

After you copied the VstsTAskSdk.psd1 folder in the right place you should be able to run a build with your task. Now there is the fun part, executing some script before the GetSouce phase of the build is run.

In this scenario I have a PowerShell scripts that kills every instance of LibreOffice that is running on the system. My build runs some integration tests that involves LibreOffice, but if, for some reason, a previous execution failed to kill properly running instances of LibreOffice, source folder in the agent cannot be clean, because soffice.exe process has open handle on source folder, thus, the GetSources task fails. The sympthom of this problem is the build failing with this error in the Get Sources:  The process cannot access the file ‘\\?\C:\vso\_work\8\s\src\Jarvis.DocumentStore.Tests\bin\Debug’ because it is being used by another process.

With latest version of the tasks, I can use the prejobexecution and postjobexecution to ask the system to run scripts at the beginning and the end of the build.

Section of task.json file showing the use of prejobexecution.

Figure 1: Relevant section of the script that uses prejobexecution to execute a script before any task runs.

Now in Figure 1 you can verify that I’m invoking the script KillLibreoffice.ps1 not only in the execution node, but also with a special node called prejobexecution and postjobexecution. These two nodes allows you to specify a script to be called before the very first task of the build is executed. To verify that everything works, here is a super simple and stupid build that uses this task.

Simple buidl that uses a libreofficekiller task then build the solution

Figure 2: Task LibreOfficeKiller used in a sample build

From the output of the build you can verify that the script that kills LibreOffice is run not only before the Get Source phase, but also after all tasks are completed.

Output of the build shows that kill libre office script is run three times, before any task, during task execution and finally after all tasks were run

Figure 3: Build output of the build with KillLibreOffice task.

From Figure 3 you can verify that the LibreOfficeKiller is run three times, the first time (1) before the Get Sources phase, then during normal task execution (2) and finally after the last task executed (3). Now my build can run just fine, because I’m able to kill any instance of LibreOffice.exe before the GetSources tasks runs.

If you are interested in the whole task, here is the code.

Happy building :).

Gian Maria.

Troubleshoot a failing build, a Winrm story

Many VSTS build and deploy tasks are based on Winrm to operate on a remote machine, one of the most common is the “Deploy Test Agent on” that will install a test agent on a remote machine.

This image shows the deploy test agent task on a standard build

Figure 1: Task to install a TestAgent on a different machine

If you are not in a domain Winrm can be a really thought opponent, especially because the target machine is not part of the same domain and is not trusted. Usually you configure the build, insert all password but when you run the build you incurr in an error like this one.

Error occured on ‘yourmachinename:5985’. Details : ‘Connecting to remote server jintegration.cyberpunk.local failed with the following error message : The WinRM client cannot process the request. If the authentication scheme is different from Kerberos, or if the client computer is not joined to a domain, then HTTPS transport must be used or the destination machine must be added to the TrustedHosts configuration setting. Use winrm.cmd to configure TrustedHosts. Note that computers in the TrustedHosts list might not be authenticated. You can get more information about that by running the following command: winrm help config. For more information, see the about_Remote_Troubleshooting Help topic.’. For troubleshooting, refer https://aka.ms/remotevstest. 

This error can happen for various reasons, firewalls, trust, misconfigured winrdm and so on, but the annoying stuff is: each time you change some configuration trying solve the problem, you usually re-schedule a build and need to wait for the build to complete to understand if the problem is gone. This way to proceed is something that actually kills your productivity.

Whenever possible, try to verify if the build is fixed without queuing another entire build.

In the new VSTS Build, the agent is running simple tasks, most of the time they are composed by PowerShell scripts, so it is really better running scripts manually to verify that your problem is gone, instead of launching another entire build. In this scenario the problem is winrm configuration, and you can use a simple winrs command from the machine where the VSTS Agent is running.

winrs -r:jintegration.cyberpunk.local /u:.\administrator /p:myP@ssword dir

This simple command will try to execute the dir command on the computer jintegration.cyberpunk.local using winrm, if you see the result of the dir command, it does mean that WinRs is configured and the computer can be contacted and it accepts WinRm commands. If you have any error, you should check your configuration and retry again. Once winrs command runs fine, communication between the two machine is ok. The important aspect is that until winrs command gives you error, you can be 100% sure that your build will not complete.

Replicating the commands issued by the build agent outside the build, greatly reduces the time needed to solve the problem.

In my situation here are the set of commands I’ve run to have Winrs command to works.

1) Ensure that winrm is enabled on both computer, do this with command winrm quickconfig
2) Verify that on target computer port 5985 is opened for connection
3) Run in both computers the command: winrm s winrm/config/client ‘@{TrustedHosts=”RemoteComputer”}’  where RemoteComputer is the name of the other computer. If you want to do a quick test you can specify * as Remote Computer name

After these three steps I was able to execute the WinRs command. Now I queued another build to verify if the task is now working ok.

The image shows the output of the step "Deploy test agent" and it shows that now the build agent was capable of using winrm to connect to target machine.

Figure 2: Deploy test agent task now runs without error

Actually I’ve done several tentative to troubleshoot the reason of the error, but since I checked each time with a simple winrs command (instead of waiting 4 minute build to run) the total time to troubleshoot the issue was few minutes instead of an hour or more.

Gian Maria.

Publish nuget package during build (.NET Standard)

In a previous post I dealt with how to build a Multitargeted dotnetcore solution in VSTS, but the build is not really complete unless you are publishing the result somewhere. Since my example was a simple library, the obvious solution is publishing everything to a nuget feed.

Publishing with nuget is really really simple with VSTS build system, because you should simply use another .NET Core task instance plus a NuGet publisher.

This image shows two tasks in the build, the dotnetpack and nuget publisher.

Figure 1: Couple of simple task is enough to publish to nuget feed

The configuration of these two task is really really simple, the dotnet pack task needs the list of the project that we want to publish and an additional command line that allows you to specify the version to publish and some other options.

image

Figure 1: .NET core task configuration

Thanks to dotnet task, it is really easy to invoke dotnet command during a build, thus, creating nuget package is really a breeze.

You should use pack command an in the projects section allows you to specify directly all the path of the csproj you want to publish, and in the arguments I’ve used several options. Here is the complete arguments commandline:

–configuration $(BuildConfiguration) –no-build /p:Version=$(NugetVersion) -o $(build.artifactstagingdirectory)\Nuget

configuration allows to decide what configuration you want to release, the –no-build option is needed to speedup the build, because the projects where built with previous tasks, there is no need to build them another time. The /p:Version parameter allows me to specify the version of the nuget package and finally the –o options allows you to specify the folder where the packages will be created.

After this task you have the Nuget Publisher task, that simply need to know the folder where the build generated the nuget packages and the target package where to publish the result.

This picture shows the configuration of nuget publisher wher I simly selected the path that contains nuget packages and the endpoint destination

Figure 3: Configuration of the NuGet Publisher task.

Et voilà, this is everything you need to do to have your package to be published and thanks to multitargeting, the nuget package automatically contains all the version of the framework you are targeting.

Package details in myget, showing that all targeted framework are included in the package

Figure 4: Package details in myget, showing that all targeted framework are included in the package

With a simple couple of task and 1 minute configuration, you have your library published to nuget feed.

Gian Maria.

Run Pester in VSTS Build

I’m not a great expert of PowerShell, but during last years I’ve written some custom utilities I’m using for various projects. The main problem is that I’ve scattered all these scripts on multiple projects and usually I need time to find the latest version of a script that does X.

Scattering PowerShell scripts all around your projects lead to error and a maintenance nightmare

To avoid this problem, the obvious solution is starting a consolidation of PowerShell scripts and the obvious location is a Git repository hosted in VSTS. Now that I’m starting script consolidation, I want also to create some Unit Test With Pester to helps me developing and obviously I want Pester Unit Tests to run inside a VSTS Build.

I had some problems running inside Hosted Build, because I got lots of errors when I tried to install Pester module. Luckly enough my friend @turibbio  gave me this link that helps me to solve the problem.

My final script is

Param(
    [string] $outputFile = 'TestRun.xml'
)
Install-PackageProvider -Name NuGet -Force -Scope CurrentUser
Install-Module -Name Pester -Force -Verbose -Scope CurrentUser

Import-Module Pester
Invoke-Pester -OutputFile $outputFile -OutputFormat NUnitXml

This simple script accepts only a single parameter, the output file for the test, it install the package provider nuget, then it install pester, and finally import pester module and invoke pester on the current directory.

While VSTS build system is really simple to extend, it is better to create a script that runs all test, so you can use both during local development and during VSTS Buidl

To have my test to be imported in VSTS build results, I’ve configured pester to output the file in NunitXml format. Creating a build is really simple, and it is composed of only three tasks.

image

Figure 1: Simple three step builds to run Pester tests on my PowerShell Scripts

As you can see I use the GitVersion task to have a nice descriptive version for my build; Pester is run a PowerShell task and finally Publish Test Results task is used to upload test result to the result of the build. Now I have a nice build results that have a GitVersion semantic name and also have the summary of the tests

image

Figure 2: Result of Pester Test run is included in build.

The ability to run PowerShell scripts inside a build, and publishing test results from various output format is what makes VSTS really simple to use to create build for every language, not only for .NET or Java but also for PowerShell.

Gian Maria.