Be sure to use latest version of Nuget Restore Task in VSTS Build

If you have in VSTS some old build that uses Nuget restore task, it is time to check if you are using the new version, because if you still use the 0.x version you are missing some interesting features.

With VSTS build it is always a good habit to periodically check if some of the tasks have new version.

Here is as an example, how the version 0 is configured

image

Figure 1: Nuget restore task in version 0

In Figure 1 you see how you configure a standard nuget restore task, if you are using version 0 (2) it is time to upgrade, especially because the old version of the task is not so flexible about the version to use (3).

Before changing version, you need to know that the newer version of this task goes hands to hands with another new Task, the Nuget Tool Installer Task, that is designed to download and install a specific version of nuget that will be used by all subsequent task in the Phase.

image

Figure 2: Thanks to the NugetToolInstaller task you can ensure that a specific version of NuGet is present in the system and used by all subsequent tasks.

Nuget Tool installer also ensure that all nuget task will use the very same version.

After the Nuget Tool Installer you can simply configure Nuget Task in version 2, see Figure 3.

image

Figure 3: Nuget task version 2 in action

As you can see you do not need to specify any version, the version used will be the one installed by the Nuget Installer task, so you are always sure that the exact version of Nuget is installed and available for the build to be used.

As usual, if you have build that sits there for long time, take your time to check if some of the tasks are available in newer and more powerful version. If you wonder how you can immediately check if some of the tasks have newer version, simply check for a little flag near the description of the task, as shown in Figure 4.

image

Figure 4: A little flag icons warn you that you are using an old version of this task.

Gian Maria.

Converting PowerShell Task in YAML

YAML Builds have many advantages over traditional build definitions, especially because YAML build definitions follows branching of code, a killer feature that is fantastic if you use GitFlow.

YAML Build definitions are stored in code, this allows them to follow branches, minimizing the need to maintain builds that should build code in different moment in time.

As an example I have a build where I have tasks to publish some Web Sites, if I had a new Web Site to publish, I can add another task in YAML build, but the build still work for older branches, especially for the master branch that represent my code in production.

To minimize impacts, I tend to use PowerShell scripts stored inside code repository to perform build tasks, this was an old trick  useful when YAML build was not present. Scripts allows you to create a generic build, add some tasks to run PowerShell scripts, and the result is that scripts follow branches.

Now that YAML Build is available, I started converting everything to YAML build, a task that is made simple tanks to a nice feature of VSTS, in the UI there is a nice View YAML button that convert all of my build definition with new YAML syntax.

image

Figure 1: Ui functionality to show the current standard build definition translated to new YAML build definition.

As you can see from Figure 1, PowerShell tasks is not converted as a task, but with a node of type powershell, followed by parameters. This is perfectly valid, because YAML build can contain a powershell node, but in my specific situation the build failed, because the conversion failed to setup working directory.

Even if YAML build allows direct call of PowerShell scripts, if you convert a working build that uses PowerShell task it is better to use the same Task inside YAML definition.

To solve my problem I simply changed generated YAML build definition to use a standard PowerShell task, this is the declaration of the task.

- task: PowerShell@2
  displayName: Pack Release
  inputs:
    targetType: filePath
    filePath: 'tools\packrelease.ps1'
    arguments: '-Configuration $(BuildConfiguration) -DestinationDir $(Build.ArtifactStagingDirectory) -StandardZipFormat $false -SkipZip $false -SkipConfigRename $false -compressionLevel 5'
    workingDirectory: 'tools'

As usual you specify a task and the type, in this example PowerShell@2. It is important the version after the @ character, it should be equal to the version of the task in the standard build, as shown in Figure 2.

image

Figure 2: Pay attention to version Task when you convert to YAML build.

To know input parameter name of the tasks (Es. targetType, arguments, workingDirectory, etc), the most immediate solution is looking in the working directory of an agent and find the task definition.

image

Figure 3: Tasks inside working folder of an agent.

Inside the folder of the chosen task there are all latest version for any major version, and inside each version folder there is a task.json file that can be inspected to know the exact version of the parameter.

After modifying YAML build to use PowerShell task the build started working as usual.

Gian Maria.

Run SonarCloud analysis in VSTS / TFS Build

Running a SonarQube analysis for TFS or VSTS is really easy because we can use a pre-made build tasks that requires few parameters and the game is done. If you have open source project it made lot of sense to use a public account in SonarCloud, so you do not need to maintain a sonar server on-premise and you can also share your public account with the community.

For open source projects, SonarCloud is available for you with zero effort and thanks to VSTS and TFS you can automate the analysis with few steps.

The first step is creating an organization in Sonar Cloud, if you prefer you can just login with your GitHub account and everything is ready. After the creation of the organization, you should create new project and generate a key to send analysis to SonarCloud server, everything is made with a simple wizard and it takes only a bunch of seconds to have your project created and ready to be used.

Once you have your project key and token  you need to add the endpoint of SonarCloud to the list of available endpoints. You only need to give the connection a name, specify https://sonarcloud.it as Server Url and add the token generated during project creation.

image

Figure 1: Configuration of sonar cloud endpoint

Now you can configure build to perform the analysis, the first task is the “prepare analysis Task”, as you can see in Figure 2. You should select the Endpoint created in previous step, fill the project key and project name, but you need also to specify a couple of properties in the advanced section . The first one is sonar.organization and it is required or the analysis will fail. This is the only difference from on-premise SonarQube server, where you do not need to add organization name.

image

Figure 2: Configuration of the prepare analysis task.

The other setting to be specified in Additional Properties is the sonar.branch.name, to perform branch based analysis, a feature that is available in sonarcloud and it is available on-premise only with enterprise version. You can simply use the $(Build.SourceBranchName) to use the current branch if you are using Git.

image

Figure 3: Analysis performed on NStore project with branch analysis enabled.

The cool part of the process is that, SonarCloud require zero installation time and less than one minute to create the first project and thanks to the VSTS / TFS build engine you can automate the analysis in less than 2 minutes.

Gian Maria.

VSTS / TFS: use Wildcards for continuous integration in Git

If you setup a build in VSTS / TFS against a git repository, you can choose to trigger the build when some specific branch changed. You can press plus button and a nice combobox appears to select the branch you want to monitor.

image

Figure 1: Adding a branch as trigger in VSTS / TFS Build

This means that if you add feature/1312_languageSelector, each time a new commit will be pushed on that branch, a new build will trigger. Actually if you use GitFlow you are interested in building each feature branch, so you do not want to add every feature branch in the list of monitored branches.

If you look closely, it turns out that the combo you use to choose branches admit wildcards. If you look at Figure 2, you can verify that, pressing the add button, a new line is added and you have a textbox that allow you to “Filter my branches”.

image

Figure 2: Filter branch text

Actually the name is misleading, because you can simply write feature/* and press enter, to instruct VSTS to monitor each branch that starts with feature/.

Using wildcards in trigger list allows you to trigger a build for every feature branch.

For people not used to VSTS, this feature goes often unnoticed, because they think that the textbox can be used only to filter one branch. With this technique you can trigger a build for each change in each standard branch in GitFlow process.

image

Figure 3: Trigger with wildcards

As you can see in Figure 3, I instructed TFS / VSTS to build develop, master and all hotfix, feature or release branches.

Gian Maria

New cool feature of VSTS to limit impact of erratic tests

I’ve blogged some time ago about running UAT testing with a mix of Build + Release in VSTS. Actually, UAT testing are often hard to write, because they can be erratic. As an example, we have a software composed by 5 services that collaborates together, CQRS and Event Sourcing, so most of the tests are based on a typical pattern: Do something then wait for something to happen.

Writing tests that interact with the UI or are based on several services interacting togheter can be difficult.

The problem is: whenever you are “wait for something to happen” you are evoking dragons. Actually we wrote a lot of extensions that helps writing tests, such as WaitForReadModel, that poll the readmodel waiting for the final result of a command, but sometimes those kind of tests are not stable because during a full run, some tests are slower or simply they fails because some other tests changed some precondition.

We spent time to make test stable, but some tests tend to have erratic failure and you cannot spent tons of time to avoid this. When they fails in VS or Nunit console, you can simply re-run again only failed tests and usually they pass without a single error. This is perfectly fine when you are running everything manually, you launch tests in a VM, then come back after a little bit, if some tests failed, just re-run failed test and verify if something was really broken.

Erratic tests are really difficult to handle in a build, because if the build was always red because of them, the team tend to ignore build output.

When a build is often failing due to false-positive, team tend to ignore that build and it starts losing value.

Thanks to a cool new feature in VSTS, we can instruct our builds to re-run automatically those kind of tests. You can read everything about it in the release notes for VSTS.

image

Figure 1: Option to re-run failed tests

It could seem a little feature, especially because usually your tests should not be erratic, but actually it is a HUGE feature for UAT testing, when having not erratic test is simply not feasible or a really huge work. Thanks to this feature, the build can automatically re-run failed tests, completely eliminating the problem of erratic tests, and showing as failed only tests that really failed constantly for more than a given amount of runs.

Another situation where this feature has tremendous impact is when you want to  migrate a huge set of tests from sequential execution to parallel execution. If your test were written assuming sequential execution, when you run them in parallel you will have many failures due to interacting tests. You can simply create a build, enable parallel execution and enable re-run failed test. This will simply helps you to spot interacting tests so you can start fixing them so they can run in parallel.

If you look at Figure 2, you can verify that all erratic tests are easily identified, because you can filter all tests that passed on rerun. This will leave your build green, but can spot erratic tests that should be investigated to make them more stable.

image

Figure 2: Erratic test can be immediately discovered from build output.

I think that this feature is really huge and can tremendously helps running UAT and Integration tests in build and releases.

If you do not see the option in figure 1, please be sure you are actually using the version number 2 of the test runner task as shown in Figure 3.

image

Figure 3: Be sure to use version 2.* of the test runner, or the option to re-run failed test is not present.

Gian Maria.