Installing a linux Agent for VSTS build was never so easy

If you installed Linux Agents for VSTS vNext build in the past, you already know that it was a simple experience, especially because the agent was installed with npm, so it is a matter of a couple of commands.

The agent is undergoing a substantial change, and in GitHub there is a project about VSTS Cross Platform Agent (CoreCLR), a new version of the agent, entirely written in CoreCLR that will substitute the closed source Windows agent and the actual XPlat agent. This version of the agent is still work-in-progress, and we can expect a preview for OSX and Linux in the next month. Until now, to run vNext build on linux, you should use the XPlat Agent.

Installing the actual xpPlat agent is really simple, just create a folder where you want to install the agent, open a bash in that folder and then issue the command

curl -skSL http://aka.ms/xplatagent | bash

This will download an install script, and does everything for you, installing prerequisite, downloading and installing everything. At the end of the script you should only execute

./run.sh

To start configuration of the agent. The official guide suggest you to use Personal Access Token, but standard alternate credentials are fine, if the user is part of the Agent Pool Administrators and Agent Pool Service Account.

Once everything is configured, the agent is ready to run your builds.

image

Figure 1: Agents configured and running on Linux CentOS machine

Gian Maria.

Adding more than one build agent in VSTS

Pricing for Build Agents in Visual Studio Team Services states that the first agent is free, subsequent on-premise agent will cost 15$/month. If you, like me, have a demo account, or you want to use more than one agent not simultaneously, you can register more than one agents, and then, from the administration panel, you can disable all agents except one.

image

Figure 1: Leave only one agent enabled at a time to be license compliant

If you need a third agent, as an example installed on a linux machine to demo linux build, you should disable the only enabled agent, then install and configure the agent on linux.

Gian Maria.

Writing a custom task for Build vNext

I wrote on how to integrate GitFlow and Semantic Versioning for my nuget packages thanks to GitVersion.exe and a simple powershell script, but still the solution is not perfect. The worst drawback is that you need to remember to include all needed PowerShell scripts and GitVersion.exe executable in project source code. This approach does not give best experience and clutter source code with build related executables.

Since the script is valid for any project, it would be a better solution to wrap it in a build task for the new vNext build system. Even in this area, the new build system shines, because writing a task is really a breeze.

First of all install the tfx-cli utility, then navigate in a folder of your pc, create a folder for the new task and use this command to create a skeleton for your first build task.

tfx build tasks create

This command will ask you several information, at the end you will have a new folder, with the same name of the task you choose, containing a bunch of files. The most important file is called task.json , because it contains all the information needed by the build system to run your task. Here is my json.file after modification.

{
  "id": "4d26fc20-dcb5-11e5-80ed-77d9cb9f353c",
  "name": "GitVersion",
  "friendlyName": "GitVersion Semantic Versioning",
  "description": "Simple script to version your task with GitVersion",
  "author": "Gian Maria Ricci",
  "helpMarkDown": "This task simply calls a script that invoke Gitversion.Exe to calculate SemVer for your repository",
  "category": "Utility",
  "visibility": [
    "Build",
    "Release"
  ],
  "demands": [],
  "version": {
    "Major": "0",
    "Minor": "1",
    "Patch": "13"
  },
  "minimumAgentVersion": "1.91.0",
  "instanceNameFormat": "GitVersion $(message)",
  "inputs": [
    {
      "name": "NugetVersionVariableName",
       "type": "string",
       "label": "Nuget Version Variable Name",
       "defaultValue": "NugetVersion",
       "required": false,
       "helpMarkDown": "If you want nuget version in a variable like $(NugetVersion) just specify its name."
    }
  ],
  "execution": {
    "Node": {
      "target": "sample.js",
      "argumentFormat": ""
    },
    "PowerShell": {
      "target": "$(currentDirectory)\\version.ps1",
      "argumentFormat": "",
      "workingDirectory": "$(currentDirectory)",
      "platforms": [ "windows" ]
    }
  }
}

This file is self explanatory; it just contains some properties, a list of the input parameter for the build task, and finally the script to run in node.js and in PowerShell. If you look at the code, I do not need any Node script, so I left the standard sample.js of the base file, but for PowerShell section I specified the version.ps1 files I’ve used to version my assembly. Powershell script is the very same of the previous example, I just changed the starting part to accept a parameter, the name of the build variable where the script will set the nuget package determined by GitVersion.exe. I also specified that the powershell runs only on windows platform. The nice aspect is I’m able to reuse the very same powershell script, with little modification.

[CmdletBinding()]
param(
    [Parameter(Mandatory=$false)][string] $NugetVersionVariableName = "NugetVersion"
)

Now I copy GitVersion.exe and all dll in the very same directory of the script, so I do not need to include GitVersion.exe in my source code to calculate semantic versioning. The final step is adding a package.json with npm to declare all the dependencies that are needed by the task to run.

{
  "name": "Git Version",
  "version": "0.1.3",
  "description": "Sample script to versioning assemblies",
  "author": "Ricci Gian Maria",
  "license": "None",
  "devDependencies": {
    "vso-task-lib": "^0.5.5"
  }
}

The only package I need is the vso-task-lib, containing all the base dependencies for vNext build system. When everything is finished I can upload the whole task directory with the command

tfx build tasks upload 

This command asks for the folder where the task is located, and upload everything to VSTS account you are logged into (you should have run tfx login to connect to your instance). It is really simple, and my new task is available to be used.

image

Figure 1: Adding new custom task to a build

In Figure 1 I showed a really simple build, that contains only my new task, the goal is verifying that it is able to execute giversion correctly and it is able to change variable NugetVersion.

From the log of the execution it can be verified that the script is actually launched from the task directory Executing the powershell script: C:\LR\MMS\Services\Mms\TaskAgentProvisioner\Tools\agents\1.96.3\tasks\GitVersion\0.1.13\version.ps1.

Gian Maria.

Save a build as a Draft

There are a lots of interesting new features in TFS / VSTS Build vNext, but surely, one of the coolest one is the ability to edit a build and save as a draft. Actually available only in the online version (Visual Studio Team Services)

Saving build as a draft

Figure 1: Saving a build as a Draft

Actually, saving build as a draft allows you to edit a build, try a new configuration / task / personalization, without distrupt the old build that works. Customizing a build can be a difficult task, and the greatest risk with older build System is having an unusable build until the new personalization is done.

Another usual technique is temporary disable tasks to reduce the time to finish the build and verify if your new customization works. Suppose you added a last task to manage artifacts publishing, you want to verify that everything works, and you disable running Unit Tests, so you can finish build faster and have a faster feedback. If you do this with the real build, until customization is not completed, all queued build will have unit test disabled.

The main problem when you edit a build, is disrupting continuous integration until your work is finished.

With the ability to save as a Draft you can avoid this type of disruption. Once you’ve saved a build as a draft, you can queue the draft, verify the outcome, and when everything works as expected, you can publish it, effectively updating the real build only when you’ve tested all modifications and you are sure that the new definition does what you really want.

Build result of Drafts build have a .DRAFT suffix to distinguish from a standard build output

Figure 2: Build result of Drafts build have a .DRAFT suffix to distinguish from a standard build output.

The net effect is: you are able to test modification in isolation, without distrupting the original working build definition. Combine this with the ability to quickly spin an agent in your machine and you will have a really pleasant build definition update experience.

1. Quick configure an agent into your local machine
2. Try your personalization and save as a Draft
3. Queue draft on your agent that is immediately able to execute the build
4. Once everything is ok, publish the build

Gian Maria.

Publishing a Nuget package to Nuget/Myget with VSO Build vNext

Publishing a package to myget or nuget with a TFS/VSO vNext build is a breeze. First of all you should create a .nuspec file that specify everything about your package and include it in your source control. Then Add a variable to the build called NugetVersion as shown in Figure 1.

Adding NugetVersion variable to the list of variables for this build.

Figure 1: Added NugetVersion variable to build definition.

In this build I disabled continuous integration, because I want to publish my package only when I decided that the code is good enough to be published. Publishing to a feed for each build is usually a waste of resources and a nice way to make the history of your package a pain. Since I want to do manual publishing I’ve checked the “Allow at Queue Time” checkbox, to be able to change Nuget Version Number at queue time.

Build vNext has a dedicated step called NugetPackager that takes care of creating your package from nuspec file, so you do not need to include nuget.exe in your repository or in the server. If you are curious where is nuget.exe stored, you can check installation folder of your build agent, and browse the Task Directory where all the tasks are contained. There you should find the NugetPackager folder where all the script used by the tasks are stored.

How to configure Nuget Packager to create your package.

Figure 2: Added Nuget Packager step to my build.

You can use wildchars as pattern to nuspec files; as an example you can specify **\*.nuspec to create a package for all nuspec file in your source directory. In this example I’have multiple nuspec in my repository, but I want to deploy only a specific package during this build, so I’ve decided to specify a single file to use. Thanks to the small button with ellipsis at the right of the textbox, you can choose the file browsing the repository.

Thanks to source browsing you can easily choose your nuspec file to create package.

Figure 3: Browsing source files to choose nuspec file to use.

Then I’ve choose $(Build.StagingDirectory) as Package folder, to be sure that resulting nupkg file will be created in staging directory, outside of the src folder. This is important, because if you do not choose to clean src folder before each build, you will end with multiple nupkg file in your agent work directory, one for each version you published in the past. If you use StagingDirectory as destination for your nupkg files, it will be automatically cleared before each build. With this configuration you are sure that staging directory contains only .nupkg files created by current build.

Finally in the Advanced tab I’ve used the Nuget Arguments textbox to specify the -version option to force using version specified in the $(NugetVersion) build parameter.

The last step is including a step of type Nuget Publisher, that will be used to publish your package to nuget / Myget.

Configuration of NugetPublisher step to publish your package to your feed

Figure 4: Final publishing step to publish nuget to your feed.

If you use Staging Directory as output folder for your Nuget Package step, you can specify a pattern of $(build.stagingDirectory)\*.nupkg to automatically publish all packages created in previous steps. If you will change the build in the future adding other NugetPackager steps to create other packages, you can use this single Nuget Publisher to automatically publish every .nupkg file found in staging directory.

Finally you need to specify the Nuget Server Endpoint; probably your combobox is empty, so you need to click the Manage link at the right of the combo to manage your endpoint.

Manage endpoint in your VSO account

Figure 5: Managing endpoint

Clicking Manage link, a new tab is opened in the service tab of Collection configuration, here you can add endpoint to connect your VSO account to other service. Since Nuget or MyGet is not in the list, you should add a new service endpoint of type Generic.

Specify your server url and your api key to create an endopint

Figure 6: Adding endpoint for nuget or myget server

You must specify the server url of your feed and your API KEY in the Password/Token Key field of the endpoint. Once you press OK the endpoint is created; no one will be able to read the API KEY from the configuration and your key is secured in VSO.

Now all Project Administrators can use this endpoint in your Nuget Publisher step to publish against that feed, without giving them API KEY or password. All endpoints have specific security so you can specify the list of the users that will be able to change that specific endpoint or list of users that will be only able to Read that specific endpoint. This is a nice way to save details of your nuget feed in VSO, specifying the list of the user that can use this feed, without giving password or token to anyone.

When everything is done, you can simply queue a new build, and choose the version number you want to assign to your Nuget Package.

You can queue the build specifying branch and Nuget Numbering

Figure 7: Queuing a build to publish your package with a specific number.

You have the ability to choose the branch you want to publish, as well as the Number of your nuget package to use. Once the build is finished your package should be published.

Feed detail in MyGet account correctly list packages published by my vNext build

Figure 8: Your package is published in your MyGet feed.

In previous example I’ve used master branch and published version number 1.3.1. Suppose you want to publish a pre-release package with new features that are not still stable. These features are usually in develop branch (especially true if you use GitFlow with git repositories), and thanks to configuration you can simply queue a new build to publish pre-release package.

Specifing developing branch and a package number ending with beta1 you can publish pre-release packages.

Figure 9: Publish a pre-release package using develop branch and a nuget version that has a –beta1 suffix.

I’ve specified to use develop branch and a nuget version number ending with –beta1, to specify that it is a pre-release package. When the build is finished you can check from your visual studio that everything is ok.

Verify that in Visual Studio stable and Pre-Release package is ok.

Figure 10: Verify in Visual Studio that everything is ok.

Thanks to Build vNext, publishing your package to myget or nuget or private nuget feed is just a matter of including a couple of steps and filling few textboxes.

Gian Maria.