VSTS Package, packages failed to publish

I have a build that publishes nuget packages on MyGet, we decided to move packages to VSTS internal package management, so I simply added another Build Task that pushes packages to VSTS internal feed. Sadly enough I got a really generic error

Error: An unexpected error occurred while trying to push the package with VstsNuGetPush.exe.
Packages failed to publish

Those two errors does not gives me real information on what went wrong, but looking in the whole log, I verified that the error happens when the task was trying to publish symbols packages (2).

image

Figure 1: Complete error log

It turns out that VSTS Package management is not still capable of publishing symbols package and this generates the error.

To solve this problem you can avoid generating symbols package, or you can simply avoid pushing them to VSTS feed. In the future VSTS feeds will probably supports symbols package and this problem will be gone.

In the meanwhile we have a issue on GitHub where we asked to the team for a much clearer error message, that can immediately point out the problem during the push.

Gian Maria.

New Nuget Task in VSTS Build

If you edit a build in VSTS where you configured Nuget Packaging and Publishing, you can notice that all the old tasks to pack and publish are marked as deprecated.

image

Figure 1: Old nuget tasks that are now deprecated.

Deprecating a package is needed when the Author decide to completely replace the entire package, changing also the id. This is needed when the task will be completely redesigned and will work in a complete different way from the old version.

For nuget package, Microsoft decides to publish a new task that can do Restore/Pack/Publish, all in one pacakge. The new package also uses a new dedicated nuget feed connection from your VSTS account to external nuget feed.

If you use VSTS internal feeds, there is no change, but if you are using an external feed, you can verify that now you can add a dedicated type of  connection for Nuget feeds (Figure 1).

image

Figure 2: External service endpoints configuration, now a dedicated Nuget feed is present.

The nice part of having a dedicated Nuget feed is that you can specify a dedicated authentication (1) and also you can Verify Connection (2) to check if all the data is good; the service knows that this is a Nuget endpoint and can verify if everything is good before triggering a build.

image

Figure 3: Configuration of Nuget Endpoint

image

Figure 4: Usage of the Nuget endpoint.

Once you defined one or more Nuget Endpoint, you can easily choose them from a Combobox in the Nuget Task, simplifying the configuration of the build.

I strongly suggest you to try this new task and to update all of your build to take advantage of it.

Gian Maria.

Publish nuget package during build (.NET Standard)

In a previous post I dealt with how to build a Multitargeted dotnetcore solution in VSTS, but the build is not really complete unless you are publishing the result somewhere. Since my example was a simple library, the obvious solution is publishing everything to a nuget feed.

Publishing with nuget is really really simple with VSTS build system, because you should simply use another .NET Core task instance plus a NuGet publisher.

This image shows two tasks in the build, the dotnetpack and nuget publisher.

Figure 1: Couple of simple task is enough to publish to nuget feed

The configuration of these two task is really really simple, the dotnet pack task needs the list of the project that we want to publish and an additional command line that allows you to specify the version to publish and some other options.

image

Figure 1: .NET core task configuration

Thanks to dotnet task, it is really easy to invoke dotnet command during a build, thus, creating nuget package is really a breeze.

You should use pack command an in the projects section allows you to specify directly all the path of the csproj you want to publish, and in the arguments I’ve used several options. Here is the complete arguments commandline:

–configuration $(BuildConfiguration) –no-build /p:Version=$(NugetVersion) -o $(build.artifactstagingdirectory)\Nuget

configuration allows to decide what configuration you want to release, the –no-build option is needed to speedup the build, because the projects where built with previous tasks, there is no need to build them another time. The /p:Version parameter allows me to specify the version of the nuget package and finally the –o options allows you to specify the folder where the packages will be created.

After this task you have the Nuget Publisher task, that simply need to know the folder where the build generated the nuget packages and the target package where to publish the result.

This picture shows the configuration of nuget publisher wher I simly selected the path that contains nuget packages and the endpoint destination

Figure 3: Configuration of the NuGet Publisher task.

Et voilà, this is everything you need to do to have your package to be published and thanks to multitargeting, the nuget package automatically contains all the version of the framework you are targeting.

Package details in myget, showing that all targeted framework are included in the package

Figure 4: Package details in myget, showing that all targeted framework are included in the package

With a simple couple of task and 1 minute configuration, you have your library published to nuget feed.

Gian Maria.

Nuget default configuration

If you use MyGet as package source or some other Nuget Feed and your code fail to build in a buildserver because it cannot retrieve some package, probably you have some problem in NuGet configuration.

If you issue a nuget.exe source list  in command line, you should be able to see all package source configured. Actually I had a problem in a TeamCity build server, after we added a new package source to a solution and pushed code to the server the build was broken. The error is due to the inability to download packages from the new source. Running nuget.exe source list command confirmed me that the new package source was not listed, so it is clear why the package was missing.

If you have this kind of a problem, just know that nuget has a default configuration file that can be used to set all sources for all users in the machine. In our build server this file is missing, so I simply created it listing all the package source we need and the build started working like a charm.

Gian Maria. 

Integrating GitVersion and Gitflow in your vNext Build

In previous article I’ve showed how to create a VSO build vNext to automatically publish a nuget package to Myget (or nuget) during a build. [Publishing a Nuget package to Nuget/Myget with VSO Build vNext]. Now it is time to create a more interesting build that automatically version your assemblies and nuget packages based on GitFlow.

GitFlow and GitVersion

GitFlow is a simple convention to manage your branches in your Git repository to support a production branch, a developement branch and Feature/Support/Release/hotfix branches. If you are completely new on the subject you can find information at the following locations:

You can find also a nice plugin for Visual Studio that will support GitFlow directly from Visual Studio IDE and also install GitFlow for your command line environment in one simple click. [VS 2015 version] [VS 2013 version]. Once you get accustomed with GitFlow, next step is having a look at Semantic Versioning,  a simple versioning scheme to manage your packages and assemblies.

The really good news, is that a free tool called GitVersion exists to do semantic versioning simply examining your git history, branches and tags. I strongly suggest you to read documentation for GitVersion online, but if you want a quick start, in this blog post I’ll show you how you can integrate with a vNext VSO build.

Thanks to GitVersion tool you can easily manage SemVer in a build vNext with little effort.

How GitVersion works at a basic level

GitVersion can be downloaded in the root folder of your repository; once it is there invokeing directly from command line with /ShowConfig parameter will generate a default configuration file.

GitVersion /ShowConfig > GitVersionConfig.yaml

This will create a default configuration file for GitVersion in the root directory called GitVersionConfig.yaml. Having a configuration file is completely optional, because GitVersion can work with default options, but it is really useful to explicit default parameter to know how Semantic Versioning is handled by the tool.

I’m not going throught the various options of the tool, you can read the doc online and I’ll blog in future post about a couple of options I usually change from default.

For the scope of this article, everything I need to know is that, invoking gitversion.exe without parameters in a folder where you have a git repository with gitflow enabled will return you a Json data. Here is a possible example:

{
  "Major":1,
  "Minor":5,
  "Patch":"0",
  "PreReleaseTag":"unstable.9",
  "PreReleaseTagWithDash":"-unstable.9",
  "BuildMetaData":"",
  "BuildMetaDataPadded":"",
  "FullBuildMetaData":"Branch.develop.Sha.8ecde89ef5b97eabcf6e0035119643334ba40c4e",
  "MajorMinorPatch":"1.5.0",
  "SemVer":"1.5.0-unstable.9",
  "LegacySemVer":"1.5.0-unstable9",
  "LegacySemVerPadded":"1.5.0-unstable0009",
  "AssemblySemVer":"1.5.0.0",
  "FullSemVer":"1.5.0-unstable.9",
  "InformationalVersion":"1.5.0-unstable.9+Branch.develop.Sha.8ecde89ef5b97eabcf6e0035119643334ba40c4e",
  "BranchName":"develop",
  "Sha":"8ecde89ef5b97eabcf6e0035119643334ba40c4e",
  "NuGetVersionV2":"1.5.0-unstable0009",
  "NuGetVersion":"1.5.0-unstable0009",
  "CommitDate":"2015-10-17"
}

This is the result of invoking GitVersion in develop branch; and now it is time to understand how these version numbers are determined.  My Master Branch is actually tagged 1.4.1 and since develop will be the next version, GitVersion automatically increments Minor versioning number (this is the default and can be configured). FullSemVer number contains the suffix unstable.9 because develop branch is usually unstable, and it is 9 commits behind the master. This will immediately gives me an idea on how much work is accumulating.

Now if I start a release 1.5.0 using command git flow release start 1.5.0  a new release/1.5.0 branch is created, and running GitVersion in that branch returns a FullSemVer of 1.5.0.0-beta0. The suffix is beta, because a release branch is something that will be released (so it is a beta) and 0 means that it is 0 commits behind develop branch. If you continue to push on release branch, the last number continues to increment.

Finally when you finish the release, the release branch is merged with master, master will be tagged 1.5.0 and finally release branch is merged back to develop. Now running GitVersion on develop returns version 1.6.0-unstable.xxx because now master is on 1.5.x version and develop will be the next version.

How you can use GitVersion on build vNext

You can read about how to integrate GitVersion with build vNext directly on GitVersion documentation, but I want to show you a slightly different approach in this article. The way I use GitVersion is, directly invoking in a Powershell build file that takes care of everything about versioning.

The main reason I choose this approach is: GitVersion can store all the information about versioning in environment variables, but in build vNext environment variables are not maintained by default between various steps. The second reason is: I already have a bulid that publish on nuget with build number specified as build variable, so I’d like to grab version numbers in my script and use it to change variable value of my build.

Thanks to PowerShell, parsing Json output is super easy, here is the simple instructions I use to invoke GitVersion and parse all json output directly into a PowerShell variable.

$Output = & ..\GitVersion\Gitversion.exe /nofetch | Out-String
$version = $output | ConvertFrom-Json

Parsing output of GitVersion inside a PowerShell variable gives you great flexibility on how to use all resulting numbers from GitVersion

Then I want to version my assemblies with versions returned by GitVersion. I starts creating some Powershell variables with all the number I need.

$assemblyVersion = $version.AssemblySemver
$assemblyFileVersion = $version.AssemblySemver
$assemblyInformationalVersion = ($version.SemVer + "/" + $version.Sha)

I’ll use the same PowerShell script I’ve described in this post to version assemblies, but this time all the versioning burden is taken by GitVersion. As you can see I’m using also the AssemblyInformationalVersion attribute that can be set as any string you want. This will give me a nice file version visible from Windows.

image

Figure 1: Versioning of the file visible in windows.

This immediately tells me: this is a beta version and gives me also the SHA1 of the commit used to create the DLL, maximum traceability with minimum effort. Now is time to use some build vNext commands to version nuget.

How Build vNext can accepts command from powershell

Build vNext infrastructure can accept commands from PowerShell script looking at the output of the script, as described in this page.

One of the coolest feature of build vNext is the ability to accept commands from console output of any script language

Write-Output in powershell is all that I need to send commands to build vNext engine. Here is how I change some build variables:

Write-Output ("##vso[task.setvariable variable=NugetVersion;]" + $version.NugetVersionV2)
Write-Output ("##vso[task.setvariable variable=AssemblyVersion;]" + $assemblyVersion)
Write-Output ("##vso[task.setvariable variable=FileInfoVersion;]" + $assemblyFileVersion)
Write-Output ("##vso[task.setvariable variable=AssemblyInformationalVersion;]" + $assemblyInformationalVersion)

If you remember my previous post on publishing Nuget Packages, you can see that I’ve used the NugetVersion variable in Build Definition to specify version number of nuget package. With the first line of previous snippet I’m automatically changing that number to the NugetVersionV2 returned by GitVersion.exe. This is everything I need to version my package with SemVer.

Finally I can use one of these two instructions to change the name of the build.

Write-Output ("##vso[task.setvariable variable=build.buildnumber;]" + $version.FullSemVer)
Write-Output ("##vso[build.updatebuildnumber]" + $version.FullSemVer)

The result of these two instructions is quite the same, the first one change build number and change also the variable build.buildnumber, while the second one only changes the number of the build, without changing the value of build.buildnumber variable.

Final result

My favourite result is: my build numbers now have a real meaning for me, instead of simply representing a date and an incremental build like 20150101.2.

image

Figure 2: Resulting builds with SemVer script

Now each build name immediately tells me the branch used to create the build, and the version of the code used. As you can see, release and master branches are in continuous deployment, because at each push the build is triggered and nuget package is automatically  published. For Develop branch the build is manual, and is done only when I want to publish a package with unstable version.

I can verify that everything is ok on MyGet/Nuget side, packages were published with the correct numbers.

image

Figure 3: SemVer is correctly applied on NuGet packages

Thanks to GitVersion I can automatically version: build number, all the assemblies and NuGet package version with few lines of powershell.

Conclusions

Thanks to build vNext easy of configuration plus powershell scripts and simple command model with script output, with few lines of code you are able to use Semantic Versioning in your builds.

This example shows you some of the many advantages you have with the new build system in VSO/TFS.

Gian Maria.