Monitor if your branch will generate merge conflicts with TFS Build

One of the greatest advantage in using Git over a centralized Version Control System is branching system. It is quite common for developers to start branching whenever they need to add a new feature, work on that branch and finally merge back to mainline when the feature is finished. One of the most famous workflow is called GitFlow, a way to work in Git that is implemented even in some GUI tool like SourceTree.

One of the main problem when you heavily use feature branches is the moment you merge back to mainline. One of the technique to easy the pain is periodically do a forward integration from mainline to feature branch, because it is usually easier doing several little merge than doing a big merge when the feature is complete.

Assuming that the mainline contains only stable code, because all feature branch merge to mainline only when they are stable, it is usually safe to periodically merge from mainline to branch to avoid a big merge at the end. In this scenario the question usually is: how frequently I have to merge? One possible solution is to take advantage of your Continuous Integration system to warn you when a modification you did on a branch introduces problem because:

  • Does not merge automatically with mainline but it generates conflicts
  • Even if the branch merge automatically the code wont compile.
  • Even if the branch merge automatically and the code compile, some tests where broken.

With TFS Build you can achieve this result quite simply, first of all configure a build with a trigger for each push, then in the Source Settings specify that this build should be triggered whenever any branch that starts with feat got any push.


Figure 1: Source code configuration, only build when a branch that starts with feat got a code increment

This build is intended to monitor the status of all feature branches. All you need to do is including a PowerShell script in source code that will be run pre-build. This script basically merge with master with command line git (Build Agent where that build runs should have git installed and available in PATH environment variable). This is a possible PowerShell script to accomplish the merge


$branchName = ""
$teststring | where { $_ -match "/(?[^/]*?):" } |
    foreach { $branchName = $Matches["bname"] }

Write-Host "BranchName = $branchName"
$rootDir = "$env:TF_BUILD_BUILDDIRECTORY\src"
Write-Host "Source folder is $rootDir"

#be sure to checkout the right branch 
$ps = new-object System.Diagnostics.Process
$ps.StartInfo.Filename = "git"
$ps.StartInfo.WorkingDirectory = $rootDir
$ps.StartInfo.Arguments = "checkout $branchName"
$ps.StartInfo.RedirectStandardOutput = $True
$ps.StartInfo.RedirectStandardError = $True
$ps.StartInfo.UseShellExecute = $false

[string] $Out = $ps.StandardOutput.ReadToEnd();
[string] $ErrOut = $ps.StandardError.ReadToEnd();
Write-Host "Output git checkout $branchName is:" $Out
Write-Host "Return value: " $ps.ExitCode
if ($ps.ExitCode -ne 0) 
    $Host.ui.WriteErrorLine("Branch cannot be checked out")

#try to merge

$ps = new-object System.Diagnostics.Process
$ps.StartInfo.Filename = "git"
$ps.StartInfo.WorkingDirectory = $rootDir
$ps.StartInfo.Arguments = " merge master"
$ps.StartInfo.RedirectStandardOutput = $True
$ps.StartInfo.RedirectStandardError = $True
$ps.StartInfo.UseShellExecute = $false

[string] $Out = $ps.StandardOutput.ReadToEnd();
[string] $ErrOut = $ps.StandardError.ReadToEnd();
Write-Host "Output git merge master:" $Out
Write-Host "Return value: " $ps.ExitCode

if ($ps.ExitCode -ne 0) 
    $Host.ui.WriteErrorLine("Branch does not merge correctly")

This is really a first tentative in this direction, there probably better way to do it, but it solves the problem without any great complexity. It just uses a regular expression to find branch name from the environment variable TF_BUILD_SOURCEGETVERSION. Once you have name of the branch just checkout it and then try to “git merge master” to merge the branch with master. Quite all git command return 0 if the operation is successful, so if the ExitCode of “git merge master” command is different from 0 it means that the merge operation did not complete correctly (conflicts occurred). If the merge is unsuccessful the script use UI.WriteErrorLine to write error on the UI and this has two effects: first the message will be reported on the Build Summary, then then build will be marked as partially failed.

Now even if your branch compile correctly and every test is green, after you push to TFS your Build Server will try to merge and run the build, if the branch does not automatically merge with master the build will partially fail and you got this:


Figure 2: Code does not automatically merge after the latest push.

This is a signal that is time to do a forward integration from mainline to your branch, because the amount of conflicts will be probably small. Another advantage is that you can merge the code while your modification is fresh in your mind. Nothing is more painfully than merge code you have written a month ago.

If the merge is successfully it does not mean that the code is in good health. Suppose you rename a method in your branch, but in the mainline another one has pushed an increment with another class that uses that method. This is the perfect example of code that gave no merge conflicts, but it does not even compile. Since the script actually does the merge before the build, the rest of the build works on the code result of the merge, now the build fails.


Figure 3: Merge was successfully, but the result of the merge does not compile

In this situation you surely have a standard build that simply works on the feature branch that is green, but the Build System can run for you this another special build that runs on the result of the merge with mainline that is Red. This means: In your branch code everything is green, but when you will merge to mainline you will have problem, so it is probably better to look at that problems now!. 

The very same happens if the result of the merge breaks some unit test. In conclusion, thanks to few PowerShell lines, you can instruct your TFS build to automatically try to merge each increment of code in your feature branches and then compile, run test and do whatever any operation you usually do on your build on the result of the merge between the feature branch and mainline. Thanks to this you can immediately merge from mainline to your branch to fix the problem as soon as you have introduced it.

This approach has some disadvantages. First of all a PowerShell script cannot make the build fails, so if the code does not merge, usually you will have a failing build because the build system try to build source code during a merge and find invalid characters in source code. Second, this special build runs only when there is a push on some feature branch, but nothing happens if someone pushes in the mainline. In that case you should run this special build for every feature branch you have in your repository.

Gian Maria.

Encrypt your password in TFS Build using Certificates

Using DPAPI to encrypt password in builds suffers from a serious drawback, the password can be decrypted only by code that runs on the very same computer used to encrypt the password. You can overcome this limitation using roaming profiles, but it is not a option in many scenario. Another technique to enable multiple build servers to decrypt a password is using certificates. In PowerShell is really easy to use a certificate to encrypt/decrypt string and this article will show you how to secure a password in TFS Build definition using Certificates.

First step is generating a valid certificate, this is easy thanks to MakeCert utility, just open a Developer Command Prompt and type

MakeCert.exe -r -pe -n “CN=www.cyberpunk.local” -sky exchange -ss my -len 2048 -e 01/01/2020

This command creates a certificate in your certificate store. To view it you should type certmgr to open the Certificates Manager Console. Your newly created certificate should now appear in the list of certificates.


Figure 1: Your newly created certificate appears in certificate store.

Double Clicking the certificate opens detail windows where you can find the value of Thumbprint property (Figure 2). This property is useful because it uniquely identifies this certificate in certificate store, and it should be used to load correct certificate in PowerShell script.


Figure 2: Grab the Thumbprint of the certificate in detail pane.

You should grab the thumbprint from the certificate details, it should be a string like: f02e4c0e13e26d25065cc0db1d03450acaef90d6. Just copy from the details pane, remove all the spaces and you are ready to use this certificate to Encrypt and Decrpyt a string. Here is the PowerShell script that uses this certificate to Encrypt and then Decrypt a string.

$cert = Get-Item -Path Cert:\CurrentUser\My\f02e4c0e13e26d25065cc0db1d03450acaef90d6 -ErrorAction Stop
$decrypted = ""
$bytesDecrypted = ""
$bytesToDecrypt = ""
$pwd = 'mySecurePwd'
$enc = [system.Text.Encoding]::UTF8
$pwdBytes = $enc.GetBytes($pwd) 

$encryptedPwdBytes = $cert.PublicKey.Key.Encrypt($pwdBytes, $true)
$EncryptedPwd = [System.Convert]::ToBase64String($encryptedPwdBytes)

Write-Output "`r`nEncrypted Password IS:"
Write-Output $EncryptedPwd

#Now decrypt.
$bytesToDecrypt = [System.Convert]::FromBase64String($EncryptedPwd)
$bytesDecrypted = $cert.PrivateKey.Decrypt($bytesToDecrypt, $true)

$decrypted = $enc.GetString($bytesDecrypted)

Write-Output "`r`nDecryptedPassword is: $decrypted"

As you can see using the certificate is straightforward, you just grab a reference to certificate with the Get-Item using the thumbprint and then it is just a matter of calling Encrypt method of the public key object stored inside the certificate and convert to Base64 to be conveniently represented as a string. To decrypt you can call Decrypt method of the Private Key object.

If you does not know how a RSA key pair works, here is the basic concept. The certificate contains two keys, one is the Public Key, and the other is the Private Key. You use the public key to encrypt a string and the resulting encrypted string can be decrypted only by the Private Key. Public key used to encrypt the string cannot be used to decrypt it. This is the reason why RSA is called an Asymmetric Algorithm.

Now return to the Certificate manager, and press “Export” button to export the certificate, choose to export only the public key, choose DER format and export the certificate to a folder of your computer. Then press “Export” again but now choose to export the private key, choose the PKCS format and then use a password to protect the exported file. A password is required because the private key is the information that you need to keep secure from the eyes of the public. Now you should have two exported files, one with .cer extension that contains only the public key, the other with .pfx extension containing also the private key.


Figure 3: Exported certificates in my filesystem.

Now remove the certificate from Certificate Manager, and try to run again the script to encrypt and decrypt the string, you should now receive an error from Get-Item, telling you that the required path does not exists. This confirms that the certificate was removed from your certificate store. Now press the “import” button on Certificate Manager and import the previously exported public key, the file public.cer in my example and run the script again to verify that you are able to encrypt a string but you cannot decrypt, because you have only the public key.


Figure 4: Now that you have only the public key, you are able only to encrypt and you could not decrypt anymore the string.

To verify that you can decrypt password using Private Key, press Import button again on the Certificate Store and import the Private.pfx certificate. You will be prompted to enter the password you use during the export, and if you look at the options you can verify that there is an important option called “mark this key as exportable”, that is turned off by default.


Figure 5: Importing a certificate with Private key in certificate store

This option is really important, because this prevents users to further export the private key to another computer. Once you imported the Private certificate, you can verify that pressing the Export button does not permit you to choose the option to export the Private Key. Once the certificate with private key is imported you are now able to decrypt the password.

Thanks to asymmetric key you can simply store the certificate with the public key in any network share, everyone can access the certificate, install it and securely encrypt a password because only people with the private key can decrypt it. Now you should login in every Build Machine with credential of TfsBuild, and import the certificate with the Private key to personal certificate store, do not mark the key as exportable. Now you can use Certificates instead of DPAPI to encrypt and decrypt password during the build. This script will work in every build server where you have installed the certificate for the user TfsBuidl. Here is the script.

[string] $url = "http://webtest1.cyberpunk.local:10000/MyService/Index",
[string] $username = "",
[string] $password = ""

Write-Host $password
$cert = Get-Item -Path Cert:\CurrentUser\My\f02e4c0e13e26d25065cc0db1d03450acaef90d6 -ErrorAction Stop
Write-Host $cert.PrivateKey.KeySize
$bytesToDecrypt = [System.Convert]::FromBase64String($password)

$bytesDecrypted = $cert.PrivateKey.Decrypt($bytesToDecrypt, $true)
$enc = [system.Text.Encoding]::UTF8
$plainText = $enc.GetString($bytesDecrypted)

Write-Host "Invoking-Service"

$retValue = Invoke-RestMethod $url"?username=$username&password=$plainText"  

Write-Host "ReturnValueIs: "$retValue.Message

This script simply decrypt the password and call a simple REST service just to verify that the password was decrypted with success. Here is the output of the build.


I want to remember again that this technique does not secure the password for people that can schedule a new build in TFS. Any person that can schedule a new build can create a PowerShell script that decrypt a password and print it in build output, grab an encrypted password from another build definition and let this script decrypt it.

This is the same technique I used with the above script where I dump the password in clear format in Build output, just to verify that decryption is ok. Any other user with permission to schedule a build could have scheduled a build running a similar script to decrypt password store by other people.

Gian Maria.

Related Articles

Tfs build fails with Database Projects

If you have a TFS Builds that fails with an error like this one, even if you have installed Visual Studio 2013 on your build server.

The imported project “C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\SSDT\Microsoft.Data.Tools.Schema.SqlTasks.targets” was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk

The reason could be that you miss Sql Server Data Tools, you only need to download them in all servers with Build Agents and install it. Now the build should runs without problems.

Gian Maria.

Automatically Deploy PowerShell modules to Build Agents in TFS

I’ve done some blog post on customizing TFS Build with PowerShell scripts and the very first question I got from this approach is

How can I store some PowerShell modules “somewhere” and have them available for all Build Agents?

This is a real good question, but sadly enough it has no out-of-the-box answer. Luckily enough you can solve this with a little bit of PowerShell knowledge. Please be aware that this solution is based on some assumption on how TFS build actually works and is not guaranteed to be stable in the future, but I’d like to share with you all if you want to try into your environment.

TFS Build controllers have a special Source Control Folder you can specify to store assemblies that should be available for Agents during the build.


Figure 1: Version control path to custom assemblies for Build Controllers

The very first step is creating a subfolder where you store all PowerShell modules you want to be available to Build Agents during a build.


Figure 2: Store your PowerShell Modules inside a specific folder of the Path to custom assemblies

This special folder is downloaded for every agent during a build and it is stored inside a specific subfolder of the temp folder of the user configured to run build agents. Once all modules are committed, it is just a matter of writing a bunch of PowerShell lines of code to identify the directory where that modules are downloaded during the build, and then copy all modules inside one of the default location supported by the PowerShell engine.

Write-Host "TempDirectoryIs"$env:Temp

$scriptRoot = Split-Path -Parent -Path $MyInvocation.MyCommand.Definition

$scriptRoot -Match "\d+"
$agentNumber = $Matches[0]
Write-Host "Build Agent Number is"$agentNumber

Write-Host "PSModulePath = "$env:PSModulePath
$userModuleLocation = $env:PSModulePath.Split(';') | where {$_.Contains('Documents')}
Write-Host "User Powershell Module location is: "$userModuleLocation

$tempBuildDirectory = $env:Temp + "\BuildAgent\" + $agentNumber + "\Assemblies\"
Write-Host "Location of custom assembly: "$tempBuildDirectory
Copy-Item -Path $tempBuildDirectory"PowershellModules\" -Destination $userModuleLocation -Recurse

Let me explain how this works. First of all I need to know the location of the temp directory, stored in Environment variables $env:temp. When the build runs this variable is something like: C:\Windows\SERVIC~2\LOCALS~1\AppData\Local\Temp. This is not the real directory where the agent downloads the content of the custom assembly folder, because each agent creates a specific subdirectory, based on its id Es: C:\Windows\ServiceProfiles\LocalService\AppData\Local\Temp\BuildAgent\290\Assemblies

Agent’s id is also used to create a separate local build folder for each agent, the first step is to store folder location where the executing script is located in variable $scriptRoot. The value is something like: C:\Builds\290\Experiments\Simple Powershell Versioning\src\BuildScripts. As you can see, id of the Agent is the first number that appears in folder structure, and extracting it with a simple regex is really straightforward.

Final step is locating all the paths where PowerShell is actually searching for modules. All locations are stored inside the Environment variable $env:PSModulePath, containing a semicolon separated list of default folders for modules. The script cannot write to all of these folders because some of them are stored in path of disk that requires elevated permission (Es: C:\ProgramFiles). Usually one of them is located under Document folder of the current user: ??%UserProfile%\Documents\WindowsPowerShell\Modules, and since it is in user folder build agent can write on it without problem. Thanks to this knowledge I’ve decided to split the path to find the one that contains the path Documents. I know that this can be a fragile assumption, but it usually works.

The last step is determine $tempBuildDirectory combining temp folder with agent id and finally use a Copy-Item cmdlets to copy all custom PowerShell modules inside the supported module location under agent user directory.

I did not use this strategy really often because it can be fragile. If you are using TFVC the best solution usually is

  • Add the custom powershell modules directory to the workflow mapping of the build
  • use the Import-Module PowerShell command to import modules from a fixed location


Figure 3: Download your custom PowerShell modules in a know location as part of the build workspace.

since I’ve stored build scripts inside $Experiments/trunk/Builds/ScriptTest/BuildScripts folder, I can use relative path to find location of PowershellModules on disk and import modules from specific locations. Sadly enough this solution does not works with Git repository and copying the modules with the above technique can be a viable solution.

Gian Maria.

Deploy from a Team Foundation Server build to Web Site without specifying password in the build

In a previous article I explained how to deploy an ASP.NET Web Site from a TFS Build thanks to MSDeploy engine. One of the great complain you can have with this solution is the need to specify UserName and password in build configuration and the need to use the AllowUntrustedCertificate=true.

The problem of the certificate is the simpler of the two to solve, you just need to use a certificate that is trusted inside your organization or a certificate issued by a trusted certificate authority (es godaddy), instead of the default one that is generated by MsDeploy configuration in IIS. This is most an administrative stuff and I’m not going to cover it in this post.

The most annoying stuff is getting rid of the password. You should start configuring the deploy user for IIS using the same user that runs TFS Build, this will give to build user the permission to execute the deploy.


Figure 1: Give to the build user publish permission.

This is not enough, if you fire your bulid you probably will receive an authorization error.

C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\Web\Microsoft.Web.Publishing.targets (4255): Web deployment task failed. (Connected to the remote computer (“webtest1.cyberpunk.local”) using the Web Management Service, but could not authorize. Make sure that you are using the correct user name and password, that the site you are connecting to exists, and that the credentials represent a user who has permissions to access the site.  Learn more at:

The dreaded ERROR_USER_UNAUTHORIZED can frustrate you for long time, because it is not so easy to solve. First of all you should check if the Windows Authentication is enabled in IIS configuration. Just go to the Web Server and verify settings of Management Service



Figure 2: You should configure Management Service to use Windows Authentication


Figure 3: You should be sure that Windows Credentials is enabled.

This is usually not enough, you should also be sure to add a value in the registry, you should locate HKLM\SOFTWARE\Microsoft\WebManagement\Server key and then add a DWORD value named WindowsAuthenticationEnabled with the value 1. Finally you should restart management service (net stop wmsvc and then net start wmsvc).

If you run the build you will probably still get the ERROR_USER_UNAUTHORIZED error, this is caused by parameters passed to MsBuild. There are two parameters you need to pass to MSBuild to have integrated security works and they are:

1) /p:UserName=””

2) /p:AuthType=NTLM

An empty username parameter is needed for it to work, if you forget to specify it, you will get ERROR_USER_UNAUTHORIZED even if the user running the script (tfsbuild) has all the right to deploy the site. The AuthType parameter is telling to the server to use Windows Authentication.

Now your build should be green and you have no credential stored inside the build definition and your are not breaking any security good practice.

Gian Maria.