Release to Azure with Azure ARM templates

Thanks to new Release Management system in VSTS / TFS creating a release to your on-premise environment is really simple (I’ve described the process here). Another option is creating a test environment in Windows Azure, and if you choose this option life can be even easier.

In this example I’m using Azure as IAAS, deploying a software on a Windows Virtual Machine. While this is probably not the best approach to cloud (PAAS is surely a better approach) to create a test environment it can be perfectly acceptable.

I’m not going to give you an introduction or explanation of Azure Resource Manager because there are tons of resources on the web and Azure moves so quickly that every information I’m going to give you will probably will be old by the time I press “publish” :). The purpose of this post is giving you a general idea on how to use Azure ARM to create a release definition that automatically generates the resource in azure and deploy your software on it.

My goal is using Azure ARM for DevOps and Automatic / Continuous Deployment and the first step is creating a template file that describes exactly all the Azure resource needed to host my application. Instead of starting to write such template file I start checking on GitHub becuse there are tons of template files ready to use.

As an example I took one of the simplest, called 101-vm-simple-windows. It creates a simple Windows Virtual Machine and nothing else. That template has various parameters that can allow you to specify VM Names and other properties, and it can be directly used by a Release Management definition.  I’ve done simple modification to the template file and in this situation it is better to first check if everything is working as expected triggering the deploy process directly from command line.

New-AzureRmResourceGroupDeployment 
	-Name JarvisRm 
	-ResourceGroupName JarvisRm
	-TemplateFile "azuredeploy.json" 
	-adminUsername alkampfer 
	-adminPassword ********** 
	-vmName JarvisCmTest 
	-storageAccount jarvisrmstorage 
	-dnsLabelPrefix jarvisrm

As you can see I need to choose the name of the resource group (JarvisRm) specify the template file (azuredeploy.json) and finally all the paramters of the template as if they are parameter of the PowerShell cmdlet. Once the script finished verify that the resource group was created correctly and all the resources are suitable to deploy your software.

image

Figure 1: Your Resource group was correctly created.

Once everything is correct, I deleted the JarvisRm resource group and I’m ready to use the template on a release definition.

Always test your ARM template directly from command line, to verify that everything is allright. When the resource are created try to use them as manually target of a deploy and only once everything is ok start automating with Release Management.

When you have a good template file the best place to store it is in your source control, this allow you to version this file along with the version of the code that is supposed to use it. If you do not need versioning you can simply store in a network share, but to avoid problem it is better to have Release Management Agent run the template from a local disk and not from a network share.

image

Figure 2: Copy template file from a network share to a local folder of the agent.

First step of the release process is copying template files from a network share to $(System.DefaultWorkingDirectory)\ARM folder so PowerShell can run against script that are placed on local disk. The second task is Azure Resource Group Deployment that uses the template to deploy all Resources to Azure.

image

Figure 3: The Azure Deployment task is used to create a Resource Group from a Template definition.

You should specify only template file (1) and  all the parameters of the template (2) such as userName, password, dns name of the VM etc. As a nice option you can choose to Enable Deployment Prerequisites (3) to have your VM being able to be used as a target for Deploy Action. You can read more about prerequisites on MSDN blog, basically when you select this action the script will configure PowerShell and other option on the target machine to being able to execute script remotely.

Virtual machines need to be configured to be used as target of deploy tasks such as remote PowerShell execution, but the Azure deployment task can take care of everything for you.

This task requires that you already connected the target Azure subscription with your VSTS account. If you never connected your TFS / VSTS account to your Azure subscription with ARM, you can follow the instruction at this link, that contains a PowerShell script that does EVERYTHING for you. Just run the script, and annotate in a safe place all the data you should insert to your TFS / VSTS instance to connect to Azure with ARM.

Antoher aspect you need to take care of, is the version of PowerShell azure tools installed in the machine where the Release Agent is running. Release Management script are tested agains specific version of Azure PowerShell tools, and since the Azure team is constantly upgrading the Tools, it coudl happen that TFS / VSTS Release Management Tasks are not compatible with the latest version of the Azure Tools.

All of these tasks are open sourced, and you can find information directly on GitHub. As an example at this link tjere are the information about the DeployAzureResourceGroup task. If you go to the bottom you can verify the PowerShell tools version suggested to run that task.

image

Figure 4: Supported version of AzureRM module version

Clearly you should install a compatible version in the machine where the Agent is installed. If you are unsure if the agents has a suitable version of Azure PowerShell tools, you can go to TFS Admin page and verify capabilities of the agent directly from VSTS / TFS

image

Figure 5: Agent capabilities contains the version of Azure PowerShell tools installed

Demand for Azure PS is present on Release Definition, but it does not specify the version, so it is not guarantee that your release is going to be successful.

image

Figure 5: Release process has a demands for Azure PowerShell tools but not to a specific version.

As a result I had problem in setting up the Release Process because my agents has PowerShell tools version 1.4 installed, but they are not fully compatible with Release Manager Task. Downgrading the tools solved the problem.

If your release fails with strange errors (such as NullReferenceException) you need to check in GitHub the version of PowerShell tools needed to run that task and install the right version in the agent (or at least you can try to change the version until you find the most recent that works)

The Azure Resource Group Deployments takes care of everything, I’ve modified the base script to apply a specific Network Security Group to the VM but the general concept is that it configures every Azure Resource you need to use. At the end of the script you have everything you need to deploy your software (Virtual Machines, sites, databases, etc).

In my example I need only a VM, and once it is configured I can simply use Copy to Azure VM task and Execute PowerShell on Azure VM Task to release my software, as I did for my on-premise environment.

image

Figure 6: Configuration of the Task used to copy files to Azure VM

You can specify files you want to copy (1) login to the machine (2) and thanks to Enable Copy Prerequisites (3) option you can let the Task takes care of every step needed to allow copy file to the VM. This option is not needed if you already choosen it in the Azure Deployment task, but it can be really useful if you have a pre-existing Virtual Machine you want to use.

Final Step is executing the release script on target machine, and it has the same option you specify to run a script on a machine on-premise.

image

Figure 7: Run the installation PowerShell script on target Azure VM

Once everything is in place you only need to create a release and wait for it to be finished.

image

Figure 8: Output of release definition with Azure ARM

With this example, since I’m using a Virtual Machine, deploy script is the same I used for on-premise release, with PAAS approach, usually you have a different script that target Azure specific resources (WebSites, DocumentDb, etc)

If the release succeeded you can login to portal.azure.com to verify that your new resource group was correctly created Figure 1, and check that in the resource group there are all the expected resources Figure 9.

image

Figure 9: Resources created inside the group.

To verify that everything is ok you should check the exact version of the software that is actually deployed on the environment. From Figure 10 I can see that the release deployed the version 1.5.2.

image

Figure 10: List of all most recents releases.

Now I can login to the VM and try to use the software to verify that it is correctly installed and that the version installed is correct.

image

Figure 11: Software is correctly installed and the version corresponds to the version of the release.

Azure Resource Management is a powerful feature that can dramatically simplify releasing your software to Azure, because you can just download scripts from GitHub to automatically creates all Azure Resources needed by your application, and lets VSTS Release Management tasks takes care of everything.

Gian Maria.

Create a release in TFS 2015 / VSTS Release Management

This is the end of the journey of the last serie of posts. I’m now at the situation where I have a build that produces a single zip file with everything I need to deploy the software and a bunch of PowerShell scripts that relase the software using that zip as a source artifact.

Now it is time to automate the process with Release Management. I want to use RM because the process is automated on a chain of environments, but also I have traceability, auditing, and verification of the release procedures. I’m not going to cover all the steps needed to create the a release definition, I want to focus on how simple is creating a Release Process when you adopt Zip+PowerShell approach. I strongly suggest to have a look at official site if you never heard of VSTS / TFS Release Management.

If you have a zip and PowerShell files that release the software using only the zip, you are ten minutes away from a working Release Management definition

Let’s start creating a first release, the most important aspect is adding a build as Artifacts source, this allows the release to consume what is produced by the build.

this image shows how a build can be linked to Release MAnagement, so the RM process can access artifacts produced by the build

Figure 1: Add your build as a artifacts source for the release process

This build produces a couple of artifacts, the zipped file with everything and the set of PowerShell scripts needed to run the software (as I suggested they should always be stored in source control and moved as build artifacts). Thus the release process is really simple and is composed by only three steps.

The entire release process is only three task, as described further in the post.

Figure 2: The entire release process is composed only by three scripts

First two steps are used to copy zip file and installation scripts in a folder of the target machine (c:\deploy\jarvis\…). as you can see I’m using local administrator user, so this technique can be used even if the machine is not in the domain. You should now run this script on the machine where the agent is running.

Set-Item WSMan:\localhost\Client\TrustedHosts -Value WebTest -Force

This instruction should be run to the machine where the build agent is running, and specify that the target machine is a Trusted Hosts. If you are using domain credentials, this is not needed.

The password is stored inside a Release variable, to use secret variable feature, so noone that can edit this release can see the password in clear text.

Password for target machine is stored as secret variable in Configuration section

Figure 3: Password for target machine is stored as secret variable in Configuration section

Final step in Figure 2 is using the task to run the installation script in a remote machine machine.

This image shows the configuration of the Remote Powershell task that allows you to execute a powershell in a remote machine.

Figure 4: Run PowerShell in a remote machine to install software

If you already tested the script manually, there is no reason why it should fail in RM process. Remember that you should never use command that interact with the shell, and you should use Write-Output instead of Write-Host in your PowerShell script to be sure that no shell is used.

With only three task I’ve created a release definition for my software

Now you can trigger a new release, or have each build trigger a new release on the dev machine. The really interesting aspect is, using GitVersion and GitFlow, when you select the build you want to release, you can choose the branch you want to deploy instead of understanding what is in a build from its number.

Thanks to gitversion when you trigger a new release you can choose the build that you want to deploy, and you can choose using a semantic versioning like 1.5.0 or 1.5.0-beta1 instead that a generic build number

Figure 5: Trigger a release manually allows to specify the build you want to release

Ad you can see from Figure 5, you can easily choose the version of the software you want to release, it is really clear from the name of the build. Once the Release is started, if the deploy is set to occur automatically, the release immediately starts

This image simply shows that the release is in progress

Figure 6: Relase is in progress

Once the release is in progress, from the summary you can verify what version of the software is released in what environment. I have only one environment for this demo, and I can verify that 4 minutes ago the version 1.5.0 is deployed in that environment.

This image shows all releases done with this definition. New release of version 1.5.0 was completed 4 minutes ago

Figure 7: New release was deployed successful

I can now verify that the new version was really deployed correctly opening the software in target machine.

Opening the software the new version confirm that the deploy was really successful

Fgiure 8: New version of the software is correctly deployed

Once you have everything in place and use simple PowerShell approach, setting up a release management process is really straightforward.

Gian Maria.

Checklists are prerequisites for Release Automation

I’ve dealt in some posts on how to deploy an application with a PowerShell script that uses an archive produced by a build. Automating a release could be simple or complex, depending on the nature of the software to be deployed, but there is a single suggestion that I always keep in my mind:

If you don’t have one or more Checklists for manual installation of a software do not even try to autmate installation process

Automatic Release and Continuous Deployment is a real key part of DevOps culture, but before starting to write automation scripts, it is necessary to ask yourself: Do I really know every single step to be done in order to release the software?

This seems a simple question, but in my experience people always tends to forget installation steps: obscure settings in windows registry or in some files, operating system settings, prerequisites, service pack and so on.

Since the devil is in the details, deploying a software manually without a cheklist is almost impossible, because the knowlege on “how to deploy the software” is scattered among team members.

Scripts are stupid, there is no human intelligence behind a script, it simply does what you told it to do, nothing more.

Before even try to write a script, you should write one or more CheckLists for the installation: PreRequisite Checklist, Installation Checklist, Post Installation Checklist and so on. Each checklist should contain a series of tests and tasks to be accomplished in order to release the software. If you are not able to take a person, give her/him installation checklists and have her/him install the software successfully, how you can think to write a script that automate the process?

The correct process to create automated script is:

1) Generate PreRequisite, Installation, PostInstallation CheckList.
2) Run them manually until they are complete and correct, write down the time needed for each step
3) Start automating the most time consuming steps.
4) Proceed until every step is automated.

Usually it is not necessary to immediately try to automate everything, as an example, if you use virtual machine, you can use golden images to use machines with all prerequisites already installed. This simply the deployment process because you can avoid writing PowerShell DSC, Puppet or Chef scripts to install Prerequisites.

If a specific installation step is difficult to automate (because there is some human intelligence behind it) let this task to be executed manually, but automate other expensive part of the checklist. There is always value in automating everything that can be easily automated.

Try to use Virtual Machine templates and sysprepped VM to simplify setup of prerequisites. Start creating automation for operations that are repeated for each upgrade, not those ones that are done only once.

Step by step you will eventually reach the point where you automated everything, and you can start to do Continous Deployment. If you followed CheckList and rigorous process, your deployment environment will be stable. If you immediately jump in writing automation scripts, even before knowing all the operation required to install the software (after all it is a simple web site), you will end with a fragile Automatic Deployment Environment, that will eventually require more maintenance time than a manual install.

Gian Maria.

Connect your TFS Release Management to Azure subscription with Update 3

In the Update 3, now in RC, you have the ability to configure your Release Management to directly access your Azure Subscription to have a list of all of your environment. The operation is really simple, you need to go to Administration tab and then choose to Manage Azure.

image

Figure 1: Adding your subscription to Release Management

Adding a new Subscription is just a matter to enter some information taken from a valid publish certificate. If you have not available a valid certificate, the easiest way to obtain a new one is going to this url https://windows.azure.com/download/publishprofile.aspx. Once authenticated a new management certificate will be downloaded. Please do not abuse this functionality, or you will end with a lot of certificates.

Once you have downloaded a valid certificate, it contains everything you need to connect your azure subscription to Release Management. You need to copy your subscription Id and your management certificate.

image

Figure 2: Copy data from your management certificate

If you installed the CTP preview of Update3 you can notice that now you have another field to fill, called Storage Account Name. You need to copy a name of one valid account storage from your azure management account.

image

Figure 3: Copy all the info from your certificate and account to Release Management.

Now you need to wait some minutes, because the sync is done by a service that runs in the background that periodically check for availability of environments. If no machine shows up after few minutes, please check the Event Viewer of the machine where Release Management Server is running and try to restart the service called Release Management Monitor.

Once the service is able to synchronize with your account you should see all of your Azure VMs now available as servers inside release management.

image

Figure 4: All of your Azure Virtual Machines are now available in the Servers list.

If you installed the Deployer Agent to some of the VM those machines are listed with a Deployer Status of Ready, all of the other machines have a Deployer Status of Offline, because no Deployer Agent is present on those machines. Machines that have no Deployer Agent deployed can be used as target of deployment using powershell DSC.

Actually Release Management load information for every VM in your account, even Linux machines. This is not weird because if you missed it, actually PowerShell DSC is being ported to Linux as you can read from this article: Announcing Windows PowerShell Desired State Configuration for Linux.

Gian Maria.

Install and configure a TFS Release Manager Deployer Agent in Azure VM

The Problem

 

You have a domain with TFS and -release management, there are no problems deploying agents on machines inside the domain, but you are not able to configure an agent for machines outside the domain.

Es: you have some Azure VMs you want to use for your release pipeline and you do not want to join them to the domain with VPN or other mechanism.

This scenario usually ends in being not able to configure Deployment Agents in those machines due to various authorization problems. The exact symptom range from getting 401 errors when you try to configure Agent on the VM. Another symptom is being able to configure the Deployment Agent, but whenever the service starts you do not see any heartbeat on the server and in the Event viewer of VM you got error like these ones

Timestamp: 6/10/2014 6:17:12 AM
Message: Error loading profile for current user: nabla-dep1\inreleasedeployer
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: NABLA-DEP1

The usual cause

 

The most common cause of the problem is a bad configuration of Shadow Accounts and authentication problem between the machine outside the domain and the machines inside the domain. I want to share with you the sequence of operation that finally made my Azure VM runs a Deployer Agent and connect to Release Management Server.

Here is my scenario

My domain is called CYBERPUNK and the user used to run deployment agents is called InReleaseDeployer.

The machine where TFS and RM Server is installed is called TFS2013PREVIEWO and Azure VM is called NABLA-DEP1.

I’ve already the user CYBERPUNK\InReleaseDeployer added as a Service user to Release Manager, and I already used to deploy an agent for a machine in the domain with no problem.

Now head for configuring everything for an Azure VM.

The solution

 

This is the solution:

You need THREE ACCOUNTS:

  1. CYBERPUNK\InReleaseDeployer: You already have this because is the standard domain account used for Domain Agent in machine joined to your domain.
  2. TFS2013PREVIEWO\InReleaseDeployer: this is the shadow account on RM serve machine. It is a local user that should be created in the machine running Release Management server.
  3. NABLA-DEP1\InReleaseDeployer: this is the shadow account on the Azure VM Machine, is a local user on the Azure VM Machine

Now be sure that these account satisfy these conditions:

  1. All three accounts must have same password
  2. NABLA-DEP1\InReleaseDeployer must be administrator of NABLA-DEP1 machine, it must also have the right o logon as a service.
  3. All three account should be added to Release Management Users with these permissions.

Domain user should have the standard Service User flagged

imageù

Then the shadow account in the RM machine should be also Release Manager and not only Service User, here is the setting.

image

Please note that the user is expressed in the form MACHINENAME\username, so it is TFS2013PREVIEWO\InReleaseDeployer, and it is a Release Manager and a Service User.

Finally you need to add the user of the Azure VM.

image

Even for this user you need to express it in the form MACHINENAME\UserName, so it is NABLA-DEP1\InReleaseDeployer. This completes the setup of the shadow account for your RM Server.

Now it is the turn of the Azure VM so connect in Remote Desktop on Azure VM, and login with credentials NABLA-DEP1\InReleaseDeployer user, do not use other users and finally configure the agent. Before configuring the agent, open credential manager and specify credentials to connect to the public address of RM Server. You need to add a Windows Credential specifying the Credentials that should be used upon connection to the Remote Release Management Server. Be sure to prefix the user with domain specification, as in the following picture (CYBERPUNK\InReleaseDeployer).

image

You should now see the credentials you just entered.

image

Actually adding Credentials in Windows Credentials is required only if you want to use RM Client to connect to the server, but I noticed that my user had problem connecting to the server if I miss this part, so I strongly suggest you to add RM Server to Windows Credentials to avoid problem.

Now the last step is configuring the agent. You must specify Nabla-dep1\InReleaseDeployer as the user used to run the service and the public address of your Release Management Server.

image

Press Apply settings and the configuration should complete with no error.

image

Once the Deployer Agent is configured you should be able to find the new agent from Release Management Client, in Configure Path –> Servers –> New –> Scan For New .

image

Everything is ok, my RM Server is able to see the deployer on the VM even if the VM is outside the network and not joined to corporate domain. Now you can select the new agent and Press Register to register with the list of valid Deployer Agents.

image

Gian Maria.