Build vNext, distributing load to different agents

One of the major benefit of the new build infrastructure of TFS and Visual Studio Online is the easy deployment of build agents. The downside of this approach is that your infrastructure become full of agents, and you should have some way to determine which agent(s) to use for a specific build. The problem is:

avoid running builds in machines that are “not appropriate” for that build.

Running on a specific agent

If you are customizing a build, or if you are interested in running the build on a specific agent in a specific machine (ex: local agent), the solution is super easy, Just edit build definition and in General tab add a demand named Agent.Name with the value of the name of the specified Agent.

image

Figure 1: Adding a demand for a specific agent

If the agent is not available, you are warned when you try to queue the build.

image

Figure 2: Warning on build queuing when there are agents problem

In this situation the system is warning me that there are agents compatible to the build, but they are offline. You can queue the build and it will be executed when the Agent will be online again, or you can press cancel and understand why the agent is not online.

If you want to run a build on a specific agent, just add Agent.Name demand on the build.

Specifying demands

The previous example is interesting because it is using Build Demands against properties of the agent. If you navigate on build agent admin page, for each agent you are able to see all associated properties.

image

Figure 3: Agent capabilities

On top there are User Capabilities that are editable. They are used to give custom capabilities to your agents. In bottom part there are System Capabilities, automatically determined by the agent itself and thus cannot be changed. If you examine all these capabilities you can see interesting capabilities such as VisualStudio that tells you if the agent is installed on a machine where Visual Studio is installed, other capabilities are also used to verify exact version of Visual Studio.

This is really important, because if you examine demands for a build you can verify that some of them are already placed in the build definition and they could not be removed.

image

Figure 4: some of the demands are read-only, you can add your demand

If you wonder why the build has some predefined, readonly demands, the answer is: they are taken from the build definition.

image

Figure 5: Build definition

If you look at the build definition, it contains a Visual Studio Build task, and it is this block that automatically adds the visualstudio demands on the build. This explain why that demand cannot be removed, it is required from a Build Step. You can try to remove Visual Studio Test task, and you can verify that the vstest demands is also gone.

Each Build Task automatically adds demands to be sure to run in compatible agents.

In my example I have some agents deployed on my office; they are used to run tests on some machines, and I use them to run build definitions composed only by build and test tasks, but no publish build artifacts. The reason is: I have really low upload bandwidth on my office, but I have fast machine on really fast SSD and tons of RAM so they are able to quickly build and test my projects.

Then I have some build used to Publish Artifacts, and I want to be sure that those build are not executed on agents in my office or they will saturate my upload Bandwidth. To avoid this problem I simply add uploader demands on these builds, and manually add this capability to agents deployed on machine that have no problem with Upload Bandwidth, Ex on agents deployed on Azure Virtual Machines.

You can use custom Demands to be sure that a build runs on agents with specific capabilities.

Using Agent pools to separate build environments

The final solution to subdivide works to your agent is using agent pools. A pool is similar to the old concept of controller, each build is associated to a default pool, and it can be scheduled only on agents bounded to that pool. Using different pools can be useful if you have really different building environments, and you want to have a strong separation between them.

A possible example is, if you have agents deployed on fast machines with fast SSD or RAMDisk to speedup build and testing, you can create a dedicated pool with a name such as FastPool. Once the pool is created you can schedule high priority build on that pool to be sure that the build will be completed in the least amount of time. You can further subdivide agents in that pool using capabilities, such as: SSD, RAMDISK, etc.

You can also create a pool called “Priority” to execute build with high priority and being sure that some slow build does not slow down high priority build and so on. If you have two different offices located very far away you can have a different pool for each office, to be sure that some builds are executed in local network for that office and so on.

With Agent Pools you can have a strong separation between your build Environment.

To conclude this post, you should use Agent Pools if you want to achieve strong separation and you are creating distinct Build Environment that shares some common strong characteristic (phisical location, machine speed, priority, etc). Inside a poll you can further subdivide work among agents using custom demands.

As final notice, as of today, with the latest update of VSO, the new build engine is not anymore in preview, the tab has no Asterisk anymore and the keyword PREVIEW is gone away :), so vNext in VSO reached GA.

image

 

Gian Maria.

TFS New Build System: vNext agents

With the latest Visual Studio Online update, the new build system is now online for all users. As I said in old post, it is completely rewritten and covering all new features really requires lots of time. Since I’m a great fan of Continuous Integration and Continuous Deploy procedures I’d like to do some post to introduce you this new build system, along with the reason why it is really superior to the old one.

In this post I’ll outline some of the improvements regarding build infrastructure management, starting from the most common problem that customers encountered with the older approach.

Each project collection needs a separate build machine

In the old build system you should install TFS bit and configure Agents and Controller to create a build machine, each machine can provide build capabilities only for a project collection.

With the new build system, Controller is part of TFS/VSO core installation, you should only deploy agents on build machines and an agent can provide build capabilities for an entire instance of TFS.

This means that an agent is connected to an entire TFS instance and it can build stuff from every Project Collection in your server. This simplify a lot your infrastructure because you can use a single build machine even if you have multiple Project Collections. If you are a small shop and you use multiple Project Collections to achieve full project separation, probably a single build machine is enough.

Deploying new agent is complex

With traditional build system you need to deploy Controllers and agents. Both of them requires a full TFS installation, followed by configuration of Build. Everything is administered trough standard TFS Administration Console.

With the new build system build controller is part of the core installation, you should only deploy agents in your machine.

Deploying an agent is a matter of downloading a file, unzip it and run a PowerShell script.

If you are interested in full installation details, you can follow step by step MSDN procedure here: https://msdn.microsoft.com/Library/vs/alm/Build/agents/windows. This aspect greatly simplify setup of a new agent.

I’ve multiple VSO accounts, I need multiple build machines

Since the agent is a simple Console Application you can install more agent simply unzipping agents file in different directories of the same machine. In the end you can have a single machine with agents connected to multiple VSO or TFS instances.

You can use a single build machine for multiple TFS2015 or VSO instances

Building is limited to windows machine

Since agents are installed with TFS Bits, the old build system only allow agents to run on a Windows Machine. Thanks to some Build Extensions you can run Ant and Maven build, but only with a Windows Box.

New build system provide agents also for non Windows machine thanks to the xPlat Agent. https://msdn.microsoft.com/Library/vs/alm/Build/agents/xplat 

You can run TFS/VSO build on non windows machine such as Linux and Macintosh.

Running local build is complex for developer

There are lots of reasons to run a build in local machines and with the old system you need to do a full TFS installation, leading to a maintenance nightmare. Usually you will end with a lot of stuff in your Continous Integration scripts, such as publishing nuget packages, and lots more. With such scenario, running a build in a local machine should be an easy task for developers.

New agent can run as a service, or run interactively with a simple double-click on the VsoAgent.exe file. Each developer can install agent in minutes and run the agents only when he/she really needs it.

Running local build is really easy because you can start local agent with a double click, only when you need to run a local build.

You can also attach a debugger to the build if you need to debug some custom code that can run during a build because it is a simple console application.

Keeping up with update is complex

If you are a large enterprise, you probably have several build machines and with the older version of TFS you should update all build agents and controller each time you updated TFS instance. With TFS 2012 Update 2 this requirement is relaxed, and older build system can target newer version of TFS. This will give you a timeframe to update all Build Machines after you updated TFS Core installation, but you really need to run TFS Update on all machines with build Agents or Controllers installed.

You will be happy to know that the new build agent has automatic update capabilities. It actually checks for new version at each startup of the agent, if a new version of the agent is available for the server you are connected to, it will automatically download, upgrade and restart the new version.

The only drawback is that, if you are running the agent as a windows service, you should restart the service for the upgrade check to take place. In future version we expect the agent to periodically check for an upgrade instead of requiring restart.

The overall feeling is that the new Build Infrastructure will be really easier to maintain and deploy, also it will give you new capabilities, like building on non-windows machine.

Gian Maria.

VSO vNext build error: You cannot run the vsoAgent.exe interactively ..

Error Symptom: You installed and configured an Agent for the new Visual Studio Online Build System, and you decided not to run as a service, but interactively. When you double click the VsoAgent.exe executable you got this error.

You cannot run the vsoAgent.exe interactively from within the Agent folder. Try running it from the parent folder

I’ve encountered this error on my main workstation machine, and I’m not sure why I got this error. Actually I’ve configured agent for TFS 2015 RC without this problem and in the same machine I got three distinct agents, but only the one targeting my VSO primary account is having this problem.

If you open a command prompt and run the program from the parent folder everything runs just fine.

image

Figure 1: Running from the parent folder solves the problem

Solution: the simplest solution to avoid opening a CLI is creating a shortcut to the executable and change the Start In location

image

Figure 2: Configure the agent with Start In main folder

This should solve the problem. Double clicking on the link now run the agent just fine.

Gian Maria.

Tfs2015 Build agent error: Access denied: xxxxx\yyyyy needs Listen permissions for pool zzzzz to perform the action

Tfs 2015 introduces a completely new and redesigned build system and one of the most important change is new lightweight agent system. Instead of installing TFS and then configure Build, to create a new agent you only need to download a zip file, uncompress and launch a PowerShell script. Another great advantage is the ability to run the agent as a service, or running it interactively in a simple console application.

If you configure a new agent you can check that everything is ok in TFS Control panel, in the new Agent pools tab. The new agent should be listed and it is Red if not active, Green if up and running.

image

Figure 1: Management of Pool and Agent in TFS Configuration

If the agent is red even if you launched the agent, you should check logs in the _diag folder.

image

Figure 2: Logs are placed in _diag folder

You should be able to understand and fix errors looking at the log. If you run the agent interactively, it could be that your user has no right permission to listen to the pool.

17:28:46.531831 Microsoft.TeamFoundation.DistributedTask.WebApi.AccessDeniedException: Access denied. CYBERPUNK\Administrator needs Listen permissions for pool Fast to perform the action. For more information, contact the Team Foundation Server administrator.

In this situation the user Administrator is in the TFS Administrator Group and it should have any permission, but new Build System is slightly different. The user that runs the agent, must be part of the Agent Pool Service Account, or it will not be able to run the agent

image

Figure 3: Permissions for Agent Pools

Simply adding the user to the AgentPoolService account should fix authorization problem

image

Figure 4: Agent is up and running.

Gian Maria