Publishing web project on disk during build

During a build you can ask MsBuild to deploy on build using the switch /p:DeployOnBuild=true as I described in previous posts. This is mainly used to deploy the site on IIS thanks to WebDeploy, but you can also use WebDeploy to deploy on a disk path. The problem is that the path is stored in publication settings file, but what about changing that path during a Build?

The answer is simple, you can use the /p:publishUrl=xxx to override what is specified inside the publication file and choose a different directory for deploy. Es.

msbuild WebApplication1.csproj /p:Deploy
OnBuild=true /p:PublishProfile=Profile1 /p:publishUrl=c:\temp\waptest

Thanks to this simple trick you can instruct MsBuild to store deployed site in any folder of the build server.

Gian Maria.

error MSB3147: Could not find required file ‘setup.bin’ in

I got this error from a powershell script when I call MsBuild.exe to publish a ClickOnce project in a TFS Build, the exact error is.

error MSB3147: Could not find required file ‘setup.bin’ in …

This is not the first time I encounter this error and it is usually caused by missing .NET SDK. The strange fact is that I have both VS 2013 and Windows SDK already installed in the machine hosting the build agent. Then I manualy added the registry key where MsBuild is looking for the bootstrapper. In the original registry, the GenericBootstrapper has only the 12.0 entry, and I added 4.0 and 4.5 for other framework version. But this is really strange because I never had to do this manually in the past.

clip_image002

Figure 1: Adding path of the Bootstrapper directly in the registry

BTW, with these registry keys the problem went away, but only to stop with another really curious error.

error MSB3482: An error occurred while signing: SignTool.exe not found

I’m started thinking that something strange is going on and the best way to troubleshoot the error was connecting in Remote Desktop to the Agent machine and launch msbuild.exe manually to start investigating the error. This time, using the standard developer command prompt, everything runs fine. The only cause could be: “the script is using wrong version of MsBuild”. The script I was using was taken from another test project of many months ago and used MsBuild from the location:

C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.EXE

Clearly, for solution built with VS 2013, the best option is using the version installed by Visual Studio located in

C:\Program Files (x86)\MsBuild\12.0\Bin\msbuild.EXE

Changing the version of MsBuild.exe solved the problem.

Gian Maria.

Take control of assembly numbering during a tfs build

One of the most important stuff in a project build, is the ability to mark the assemblies with unique numbers that permits us to reproduce the build. Tfs does not have a standard way of doing this, but with a couple of MsBuild actions it is really simple to overcome this limitation. This is a good example that shows how you can extend build script to do complex task.

First of all I want to change only AssemblyFileVersion and not the AssemblyVersion, in this way all builds are compatible until someone manually changes AssemblyVersion. A standard technique I like very much is letting the programmers to manage major and minor number manually, and letting my builds generates build and revision ones. For build number I want to be able to generate a unique number each build, a sequential generator will be fine; but for revision number I want to use the changeset used to generate the build. To accomplish this we need essentially four macro steps.

In the first step I need to generate unique integer build number, most of the time sequential generator is ok, then I need also to find a way to correlate this generated number with the build label of the TFS. Step two is used to find latest changeset, then in step three we need to check-in modified files (the one used by the generator), being sure that this check-in does not trigger another build, finally we need to modify a file named ProjectVersion.cs that is used by all projects.

To modify AssemblyFileVersion for a project I love this technique: I remove AssemblyFileVersion and AssemblyVersion attributes from assemblyinfo.cs, put them in a single ProjectVersion.cs file stored in the root of the team project. Here is a typical content for the ProjectVersion.cs .

using System.Reflection;
[assembly: AssemblyVersion("1.2.0.0")]
[assembly: AssemblyFileVersion("1.2.0.0")]

Next I import this file as link in every project that belong to this team project

image

With this little trick to change AssemblyFileVersion attribute for every project I need only to change one file. Now it is time to build some custom tasks that will help us to manage the whole process, first of all the task that generates unique numbers and correlate them with build label.

public class BuildVersionNumberManagerTask : Task
{
    /// <summary>
    /// Filename that will store the versions
    /// </summary>
    [Required]
    public String VersionFileName { get; set; }

    /// <summary>
    /// label of the current build.
    /// </summary>
    [Required]
    public String BuildLabel { get; set; }

    /// <summary>
    /// new incremental version numer.
    /// </summary>
    [Output]
    public Int32 NewVersionNumber { get; set; }

    public override bool Execute()
    {
        if (!File.Exists(VersionFileName))
        {
            Log.LogError("The file {0}[{1}] with version number does not exists", VersionFileName, Path.GetFullPath(VersionFileName));
            return false;
        }
        String[] allLines = File.ReadAllLines(VersionFileName);
        String startChar = "";
        if (allLines.Length > 0 && !String.IsNullOrEmpty(allLines[allLines.Length - 1]))
        {
            startChar = "\n";
            String lastline = allLines[allLines.Length - 1];
            String lastNum = lastline.Substring(0, lastline.IndexOf("|"));
            Int32 lastNumber;
            if (!Int32.TryParse(lastNum, out lastNumber))
            {
                Log.LogError("There are errors in the version file, the last line does not contains lastnum|lastlabel valid format");
                return false;
            }
            NewVersionNumber = lastNumber + 1;
        } else
        {
            //There is no lines, or the last line is empty.
            NewVersionNumber = 1;
        }
        String newLastLine = String.Format("{0}{1}|{2}",startChar,  NewVersionNumber, BuildLabel);
        File.AppendAllText(VersionFileName, newLastLine);
        return true;
    }
}

The BuildVersionNumberManagerTask is responsible of the generation of a sequential number, as well as storing in a file the relationship between autogenerated numbers and build labels. It use a simple text file and write a line for each generated number. Here is an example of file content.

...
3|Standard build for CI_20090820.20
4|Standard build for CI_20090820.21
5|Standard build for CI_20090820.24
...

With such a technique, we can immediately find the build label associated with each auto generated number. Now if you have problem with an assembly that have 4 as the build number, I immediately find in the file that it was build by “Standard build for CI_20090820.21” Here is how I call this task into TFSBuild.proj file

<PropertyGroup>
        <tfTool>"C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\tf.exe"</tfTool>
        ...
<Target Name="BeforeCompile" Condition=" '$(IsDesktopBuild)' != 'true' ">
        <Message Text="Beginning generation of new AssemblyFileVersionAttribute" />
        <Exec Command="$(tfTool) checkout ..\sources\src\StandardBuildForCIVersion.txt"  />
        <BuildVersionNumberManagerTask
            VersionFileName="..\sources\src\StandardBuildForCIVersion.txt"
            BuildLabel="$(BuildNumber)">
            <Output TaskParameter="NewVersionNumber" PropertyName="NewVersionNumber" />
        </BuildVersionNumberManagerTask>

I override the BeforeCompile task, since I need to change version number before the compile phase. As you can see I use the Exex command to call the tf.exe tool to checkout the file StandardBuildForCIVersion.txt, the one used to store autogenerated numbers. In this way I can use a different file for each build definition or I can use same file for different builds, I have great flexibility. The checkout is needed because I want to check in modified file at the end of the process, in this way the autogenerated number can be accessed from other build agents. Then I need another custom task capable to modify the ProjectVersion.cs file.

public class AssemblyInfoVersionManagerTask : Task
{
    [Required]
    public String AssemblyInfoFileName { get; set; }

    [Required]
    public Int32 NewBuildNumber { get; set; }

    [Required]
    public Int32 NewRevisionNumber { get; set; }


    public override bool Execute()
    {
        if (!File.Exists(AssemblyInfoFileName))
        {
            Log.LogError("The file {0} does not exists", AssemblyInfoFileName);
            return false;
        }
        String filecontent = File.ReadAllText(AssemblyInfoFileName);
        Match curVersionMatch =
            Regex.Match(filecontent, "AssemblyVersion\\(\"(?<curversion>.*?)\"\\)");
        if (!curVersionMatch.Success)
        {
            Log.LogError("The content of file {0} does not contains valid Assemblyversion attribute");
            return false;
        }
        String curversion = curVersionMatch.Groups["curversion"].Value;
        String[] versionParts = curversion.Split('.');
        String newVersion = String.Format("{0}.{1}.{2}.{3}", versionParts[0], versionParts[1], NewBuildNumber, NewRevisionNumber);
        String newFileContent = Regex.Replace(
            filecontent,
            "AssemblyFileVersion\\(\"(?<curversion>.*?)\"\\)",
            String.Format(@"AssemblyFileVersion(""{0}"")", newVersion));
        FileAttributes currentFileAttributes = File.GetAttributes(AssemblyInfoFileName);
        File.SetAttributes(AssemblyInfoFileName, FileAttributes.Normal);
        File.WriteAllText(AssemblyInfoFileName, newFileContent);
        File.SetAttributes(AssemblyInfoFileName, currentFileAttributes);
        return true;
    }
}

This is another simple task, it uses a little bit of regular expression to find actual version number stored in the ProjectVersion.cs, then it creates another version number with simple composition. Major and minor number are taken from the original content, while build and revision are passed by the caller. Remember also to remove the Readonly attribute from the file, because it is usually readonly since it is under version control. Now I need the last piece, a simple TfTask that permits me to grab the changeset number associated with the current project.

public class TfTask : ToolTask
{
    public TfTask()
    {
        ToolPath = @"C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\";
    }

    [Required]
    public String Operation { get; set; }

    [Output]
    public String TfOutput { get; set; }

    public String TfsUrl { get; set; }

    public String TeamProject { get; set; }

    protected override int ExecuteTool(string pathToTool, string responseFileCommands, string commandLineCommands)
    {
        TfTaskOperation operation = (TfTaskOperation)Enum.Parse(typeof(TfTaskOperation), Operation);
        String commandline;
        switch (operation)
        {
            case TfTaskOperation.GetLatestChangeset:
                commandline = String.Format(
                    "history /s:{0} /stopafter:1 /noprompt /recursive /version:T $/{1}", TfsUrl, TeamProject);
                break;
            default:
                throw new NotSupportedException();
        }
        using (System.Diagnostics.Process process = new System.Diagnostics.Process())
        {
            process.StartInfo.FileName = Path.GetFullPath(pathToTool);
            process.StartInfo.Arguments = commandline;
            process.StartInfo.WorkingDirectory = Path.GetDirectoryName(pathToTool);
            process.StartInfo.WindowStyle = ProcessWindowStyle.Normal;
            process.StartInfo.UseShellExecute = false;
            //process.StartInfo.ErrorDialog = false;
            //process.StartInfo.CreateNoWindow = true;
            process.StartInfo.RedirectStandardOutput = true;
            process.Start();
            process.WaitForExit();
            TfOutput = process.StandardOutput.ReadToEnd();
        }
        return 0;
    }

    protected override string GenerateFullPathToTool()
    {
        throw new NotImplementedException();
    }

    protected override string ToolName
    {
        get { return "tf.exe"; }
    }
}

public enum TfTaskOperation
{
    GetLatestChangeset = 0,
}

This is a simple wrapper to the tf.exe tool, and since I invoke it with the System.Diagnostic.Process class, I’m able to intercept the output. There are a lot of possibilities on how to get latest changeset, but the simplest is to invoke tf.exe with a command line like this: tf.exe history /s:http://tfsalkampfer:8080 /stopafter:1 /noprompt /recursive /version:T $/MsBuildExtension

This This command gives a result like this one.

Changeset User          Date       Comment
--------- ------------- ---------- ---------------------------------
129       TfsService    8/21/2009  ***NO_CI***

That can be parsed with a simple regular expression, here is the remaining of the build script.

<TfTask 
    TfsUrl="$(TeamFoundationServerUrl)" 
    TeamProject="$(TeamProject)" 
    Operation="GetLatestChangeset">
    <Output    TaskParameter="TfOutput" PropertyName="historyOutput"/>
</TfTask>

<Message Text="historyOutput is $(historyOutput)" />

<RegexTask TextToMatch="$(historyOutput)" Regex="\n(?&lt;changeset&gt;\d+)" FailIfNoMatch="true" GroupName="changeset">
    <Output
         TaskParameter="Result"
         PropertyName="lastChangeset"/>
</RegexTask>

<AssemblyInfoVersionManagerTask
    AssemblyInfoFileName="..\sources\src\ProjectVersion.cs"
    NewBuildNumber="$(NewVersionNumber)"
    NewRevisionNumber="$(lastChangeset)">
</AssemblyInfoVersionManagerTask>

<Exec Command='$(tfTool) checkin /comment:"***NO_CI***" ..\sources\src\StandardBuildForCIVersion.txt' />

IT is quite simple, I first invoke the TfTask, then I use a simple RegexTask to find the changeset number from tf output. Thanks to TfTask custom task, the script remains simple, I need only to specify tfsurl, team project name and the operation I want to execute (in this situation GetLatestChangeset), and the task will store in the historyOutput property the full output of the tf.exe command.

Finally I use my AssemblyInfoVersionManager task to change the ProjectVersion.cs file and finally I do a check-in of the file that contains the new generated sequential build number with comment ***NO_CI*** to avoid going in loop with continuos integration engine.

After a build you can verify that everything is gone ok.

image

Now if you deploy this assembly into a customer computer, if you have a problem in the future, you can immediately verify that this was compiled with the build called 8, then you check the StandardBuildForCIVersion.txt

7|Standard build for CI_20090820.37
8|Standard build for CI_20090820.39
9|Standard build for CI_20090821.2

You can immediately find the label of the build, but the most important thing in my opinion is the revision number, 128 in the picture above, because it is the changeset that generates this assembly, to replicate and debug any problem, you can simply do a Get Specific Version of that changeset, and you can work with the exact source code and build tools that generates that assembly.

This example shows how simple can be extending build process with the creation of some ad-hoc tasks and a good use of the tf.exe command line tool.

A zip of all the repository can be downloaded here.

Alk.

Tags:

Writing extension for Msbuild

Msbuild is microsoft build engine, and I showed some time ago how you can write a custom task to post in twitter the outcome of a build result. Now it is time to give a greater focus on how to write a good task.

Creating a Task is a simple matter of inheriting from the Task class but there are some key points you should keep in mind, first of all you should never throw an exception when executing the task, except for some specific system exception like  OutOfMemoryException or StackOverflowException, then you should provide good logs to make simple for the user of your task to understand causes of failures.

Some of the tasks I miss most from transition to Nant to Msbuild is the couple XmlPeek and XmlPoke, used to read and manipulate xml files with xpath. Replicate them with LINQ to XML is a breeze. To actually build the task I first included all the logic in a different class.

image

With such an arrangement I can easily test my XML logic

[TestMethod]
public void TestXmlPeek()
{
    XElement element =  XElement.Parse("<root>value</root>");
    String value = XmlFunctions.Peek( element,  "/");
    Assert.AreEqual("<root>value</root>", value);
}

 

After you have carefully tested your logic function it is time to test logic of the task, here is the full code of the task.

public class XmlPeek : Task
{
    [Required]
    public String XPath { get; set; }

    [Required]
    public String FilePath { get; set; }

    [Required]
    public Boolean FailIfError { get; set; }

    [Output]
    public String Value { get; set; }

    public override bool Execute()
    {
        try
        {
            if (!File.Exists(FilePath))
            {
                Log.LogError("The file {0} cannot be found", FilePath);
                return !FailIfError;
            }
            //can load the file
            XElement element = XElement.Load(FilePath);
            String foundValue = XmlFunctions.Peek(element, XPath);
            if (foundValue == null)
            {
                Log.LogError("No node found with XPath={0}.", XPath);
                return !FailIfError;
            }
            Log.LogMessage("Found node with XPath={0}", XPath);
            Value = foundValue;
            return true;
        }
        catch (OutOfMemoryException ex)
        {
            throw;
        }
        catch (StackOverflowException ex)
        {
            throw;
        }
        catch (Exception ex)
        {
            Log.LogErrorFromException(ex);
            return false;
        }
    }
}

I enclosed the main body of the action in a try catch, but I need to rethrow exceptions that really cannot be catched. Then for every other exception I simply log it and then return false to the caller. For the main path of the action you should carefully validate input parameters, and give detailed error with the Log property from Task base class. Since good logs are really important, you should test them carefully.

public void TestNotFailDefault()
{
	XmlPeek sut = new XmlPeek() {FilePath="notExists"};
	sut.BuildEngine = MockRepository.GenerateStub<IBuildEngine>();
	
	Assert.IsTrue(sut.Execute());
	sut.BuildEngine.AssertWasCalled(
		be => be.LogErrorEvent(null), 
		e => e.IgnoreArguments().Repeat.Once());
}

Thanks to Rhino Mock I create an implementation of IBuildEngine to pass to my action to simulate the msbuild execution environment, then I call my task with a file name that does not exists, and verify that the LogErrorEvent was called in the BuildEngine.

Alk.

Tags:

Comparison between nant and msbuild

You can find at this link, a table that compare nant tasks with msbuild tasks. If you look at this table it seems that msbuild lacks a lot of things, but actually there are specific visual studio tasks that have no equivalent in nant, like those one for tfs (open issue etc), publishing with clickonce, run a data generation plan and many more.

In the end msbuild is a great product, but probably it lacks a little on some basic tasks like XML manipulation with xmlpeek and xmlpoke.

Alk.