Fix of ChangeConnectionString resource in DSC Script to deploy Web Site

In the second part of this series I’ve received a really good comment by Rob Cannon, that warn me about an error in my ChangeConnectionString resource. In that article I told you that is ok for the Test part to return always False, so the Set Script is always run, because it is idempotent. This is true if you are using the Push Model, but if you are using the Pull Model instead, every 30 minutes the DSC will be applied and web config will be changed, so your application pool will be restarted. This is not a good situation, so I decided to change the script fixing the Test Part.

    Script ChangeConnectionString 
    {
        SetScript =
        {    
            $path = "C:\inetpub\dev\tailspintoys\Web.Config"
            1$xml = Get-Content $path 

            $node = $xml.SelectSingleNode("//connectionStrings/add[@name='TailspinConnectionString']")
            $node.Attributes["connectionString"].Value = "Data Source=localhost;Initial Catalog=TailspinToys;User=sa;pwd=123abcABC;Max Pool Size=1000"
            $xml.Save($path)
        }
        TestScript = 
        {
            $path = "C:\inetpub\dev\tailspintoys\Web.Config"
            1$xml = Get-Content $path 

            $node = $xml.SelectSingleNode("//connectionStrings/add[@name='TailspinConnectionString']")
            $cn = $node.Attributes["connectionString"].Value
            $stateMatched = $cn -eq "Data Source=localhost;Initial Catalog=TailspinToys;User=sa;pwd=123abcABC;Max Pool Size=1000"
            return $stateMatched
        }
        GetScript = 
        {
            return @{
                GetScript = $GetScript
                SetScript = $SetScript
                TestScript = $TestScript
                Result = false
            }
        } 
    }

The test part is really simple, it loads the xml file, verify if the connection string has the correct value and return true if the state was matched, false if the state was not matched. Running this new version of the script always runs the Set Part of ChangeConnectionString as before, nothing was changed. At a first time I though of a bug in the Test part, but after a moment I realized that the File resource actually overwrites the web config with the original one whenever the script runs because it was changed. This is how DSC is supposed to work, the file resource forces a Destination Directory to be equals to a source directory.This confirms me that the technique to download a base web.config with Node resource, and change it with a Script resource is suitable only for test server and if you use Push configuration. Actually to use Pull configuration the right web.config should be uploaded in the original location, so you do not need to change it after it was copied with the File Resource.If you are interested in a quick fix, the solution could be using two distinct file resources, the first one copies all needed files from the original location to a temp directory, then the ChangeConnectionString operates on web.config file present in this temp directory, finally another File Resource copies files from the temp directory to the real IIS directory.

 File TailspinSourceFilesShareToLocal
    {
        Ensure = "Present"  # You can also set Ensure to "Absent"
        Type = "Directory“ # Default is “File”
        Recurse = $true
        SourcePath = $AllNodes.SourceDir + "_PublishedWebsites\Tailspin.Web" # This is a path that has web files
        DestinationPath = "C:\temp\dev\tailspintoys" # The path where we want to ensure the web files are present
    }

    
    #now change web config connection string
    Script ChangeConnectionString 
    {
        SetScript =
        {    
            $path = "C:\temp\dev\tailspintoys\Web.Config"
            1$xml = Get-Content $path 

            $node = $xml.SelectSingleNode("//connectionStrings/add[@name='TailspinConnectionString']")
            $node.Attributes["connectionString"].Value = "Data Source=localhost;Initial Catalog=TailspinToys;User=sa;pwd=123abcABC;Max Pool Size=1000"
            $xml.Save($path)
        }
        TestScript = 
        {
            $path = "C:\temp\dev\tailspintoys\Web.Config"
            1$xml = Get-Content $path 

            $node = $xml.SelectSingleNode("//connectionStrings/add[@name='TailspinConnectionString']")
            $cn = $node.Attributes["connectionString"].Value
            $stateMatched = $cn -eq "Data Source=localhost;Initial Catalog=TailspinToys;User=sa;pwd=xxx;Max Pool Size=1000"
            return $stateMatched
        }
        GetScript = 
        {
            return @{
                GetScript = $GetScript
                SetScript = $SetScript
                TestScript = $TestScript
                Result = false
            }
        } 
    }
    
    
    File TailspinSourceFilesLocalToInetpub
    {
        Ensure = "Present"  # You can also set Ensure to "Absent"
        Type = "Directory“ # Default is “File”
        Recurse = $true
        SourcePath = "C:\temp\dev\tailspintoys" # This is a path that has web files
        DestinationPath = "C:\inetpub\dev\tailspintoys" # The path where we want to ensure the web files are present
    }

Now the ChangeConnectionString resource runs always, as we saw before, because each time that the File Resource runs it updates all the file with content of the original files. Changing this web.config file at each run is not a problem, because it is a temporary directory so not Worker Process Recycle happens. The final File Resource now works correctly and copies the files only if they are modified.This is what happens during the first run.

image

Figure 1:  During the first run all three resources were run, the first one copies files from the share to local temp, the second one changes web.config located in temp folder and finally the third one copies all files from temp folder to the folder monitored by IIS.If you run the configuration again without changing anything in the target node you got this result.

image

Figure 2: During second run, the first two resources are run, but the third one that actually copies file to the folder where the site resides was skipped, avoiding recycling the worker process.

The important aspect in previous picture is the third arrow, that highlight how the set part of the resource that copies files from temp directory to the local folder where IIS points is skipped, so no worker process recycle will happen. Thanks to this  simple change, now the script can be used even in a Pull process without too many changes.

Gian Maria.

Deploying Web Site With PowerShell DSC part 3

In this last part of this series I’ll explain how to deploy database projects output to local database of node machine. It was the most difficult due to some errors present in the xDatabase resource. Actually I have a couple of Database Projects in my solution, the first one define the structure of the database needed by my application while the second one reference the first and installs only some test data with a Post Deploy Script. You can read about this technique in my previous post Manage Test Data in Visual Studio Database Project Sadly enough, the xDatabase resource of DSC is still very rough and grumpy.

I’ve found two distinct problems:

The first one is that DatabaseName is used as key property of the resource, this means that it is not possible to run two different DacPac on the same database because of duplicate key violation. This is usually a no-problem, because I could have deployed only the project with test data and since it reference the dacpac with the real structure of the site, both of them should deploy correctly. Unfortunately this does not happens, because you need to add some additional parameters deploy method, and xDatabase resource still not supports DacDeployOptions class. The fix was trivial, I changed the resource to use the name of the DacPac file as the key and everything just works.

The second problem is more critical and derives from usage of the DacService.Register method inside the script. After the first successful deploy, all the subsequent ones gave me errors. If you got errors during Start-DscConfiguration the output of the cmdlet, even in verbose mode, does not gives you details of real error that happened to target node where the configuration was run.  Usually what you get is a message telling: These errors are logged to the ETW channel called
Microsoft-Windows-DSC/Operational. Refer to this channel for more details

It is time to have a look to Event Viewer of the nodes where the failure occurred. Errors are located in Application And Service Logs / Microsoft / Windows / Desired State Configuration. Here is how I found the real error that xDatabase is raising on the target node.

image

Figure 1: Errors in event viewer of Target Node.

The error is in the update, DacServices.Deploy failed to update the database because it was registered as a Data Tier application and the Deploy command does not update its registration accordingly. This problem was easy to solve, because I need only to Specify RegisterDataTierApplication with a DacDeploymentOptions. I’ve added even this fix to the original xDatabase resource and I’ve added also more logging, so you are able to verify, when dsc runs, what DacServices class is really doing.

If you like I’ve posted my fix at this address: http://1drv.ms/1osn09U but remember that my fix are not thoroughly tested, and are not official Microsoft Correction in any way. So feel free to use them at your own risk. Clearly all these error will be fixed when the final version of xDatabase will be released (I remember you that these resources are pre-release, and this is the reason why they are prefixed with an x).

Now that xDatabase Resource works good, I can define a couple of resources to deploy my two dacpacs to target database.

xDatabase DeployDac 
{ 
    Ensure = "Present" 
    SqlServer = "." 
    SqlServerVersion = "2012" 
    DatabaseName = "TailspinToys" 
    Credentials = (New-Object System.Management.Automation.PSCredential("sa", (ConvertTo-SecureString "xxxxx" -AsPlainText -Force)))
    DacPacPath =  $AllNodes.SourceDir + "Tailspin.Schema.DacPac" 
    DacPacApplicationName = "Tailspin"
} 
    

xDatabase DeployDacTestData
{ 
    Ensure = "Present" 
    SqlServer = "." 
    SqlServerVersion = "2012" 
    DatabaseName = "TailspinToys" 
    Credentials = (New-Object System.Management.Automation.PSCredential("sa", (ConvertTo-SecureString "xxxxx" -AsPlainText -Force)))
    DacPacPath =  $AllNodes.SourceDir + "Tailspin.SchemaAndTestData.DacPac" 
    DacPacApplicationName = "Tailspin"
} 

Shame on me, I’m using explicit UserName and password again in DSC scripts, but actually if I omit Credentials to use integrated security, the xDatabase script fails with a NullReferenceException. Since this is a test server I accept to use clear text password until the xDatabase resource will not be fixed to support integrated authentication.

Here is the link to the full DSC script: http://1drv.ms/1osoIYZ. Have fun with DSC.

Gian Maria.

How to Deploy Web Site with PowerShell DSC

I do not want to create another tutorial on DSC and I suggest you reading some introductory articles like: Introducing PowerShell Desired State Configuration before reading this article. Since I’m pretty new with PowerShell and I’m starting experimenting with DSC I decided to start creating a script to deploy my favorite test application (TailspinToys :) ) on a single Windows 2012 R2 server using only DSC. This post aims to share my thought on the subject.

I was able to complete the script, even if I encountered some difficulties and I manage to automate almost everything, except the installation of Sql Server 2012 (I’m working on it). The goal is being able to deploy an application that uses a SQL server database written in Asp.Net 4.5 to a Windows Server with a fresh install, using only DSC Goodness.

First of all I warn you that most of the resources I needed to deploy my site are not available in basic distribution of PoweShell and should be downloaded from Microsoft. To download all the resources in a single package there is a single page in MSDN to download the entire DSC Resource Kit.

After you downloaded the resource kit you should care about a couple of important points, the first one is that these resources are not production ready and they are all experimental. This is the reason why all these resources starts with an x. So do not expect any official program to support them, if you have problem you should ask people in the forum and you will found solution. The other aspect is: if you, like me, appreciate the push model, you need to install all of these modules to all target servers. This violates in a certain way my requirement of being able to install in a clean server, because the server is not really “clean” if you need to have DSC resources deployed on it. This problem will be mitigated with WMF 5.0 that introduces the concept of PowerShellGet to automatically discover, install and update Powershell Modules, so it is really a no-problem.

Once everything is in place, I started creating the script, the first part is the standard one you can find in every PowerShell DSC related article, plus some import instructions to import all the DSC resources I want to use in the package.

Configuration TailspinToys
{
   
  Import-DscResource -Module xWebAdministration
  Import-DscResource -Module xNetworking
  Import-DscResource -Module xSqlPs
  Import-DscResource -Module xDatabase
  #http://www.vexasoft.com/blogs/powershell/9561687-powershell-4-desired-state-configuration-enforce-ntfs-permissions
  Import-DscResource -Module NTFSPermission

  Node $AllNodes.NodeName 
  { 
    

    #Install the IIS Role 
    WindowsFeature IIS 
    { 
      Ensure = “Present” 
      Name = “Web-Server” 
    } 

    # Required for SQL Server 
    WindowsFeature installdotNet35 
    {             
        Ensure = "Present" 
        Name = "Net-Framework-Core" 
        Source = "\\neuromancer\Share\Sources_sxs\?Win2012R2" 
    } 

    #Install ASP.NET 4.5 
    WindowsFeature ASP 
    { 
      Ensure = “Present” 
      Name = “Web-Asp-Net45” 
    } 

In the beginning of the script the Import-DscResource allow me to import the various resources I’ve installed, and NTFS Permission resource is taken from an article on VexaSoft site; many thanks to the author for authoring this module. That article is really useful because it shows how easy is create a resource for DSC in the situation where there is nothing already pre-made to obtain your purpose.

I use a configuration resource and the special name $AllNodes will contain the name of the single server I want to use for the installation. The above part of the scripts takes care of all of the prerequisites of my TailspinToys application. I’m installing .NET 3.5 because it is needed for Sql Server installation, but sadly enough I was not able to make the xSqlServerInstall works, to automatically install Sql Server (Actually it asks me to reboot and even rebooting the DSC scripts stops to run). I’ve decided to install Sql Server manually and wait for a better and more stable version of xSqlServerInstall. Then I request IIS and asp.net 4.5.

Running the above script with the right configuration data produces a mof file that can be used to actually configure the target. Here is the configuration I’,m using.

$ConfigurationData = @{
    AllNodes = @(
        @{
            NodeName="WebTest2"
            SourceDir = "\\neuromancer\Drops\TailspinToys_CD_WebTest1\TailspinToys_CD_WebTest1_20140213.1\"
            PSDscAllowPlainTextPassword=$true
            RebootNodeIfNeeded = $true
         }
   )
}

I need the name of the server and the source directory where I stored the distribution of my WebSite. In this example I’m using a standard Drop Folder of a TFS Build, so I have my binaries indexed with my symbol server. The creation of the mof file is simply triggered calling the new defined function TailspinToys passing the configuration above..

TailspinToys -ConfigurationData $ConfigurationData 

Now I have a mof file that contains everything I need to create the deploy, and I can push configuration to desired nodes with:

Start-DscConfiguration -Path .\TailspinToys -Wait -Verbose

This will start configuration, connect to all the nodes (in this example the single machine WebTest2) and “make it so”, moving the state of the nodes to desired state. The cool part of DSC is that you specify the state you desire on the target nodes, without taking care on how this state will be achieved, this is done by the various resources. Another interesting aspect is, if a resource is already in desired state, the Start-DscConfiguration will do nothing. When you run the above script the first time it needs a little bit time, because it will install IIS, but if IIS is already installed in target node, nothing happens.

With few lines of PowerShell I was able to install IIS and Asp.NET 4.5 plus .NET 3.5 to my machines.

In the next article I’ll deal on how to deploy the website bits.

Gian Maria.

The advantage of Scrum in evaluating “new stuff”

One of the greatest problem in Software industry is the rapid change in Technologies and Tools, a change so rapid that your knowledge become stale or obsolete before you have the time to fully master your new toys. In this scenario the major problem is how to evaluate some new “stuff”, while not losing money and, most important, how to understand if the new “stuff” is really better than the old one so it worth continue spending time on it and not moving on other newer “stuff”.

In traditional long term planning approach, like waterfall, this type of evaluation is really hard to obtain. You have several options, but no one is really good. Suppose you have a six month project, and you heard of some new “stuff” that seems promising, how you should handle this new “stuff”?

Waterfall or traditional approach

imageGain a complete knowledge of the “stuff” before starting the project: With this approach you assign some tasks to resources for learning the new “stuff”. While people are doing Requirement Analysis, some resources starts learning and when it is time to outline the architecture of the software, they should decide if the new “stuff” should be used in the project. The major drawback of this approach, is that you had no real experience of how this new Stuff will behave in your specific project with your real requirements, the decision will be made with incomplete information and the risk is discovering in the middle of the project, that the new “Stuff” actually is worst than the previous one.

Learn as you go, planning for a little delay in the project due to the introduction of the new “stuff”: with this approach you actually believe that the new “stuff” is useful, probably you got recommendations from other teams. Armed with this knowledge you decide to introduce it in your project while accommodating a little delay in development phase due to the time needed by developers to learn the new “stuff”. This is a risky path, because what happens if in the middle of the project something went wrong with the new stuff? How much delay you can expect and most important, how you can justify the delay to the customer? If at the end of the project the team will tell you that the new “stuff” is not good you probably wasted a great amount of time.

Use the new stuff in little part of the project, not a critical one, to evaluate it: This is the most conservative approach, you design your product to use the new “stuff” in a limited and not critical part. This will give to the development team the ability to learn it on a real Project, but if the new “stuff” proven to be bad for the team, risks are mitigated. The drawback is that you are evaluating the new stuff in an unimportant area. If everything went good no one can assure you that it will works for the critical part of the software.

All these approaches suffers of several problems, but the most dangerous is: after the project is finished, (usually with delay in schedule) it is difficult if not impossible to answer the question: the introduction of the new “stuff” helped the team in the project? Some team member will answer Yes, some other No, because after such a long time and in projects that quite are late on scheduling, it is nearly impossible evaluate the new “stuff” in isolation and it is easy to blame the new “stuff” as the root cause  Ex: “we are late, but we introduced XXX and YYY so the estimation was incorrect due to missing knowledge of these new tools”.

Scrum to the rescue

image

If you manage your project with Scrum the situation is really different. First of all you usually introduce the new “stuff” because there is some evidence that it can be useful to solve a User Story of the current Sprint (or one that probably will be done in couple of sprint). Maybe the customer asked for some advanced search functionalities, with OR AND and advanced syntax and the team knows that there are some tools based on Lucene (Es Elastic Search) that can tremendously helps the team in implement the User Story. If you start evaluating some new “stuff” on a real User Story you have two great advantages: you start using it where you really need it and you start using on a limited amount of the project. If the new stuff is really Wrong, at least some user stories will slip in the next Sprint, this minimize the risk.

Another advantage of Scrum is Sprint Retrospective; when the team answer couple of  questions: What went well and What could be improved. Scrum process gives you the opportunity and the time to discuss on the new “stuff” as soon as it is is used to accomplish real Useful Work, giving you honest and useful feedback. After a new “stuff” is introduced, at the end of the first sprint the team probably still not mastered it, but they can probably know if it worth spending more time or if it was a waste of time. Sprint after sprint the knowledge of the new “stuff” improves and the team can refine the judgment. In each sprint you can extend the use of the new “Stuff” in other areas of the project if the team think that it can help implementing User Stories. This will permits to use the new “stuff” where you really need it and introduce it gradually in your team. The key part is that the evaluation is based on how the new “stuff” helped the team to implement User Stories and satisfy the customer.

In a rapid evolving industry, like software development, using Scrum is a really good solution to constantly verify new technologies in your team, with little risks, but with real world feedback at the same time.

If you do not know what Scrum is, or you want to refine knowledge on it I suggest you to read the guide at http://www.scrum.org as well reading some introductory book Es: Software In 30 Days.

Gian Maria.

Continuous Delivery in Agile procedures

“The hardest single part of building a software system is deciding precisely what to build.”

Frederick P. Brooks, The Mythical Man Month

This simple sentence contains big truth, the process of building a software has many pitfalls, but the vast majorities of them are related to requirements and gaining a deep understanding of the real problem that the software is going to solve. To build a great software you need to be aware that the act of writing code is just a minimal part of the problem and mastering Application Lifecycle Management is the key of success.

In these years Scrum emerged as a lead process to rationalize development. One of the key of its success is the ability to gather feedback early, to continuously steer your requirements (Backlog Items) because iteration after iteration you have a better comprehension of what needed to be build. If you want more detail on Scrum process you can read the Official Scrum Rulebook that will give you a good introduction over the process.

One of the key aspect is having a “potentially shippable product increments” at the end of each Sprint (time boxed interval of time). This maximize the visibility of the process, because at the end of each Sprint the progress the team will inspect what they have achieved and they can gather feedback from Product Owner and users. The key aspect is “potentially shippable”, the increment produced during a sprint should be ready to be moved in production easily, and at least you should deploy in Test and/or pre-production Environments for Beta Testing. One of the major risks of implementing Scrum or in general an iterative process is a “non potentially shippable” increment at the end of an iteration (I suggest you to read Software in 30 days).

The key aspect of Scrum is empiricism and transparency; you absolutely need to require that software is in Shippable State after an iteration. If this requirements is not met, Product Backlog Item from the Sprint Backlog will be moved again to the Product Backlog as “unfinished or Undone” to reflect the need of more work.

Scrum teams usually do demo at the end of the Sprint, to demonstrate what they achieved with the increment and one of the most useful helps to have “potentially shippable software” is defining what exactly is a “demo”.

This is just another aspect of Definition of Done, because doing a Demo of the sprint in a developer’s laptop can be acceptable, but it suffers from: It works on my machine syndrome. With modern software, the act of moving code from dev machine to production machines can be as simple as using Publish functionality of Visual Studio, or as complex as doing manual update of multiple environments (database, AT, Client code). In these days DevOps movement gains lots of popularity, and we are aware that all barriers between Dev (engineering) and Ops (Operations) should be removed as early as possible. The risk for the team is focusing only on Engineering side, forgetting that we need works to move and maintain code in production.

image

In the above picture most of the impediments that the team encounters towards the Goal of satisfying Product Owner and user are depicted. But this picture is incomplete, because it does not account for Operations related impediments.

image

This is a more realistic situation, where another important impediment is: the increment produced in a Sprint does not met Operations Readiness requirements. Such impediments derives from many disparate reason; some related to technological problem (Sql database schema updates or in general change in schema of saved data) some other arise from poor communication between Dev and Ops (software configurations spread over multiple sources: web.config, database, etc without any documentation). Quite often developers simply underestimate the cost of a change. A modification that can be easy done on dev machine, can be much more complex or problematic in production environments (Ex. Big database in production, complex multi machine deployment, 99,999% uptime and so on).

Continuous delivery is one of the key procedures that helps you to guarantee that the software is always in a “shippable state”. It basically consists of automatic procedures that bring code changes from source control and deploy automatically to Test and production machines. The whole process can be automated or can require the press of a button, but it should not require more than this. Nothing is much frustrating that finishing a sprint and realize that there are some barriers that makes really difficult or even prevent the code to be deployed in Test or Production environments. One of the classical example is changing the structure of a Relational Database making it not compatible with other piece of the software developed by other teams.

Another risk of demo done in “developer computer” is that you have no way to verify if the increment is stable enough, or has problem once exercised in production-similar environments. Thanks to Continuous Deployment you can deploy on multiple environments, test upgrade with production data test lab and maintain one or more beta environments where beta tester can assure that the software is stable and meets QA requirements.

Here is some improvement you will gain if you start moving to Continuous Delivery

  • No more deployment nightmare
  • Detailed log of deploy steps
  • Ability for dev to one-click deploy in test or preproduction environment to test with Real-Data
  • Better repro and fix of bug in Production
  • Ability to rollback a deploy
  • Turn features on/off easily with feature flags.
  • Early feedback from Products Owner / Stakeholders
  • Easy maintenance of Beta Product

Having a Continuous Deployment architecture in place makes the team confident of being able to move iteration increments from dev machine to shared environments. Team Foundation Server helps a lot with Lab Management that permits you to manage environments and automatically run workflow of Build-Apply baseline snapshot-Deploy on Virtual machines. With TFS 2013 Microsoft acquired in-release to give you a more complete solution to manage workflows to bring code from TFS to production Server.

Moving product increments automatically from dev machines to a chain of environments (test, pre-production, production) and maintaining them is a key part of the whole iterative process and helps the team maintaining high software quality.

Gian Maria.