Avoid using Shell command in PowerShell scipts

I have setup scripts that are used to install software, they are simply based on this paradigm

The build produces a zip file that contains everything needed to install the software, then we have a script that accepts the zip file as parameter as well as some other parameters and does install sofwtare on a local machine

This simple paradigm is perfect, because we can manually install a software launching powershell, or we can create a Chocolatey package to automate the installation. Clearly you can use the very same script to release the software inside TFS Release Management.

When PowerShell script runs in the RM Agent (or in a build agent) it has no access to the Shell. This is usually the default, because the agents does not run interactively and usually in Release Management, PowerShell scripts are executed remotely. This fact imply that you should not use anything related to shell in your script. Unfortunately, if you looks in the internet for code to unzip a zip file with PowerShell you can find code that uses shell.application object:

function Expand-WithShell(
    [string] $zipFile,
    [string] $destinationFolder,
    [bool] $deleteOld = $true,
    [bool] $quietMode = $false) 
{
    $shell = new-object -com shell.application

    if ((Test-Path $destinationFolder) -and $deleteOld)
    {
          Remove-Item $destinationFolder -Recurse -Force
    }

    New-Item $destinationFolder -ItemType directory

    $zip = $shell.NameSpace($zipFile)
    foreach($item in $zip.items())
    {
        if (!$quietMode) { Write-Host "unzipping " + $item.Name }
        $shell.Namespace($destinationFolder).copyhere($item)
    }
}

This is absolutely a problem if you run a script that uses this function in a build or RM agent. A real better alternative is a funcion that uses classes from Framework.

function Expand-WithFramework(
    [string] $zipFile,
    [string] $destinationFolder,
    [bool] $deleteOld = $true
)
{
    Add-Type -AssemblyName System.IO.Compression.FileSystem
    if ((Test-Path $destinationFolder) -and $deleteOld)
    {
          Remove-Item $destinationFolder -Recurse -Force
    }
    [System.IO.Compression.ZipFile]::ExtractToDirectory($zipfile, $destinationFolder)
}

With this function you are not using anything related to Shell, and it can be run without problem even during a build or a release management. As you can see the function is simple and probably is a better solution even for interactive run of the script.

Another solution is using 7zip from command line, it gives you a better compression, it is free, but you need to install into any server where you want to run the script. This imply that probably zip files are still the simplest way to package your build artifacts for deployment.

Gian Maria.

Grant right to use $eval on Mongodb 3.2

One of the side effect of enabling authorization on MongDb is that, even if you create a user with “root” right, this account is not able to execute the $eval command. The simpthom is, when you try to execute $eval you got this error

mongodb Command '$eval' failed: not authorized on jarvis-framework-saga-test to execute command

This happens because $eval is somewhat deprecated, and it should not be used. Since it is a dangerous command, a user should have access to all action on all resources, and you need to create a role that has anyAction on anyResource.

If you really need to use $eval, you should create a role, just connect to the admin database and create a new role with the command.

db.createRole( 
	{ 
		role: "executeEval", 
		privileges: [ { 
			resource: { anyResource: true }, 
			actions: [ "anyAction" ] } ], 
		roles: []
 } ) 

Now that you have this new role, just add to all the users that need to use $eval, as an example, if you have a single admin user in admin database, just run this against the admin db.

db.grantRolesToUser("admin", [ { role: "executeFunctions", db: "admin" } ])

And now the admin user can execute $eval against all databases.

Gian Maria.

Secure your MongoDb installation

In last months a lots of rumor spreads about MongoDb and Data Leak because people found lots of MongoDb exposed on the internet without any protection.

The root of the problem is probably a bad default for MongoDb that actually starts without any autentication by default. Developers usually download mongodb, configure without authentication and access MongoDb instance without any knowledge of MongoDb security model. This simplicity of usage can lead to unsecure installation in production.

While this can be tolerable for MongoDb instances that lives in intranets, it is always not a good strategy to leave MongoDb completely unauthenticated.  It turns out that enabling a really basic authentication is really simple even in the community edition.

Once you started your MongoDb instance without authentication just connect with your tool of choice (ex robomongo) and create a user admin in the admin database.

use admin
db.createUser(
  {
    user: "admin",
    pwd: "mybeautifulpassword",
    roles: [ { role: "root", db: "admin" } ]
  }
)

Once this user is created, just stop MongoDb, change configuration to enable authentication.

security:
   authorization: enabled

If authorization is enabled in the configuration file, MongoDb requires that all of your connection to the server is authenticated. There is a nice tutorial in MongoDb site, but basically once authorization is enabled you can authenticate on a single database or to the admin db. With the above instruction I’ve created a user admin on the admin database with the role root. This is the minimum level of authentication you should have, a single user that can do anything. 

This configuration is far to be really secure, but at least avoid to access MongoDb instance without password. It is equivalent to enable only the user “sa” on a Sql Server.

The next step is changing your connection string inside your sofware to specify user and password. The format of the url is this:

mongodb://user:password@localhost/newDb?authSource=admin

As for native authentication in Sql Server, username and password are stored in connection string, and pay attention to the authSource parameter of the connection string. If you omit that parameter C# driver try to authenticate against specified database (newDb in this example) and it fails because the only user is in the admin database. Thanks to the authSource parameter you are able to specify the database to use to authenticate.

You don’t need to change anything else in your code, because the connectionstring contains all the information to authenticate the connection.

To avoid having unsecure instance of mongoDb in production, starts immediately to secure database directly during developing phase, so every person included in the process knows that he need a password to access the database.

Gian Maria.

Decide when Castle.Windsor Startable Facility starts your components

Castle.Windsor Startable facility is a nice facility that automatically starts component that implements a specific interface (IStartable) or components registered with specific extensions method (ex StartUsingMethod). 

This approach is really nice, and the Facility has different way to work, the old aggressive mode that try to start a component immediately after is registered, and another, more useful, that starts a component only when all its dependencies are registered. This feature is helpful, because it avoid you to worry about order of registration.

Basically the problem is: if component A implements IStartable and depends from service B and service B is registered after A, Houston we have a problem.  In that scenario the Startable facility try to Instantiate A to call Start(), but A cannot be resolved because it still misses B dependency. To avoid this problem Startable facility support a deferred start, where A is instantiated (and started) only after all dependencies (in this scenario B) are correctly registered.

But this is not enough in some scenario. I have a problem because I not only need that the component is started after all dependencies are registered, but I want also to be sure that the component is started after I’ve started Rebus IBus interface.

Generally speaking there are a lot of legitimate situations where you want to control WHEN the Startable Facility actually instantiate Startable Components.

A standard solution is not using the facility at all, when you want to start IStartable components, you can simple scan all registered components in Castle to find those ones that implements IStartable, create and intsantiate it.

This approach is wrong, because it has a couple of problem: the first one is that it does not work for components registered with StartUsingMethod fluent interface, the second problem is that the startable facility also takes care of calling stop during decommition phase.

To overcome this problem you can write a modified version of Startable Facility, with a simple method that has to be manually called to start everything. Here is a possible implementation

public class MyStartableFacility
    : AbstractFacility
{
	private ITypeConverter converter;
     
    protected override void Init()
    {
        converter = Kernel.GetConversionManager();
        Kernel.ComponentModelBuilder.AddContributor(new StartableContributor(converter));
    }

    public void StartAllIStartable()
    {
        IHandler[] handlers = Kernel.GetAssignableHandlers(typeof(object));
            
        foreach (var handler in handlers)
        {
            if (typeof(IStartable).IsAssignableFrom(handler.ComponentModel.Implementation) ||
                IsStartable(handler))
            {
                handler.Resolve(CreationContext.CreateEmpty());
            }
        }
    }

    public static bool IsStartable(IHandler handler)
    {
        var startable = handler.ComponentModel.ExtendedProperties["startable"];
        var isStartable = (bool?)startable;
        return isStartable.GetValueOrDefault();
    }
}

Most of this code is actually taken from the original implementation of the facility. It actually add the StartableContributor to the kernel as for the original IStartable interface, but it does not start anything automatically. To start everything you need to call StartAllIStartable method, that simply scans all registered component, and if it is startable it just resolve the object.

Actually all the dirty work is done by basic accessories classes of IStartable (StartableContributor) and you only need to resolve IStartable object to have everything works as expected. To understand if a component is IStartable you can simply check for Extended property “startable” that is inserted by the StartableControbutor.

Gian Maria.

No agent could be found with the following capabilities

In TFS 2015 / VSTS new build system each task contains a series of requirements that needs to be matched by agents capabilities for the task to run. Usually you install Visual Studio in the machine with the build agent and you can schedule standard .NET builds without problem, but what happens when the build starts to evolve?

When you start creating more complex build, you can find that your agent does not meets requirements because it miss some of the required capabilities. As an example, in TFS 2015 Build I’ve added task to run Sonar Qube Analysis on my code.

Build definition with Sonar Qube analysis enabled

Figure 1: A build with SonarQube analysis enabled

Now if I queue a build manually, TFS warned me that it is not able to find a suitable agent, and if you ignore that warning and queue the build here is the result.

The build failed because no agent with required capabilities was found in the pool.

Figure 2: Build failed because no suitable agent was find

The build failed because there are no agent capable to run it. Now you should go to the Project Collection administration page and verify all the agents.

Thanks to Capabilities tab in the administration page I can see a complete list of all agent capability.

figure 3: View agents capabilities in administration page

From that picture I verified that the only agent I’ve configured missed the Java capability, so I simply remote desktop to the server, installed java on the machine and then restart the agent service (VSO-Agemt-TFS2013Preview). After agent is restarted the build rans just fine.

Thanks to the new build system, TFS Build Agents can automatically determines some known capabilities (such as Java installed) and this is atuomatically matched with all the requirements that are contained in tasks you use in the build, so TFS can choose to run the build only in the agent that satisfies requirements, and if no agent is found it warns you with a clear error.

Gian Maria.