Another big advantages of using MMS to manage your mongo deployments is the ability to automatic upgrade agents and instances. For agents you can simply open the deployment area of a group and select edit mode. If some agent is outdated, the interface will immediately warn you with an alert on the top of the page as you can see in Figure 1.

image

Figure 1: Mms is telling you that agents are out of date

If you choose to upgrade agents pressing the update link at the top of the pages, you should see some unpublished changes appears in the Ui.

SNAGHTML167b66d

Figure 2: Upgrading agents is a simple modification of the deploy 

Remember that each modification you do through mms will be downloaded by all the agents and should be reviewed and confirmed for deploy. Pressing Review & Deploy will show pending operations:

image

Figure 3: Pending modification to deploy

When you press confirm and deploy, mms will confirm all modifications and the next time that the agents will poll the site, the new deploy instructions will be downloaded and applied from the various automation agents.

Upgrading automation and monitoring agents is not the only upgrade you can do from the web interface, the real killer feature is  upgrading the version of mongo that your instances are running. In Figure 4 you can see that mongo mms shows my current version as red, because is not the latest one available.

image

Figure 4: Mms interfaces shows mongo version in red because is not the latest one available.

If you come back to edit mode you can edit any monitored instance, in this example I want to edit configuration for the entire replica set, upgrading mongo to the latest version.

image

Figure 5: In edit mode you can change configuration of your mongo deployment

Now you can simply edit the configuration and choose whatever version you like to install. You can install newer version, but you can also install older version if you want to rollback your environment to previous version.

image

Figure 6: Editing of configuration of your replica set.

image

Figure 7: From edit panel you can change version to whatever version you like.

Once you have done your changes and pressed Apply button, you should review your modification and confirm as for agent upgrade.

image

Figure 8: Changes of your replica set ready to be deployed, version will be upgraded from 2.6.2 to 2.6.5

As usual mms interface starts showing what is happening to your machines, in Figure 9 you can verify that the three machines of the replica set are downloading the new version of mongo.

image

Figure 9: Environment is upgrading to new version

image

Figure 10: Not all the machine have same speed, rs1-red is already in goal state, while the other are still installing new version

image

Figure 11: Replica set was upgraded to latest version, primary has changed

After a little bit, all of your machines are upgraded to the latest version. Thanks to Mongo MMS you can manage upgrade or downgrade of your instances with few mouse clicks, because automation agents will take care of all operations needed to upgrade your servers.

Gian Maria.

Tags: ,

No comments

In an old post I’ve shown how to easily deploy mongo thanks to MMS online services. Mms is not only useful to deploy mongo instances, but it is exceptional also for monitoring and configuring. One of the most interesting features is managing users easily from mms web interface. Suppose you had installed mongo on an Azure Virtual Machine and you do not want everyone being able to access your instance. A possible solution is enabling authentication.

Thanks to mms you can simply go to Deployment / Authentication & Users and enable authentication. With the green Add User button you can add how many user you want and you can give them permission using roles.

image

Figure 1: Manage mongodb users through mms web interface.

Once everything is configured, mms will deploy changes to all of your instances that belong to the same group, so every database shares the same sets of users. Wait a little bit and then try to authenticate to your instance, ex with robomongo.

image

Figure 2: Test auth from Robomongo

Et voilà, with few clicks you have now authentication enabled on your mongo databases of that group, it could not be easier than this :).

Gian Maria.

Tags: ,

No comments

One of the greatest advantage in using Git over a centralized Version Control System is branching system. It is quite common for developers to start branching whenever they need to add a new feature, work on that branch and finally merge back to mainline when the feature is finished. One of the most famous workflow is called GitFlow, a way to work in Git that is implemented even in some GUI tool like SourceTree.

One of the main problem when you heavily use feature branches is the moment you merge back to mainline. One of the technique to easy the pain is periodically do a forward integration from mainline to feature branch, because it is usually easier doing several little merge than doing a big merge when the feature is complete.

Assuming that the mainline contains only stable code, because all feature branch merge to mainline only when they are stable, it is usually safe to periodically merge from mainline to branch to avoid a big merge at the end. In this scenario the question usually is: how frequently I have to merge? One possible solution is to take advantage of your Continuous Integration system to warn you when a modification you did on a branch introduces problem because:

  • Does not merge automatically with mainline but it generates conflicts
  • Even if the branch merge automatically the code wont compile.
  • Even if the branch merge automatically and the code compile, some tests where broken.

With TFS Build you can achieve this result quite simply, first of all configure a build with a trigger for each push, then in the Source Settings specify that this build should be triggered whenever any branch that starts with feat got any push.

image

Figure 1: Source code configuration, only build when a branch that starts with feat got a code increment

This build is intended to monitor the status of all feature branches. All you need to do is including a PowerShell script in source code that will be run pre-build. This script basically merge with master with command line git (Build Agent where that build runs should have git installed and available in PATH environment variable). This is a possible PowerShell script to accomplish the merge

$teststring = $env:TF_BUILD_SOURCEGETVERSION

$branchName = ""
$teststring | where { $_ -match "/(?[^/]*?):" } |
    foreach { $branchName = $Matches["bname"] }

Write-Host "BranchName = $branchName"
$rootDir = "$env:TF_BUILD_BUILDDIRECTORY\src"
Write-Host "Source folder is $rootDir"

#be sure to checkout the right branch 
$ps = new-object System.Diagnostics.Process
$ps.StartInfo.Filename = "git"
$ps.StartInfo.WorkingDirectory = $rootDir
$ps.StartInfo.Arguments = "checkout $branchName"
$ps.StartInfo.RedirectStandardOutput = $True
$ps.StartInfo.RedirectStandardError = $True
$ps.StartInfo.UseShellExecute = $false
$ps.start()

[string] $Out = $ps.StandardOutput.ReadToEnd();
[string] $ErrOut = $ps.StandardError.ReadToEnd();
Write-Host "Output git checkout $branchName is:" $Out
Write-Host "Return value: " $ps.ExitCode
if ($ps.ExitCode -ne 0) 
{
    $Host.ui.WriteErrorLine("Branch cannot be checked out")
}

#try to merge


$ps = new-object System.Diagnostics.Process
$ps.StartInfo.Filename = "git"
$ps.StartInfo.WorkingDirectory = $rootDir
$ps.StartInfo.Arguments = " merge master"
$ps.StartInfo.RedirectStandardOutput = $True
$ps.StartInfo.RedirectStandardError = $True
$ps.StartInfo.UseShellExecute = $false
$ps.start()

[string] $Out = $ps.StandardOutput.ReadToEnd();
[string] $ErrOut = $ps.StandardError.ReadToEnd();
Write-Host "Output git merge master:" $Out
Write-Host "Return value: " $ps.ExitCode

if ($ps.ExitCode -ne 0) 
{
    $Host.ui.WriteErrorLine("Branch does not merge correctly")
}

This is really a first tentative in this direction, there probably better way to do it, but it solves the problem without any great complexity. It just uses a regular expression to find branch name from the environment variable TF_BUILD_SOURCEGETVERSION. Once you have name of the branch just checkout it and then try to “git merge master” to merge the branch with master. Quite all git command return 0 if the operation is successful, so if the ExitCode of “git merge master” command is different from 0 it means that the merge operation did not complete correctly (conflicts occurred). If the merge is unsuccessful the script use UI.WriteErrorLine to write error on the UI and this has two effects: first the message will be reported on the Build Summary, then then build will be marked as partially failed.

Now even if your branch compile correctly and every test is green, after you push to TFS your Build Server will try to merge and run the build, if the branch does not automatically merge with master the build will partially fail and you got this:

image

Figure 2: Code does not automatically merge after the latest push.

This is a signal that is time to do a forward integration from mainline to your branch, because the amount of conflicts will be probably small. Another advantage is that you can merge the code while your modification is fresh in your mind. Nothing is more painfully than merge code you have written a month ago.

If the merge is successfully it does not mean that the code is in good health. Suppose you rename a method in your branch, but in the mainline another one has pushed an increment with another class that uses that method. This is the perfect example of code that gave no merge conflicts, but it does not even compile. Since the script actually does the merge before the build, the rest of the build works on the code result of the merge, now the build fails.

image

Figure 3: Merge was successfully, but the result of the merge does not compile

In this situation you surely have a standard build that simply works on the feature branch that is green, but the Build System can run for you this another special build that runs on the result of the merge with mainline that is Red. This means: In your branch code everything is green, but when you will merge to mainline you will have problem, so it is probably better to look at that problems now!. 

The very same happens if the result of the merge breaks some unit test. In conclusion, thanks to few PowerShell lines, you can instruct your TFS build to automatically try to merge each increment of code in your feature branches and then compile, run test and do whatever any operation you usually do on your build on the result of the merge between the feature branch and mainline. Thanks to this you can immediately merge from mainline to your branch to fix the problem as soon as you have introduced it.

This approach has some disadvantages. First of all a PowerShell script cannot make the build fails, so if the code does not merge, usually you will have a failing build because the build system try to build source code during a merge and find invalid characters in source code. Second, this special build runs only when there is a push on some feature branch, but nothing happens if someone pushes in the mainline. In that case you should run this special build for every feature branch you have in your repository.

Gian Maria.

Tags: ,

1 Comment
Deploy mongo easily with mms
on November 1st, 2014
On category: NoSql

One of the reason why Mongo is gaining a lot of momentum in the industry is the easy of use. Just download the package, start mongod and you are donw. When you are in production, you cannot surely simply install your mongo as a service, start it and then forget it without any kind of monitoring, because this will surely lead to problems. When is time to create more complex topologies, like sharding or replica set you can commit some mistakes and you have some tedious work to do. I strongly suggest you to have a look at Mongo Mms, because the online version offers a lot of functionalities for free, and every developer can use it to simplify even deployment of dev servers.

The new version introduced some interesting capabilities to automatically deploy and manage your mongo instances with few clicks of your mouse. Suppose you want to create a classic replica set of 3 machines using 3 VM with Linux Ubuntu. After you register with MMS you can simply proceed to install the MMS automation agent on your machine with few steps

image

Figure 1: Instruction to install automation agent on Ubuntu

Once you installed the automation agent on all of your machines, you can simply start creating your mongo deployment using MMS web interface. Agents poll server periodically for instructions, this implies that mms only requires your machines to be able to reach mms site to poll instructions and to send monitoring data. All of your machines can be in a private NAT network. If you go in the Administration section of MMS web interface you can check status of your agents.

2014-10-29_17-04-10

Figure 2: List of all of your active agents

In this picture you can check not only automation agents, but also monitoring and backup ones. The automation one is the most important, because is the responsible for installing other agents and configuring your mongo instance. Now you can move to the deployment tab, and if some of your agents are out of date you can simply upgrade with a simple click.

2014-10-29_17-04-35

Figure 3: List of all of your deployment servers and in the top area of the page you can check if some of the agents are old

In the upper right part of the deployment area, there is a button called Add that can be used to create a new deployment of Mongo, in this example I want to create a new Replica Set.

2014-10-29_17-07-23

Figure 4: Create a replica set from the web interface.

Now it is just a matter of configuring all the options. The most important one is the Elegible Server RegExp where you should specify a regex that will select all the servesr that will take part of the replica set. In this example I used three distinct names, and I separated with a pipe to create a regex that choose all the servers, you can use prefix for convenience.

 2014-10-29_17-09-34

Figure 5: Options to create a replica set.

You can configure a lot of options and once you are done you can simply press Apply. This will create a script to create your new deployment, and you can review it before agents starts applying it to destination machines.

SNAGHTML12f332

Figure 6: Pressing apply creates the script but you still have time to review before starting deploy

2014-10-29_17-09-56

Figure 7: Reviewing the script tells you every operation that will be triggered in your environment

Pressing Confirm & Deploy activates the script, and the web interface starts showing you the status of the various machines that will take part of the deploy.

2014-10-29_17-10-16

Figure 8: Deploy starts and the web interface starts waiting for it to finish.

 The status is pushed from the automation client to the mms server and you can see the status of the single machines.

2014-10-29_17-23-02

Figure 9: One of the machine is in goal state, the other two still needs time

In Figure 9 you can see that the machine avalanche reached Goal State, this means that the automation agent finished applying modification to the system. The other two machines are in WaitRsInit, this means that mongo is started and they are starting the replica set. Once everything is ok, all the machines are in the Goal State and the version of mongo installed in shown in Version column.

Now you can switch to View mode, where you can look at your deployment.

image

Figure 10: View mode shows the status of your servers, as well as witch instance is primary and which instance is secondary

You can change every aspect of your deployment after it was provisioned, as an example I decided to change priority of one of my server after the replica set was up and running because avalanche VM is stored on an SSD and writing speed is higher than the other two VM.

image

Figure 11: Changing options after replica set is created.

You can also change the installed version of mongo, and the automation agents will take care of all operations needed to update or downgrade your instance of mongo.

If you use Amazon AWS, Mongo MMS will be also able to automatically provision your virtual machines in the cloud, automatically installing automation agents on it, so you can really deploy everything, from virtual machines to mongo with few clicks.

I must admit that I’m really impressed from deployment capabilities of Mongo MMS especially because you can manage up to 8 servers for free.

Gian Maria.

Tags:

1 Comment

One of the most interesting news of VSO in last months are service hooks and REST API to integrate VSO with third party tools. This capability is a key for success because ALM is really a complicated subject and rarely you can manage all of your applications with a single tools.

VSO and TFS are no exception to this rule, they are more a set of suites and tools glued together under a single brand. TFS is composed by a Work Item Store plus Two options for source control (TFVC/Git) plus a Continuous Integration service (Build service) and a Release Management Tool for release, it integrates with SCOM to keep Ops and Devs in contact, plus much more.

But no tool can cover all of the need of every possible user, and VSO solves this problem giving full access to every data through Managed and Java API and now even with a REST interface. In this article I’ll tell you how you can integrate with Trello, a widely used online tool to manage Kanban boards. VSO has support for Kanban, but is still rough and many people around the world are using Trello to manage Kanban Cards. If you like VSO but really prefer using Trello for your Kanban you can follow this guide to link your VSO account to Trello.

Possibilities are infinte, as an example I can configure VSO to create a particular card in trello whenever a PBI is created in my project

image 

Figure 1: Automatic creation of card upon creation of PBI in VSO

If you start adding PBI in VSO

image

Figure 2: Create PBI in VSO account

You will find corresponding card in Trello automatically created.

image

Figure 3: Card are automatically created with a Link to the corresponding PBI

Actually the integration is still rough, because I’ve no way to keep the two synchronized, but the direction is promising.

Gian Maria.

Tags: ,

No comments