Deploy mongo easily with mms
on November 1st, 2014
On category: NoSql

One of the reason why Mongo is gaining a lot of momentum in the industry is the easy of use. Just download the package, start mongod and you are donw. When you are in production, you cannot surely simply install your mongo as a service, start it and then forget it without any kind of monitoring, because this will surely lead to problems. When is time to create more complex topologies, like sharding or replica set you can commit some mistakes and you have some tedious work to do. I strongly suggest you to have a look at Mongo Mms, because the online version offers a lot of functionalities for free, and every developer can use it to simplify even deployment of dev servers.

The new version introduced some interesting capabilities to automatically deploy and manage your mongo instances with few clicks of your mouse. Suppose you want to create a classic replica set of 3 machines using 3 VM with Linux Ubuntu. After you register with MMS you can simply proceed to install the MMS automation agent on your machine with few steps

image

Figure 1: Instruction to install automation agent on Ubuntu

Once you installed the automation agent on all of your machines, you can simply start creating your mongo deployment using MMS web interface. Agents poll server periodically for instructions, this implies that mms only requires your machines to be able to reach mms site to poll instructions and to send monitoring data. All of your machines can be in a private NAT network. If you go in the Administration section of MMS web interface you can check status of your agents.

2014-10-29_17-04-10

Figure 2: List of all of your active agents

In this picture you can check not only automation agents, but also monitoring and backup ones. The automation one is the most important, because is the responsible for installing other agents and configuring your mongo instance. Now you can move to the deployment tab, and if some of your agents are out of date you can simply upgrade with a simple click.

2014-10-29_17-04-35

Figure 3: List of all of your deployment servers and in the top area of the page you can check if some of the agents are old

In the upper right part of the deployment area, there is a button called Add that can be used to create a new deployment of Mongo, in this example I want to create a new Replica Set.

2014-10-29_17-07-23

Figure 4: Create a replica set from the web interface.

Now it is just a matter of configuring all the options. The most important one is the Elegible Server RegExp where you should specify a regex that will select all the servesr that will take part of the replica set. In this example I used three distinct names, and I separated with a pipe to create a regex that choose all the servers, you can use prefix for convenience.

 2014-10-29_17-09-34

Figure 5: Options to create a replica set.

You can configure a lot of options and once you are done you can simply press Apply. This will create a script to create your new deployment, and you can review it before agents starts applying it to destination machines.

SNAGHTML12f332

Figure 6: Pressing apply creates the script but you still have time to review before starting deploy

2014-10-29_17-09-56

Figure 7: Reviewing the script tells you every operation that will be triggered in your environment

Pressing Confirm & Deploy activates the script, and the web interface starts showing you the status of the various machines that will take part of the deploy.

2014-10-29_17-10-16

Figure 8: Deploy starts and the web interface starts waiting for it to finish.

 The status is pushed from the automation client to the mms server and you can see the status of the single machines.

2014-10-29_17-23-02

Figure 9: One of the machine is in goal state, the other two still needs time

In Figure 9 you can see that the machine avalanche reached Goal State, this means that the automation agent finished applying modification to the system. The other two machines are in WaitRsInit, this means that mongo is started and they are starting the replica set. Once everything is ok, all the machines are in the Goal State and the version of mongo installed in shown in Version column.

Now you can switch to View mode, where you can look at your deployment.

image

Figure 10: View mode shows the status of your servers, as well as witch instance is primary and which instance is secondary

You can change every aspect of your deployment after it was provisioned, as an example I decided to change priority of one of my server after the replica set was up and running because avalanche VM is stored on an SSD and writing speed is higher than the other two VM.

image

Figure 11: Changing options after replica set is created.

You can also change the installed version of mongo, and the automation agents will take care of all operations needed to update or downgrade your instance of mongo.

If you use Amazon AWS, Mongo MMS will be also able to automatically provision your virtual machines in the cloud, automatically installing automation agents on it, so you can really deploy everything, from virtual machines to mongo with few clicks.

I must admit that I’m really impressed from deployment capabilities of Mongo MMS especially because you can manage up to 8 servers for free.

Gian Maria.

Tags:

No comments

One of the most interesting news of VSO in last months are service hooks and REST API to integrate VSO with third party tools. This capability is a key for success because ALM is really a complicated subject and rarely you can manage all of your applications with a single tools.

VSO and TFS are no exception to this rule, they are more a set of suites and tools glued together under a single brand. TFS is composed by a Work Item Store plus Two options for source control (TFVC/Git) plus a Continuous Integration service (Build service) and a Release Management Tool for release, it integrates with SCOM to keep Ops and Devs in contact, plus much more.

But no tool can cover all of the need of every possible user, and VSO solves this problem giving full access to every data through Managed and Java API and now even with a REST interface. In this article I’ll tell you how you can integrate with Trello, a widely used online tool to manage Kanban boards. VSO has support for Kanban, but is still rough and many people around the world are using Trello to manage Kanban Cards. If you like VSO but really prefer using Trello for your Kanban you can follow this guide to link your VSO account to Trello.

Possibilities are infinte, as an example I can configure VSO to create a particular card in trello whenever a PBI is created in my project

image 

Figure 1: Automatic creation of card upon creation of PBI in VSO

If you start adding PBI in VSO

image

Figure 2: Create PBI in VSO account

You will find corresponding card in Trello automatically created.

image

Figure 3: Card are automatically created with a Link to the corresponding PBI

Actually the integration is still rough, because I’ve no way to keep the two synchronized, but the direction is promising.

Gian Maria.

Tags: ,

No comments

If you need to do an aggregation in MongoDb on a DateTime property and you want to aggregate only by Date part, you can use a simple trick and use a filter to compose the date part before the grouping. Here is the code

 

db.mycollection.aggregate(
{
    "$project" :
    {
       _id : 0,
       "datePartDay" : {"$concat" : [ 
           {"$substr" : [{"$dayOfMonth" : "$LastUpdate"}, 0, 2]}, "-",
           {"$substr" : [{"$month" : "$LastUpdate"}, 0, 2]}, "-",
           {"$substr" : [{"$year" : "$LastUpdate"}, 0, 4]}
      ] }
    }
},
{ "$group" : 
    { "_id" : "$datePartDay", "Count" : { "$sum" : 1 } } 
    }
)

Thanks to the power of aggregation framework, we can construct a pipeline where the first stage create a document with a new property that is composed only by the day-month-year. That new document can have id equal to zero if we only need a count based on that date field, or you can assign original id of the document or whatever else. The subsequent stage of the pipeline is a simple count where the id is the new field that contains only date part of the original DateTime property.

Gian Maria.

Tags:

No comments

Today I encountered a strange error during the configuration of a Build Controller in TFS. We installed and configured the first Build Controller for a TFS Instance, everything went good, but both controllers and agent are marked with stopped icon, even if status is “ready”

image

Figure 1: Controller and agents are marked as stopped even if they are in Ready State

I immediately looked into Event Viewer, but absolutely no clue of what is happening. I tried creating and scheduling a build, but it starts, then remains silent forever. The build system was not working. I remember a post by Richard where he had the same problem, but I’m not in that scenario. I checked DNS, tried to ping the server and everything is ok, but builds never starts and there are absolutely no error in event viewer.

Then I noticed that in the upper section of the Build Server there is another link called Details… that usually is not there. If I clicked on that link it told me that the controller is not able to communicate with TFS because he got a 500 internal error response.

This is extremely painful, because it means that something in the Application Tier is not working properly, so I immediately remote desktop into the TFS machine and looked at the Event Viewer of the server. This time the error is there and luckily enough it was simple to fix.

System.ServiceModel.ServiceHostingEnvironment+HostingManager/42931033
 Exception: System.ServiceModel.ServiceActivationException: The service '/tfs/queue/test/Services/v4.0/MessageQueueService2.svc' cannot be activated due to an exception during compilation.  The exception message is: Memory gates checking failed because the free memory (176160768 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.. ---> System.InsufficientMemoryException: Memory gates checking failed because the free memory (176160768 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.
   at System.ServiceModel.Activation.ServiceMemoryGates.Check(Int32 minFreeMemoryPercentage, Boolean throwOnLowMemory, UInt64& availableMemoryBytes)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CheckMemoryCloseIdleServices(EventTraceActivity eventTraceActivity)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity)
   --- End of inner exception stack trace ---
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity)
   at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath, EventTraceActivity eventTraceActivity)
 Process Name: w3wp
 Process ID: 2768

This is a typical error you can encounter if you install TFS in a single machine configuration. If you follow general guidance on MSDN the single server approach is ok for groups up to 500 users, with 4 GB of ram and 1 disk at 10k. Single server maintenance is easier and for small team is probably the best configuration, but you need to be aware of one possible problem: SQL Server is greed about memory.

The problem is that SQL Server tends to use all available memory, until the system starts becoming really, really slow because it has no free memory for other processes. Whenever you install TFS in a single machine environment, is a good suggestion to limit maximum amount of memory available to SQL Server, leaving space for the AT to work properly. I have no gold number to give you, but if you have a single machine with 4 GB of RAM, usually I limit SQL Server to a maximum of 2 GB. In this specific situation I remember talking about this configuration, but it was never done; this results in SQL Server using about 3 GB of RAM in a 4 GB machine, leaving no space for WCF Service to starts.

Lesson learned: Whenever something goes wrong in TFS, always have a look at events viewer of all machines involved in the process (AT, SQL, Build, etc) because root error could originates in another machine and not in the one you are looking at. As a rule of thumb, if something went wrong, always look at the AT machine Event Viewer.

Gian Maria.

Tags:

1 Comment

During a build you can ask MsBuild to deploy on build using the switch /p:DeployOnBuild=true as I described in previous posts. This is mainly used to deploy the site on IIS thanks to WebDeploy, but you can also use WebDeploy to deploy on a disk path. The problem is that the path is stored in publication settings file, but what about changing that path during a Build?

The answer is simple, you can use the /p:publishUrl=xxx to override what is specified inside the publication file and choose a different directory for deploy. Es.

msbuild WebApplication1.csproj /p:Deploy
OnBuild=true /p:PublishProfile=Profile1 /p:publishUrl=c:\temp\waptest

Thanks to this simple trick you can instruct MsBuild to store deployed site in any folder of the build server.

Gian Maria.

Tags:

No comments