If you need to do an aggregation in MongoDb on a DateTime property and you want to aggregate only by Date part, you can use a simple trick and use a filter to compose the date part before the grouping. Here is the code

 

db.mycollection.aggregate(
{
    "$project" :
    {
       _id : 0,
       "datePartDay" : {"$concat" : [ 
           {"$substr" : [{"$dayOfMonth" : "$LastUpdate"}, 0, 2]}, "-",
           {"$substr" : [{"$month" : "$LastUpdate"}, 0, 2]}, "-",
           {"$substr" : [{"$year" : "$LastUpdate"}, 0, 4]}
      ] }
    }
},
{ "$group" : 
    { "_id" : "$datePartDay", "Count" : { "$sum" : 1 } } 
    }
)

Thanks to the power of aggregation framework, we can construct a pipeline where the first stage create a document with a new property that is composed only by the day-month-year. That new document can have id equal to zero if we only need a count based on that date field, or you can assign original id of the document or whatever else. The subsequent stage of the pipeline is a simple count where the id is the new field that contains only date part of the original DateTime property.

Gian Maria.

Tags:

No comments

Today I encountered a strange error during the configuration of a Build Controller in TFS. We installed and configured the first Build Controller for a TFS Instance, everything went good, but both controllers and agent are marked with stopped icon, even if status is “ready”

image

Figure 1: Controller and agents are marked as stopped even if they are in Ready State

I immediately looked into Event Viewer, but absolutely no clue of what is happening. I tried creating and scheduling a build, but it starts, then remains silent forever. The build system was not working. I remember a post by Richard where he had the same problem, but I’m not in that scenario. I checked DNS, tried to ping the server and everything is ok, but builds never starts and there are absolutely no error in event viewer.

Then I noticed that in the upper section of the Build Server there is another link called Details… that usually is not there. If I clicked on that link it told me that the controller is not able to communicate with TFS because he got a 500 internal error response.

This is extremely painful, because it means that something in the Application Tier is not working properly, so I immediately remote desktop into the TFS machine and looked at the Event Viewer of the server. This time the error is there and luckily enough it was simple to fix.

System.ServiceModel.ServiceHostingEnvironment+HostingManager/42931033
 Exception: System.ServiceModel.ServiceActivationException: The service '/tfs/queue/test/Services/v4.0/MessageQueueService2.svc' cannot be activated due to an exception during compilation.  The exception message is: Memory gates checking failed because the free memory (176160768 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.. ---> System.InsufficientMemoryException: Memory gates checking failed because the free memory (176160768 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.
   at System.ServiceModel.Activation.ServiceMemoryGates.Check(Int32 minFreeMemoryPercentage, Boolean throwOnLowMemory, UInt64& availableMemoryBytes)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CheckMemoryCloseIdleServices(EventTraceActivity eventTraceActivity)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity)
   --- End of inner exception stack trace ---
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity)
   at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath, EventTraceActivity eventTraceActivity)
 Process Name: w3wp
 Process ID: 2768

This is a typical error you can encounter if you install TFS in a single machine configuration. If you follow general guidance on MSDN the single server approach is ok for groups up to 500 users, with 4 GB of ram and 1 disk at 10k. Single server maintenance is easier and for small team is probably the best configuration, but you need to be aware of one possible problem: SQL Server is greed about memory.

The problem is that SQL Server tends to use all available memory, until the system starts becoming really, really slow because it has no free memory for other processes. Whenever you install TFS in a single machine environment, is a good suggestion to limit maximum amount of memory available to SQL Server, leaving space for the AT to work properly. I have no gold number to give you, but if you have a single machine with 4 GB of RAM, usually I limit SQL Server to a maximum of 2 GB. In this specific situation I remember talking about this configuration, but it was never done; this results in SQL Server using about 3 GB of RAM in a 4 GB machine, leaving no space for WCF Service to starts.

Lesson learned: Whenever something goes wrong in TFS, always have a look at events viewer of all machines involved in the process (AT, SQL, Build, etc) because root error could originates in another machine and not in the one you are looking at. As a rule of thumb, if something went wrong, always look at the AT machine Event Viewer.

Gian Maria.

Tags:

1 Comment

During a build you can ask MsBuild to deploy on build using the switch /p:DeployOnBuild=true as I described in previous posts. This is mainly used to deploy the site on IIS thanks to WebDeploy, but you can also use WebDeploy to deploy on a disk path. The problem is that the path is stored in publication settings file, but what about changing that path during a Build?

The answer is simple, you can use the /p:publishUrl=xxx to override what is specified inside the publication file and choose a different directory for deploy. Es.

msbuild WebApplication1.csproj /p:Deploy
OnBuild=true /p:PublishProfile=Profile1 /p:publishUrl=c:\temp\waptest

Thanks to this simple trick you can instruct MsBuild to store deployed site in any folder of the build server.

Gian Maria.

Tags:

No comments

Some time ago a friend asked me the easiest way to get code from a specific folder and a specific version in TFVC. The goal is avoiding using Get Specific Version because he do not want to overwrite the Workspace folder he is using, he want also to avoid creating another workspace only to do a one-shot get of a folder.

It turns out that the easiest way to accomplish this task is from Web Interface, because it has the capability of browsing and downloading code as zip. You can simply navigate to the CODE hub in web interface, choose the folder you want to download and use the context menu of desired folder to download everything as a single zip file.

image

Figure 1: Browse and download code from the Web Interface.

But wait, this will download the latest version of the code, not a specific version. The cool part of the web interface is that if you just append #version=xxx where xxx is the changeset-id you want to download, you can browse the code of that specific changeset, and you can also download as Zip that specific version. If you just look at the url, you can easily spot out that downloading the code as zip is just a matter of calling the right url

https://gianmariaricci.visualstudio.com/DefaultCollection/Experiments/_api/_versioncontrol/itemContentZipped?repositoryId=&path=%24%2FExperiments%2Ftrunk&version=96&__v=5

You can simply change the version parameter and you are able to download every version of your code as zip with a simple call and without resorting to API or external tool. Just copy and paste url inside your browser and you are done. If you have not previously authenticated with your TFS or VSO you will be prompted for credentials, then the file is downloaded.

I’ve showed you examples with Visual Studio Online, but you can use the very same technique against your on-premise TFS.

Gian Maria.

Tags:

Comments Off

It could happens when you clone a Git Repository with submodules, issuing a git submodule update command, you are prompted with this error error.

Cloning into ‘src/xxxx’…
Warning: Permanently added the RSA host key for IP address xxx.xxx.xxx.xxx to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

If you search in the internet for the cause of errors, you can find some people suggesting that the url specified in .gitmodules file is wrong and should be changed, here is my .gitmodule

[submodule "src/CQRS"]
     path = src/CQRS
     url = git@github.com:xxxxxx/cqrs.git
     branch = master
    

You could change the url configuration to https url and everything works, but this is not the perfect solution, because the address git@github.com is perfectly valid, but probably there is some problem with your RSA keys stored in Github (or you never configured RSA Keys for your account). In my situation, my RSA Keys had some problem and I needed to recreate another one. If you do not know what a RSA key is and how to create a RSA Key to connect to github I strongly suggest you reading the guide: Generating SSH Keys.

Once you configure a valid certificate in github your submodule should word without problem.

Gian Maria.

Tags: ,

Comments Off