Git for windows, getting Invalid username or password with Wincred

If you use Https to communicate with your git repository, Es, Github or VisualStudioOnline, you usually setup credential manager to avoid entering credential for each command that contact the server. With latest versions of git you can configure wincred with this simple command.

git config --global credential.helper wincred

This morning I start getting error while I’m trying to push some commits to GitHub.

$ git push
remote: Invalid username or password.
fatal: Authentication failed for 'https://github.com/ProximoSrl/Jarvis.DocumentS
tore.git/'

If I remove credential helper (git config –global credential.helper unset) everything works, git ask me for user name and password and I’m able to do everything, but as soon as I re-enable credential helper, the error returned. This problem is probably originated by some corruption of stored credentials, and usually you can simply clear stored credentials and at the next operation you will be prompted for credentials and everything starts worked again. The question is, where are stored credential for wincred?

If you use wincred for credential.helper, git is storing your credentials in standard windows Credential Manager

You can simple open credential manager on your computer,

image

Figrue 1: Credential manager in your Control Panel settings

Opening Credential manager you can manage windows and web credentials. Now simply have a look to both web credentials and windows credentials, and delete everything related to GitHub or the server you are using. The next time you issue a git command that requires authentication, you will be prompted for credentials again and the credentials will be stored again in the store.

Gian Maria.

Monitor if your branch will generate merge conflicts with TFS Build

One of the greatest advantage in using Git over a centralized Version Control System is branching system. It is quite common for developers to start branching whenever they need to add a new feature, work on that branch and finally merge back to mainline when the feature is finished. One of the most famous workflow is called GitFlow, a way to work in Git that is implemented even in some GUI tool like SourceTree.

One of the main problem when you heavily use feature branches is the moment you merge back to mainline. One of the technique to easy the pain is periodically do a forward integration from mainline to feature branch, because it is usually easier doing several little merge than doing a big merge when the feature is complete.

Assuming that the mainline contains only stable code, because all feature branch merge to mainline only when they are stable, it is usually safe to periodically merge from mainline to branch to avoid a big merge at the end. In this scenario the question usually is: how frequently I have to merge? One possible solution is to take advantage of your Continuous Integration system to warn you when a modification you did on a branch introduces problem because:

  • Does not merge automatically with mainline but it generates conflicts
  • Even if the branch merge automatically the code wont compile.
  • Even if the branch merge automatically and the code compile, some tests where broken.

With TFS Build you can achieve this result quite simply, first of all configure a build with a trigger for each push, then in the Source Settings specify that this build should be triggered whenever any branch that starts with feat got any push.

image

Figure 1: Source code configuration, only build when a branch that starts with feat got a code increment

This build is intended to monitor the status of all feature branches. All you need to do is including a PowerShell script in source code that will be run pre-build. This script basically merge with master with command line git (Build Agent where that build runs should have git installed and available in PATH environment variable). This is a possible PowerShell script to accomplish the merge

$teststring = $env:TF_BUILD_SOURCEGETVERSION

$branchName = ""
$teststring | where { $_ -match "/(?[^/]*?):" } |
    foreach { $branchName = $Matches["bname"] }

Write-Host "BranchName = $branchName"
$rootDir = "$env:TF_BUILD_BUILDDIRECTORY\src"
Write-Host "Source folder is $rootDir"

#be sure to checkout the right branch 
$ps = new-object System.Diagnostics.Process
$ps.StartInfo.Filename = "git"
$ps.StartInfo.WorkingDirectory = $rootDir
$ps.StartInfo.Arguments = "checkout $branchName"
$ps.StartInfo.RedirectStandardOutput = $True
$ps.StartInfo.RedirectStandardError = $True
$ps.StartInfo.UseShellExecute = $false
$ps.start()

[string] $Out = $ps.StandardOutput.ReadToEnd();
[string] $ErrOut = $ps.StandardError.ReadToEnd();
Write-Host "Output git checkout $branchName is:" $Out
Write-Host "Return value: " $ps.ExitCode
if ($ps.ExitCode -ne 0) 
{
    $Host.ui.WriteErrorLine("Branch cannot be checked out")
}

#try to merge


$ps = new-object System.Diagnostics.Process
$ps.StartInfo.Filename = "git"
$ps.StartInfo.WorkingDirectory = $rootDir
$ps.StartInfo.Arguments = " merge master"
$ps.StartInfo.RedirectStandardOutput = $True
$ps.StartInfo.RedirectStandardError = $True
$ps.StartInfo.UseShellExecute = $false
$ps.start()

[string] $Out = $ps.StandardOutput.ReadToEnd();
[string] $ErrOut = $ps.StandardError.ReadToEnd();
Write-Host "Output git merge master:" $Out
Write-Host "Return value: " $ps.ExitCode

if ($ps.ExitCode -ne 0) 
{
    $Host.ui.WriteErrorLine("Branch does not merge correctly")
}

This is really a first tentative in this direction, there probably better way to do it, but it solves the problem without any great complexity. It just uses a regular expression to find branch name from the environment variable TF_BUILD_SOURCEGETVERSION. Once you have name of the branch just checkout it and then try to “git merge master” to merge the branch with master. Quite all git command return 0 if the operation is successful, so if the ExitCode of “git merge master” command is different from 0 it means that the merge operation did not complete correctly (conflicts occurred). If the merge is unsuccessful the script use UI.WriteErrorLine to write error on the UI and this has two effects: first the message will be reported on the Build Summary, then then build will be marked as partially failed.

Now even if your branch compile correctly and every test is green, after you push to TFS your Build Server will try to merge and run the build, if the branch does not automatically merge with master the build will partially fail and you got this:

image

Figure 2: Code does not automatically merge after the latest push.

This is a signal that is time to do a forward integration from mainline to your branch, because the amount of conflicts will be probably small. Another advantage is that you can merge the code while your modification is fresh in your mind. Nothing is more painfully than merge code you have written a month ago.

If the merge is successfully it does not mean that the code is in good health. Suppose you rename a method in your branch, but in the mainline another one has pushed an increment with another class that uses that method. This is the perfect example of code that gave no merge conflicts, but it does not even compile. Since the script actually does the merge before the build, the rest of the build works on the code result of the merge, now the build fails.

image

Figure 3: Merge was successfully, but the result of the merge does not compile

In this situation you surely have a standard build that simply works on the feature branch that is green, but the Build System can run for you this another special build that runs on the result of the merge with mainline that is Red. This means: In your branch code everything is green, but when you will merge to mainline you will have problem, so it is probably better to look at that problems now!. 

The very same happens if the result of the merge breaks some unit test. In conclusion, thanks to few PowerShell lines, you can instruct your TFS build to automatically try to merge each increment of code in your feature branches and then compile, run test and do whatever any operation you usually do on your build on the result of the merge between the feature branch and mainline. Thanks to this you can immediately merge from mainline to your branch to fix the problem as soon as you have introduced it.

This approach has some disadvantages. First of all a PowerShell script cannot make the build fails, so if the code does not merge, usually you will have a failing build because the build system try to build source code during a merge and find invalid characters in source code. Second, this special build runs only when there is a push on some feature branch, but nothing happens if someone pushes in the mainline. In that case you should run this special build for every feature branch you have in your repository.

Gian Maria.

Git submodule update error: Permission Denied Publickey

It could happens when you clone a Git Repository with submodules, issuing a git submodule update command, you are prompted with this error error.

Cloning into ‘src/xxxx’…
Warning: Permanently added the RSA host key for IP address xxx.xxx.xxx.xxx to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

If you search in the internet for the cause of errors, you can find some people suggesting that the url specified in .gitmodules file is wrong and should be changed, here is my .gitmodule

[submodule “src/CQRS”]
     path = src/CQRS
     url = git@github.com:xxxxxx/cqrs.git
     branch = master
    

You could change the url configuration to https url and everything works, but this is not the perfect solution, because the address git@github.com is perfectly valid, but probably there is some problem with your RSA keys stored in Github (or you never configured RSA Keys for your account). In my situation, my RSA Keys had some problem and I needed to recreate another one. If you do not know what a RSA key is and how to create a RSA Key to connect to github I strongly suggest you reading the guide: Generating SSH Keys.

Once you configure a valid certificate in github your submodule should word without problem.

Gian Maria.

Git showing file as modified even if it is unchanged

This is one annoying problem that happens sometimes to git users: the symptom is: git status command shows you some files as modified (you are sure that you had not modified that files), you revert all changes with a git checkout — . but the files stills are in modified state if you issue another git status.

This is a real annoying problem, suppose you want to switch branch with git checkout branchname, you will find that git does not allow you to switch because of uncommitted changes.

This problem is likely caused by the end-of-line normalization (I strongly suggest you to read all the details in Pro Git book or read the help of github). I do not want to enter into details of this feature, but I only want to help people to diagnose and avoid this kind of problem.

To understand if you really have a Line Ending Issue you should run git diff -w command to verify what is really changed in files that git as modified with git status command. The -w options tells git to ignore whitespace and line endings, if this command shows no differences, you are probably victim of problem in Line Ending Normalization.

This is especially true if you are working with git svn, connecting to a subversion repository where developers did not pay attention to line endings and it happens usually when you have files with mixed CRLF / CR / LF. If you work in mixed environment (Unix/Linux, Windows, Macintosh) it is better to find files that are listed as modified and manually (or with some tool) normalize Line Endings. If you do not work in mixed environment you can simply turn off eol normalization for the single repository where you experience the problem. To do this you can issue a git config –local core.autocrlf false  but it works only for you and not for all the other developers that works to the project. Moreover some people reports that they still have problem even with core.autocrlf to false.

Remember that git supports .gitattributes files, used to change settings for a single subdirectory. If you set core.autocrlf to false and still have line ending normalization problem, please search for .gitattribuges files in every subdirectory of your repository, and verify if it has a line where autocrlf is turned on:

* text=auto

now you can turn off in all .gitattributes files you find in your repository

* text=off

To be sure that every developer of the team works with autocrlf turned off, you should place a .gitattributes file in repository root with autocrlf turned off. Remember that it is a better option to normalize files and leave autocrlf turned on, but if you are working with legacy code imported from another VCS, or you work with git svn, git-tf or similar tools, probably it is better turn autocrlf to off if you start experiencing that kind of problems.

Gian Maria.

Moving between different VCS to Team Foundation Server

In an old post I’ve already demonstrated how you can move a project from a standard TFS Version Control System to a Git repository hosted on TF Service (or wherever else). This is especially useful if you are working with TFS but you really need features of a Distributed Version Control System. The original post is here: move source from a TFVCS based Team Project to a TF Service Git Based Team Project.

Now that you have support for Git even on on-premise installation of TFS 2013, you can use the very same procedure to move source control from a team project based on TFVC to another one based on Git. But there is more: thanks to support of git-tf, a java based bridge between Git and TFVC you can support even more scenarios. You can use Git-Svn to convert from a Subversion Repository and move code to a Team Project based on Git or TFVC. With a little setup you can also handle external companies that works with Subversion and you can import their work in your TFS maintaining even the connection between Subversion commit and Work Items. I’ve described this process in a post called Git as a bridge between Subversion and TFS. You can use Hg-git to convert from Mercurial to Git or TFVC, and in general you can move code to your TFS from any VCS that supports conversion to Git.

The primary drawbacks of this approach are:

Time compression: if you move to a TFVC based Team Project you are not able to maintain the original timestamp of commit / check-in, and the timestamp will reflect the moment you checking from git to TFVC with git-tf.

User Mapping: checkin to the system done with git-tf are done with the credential of the person that is doing the check-in.

Since Git is a real good tool, I strongly suggest to move code to a Git based Team Project, so you can maintain full history. Then if you really need to work with TFVC and you cannot use Git, you can move only the tip to a TFCV based team project, and the history will be maintained with full fidelity in the Git Repository.

Gian Maria.