Leaving a VSTS Account

As a VSTS trainer, it is quite common for me to made students create VSTS Accounts, play with them and being enlisted in those account to help them in various stuff. It happens also that some customer gives me temporary access to the VSTS account, and in all those years, many of them forgot to remove me from the account.

This is annoying because each time Visual Studio or other tools try to understand VSTS accounts I have right to access, the list is really long. Luckily enough few months ago, the team added the ability to leave an account directly from your profile list.

SNAGHTML234a2d0

Figure 1: Profile page where you can leave all the account you are not interested into.

So if you, like me, have access to VSTS accounts you do not care of, proceed to cleanup your access list leaving all accounts you are not interested into.

Gian Maria.

Public projects in VSTS

This is a long and awaited feature, not because VSTS should fight GitHub as Open Source repository standard, but because an organization is often composed by many private projects and some projects that are public and Open Source.

You can read about the new feature in this blog post, the feature is in preview, but will become available to every account in the future. Enjoy it.

Gian Maria.

VSTS / TFS: use Wildcards for continuous integration in Git

If you setup a build in VSTS / TFS against a git repository, you can choose to trigger the build when some specific branch changed. You can press plus button and a nice combobox appears to select the branch you want to monitor.

image

Figure 1: Adding a branch as trigger in VSTS / TFS Build

This means that if you add feature/1312_languageSelector, each time a new commit will be pushed on that branch, a new build will trigger. Actually if you use GitFlow you are interested in building each feature branch, so you do not want to add every feature branch in the list of monitored branches.

If you look closely, it turns out that the combo you use to choose branches admit wildcards. If you look at Figure 2, you can verify that, pressing the add button, a new line is added and you have a textbox that allow you to “Filter my branches”.

image

Figure 2: Filter branch text

Actually the name is misleading, because you can simply write feature/* and press enter, to instruct VSTS to monitor each branch that starts with feature/.

Using wildcards in trigger list allows you to trigger a build for every feature branch.

For people not used to VSTS, this feature goes often unnoticed, because they think that the textbox can be used only to filter one branch. With this technique you can trigger a build for each change in each standard branch in GitFlow process.

image

Figure 3: Trigger with wildcards

As you can see in Figure 3, I instructed TFS / VSTS to build develop, master and all hotfix, feature or release branches.

Gian Maria

New cool feature of VSTS to limit impact of erratic tests

I’ve blogged some time ago about running UAT testing with a mix of Build + Release in VSTS. Actually, UAT testing are often hard to write, because they can be erratic. As an example, we have a software composed by 5 services that collaborates together, CQRS and Event Sourcing, so most of the tests are based on a typical pattern: Do something then wait for something to happen.

Writing tests that interact with the UI or are based on several services interacting togheter can be difficult.

The problem is: whenever you are “wait for something to happen” you are evoking dragons. Actually we wrote a lot of extensions that helps writing tests, such as WaitForReadModel, that poll the readmodel waiting for the final result of a command, but sometimes those kind of tests are not stable because during a full run, some tests are slower or simply they fails because some other tests changed some precondition.

We spent time to make test stable, but some tests tend to have erratic failure and you cannot spent tons of time to avoid this. When they fails in VS or Nunit console, you can simply re-run again only failed tests and usually they pass without a single error. This is perfectly fine when you are running everything manually, you launch tests in a VM, then come back after a little bit, if some tests failed, just re-run failed test and verify if something was really broken.

Erratic tests are really difficult to handle in a build, because if the build was always red because of them, the team tend to ignore build output.

When a build is often failing due to false-positive, team tend to ignore that build and it starts losing value.

Thanks to a cool new feature in VSTS, we can instruct our builds to re-run automatically those kind of tests. You can read everything about it in the release notes for VSTS.

image

Figure 1: Option to re-run failed tests

It could seem a little feature, especially because usually your tests should not be erratic, but actually it is a HUGE feature for UAT testing, when having not erratic test is simply not feasible or a really huge work. Thanks to this feature, the build can automatically re-run failed tests, completely eliminating the problem of erratic tests, and showing as failed only tests that really failed constantly for more than a given amount of runs.

Another situation where this feature has tremendous impact is when you want to  migrate a huge set of tests from sequential execution to parallel execution. If your test were written assuming sequential execution, when you run them in parallel you will have many failures due to interacting tests. You can simply create a build, enable parallel execution and enable re-run failed test. This will simply helps you to spot interacting tests so you can start fixing them so they can run in parallel.

If you look at Figure 2, you can verify that all erratic tests are easily identified, because you can filter all tests that passed on rerun. This will leave your build green, but can spot erratic tests that should be investigated to make them more stable.

image

Figure 2: Erratic test can be immediately discovered from build output.

I think that this feature is really huge and can tremendously helps running UAT and Integration tests in build and releases.

If you do not see the option in figure 1, please be sure you are actually using the version number 2 of the test runner task as shown in Figure 3.

image

Figure 3: Be sure to use version 2.* of the test runner, or the option to re-run failed test is not present.

Gian Maria.

VSTS Package, packages failed to publish

I have a build that publishes nuget packages on MyGet, we decided to move packages to VSTS internal package management, so I simply added another Build Task that pushes packages to VSTS internal feed. Sadly enough I got a really generic error

Error: An unexpected error occurred while trying to push the package with VstsNuGetPush.exe.
Packages failed to publish

Those two errors does not gives me real information on what went wrong, but looking in the whole log, I verified that the error happens when the task was trying to publish symbols packages (2).

image

Figure 1: Complete error log

It turns out that VSTS Package management is not still capable of publishing symbols package and this generates the error.

To solve this problem you can avoid generating symbols package, or you can simply avoid pushing them to VSTS feed. In the future VSTS feeds will probably supports symbols package and this problem will be gone.

In the meanwhile we have a issue on GitHub where we asked to the team for a much clearer error message, that can immediately point out the problem during the push.

Gian Maria.