Converting a big project to .NET Standard without big bang

When you have a big project in .NET full framework and you want to convert to .NET standard / core, usually MultiTargeting can be a viable solution to avoid a Big Bang conversion. You starts with the very first assembly in the chain of dependency, the one that does not depends on any other assembly in the project, and you start checking compatibility with .NET standard for all referenced NuGet Packages. Once the first project is done you proceed with the remaining.

The very first step is converting all project files to new project format, leave all project to target full framework, then you can use a nice technique called MultiTargeting starting with the aforementioned first assembly of the chain. 

Multitargeting allows you to target both framework with a single Visual Studio Project

To enable it just edit project files, and change TargetFramework to TargetFrameworks (mind the final s) and specify that you want that project compiled for Full Framework and .NET Standard. Including .NET Standard in the list of target framework requires removal of all code that is dependent on Full Framework, but this is not always obtainable in a single big bang conversion, because the amount of code could be really high. 

image

Figure 1: Multi Targeting in action

To easy the transition I usually add some constants so I’ll be able to use #ifdef directive to isolate some piece of code only when the project is targeting Full Framework.

  <PropertyGroup Condition=" '$(TargetFramework)' == 'netstandard2.0'">
    <DefineConstants>NETCORE;NETSTANDARD;NETSTANDARD2_0</DefineConstants>
  </PropertyGroup>

  <PropertyGroup Condition=" '$(TargetFramework)' == 'net461'">
    <DefineConstants>NET45;NETFULL</DefineConstants>
  </PropertyGroup>

Thanks to conditional compile we can have some of the code that is compiled only for full framework, or we can have different implementation for a given class (full or netstandard)

After multitarget is enabled and netstandard is one of the target,  project usually stops compiling, because it usually contains some code that depends on full framework. There are two distinct problems: A) nuget packages that does not support netstandard, B) references to full framework assembly. To solve the problem you must use conditional referencing, setting the references only for specific framework version.

image

Figure 2: Conditional reference in action.

As you can see in Figure 2 I can reference difference nuget packages or assemblies depending on the version of framework used. The nice part is being able to reference different version of a library (ex System.Buffers) for full framework or .net standard framework.

At this point the project usually stops compiling because code references classes that are not available in netstandard, as an example in Figure 2 you can verify that netstandard misses lots of references like System.Runtime.Caching and System.Runtime.Remoting. For all code that uses references not available for netstandard project just use a #ifdef NETFULL to compile those classes only with full framework. This is not a real complete solution, but at the end you will have your project that is still fully functional with .NET full framework, and a nice list of #ifdef NETFULL that identify the only parts that are not available in netstandard. Now you can continue working as usual while slowly removing and upgrading all the code to netstandard.

Now repeat the process this for every project in the solution, usually there are two scenarios:

1) The vast majority of the code is full functional with netstandard, there are very few point in the code where you use #ifdef NETFULL.
2) Some of the classes / functionality available only in full framework are extensively used in all of your code, the amount of code with #ifdef NETFULL is becoming too big

Point 1 is the good side, you can continue working with full framework then convert remaining code with your pace. If you find yourself in point 2, as an example if some code uses MSMQ you should isolate the full framework depending code in a single class, then use Inversion Of Control to inject concrete class. As an example, instead of having lots of points in the code that uses MSMQ simply abstract all the code in a IQueue interface, create a MSMQQueue class and you have a single point of code that is not available for netstandard. You can then write code that uses Rabbit MQ and the problem is gone with a simple class rewrite.

Lets do an example: I have a InMemoryCacheHelper to abstract the usage of MemoryCache, and since MemoryCache class from System.Runtime.Caching is not available in netstandard, I simply protect the class with #if NETFULL conditional compiling.

image

Figure 3: Conditional compilation, this is version of the class for NET Full.

Looking in the documentation there is a nuget package called Microsoft.Extensions.Caching.Memory meant to be a replacement for MemoryCache standard full framework class. I can use this NuGet packages to create an implementation of InMemoryCacheHelper compatible with netstandard.

When you enable multitargeting it is quite common to manually edit references in project file, because references are different from Full Framework build and NetStandard build.

Remember that you need to reference the full framework version (1) for net461 compilation, but reference nuget package for netstandard version. This can be done manually editing references in csproj file Figure 4.

image

Figure 4: Reference the correct assembly or NugetPackage based on framework version.

Now you come back to the InMemoryCacheHelper class add an #else branch to the #ifdef NETFULL directive and start writing a version of the class that uses Microsoft.Extensions.Caching.Memory. One class after another you will have all of your code that is able to target both net full and netcore. You can rewrite the entire class or you can use the same class and use #if NETFULL inside each method, I prefer the first approach but this is what happens when I’m starting editing the netstandard version of the class

image

Figure 5: No highlight and no intellisense because we are in a conditional compilation branch that evaluates to false.

Ouch, since we are in a branch of conditional compilation that evaluate to false, VS does not offer highligh, no intellisense, everything is greyed out. At this point you should be aware that, when you use multitargeting, the order of the frameworks in the project file matters. The first framework in <TargetFrameworks> node is the one that VS would use to evaluate conditional compilation. This means that, when you are working on classes or piece of code that should target both full framework and NET standard, you need to change the order to fit your need.

In this example I needed to change <TargetFrameworks>net461;netstandard2.0;</TargetFrameworks> to <TargetFrameworks>netstandard2.0;net461</TargetFrameworks>, save project file and unload and reload the project (sometimes you need to force a project reload) and Visual Studio will consider netstandard2.0 during code editing.

image

Figure 6: Reversing the order of the frameworks, will enable intellisense for netstandard branch of the conditional compilation directive.

Now you can edit the netstandard version of your class and convert one by one all parts of the code that does not natively run on netstandard.

Happy coding.

Converting a big project to new VS2017 csproj format

Visual Studio 2017 introduced a new .csproj file format for C#  project that is the key to move to .NET Standard but it is useful also for project with Full Framework, because is more human manageable.

The main drawback of this approach is that you end compatibility with older version of VS, if you open the .csproj with VS2015 you are not able to compile the project anymore. For this reason, switching to new format should be a decision that is well discussed in the team.

New project format is really human friendly, the only drawback is losing compatibility with older VS.

Luckily enough, when you have a solution with 68 project, you do not need to do all the work manually thanks to a nice tool found on GitHub, CsprojToVs2017. I’ve done manual conversion in the past, it is time consuming and frustrating, the above tools is not perfect but it does most of the work for you.

First of all I’ve run the tool to do a conversion of the entire solution without creating backup files (why should you take a backup when you can do all the test in a Git branch isolated from the rest of the code?).

csproj-to-2017 --no-backup .\mysln.sln

The tool output lots of information, warning and other suggestion, I simply stored in a file as reference but I immediately opened the solution to verify that it compiles successfully and the answer is no, but most of the work was already done for me.

Automated tool can do most of the work for you, but you will always do some manual work to finish and fine tune the conversion.

First of all the tool remove all the information of assembly from assemblyinfo.cs file, and in my situation it generates an error because I have AssemblyInformationalVersion in my assemblyinfo.cs, value of this attribute is “localCompile” and here what the tools generates.

image

Figure 1: Incorrectly Version generated by conversion tool

I still prefer all version information to be stored inside assemblyinfo.cs, so I removed all the version information from all .csproj file, added the attribute <GenerateAssemblyInfo>false</GenerateAssemblyInfo> to avoid automatic generation of assemblyinfo and finally reverted all the modification done to all assemblyinfo.cs to revert to previous content.

Then I start having a strange error:

Assets file …\obj\project.assets.json' not found. Run a NuGet package restore to generate this file

This error can be solved simply running a dotnet restore and in this particular situation it probably happens because in the sln file there are still a couple of web project that were not converted because web project still should use the old format. If we removed those two project from the solution everything compiles correctly, probably VS2017 got confused when a solution mixed the two types of project. The funny thing is that this error did not happened to every member of the team and happens only when we switched branch from a branch with the old format to the new format. This is usually not a big problem, when we switch branch we should only issue a git clean –xdf to clean every file not in source control, dotnet restore and we are ready to go. Now that we do not have active branches with the old format, this issue is gone away.

Then we start having some compilation errors because the new project formats includes automatically all .cs files inside project folders and during the years, some files were probably removed from the project , but they are still in the file system as you can see in following picture, where we have in the left the converted project to the right the original one.

image

As you can verify the file ConfigurationManagerProcessManagerConfiguration is present in file system but it is excluded from the project (right part of the picture) and it is incorrectly included by the new project format.

To solve this problem you should open side by side converted version of the project and original version and manually remove all cs files that are still in source code but are not referenced by the old project format, to avoid them to be incorrectly included after conversion.

One nice side effect of the conversion is that we found a bunch of files still in source control but not included in the solution

Then we have problem with Post Build Action, because we use Pre and Post build action to xcopy some files around but with the new format it seems not to work anymore. It turns out that in post build and pre build action some of the properties, as $(ConfigurationName) were not correctly populated, thus all the post and pre build actions generate error. The solution is really simple, convert them to a standard MsBuild Target as seen in the following snippet, where you can see the correct way to use xcopy and the original code that gets converted.

image

Conversion is really simple, original part is commented in the lower part of previous snippet, and it is composed by a series of xcopy command inside a PreBuildEvent. To made it work with the new csproj format the solution is adding a Target with a BeforeTarget=”PreBuildEvent” that will be run before the pre build event, then add a Exec task for each xcopy instruction. Remember also that the output folder is not bin/debug but you need to append the net461 (framework version).

The whole conversion took almost a couple of  hours for a 68 project solution and thanks to the automatic conversion tool, most of the work was automated, we only need to tweak a little bit project files and solves some problems due to the fact that this is a 5 years old solution.

This conversion forces also you to review your build in VSTS or whatever build system you are using. I encountered a couple of modification to do to every build.

Once you switch to the new project version it is time to check the all automated builds.

  1. First of all I need to be sure that the version of NuGet used to restore dependencies should be set at minimum to 4.6. to be sure of the NuGet version used, you should use the couple of task  NuGet Tool Installer plus Nuget, as I described in a previous post.

image

If you are using a wrong nuget version, probably the build solution will fail with an error like: Assets file ……\project.assets.json’ not found. Run a NuGet package restore to generate this file.

Another problem is due to switching between a branch that still uses old project format and branch that use the new format. It is imperative that obj folders are cleared because building again, or the build probably will fail. To accomplish this, I simply decide to clear everything before a get sources. (This was related to the issue I explained before.)

image

Symptoms of not clearing the obj folder are weird error like: Your project file doesn’t list ‘win’ as a “RuntimeIdentifier”. You should add ‘win’ to the “RuntimeIdentifiers” property in your project file and then re-run NuGet restore.

The overall process went smooth, and now we are ready to start the migration to dotnetcore.

Gian Maria.

Test Explorer in VS 15.6

The latest version of VS introduces some new cool feature, and it is quite big (almost 1 GB of download). If I should give you a single reason to upgrade to this new version, is the new Test Runner that now has the ability to show tests hierarchically

image

Figure 1: Hierarchical view in action in Test Explorer.

Since I’m a great NUnit fan, I always like the ability to see my tests following the namespace structure of my test projects, this is surely much more better than a flat list. Actually I can detatch test runner from VS IDE and keep it on my second monitory to have a quick look at my test outcome.

The full list of bugfix and improvements can be found here.

Gian Maria

Nuget packages for TFS / VSO Client Object Model

Finally the Client Object Model for TFS / VSO is finally distributed with Nuget Packages, as you can read here. This is a great news especially because the Dll are finally redistributable, and your tool does not need to require a previous installation of Visual Studio or Team Explorer, or Client Object Model Package.

Another interesting fact, is that REST API are now supported for TFS 2015 / VSO (previous version of TFS does not support REST API). If you have traditional application that uses Client Object Model, you can remove all the references to the old dll and directly reference the ExtendedClient Package and you are ready to go.

 

Gian Maria.

Unable to debug dll source code with symbol server

I’ve blogged in the past about using a Symbol server and I recommend to all people to use symbol servers whenever possible, to helping people troubleshooting problem on dependencies. Basically with a symbol server you can reference a dll in your project, but you can debug original source code as if you have the original project linked instead of having the dll.

Sometimes this process just don’t work. Recently I’have a customer that had problem with this scenario, and the real strange stuff is: I’m able to step in dll source code without problem from any machine, but noone of the customer’s developers are able to make it work. After I’ve sent them detailed instruction it worked, and we were able to track down the problem.

Visual Studio has a nice option to cache symbols in local directory to avoid downloading each time from the server. Here are my usual settings.

Visual Studio options to use local folder as a symbol cache

Figure 1: Visual Studio symbols settings

Developers in customer sites decided to use a subfolder of %TEMP% directory and this was the cause. As soon as they moved symbol cache to something like c:\symbols everything starts working. The underling cause is probably due to long paths.

If you have problem using symbol server, try using a really short path for your Symbols Local Cache directory.

In this specific situation we are using free symbols server in conjunction with MyGet nuget package feeds. In my machine here is the location for a source file during debugging.

Z:\Symbols\src\pdbsrc\MyGet\alkampfer\11111111-1111-1111-1111-111111111111\BIGEND\GianMariaRicci\Jarvis.Framework.Kernel\2C5AB5CBD1C74688974B2DDB55F51EDA1\Jarvis.Framework.Kernel\ProjectionEngine\ConcurrentProjectionEngine.cs

Usual %TEMP% variable is something like c:\users\gianmaria.ricci\appdata\local\temp (this is my system and it is long 44 characters), so it is not a good idea to use it for symbol source cache.

Since it is really easy to have really long path for your source when you use a symbol server, it is always a good idea using a short path for symbols cache directory, something like x:\SymSrc is probably the best solution.

If this does not solves your problem, another suggestion is using Fiddler to inspect the traffic between your Visual Studio and the Source Server to understand what is happening.

Gian Maria.