Deploy click-once application on Azure Blob with Azure DevOps

It was a long time ago I blogged on how to publish a click-once application from a VSTS Build to Azure Blob, long time was passed, and lots of stuff changed. The whole process is now simpler, thanks to many dedicated tasks that avoid doing any manual work.

My new build always start with a  GitVersion custom tasks, that populates some environment variables with version numbers generated by GitVersion, this will allow me to simply add an MsBuild task in the build to publish click-once using automatic GitVersion versioning.

image

Figure 1: MsBuild task to publish click once application

You need to build the csproj that contains the project, using $(BuildConfiguration) to build in the right configuration and adding custom argument arguments (3) as the following list

/target:publish 
/p:ApplicationVersion=$(AssemblyVersion)  
/p:PublishURL=https://aaaaaaa.blob.core.windows.net/atest/logviewer/  
/p:UpdateEnabled=true  
/p:UpdateMode=Foreground 
/p:ProductName=Jarvis.LogViewer 

ApplicationVersion is set to variable $(AssemblyVersion) that was populated by GitVersion task, publish Url is a simple address of an azure blob storage, that contains a blob container called atest, with a  folder named logviewer that has public read access.

image

Figure 2: Public read access for a simple blob in my azure subscription

This will allows me to have a simple public blob, with a public address, that everyone can read, so everyone can download my application.

Azure blob can be given a public read access to everyone, making your click-once application available to everyone.

MsBuild task simply creates a app.publish subfolder that contains all the files that needs to be copied into the blob azure storage, the first step is a Copy file Task to copy them in the artifacts directory, as for the following picture.

image

Figure 3: All files of click-once publishing were copied in the artifacts staging directory.

Now everything is ready, just add a final task of type Azure File Copy and configure to copy everything from artifacts directory to the right subfolder of azure blob.

SNAGHTML18f3d81

Figure 4: Configuration of Azure Blob File to deploy click-once generated files.

Configuration is really simple, because thanks to a direct connection to Azure Subscription (3) we can simply connect Azure Dev Ops to an azure subscription so we can simply select storage account (4) blob name (5) and subfolder of the blob (6).

One of the advantage of the dedicated Azure File Copy Task, is that it can simply use one of  the Azure account linked to the Azure DevOps account, making super-simple to choose right blob where to upload click-once package.

Once you’ve finished configuration, you can simply run a build and your application will be automatically published.

image

Figure 5: Summary of the build with click once publishing.

Now you can simply point to the setup.exe file with the address of the blob and the game is done.

Gian Maria.

Run code coverage for Python project with Azure DevOps

Creating a simple build that runs Python tests written with PyTest framework is really simple, but now the next step is trying to have code coverage. Even if I’m pretty new to Python, having code coverage in a build is really simple, thanks to a specific task that comes out-of-the-box with Azure DevOps: Publish Code Coverage.

In Azure DevOps you can create build with Web Editor or with simple YAML file, I prefer YAML but since I’ve demonstrated in the old post YAML build for Python, now I’m creating a simple build with standard Web Editor

Instead of creating a Yaml Build, this time I’m going to demonstrate a classic build: here is the core part of the build.

image

Figure 1: Core build to run tests and have code coverage uploaded to Azure DevOps

As you can see, I decided to run test with a Bash script running on Linux, here is the task configuration where I’ve added Pytest options to have code coverage during test run.

image

Figure2: Configuration of Bash script to run Pytests

The task is configured to run an inline script (1), command line (2) contains –cov options to specify Python modules I want to monitor for code coverage, then a couple of –cov-report options to have output in xml and HTML format. Finally I’ve specified the subfolder that contains the module I want to test (3) and finally I’ve configured the task con Continue on Error (4), so if some of the tests fails the build will be marked as Partially failed.

Thanks to Pytest running code coverage is just a matter of adding some options to command line

After a build finished you can find in the output how Pytest generates Code Coverage reporting, it create a file called coverage.xml then an entire directory called htmlcov that contains a report for code coverage.

image

Figure 3: Result of running tests with code coverage.

If you look at Figure 1 you can see that the build final task is a Publish Code Coverage Task, whose duty is to grab output of the Pytest run and upload to the server. Configuration is really simple, you need to choose Cobertura as Code coverage tool (the format used by Pytest) and the output of test run. Looking at output of Figure 3 you can double check that the summary file is called coverage.xml and all the report directory is in htmlcov subdirectory.

image

Figure 4: Configuration for Publish Code Coverage task.

Once you run the build, you can find Code Coverage result on the summary page, as well as Code Coverage Report published as Build artifacts, the whole configuration will take you no more than 10 minutes.

image

Figure 5: Artifacts containing code coverage reports as well as code coverage percentage are accessible from Build Summary page.

Finally you have also a dedicated tab for Code Coverage, showing the HTML summary of the report

image

Figure 6: Code coverage HTML report uploaded in a dedicated Code Coverage tab in build result

Even if the code coverage output is not perfectly formatted you can indeed immediately verify percentage of code coverage of your test.

Gian Maria.

Run Python test with Azure DevOps pipeline

The beauty of Azure DevOps is it support to many technologies and all of major language.s I have a simple git repository where I’m experimenting Python code, in that repository I have several directories like 020_xxxx 010_yyy where I’m playing with Python code.

Each folder contains some code and some unit tests written in Pytest, my goal is creating an Azure Pipeline that can automatically run all pytest for me automatically each time I push some code to the repository.

Even if Phyton is a script languages, it has several Unit Testing frameworks that can be used to verify that the code you wrote works as expected

Creating a build in Azure DevOps is really simple, just create a build that points to a yaml file in your repository that contains the definition.

queue:
  name: Hosted Linux Preview

trigger:
- master
- develop
- features/*
- hotfix/*
- release/*

steps:

- task: UsePythonVersion@0
  displayName: 'Use Python 3.x'

- bash: |
   pip install pytest 
   pip install pytest-cov 
   pip install pytest-xdist 
   pip install pytest-bdd 

  displayName: 'Install a bunch of pip packages.'


- bash: 'pytest --junitxml=$(Build.StagingDirectory)/010.xml' 
  workingDirectory: '010_CrackingCodeBasic'
  displayName: 'Run test 010'

- bash: 'pytest --junitxml=$(Build.StagingDirectory)/020.xml' 
  workingDirectory: '020_CrackingIntermediate'
  displayName: 'Run test 020'

- task: PublishTestResults@2
  displayName: 'Publish test result /010.xml'
  inputs:
    testResultsFiles: '$(Build.StagingDirectory)/010.xml'
    testRunTitle: 010

- task: PublishTestResults@2
  displayName: 'Publish test result /020.xml'
  inputs:
    testResultsFiles: '$(Build.StagingDirectory)/020.xml'
    testRunTitle: 020





This is a real simple yaml buid definition where I’m simply requiring the usage of python 3.x, then install some packages with pip and finally a series of pytest tests run for each folder. As you can see I specified also the trigger to automatically build all branches typical of GitFlow naming convention.

The trick to have the result of the tests published directly in your build result is using a Pytest option to create a result file with JUNIT xml file format; once you have test result as a JUNIT xml files you can use standard PublishTestResults task to publish results in the build.

After the build completed you can simply looks at the output, if everything is ok the build is Green.

image

Figure 1: Test results in my build that shows results of my python unit tests.

The build will run all python tests in all of my source code folder, thanks to Pytest that does everything for you, both discovering and run all tests in the folder.

If code does not compile, unit test will fail and you have a failing build.

image

Figure 2: Not compiling code will lead to a failing build.

The problem with this approach is that the build stops at the very first error, so if an error happens in 010 directory my 020 directory will not be tested because at the very first failed test the execution of the build stopped.

This condition is usually too strict, for unit testing it is usually a better approach to configure the build to continue even if run test failed. To accomplish this, just add continueOnError: true after each bash task used to run tests. With continueOnError equal to true, the build will continue and if some of the test task fails, the build is marked as Partially Failed, and from the summary you can easily check the run that generates the error.

image

Figure 3: Continue on error true will make the build continue on test error, in the summary you can verify what failed,

The reason why I choose to launch a different Pytest run for each folder and upload each result with a different task is to have separate run in build result.

image

Figure 4: Test runs are distinct for each folder.

Even if this will force me to add two task for each folder (one for the run and the other for the publish) this approach will give me a different result for each folder so I can immediately understand where is the error.

Gian Maria.

Analyze your GitHub project for free with Azure DevOps and SonarCloud

I’ve blogged some weeks ago on how to analyze OS code with SonarCloud, but it is time to update the post, because if you want to use SonarCloud you have a dedicated extension in the marketplace.

image 

Figure 1: Official SonarCloud extension in the marketplace.

One of the great feature of Azure DevOps is its extendibility, that allows people external to Microsoft to create extensions to expand the possibility of the tool. Once you’ve added the SonarCloud extension to your account, you have a whole bunch new build templates you can use:

image

Figure 2: Build template based on Sonar Cloud

Having a template make super easy to create a build, you just choose .NET Desktop with SonarCloud and you are ready to go. As you can see in Figure 2 you can also use Azure DevOps pipeline to build with Gradle, maven or .NET core, so you are not confined to microsoft tooling.

In Figure 3 there is the build created by .NET desktop project template (remember that this template can be used also for web application, and for every .NET application).

image

Figure 3: .NET Sonar Cloud analysis template.

The only task you need to configure for Sonar Cloud analysis is the Prepare analysis on Sonar Cloud. As you can see in Figure 4, you should first create an endpoint that connect Azure DevOps to your SonarCloud account.

SNAGHTML1d1ca3

Figure 4: In task configuration you have a nice button to create the connection to your SonarCloud account

Configuring the connection is really simple, just give a name to the connection and specify the access token (you should first generate a token in SonarCloud). Then, as shown in Figure 5, press Verify Connection to check that everything is ok.

image

Figure 5: Configuration and test of the connection between Azure DevOps and SonarCloud.

Thanks to the concept of external services, you can configure one or more connection to SonarCloud and having it available in the build without disclosing tokens.

Once you’ve selected the connection, just specify name and key of the project, and other optional parameters if you need to do a custom analysis. In less than a couple of minutes you have a build up and running. Just configure the agent to use Hosted VS2017 pipeline and queue a first build to verify that everything is ok.

Once you have configured the build with the visual web designer, you can convert to Yaml build with few steps.

Clearly I prefer to have a YAML build for a lot of reasons, once the build is up and running simply press the YAML button in the build definition to have your build converted to YAML.

# .NET Desktop
# Build and run tests for .NET Desktop or Windows classic desktop solutions.
# Add steps that publish symbols, save build artifacts, and more:
# https://docs.microsoft.com/azure/devops/pipelines/apps/windows/dot-net

pool:
  vmImage: 'VS2017-Win2016'

trigger:
- master
- develop
- release/*
- hotfix/*
- feature/*

variables:
  solution: 'migration/MigrationPlayground.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:

- task: GitVersion@1
  displayName: GitVersion 
  inputs:
    BuildNamePrefix: 'MigrationCI'

- task: SonarSource.sonarcloud.14d9cde6-c1da-4d55-aa01-2965cd301255.SonarCloudPrepare@1
  displayName: 'Prepare analysis on SonarCloud'
  inputs:
    SonarCloud: 'SonarCloud'
    organization: 'alkampfergit-github'
    projectKey: MigrationPlayground
    projectName: MigrationPlayground
    projectVersion: '$(AssemblyVersion)'

- task: NuGetToolInstaller@0

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: SonarSource.sonarcloud.ce096e50-6155-4de8-8800-4221aaeed4a1.SonarCloudAnalyze@1
  displayName: 'Run Code Analysis'

- task: SonarSource.sonarcloud.38b27399-a642-40af-bb7d-9971f69712e8.SonarCloudPublish@1
  displayName: 'Publish Quality Gate Result'




Finally, if you still have not installed Azure Devops Pipeline in your GitHub account, I strongly suggest you to do so, just follow the instruction of this article, it is free and gives you free hosted pipelines to run your build for free.

Gian Maria

Azure DevOps pipelines and Sonar Cloud gives free analysis to your OS project

In previous post I’ve shown how easy is to create a YAML definition to create a build definition to build your GitHub Open Source project in Azure DevOps, without the need to spend any money nor installing anything on you server.

Once you create a default build that compile and run tests, it would be super nice to create a free account in SonarCloud to have your project code to be analyzed automatically from the Azure Pipeline you’ve just created. I’ve already blogged on how to setup SonarCloud analysis for OS project with VSTS build and the very same technique can be used in YAML build.

Once you have free YAML Azure DevOps pipeline, it makes sense to enable analysis with SonarCloud

First of all you need to register to SonarCloud, create a project, setup key and create a token to access the account. Once everything is in place you can simply modify YAML build to perform the analysis.

image

Figure 1: Task to start sonar cloud analysis.

The above task definition can be obtained simply creating a build with standard graphical editor, then press the YAML build to have the  UI generate the YAML for the task.

Actually YAML build does not have an editor, but it is super easy to just create a fake build with standard editor, drop a task into the definition, populate properties then let the UI to generate YAML that can be copied into the definition.

Once the analysis task is in place, you can simply place the “Run code analysis task” after build and test tasks. The full code of the build is the following.

# .NET Desktop
# Build and run tests for .NET Desktop or Windows classic desktop solutions.
# Add steps that publish symbols, save build artifacts, and more:
# https://docs.microsoft.com/azure/devops/pipelines/apps/windows/dot-net

pool:
  vmImage: 'VS2017-Win2016'

trigger:
- master
- develop
- release/*
- hotfix/*
- feature/*

variables:
  solution: 'migration/MigrationPlayground.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:

- task: GitVersion@1
  displayName: GitVersion 
  inputs:
    BuildNamePrefix: 'MigrationCI'

- task: SonarSource.sonarqube.15B84CA1-B62F-4A2A-A403-89B77A063157.SonarQubePrepare@4
  displayName: 'Prepare analysis on SonarQube'
  inputs:
    SonarQube: 'SonarCloud'
    projectKey: xxxxxxxxxxxxxxxxxxx
    projectName: MigrationPlayground
    projectVersion: '$(AssemblyVersion)'
    extraProperties: |
     sonar.organization=alkampfergit-github
     sonar.branch.name=$(Build.SourceBranchName)

- task: NuGetToolInstaller@0

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: SonarSource.sonarqube.6D01813A-9589-4B15-8491-8164AEB38055.SonarQubeAnalyze@4
  displayName: 'Run Code Analysis'




Once you changed the build just push the code and let the build run, you should check if the build completes without error, then verify if analysis is present in SonarCloud dashboard.

A couple of suggestion are useful at this point: first of all you can encounter problem with endpoint authorization, if you have such problem check this link. Another issue is that you should analyze master branch for the first analysis for SonarCloud to work properly. Until you do not analyze master branch, no analysis will be shown to SonarCloud.

If everything is green you should start seeing analysis data on SonarCloud UI.

image

Figure 2: Analysis in SonarCloud after a successful master build

As you can see just a few lines of YAML and I have my code automatically analyzed in SonarCloud, thanks to Azure DevOps pipelines that already have tasks related to SonarCube integration.

A nice finishing touch is to grab the badge link for SonarCloud analysis and add it to your github readme.md.

image

Figure 3: SonarCloud badge added to readme.md of the project.

Gian Maria.