Sonar Analysis of Python with Azure DevOps pipeline

Once you have test and Code Coverage for your build of Python code, last step for a good build is adding support for Code Analysis with Sonar/SonarCloud. SonarCloud is the best option if your code is open source, because it is free and you should not install anything except the free addin in Azure Devops Marketplace.

From original build you need only to add two steps: PrepareAnalysis onSonarCloud and Run SonarCloud analysis, in the same way you do analysis for a .NET project.

image

Figure 1: Python build in Azure DevOps

You do not need to configure anything for a standard analysis with default options, just follow the configuration in Figure 2.:

image

Figure 2: Configuration of Sonar Cloud analysis

The only tricks I had to do is deleting the folder /htmlcov created by pytest for code coverage results. Once the coverage result was uploaded to Azure Devops server I do not needs it anymore and I want to remove it from sonar analysis. Remember that if you do not configure anything special for Sonar Cloud configuration it will analyze everything in the code folder, so you will end up with errors like these:

image

Figure 3: Failed Sonar Cloud analysis caused by output of code coverage.

You can clearly do a better job simply configuring Sonar Cloud Analysis to skip those folder, but in this situation a simple Delete folder task does the job.

To avoid cluttering SonarCloud analysis with unneeded files, you need to delete any files that were generated in the directory and that you do not want to analyze, like code coverage reports.

Another important settings is the Advances section, because you should specify the file containing code coverage result as extended sonar property.

image

Figure 4: Extra property to specify location of coverage file in the build.

Now you can run the build and verify that the analysis was indeed sent to SonarCloud.

image

Figure 5: After the build I can analyze code smells directly in sonar cloud.

If you prefer, like me, YAML builds, here is the complete YAML build definition that you can adapt to your repository.

queue:
  name: Hosted Ubuntu 1604

trigger:
- master
- develop
- features/*
- hotfix/*
- release/*

steps:

- task: UsePythonVersion@0
  displayName: 'Use Python 3.x'

- bash: |
   pip install pytest 
   pip install pytest-cov 
   pip install pytest-xdist 
   pip install pytest-bdd 
  displayName: 'Install a bunch of pip packages.'

- task: SonarSource.sonarcloud.14d9cde6-c1da-4d55-aa01-2965cd301255.SonarCloudPrepare@1
  displayName: 'Prepare analysis on SonarCloud'
  inputs:
    SonarCloud: SonarCloud
    organization: 'alkampfergit-github'
    scannerMode: CLI
    configMode: manual
    cliProjectKey: Pytest
    cliProjectName: Pytest
    extraProperties: |
     # Additional properties that will be passed to the scanner, 
     # Put one key=value per line, example:
     # sonar.exclusions=**/*.bin
     sonar.python.coverage.reportPath=$(System.DefaultWorkingDirectory)/coverage.xml

- bash: 'pytest --junitxml=$(Build.StagingDirectory)/test.xml --cov --cov-report=xml --cov-report=html' 
  workingDirectory: '.'
  displayName: 'Run tests with code coverage'
  continueOnError: true

- task: PublishTestResults@2
  displayName: 'Publish test result /test.xml'
  inputs:
    testResultsFiles: '$(Build.StagingDirectory)/test.xml'
    testRunTitle: 010

- task: PublishCodeCoverageResults@1
  displayName: 'Publish code coverage'
  inputs:
    codeCoverageTool: Cobertura
    summaryFileLocation: '$(System.DefaultWorkingDirectory)/coverage.xml'
    reportDirectory: '$(System.DefaultWorkingDirectory)/htmlcov'
    additionalCodeCoverageFiles: '$(System.DefaultWorkingDirectory)/**'

- task: DeleteFiles@1
  displayName: 'Delete files from $(System.DefaultWorkingDirectory)/htmlcov'
  inputs:
    SourceFolder: '$(System.DefaultWorkingDirectory)/htmlcov'
    Contents: '**'

- task: SonarSource.sonarcloud.ce096e50-6155-4de8-8800-4221aaeed4a1.SonarCloudAnalyze@1
  displayName: 'Run Sonarcloud Analysis'

The only settings you need to adapt is the name of the SonarCloud connection (in this example is called SonarCloud) you can add/change in Project Settings > Service Connections.

image

Figure 6: Service connection settings where you can add/change connection with Sonar Cloud Servers.

A possible final step is adding the Build Breaker extension to your account that allows you to made your build fails whenever the Quality Gate of SonarCloud is failed.

Thanks to Azure DevOps build system, creating a build that perform tests and analyze your Python code is extremely simple.

Happy Azure Devops.

Gian Maria

Run SonarCloud analysis in VSTS / TFS Build

Running a SonarQube analysis for TFS or VSTS is really easy because we can use a pre-made build tasks that requires few parameters and the game is done. If you have open source project it made lot of sense to use a public account in SonarCloud, so you do not need to maintain a sonar server on-premise and you can also share your public account with the community.

For open source projects, SonarCloud is available for you with zero effort and thanks to VSTS and TFS you can automate the analysis with few steps.

The first step is creating an organization in Sonar Cloud, if you prefer you can just login with your GitHub account and everything is ready. After the creation of the organization, you should create new project and generate a key to send analysis to SonarCloud server, everything is made with a simple wizard and it takes only a bunch of seconds to have your project created and ready to be used.

Once you have your project key and token  you need to add the endpoint of SonarCloud to the list of available endpoints. You only need to give the connection a name, specify https://sonarcloud.it as Server Url and add the token generated during project creation.

image

Figure 1: Configuration of sonar cloud endpoint

Now you can configure build to perform the analysis, the first task is the “prepare analysis Task”, as you can see in Figure 2. You should select the Endpoint created in previous step, fill the project key and project name, but you need also to specify a couple of properties in the advanced section . The first one is sonar.organization and it is required or the analysis will fail. This is the only difference from on-premise SonarQube server, where you do not need to add organization name.

image

Figure 2: Configuration of the prepare analysis task.

The other setting to be specified in Additional Properties is the sonar.branch.name, to perform branch based analysis, a feature that is available in sonarcloud and it is available on-premise only with enterprise version. You can simply use the $(Build.SourceBranchName) to use the current branch if you are using Git.

image

Figure 3: Analysis performed on NStore project with branch analysis enabled.

The cool part of the process is that, SonarCloud require zero installation time and less than one minute to create the first project and thanks to the VSTS / TFS build engine you can automate the analysis in less than 2 minutes.

Gian Maria.

Exclude Folders from SonarQube analysis

Creating a build that is capable of perform a SonarQube analysis on a VSTS /  TFS is a really simple task, thanks to the two tasks that are present out-of-the box.

image

Figure 1: Build that uses Sonarqube tasks to perform analysis

The problem in a project that was alive for more than a couple of years is that you usually have a really bad report when you do your first analysis. This happens because, without a constant analysis, the code have surely some smells.

Sometimes you get really discouraged, because the number of issue is really high, but before losing any hope, check if the errors are really part of your code. In a project where I’m working, we got really bad numbers and I was 100% sure that it is not a problem of our code.

When you analyze your project for the first time, sometimes the number of issue is so high that you really are discouraged. Before giving up, check if the errors are really part of your code.

To diagnostic the problem, simply login to the project, then go to Code view (something like http://build:9000/code/?id=projectName), then you will see a summary of all the bugs, but unfortunately you cannot order for the number of the bug, so Just scroll down to see the part of the code with the most errors.

SNAGHTML139609

Figure 2: 185 bugs are located in scripts folder

In this situation, we have 185 bugs signaled by the javascript analyzer under the scripts folder, but in that folder we have our code but also third party code. This is not the very first analysis, because the first analysis shows a folder where there are more than 3k of errors and it was the folder that stores all third party javascript libraries.

If you do not use npm it is quite normal to have third party javascript code in your source code and if you are using SonarQube you absolutely need to configure it to exclude that directories. Just go to the administration page of the project and in the Analysis Scope you find the Source File Exclusions section that allows you to exclude some directories from the analysis.

SNAGHTML16d611

Figure 3: Exclude folder for analysis

In your situation the vast majority of errors comes from the angular library, from all the script of the skin we use and for third party libraries stored under the /app/scripts/lib folder. After exclusion, the number of bugs dropped from almost 7k to 500.

If you add Sonarqube analysis to existsting project that have third party javascript code in source code repository, please always check where the errors are and exclude folder accordingly.

Gian Maria.

SonarQube UTF-8 error after upgrading

I’ve upgraded a SonarQube instance, and then, suddently I realyze that some builds start failing due to to a strange error in the SonarQube complete analysis task.

2016-12-09T11:41:00.8695497Z ##[error]WARN:
 Invalid character encountered in file C:\vso\_work\3\s\src\Jarvis.ConfigurationService.Client.CastleIntegration\Properties\AssemblyInfo.cs at line 25 for encoding UTF-8.
 Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.

Thiserror in turns made the entire task execution failing with a really strange error

2016-12-09T11:41:03.5135467Z ##[error]ERROR: Error during SonarQube Scanner execution
2016-12-09T11:41:03.5145430Z ##[error]java.lang.IllegalArgumentException: 5 is not a valid line offset for pointer. File [moduleKey=Jarvis.ConfigurationService:Jarvis.ConfigurationService:893ED554-3D86-47C8-B529-965329DB32AF, relative=Properties/AssemblyInfo.cs, basedir=C:\vso\_work\3\s\src\Jarvis.ConfigurationService.Client.CastleIntegration] has 1 character(s) at line 2
2016-12-09T11:41:03.5145430Z ##[error]at org.sonar.api.internal.google.common.base.Preconditions.checkArgument(Preconditions.java:145)
2016-12-09T11:41:03.5145430Z ##[error]at org.sonar.api.batch.fs.internal.DefaultInputFile.checkValid(DefaultInputFile.java:215)
2016-12-09T11:41:03.5145430Z ##[error]at org.sonar.api.batch.fs.internal.DefaultInputFile.newPointer(DefaultInputFile.java:206)

This is really annoying, but actually I start investigating the error logging to the build server and checking the file that generates the error. When you have strange encoding erorr I strongly suggested you to visualize the file with some Hex editor, and not a standard editor.

I immediately realized that the error is due to my GitVersion task script, because it manipulates assemblyinfo.cs files to change version numbers and save back the assemblyinfo.cs in UTF-16 encoding. If you do not know Byte Order Mark I strongly suggest you to take a look in wikipedia, because this is an important concepts for Unicode files.

I immediately checked with an hex editor, and verified that the original assemblyinfo.cs has a BOM for UTF-8 but the PowerShell script modified it converting to UTF-16 but the BOM is correct. The annoying stuff is that the build worked perfectly until I updated SonarQube (server + analyzers) this means that for some reason, the most recent version of Sonar somewhat does not check the BOM to understand file encoding.

I’ve solved the problem simply changing the Set-Content call to force UTF8 and the problem is gone.

Set-Content -Encoding UTF8 -Path $file.FullName -Force

I really like SonarQube, but you should always verify that everything works after every upgrade.

Gian Maria.

Upgrading SonarQube from 5.1 to 6.0

SonarQube is a really nice software, but for what I experienced it does not play well with Sql Server. Even if Sql Server is fully supported, there are always some little problem in setting everything up, and this is probably due to the fact that most of the people using SonarQube are using MySql as Database Engine.

Today I was upgrading a test instance from version 5.1 to 6.0, I’ve installed the new version, launched database upgrade procedure, and after some minutes the upgrade procedure stopped with a bad error

image

Figure1: Databse error during upgrade.

Remember that SonarQube upgrade procedure does not have a rollback procedure, so it is mandatory that you take a full backup of the system before performing the upgrade.

From what I’ve learned in previous situation, whenever something does not work in SonarQube, you need to look at the log. The above error message is really misleading, because it seems that there were some connection problems. Reading the log I discovered that the error is really different.

2016.08.11 19:24:00 ERROR web[o.s.s.d.m.DatabaseMigrator] Fail to execute database migration: org.sonar.db.version.v60.CleanUsurperRootComponents
com.microsoft.sqlserver.jdbc.SQLServerException: Cannot resolve the collation conflict between "SQL_Latin1_General_CP1_CS_AS" and "Latin1_General_CS_AS" in the equal to operation.

Whenever you have problem with SonarQube do not forget to read the log, because only in the log you can understand the real cause of errors.

Collation problem are quite common with SonarQube, documentation is somewhat not correct, because it tell you that you need to use an Accent and Case sensitive collation, but does not specify the collation. My SonarQube 5.1 instance worked perfectly with SQL_Latin1_General_CP1_CS_AS, but sadly enough, script to upgrade db to version 6.0 fails because it is expecting Latin1_General_CS_AS.

If you install SonarQube with Sql Server, it is better to chose Latin1_General_CS_AS as collation to avoid problems.

Luckly enough you can change database collation for an existing database, the only caveat is that you need to set the database in Single User Mode to perform this operation.

USE master;  
GO  
  
ALTER DATABASE Sonar SET SINGLE_USER WITH ROLLBACK IMMEDIATE; 
GO 

ALTER DATABASE Sonar  COLLATE Latin1_General_CS_AS ;  
GO 

ALTER DATABASE Sonar SET MULTI_USER; 
GO  

Clearly you should have a backup of your database before the migration, or you will end with a corrupted database and nothing to do. So I restored the database from the backup, run the above script, restart SonarQube and try to perform the upgrade again.

image

Figure 2: Database is upgraded correctly

My instance is now running 6.0 version.

Gian Maria.