Continuous Integration in GitHub Actions, deploy in AzureDevops

My dear friend Matteo just published an interesting article on integration between GitHub actions and Azure Devops Pipeline here. I have a different scenario where I’ve already published a GitHub release from a GitHub action, but I have nothing ready in GitHub to release in my machines.

While GitHub is really fantastic for source code and starts having a good support for CI with Actions, in the release part it still miss a solution. Usually this is not a problem, because we have Azure DevOps or other products that can fill the gap.

This specific project is a firewall that closes every port in a machine and listen on UDP ports for a specific message to open other ports, thus, a machine where the service is installed has no way to be contacted unless you use the appropriate client to ask for port opening. I want the deploy to be automatic, no way I’m going to all my machines, login in RDP and then update the service, everything should happen automatically.

The really nice aspect of Azure DevOps release pipelines, is that, once you installed agents in one or more machines, those machines will contact Azure DevOps and pull works to do without the need for the machine to be contacted from outside world.

This is a key point of Azure DevOps release pipeline, you do not need to have any special setup in deploy target, you should simply let the target to be able to contact Azure DevOps site (https://dev.azure.com).

Another nice aspect of Azure DevOps release pipeline, is that it can use many sources for artifacts, not only those produced by Azure DevOps CI pipeline. When you add an artifacts, you can choose a GitHub release as well as Jenkins and other CI providers like Azure Artifacts (check Matteo’s article to see how to publish in azure artifacts from a GitHub Action)

image

Figure 1: Choose GitHub release as artifact source

To use GitHub as source you should have already connected your azure DevOps to GitHub with a service connection, another cool feature of Azure DevOps. As an administrator you can connect Azure DevOps account to GitHub, then give permission to specific people to use that service connection, without requiring them to know the real credentials to connect to the service (Github in this example). Once you have one or more connection active you can simply choose the repository to use. In Figure 2 You can see the configuration I choose for my project.

image

Figure 2: Configure GitHub release as artifact source.

Settings are: Repository (1), default version of the release to use (2) and finally alias you use for that specific artifact in your release (3). Remember that a release can have more than a single artifact as source, if you have a simple project like this, probably you have a single artifact.

Now you have the full power of Azure DevOps pipeline at your fingertips, in this specific example I just need to deploy a Windows Service and this is the pipeline to release in my stages.

image

Figure 3: Release pipeline for a Windows Service

This is a standard  four phase for a service release, first step is needed to stop the service if it is running, then I extract the artifacts coming from GitHub as 7zipped files, then I overwrite the directory where I’ve installed the service and finally I install the service if needed and restart it.

Before launching the release, you need to be sure that you have at least one release associated to that repository, in this example I have release 0.4.1 and others available.

image

Figure 4: Available releases for my GitHub repository

When you create a release (if the release is launched manually) you can choose GitHub release you want to use (if the release is automatic it will use release configured in the artifact configuration, usually latest one), the connection is done by Azure DevOps for you, no need to know credentials of GitHub, just choose the version you want to install and Bam, you are ready.

image

Figure 5: Choose the release you want to use directly from Azure DevOps

When the release starts, your target will download the workflow, it will instruct the agent to download artifacts from GitHub and then your scripts will run releasing the software.

image

Figure 6: Release completed, version 0.4.1 is now released on my machines.

As you can verify from detail page, artifacts are indeed downloaded by a GitHub standard release.

image

Figure 7: Artifacts downloaded directly from GitHub.

If everything runs successfully, you will have the new version installed on all machines part of deployment group used.

image

Figure 8: All steps executed successfully.

As you can see, Azure DevOps has a real powerful way to connect other services like GitHub and this is ideal to compensate the gap that other tools has at the moment. This leaves you free to compose your tooling chain, using the service that is best for the specific part.

Gian Maria.

Sample report for Azure DevOps

Reporting was always a pain point in Azure DevOps, because people used on SQL Server reporting Services for the on-premise version, missed a similar ability to create custom reports in Azure Dev Ops.

Now you have a nice integration with Power BI and a nice article here that explains how to connect Power BI to your instance and create some basic query. The nice part is that you can use a query that will connect directly with the OData feed, no need to install anything.

Power BI - Advanced Editor - Replace strings in query

Figure 1: Sample OData query that directly connects to your organization account to query for Work Item Data.

I strongly encourage you to experiment with Power BI because is a powerful tool that can create really good report, closing the gap with on-premise version and old Reporting Services.

Gian Maria.

Azure DevOps is now 150 sprints old

I remember old days when Azure DevOps was still in private preview, and yet it was really a good product, now 150 sprints passed, and the product is better than ever. Not everything is perfect, but, as users, we can expect new feature to being deployed each 3 weeks, the duration of Microsoft Sprint.

This means that now the product is 450 Weeks old, and finally we got a little nice feature that shows up news in the front page.

image

Figure 1: Widget with new feature of newest deployed sprint

This allows users to immediately being notified of new feature in their accounts, with a nice summary with key new features. In this sprint we have the new Task Assistant to help editing YAML pipelines, and many new feature, like the new agent administration ui.

image

Figure 1: New administration page in action.

The new page is more consistent with the look and feel of the rest of the service, also it shows wait time and build duration when you drill down in a pool.

As always I cannot stress out how good is to have all of your project administration tool in the Cloud, no time spent to upgrade, no time spent to verify and check backup policies, and, completely free for the first 5 users.

Gian Maria.

Troubleshoot YAML Build first run

Scenario: You create a branch in your git repository to start with a new shiny YAML Build definition for Azure Devops, you create a yaml file, push the branch in Azure Devops and Create a new Build based on that YAML definition. Everything seems ok, but when you press the run button you got and error

Could not find a pool with name Default. The pool does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz.

image

Figure 1: Error running your new shiny pipeline

Ok this is frustrating and following the link gives you little clue on what really happened. The problem is that, with the new editor experience, when you navigate to the pipeline page, all you see is the editor of YAML build and nothing more.

image

Figure 2: New Editor page of YAML pipeline, advanced editor and nothing more.

The new editor is fantastic, but it somewhat hides standard configuration parameters page, where the default branch can be set. As you can see from Figure 2 you can specify pool name (default) and triggers directly in YAML build so you think that this is everything you need, but there is more. Clicking on the three buttons in the right upper corner you can click on the trigger menu to open the old editor.

image

Figure 3: Clicking on the Triggers menu item will bring on the old UI

This is where the YAML pipeline experience still needs some love, you are surely puzzled why you need to click triggers menu item if you already specified triggers directly in the YAML definition, but the reason is simple, it will open the old pipeline editor page.

The new editor page with YAML editor is fantastic, but you should not forget that there are still some parameters, like default branch, that are editable from the old interface

Trigger page is not really useful, it only gives you the ability to override the YAML configuration, but the important aspect is that we can now access the first tab of the YAML configuration to change default branch.

image

Figure 4: Trigger page is not useful, but now we can access default configuration for the pipeline.

image

Figure 5: Default configuration tab where you can edit default branch

In Figure 5 you can now understand what went wrong, the wizard created my pipeline using master as default branch, but clearly my buid YAML file does not exists in master, but exists only in my feature branch. Yust change the default build to the branch that contains your build definition file, save and queue again; now everything should word again.

This trick works also when you got errors not being authorized to use endpoints, like sonar endpoint, nuget endpoint etc.

Happy YAML Building experience.

Gian Maria.

YAML Build in Azure DevOps

I’ve blogged in the past about YAML build in azure DevOps, but in that early days, that kind of build was a little bit rough and many people still preferred the old build based on visual editing in a browser. One of the main complaint was that the build was not easy to edit and there were some glitch, especially when it is time to access external services.

After months from the first version, the experience is really improved and I strongly suggest you to start trying to migrate existing build to this new system, to take advantage of having definition of build directly in the code, a practice that is more DevOps oriented and that allows you to have different build tasks for different branches.

YAML Build is now a first class citized in Azure DevOps and it is time to plan switching old build to the new engine.

You can simply start with an existing build, just edit it, select one of the phases (or the entire process) then press View YAML button to grab generated YAML build.

SNAGHTML2fecbc

Figure 1: Generate YAML definition from an existing build created with the standard editor

Now you can simply create a yaml file in any branch of your repository, paste the content in the file, commit to the branch and create a new build based on that file. I can clearly select not only AzDO repositories, but I can build also GitHub and GitHub enterprise

image

Figure 2: I can choose GitHub as source repository, not only azure repos

Then I can choose the project searching in all the project I have access to with my access token used to connect GitHub

image

Figure 3: Accessing GitHub repositories is simple, once you connected the acount with an access token AzDO can search in repositories

Just select a repository and select the option Existing Azure Pipelines, if you are starting from scratch you can create a starter pipeline.

image

Figure 4: Choose the option to use an existing pipeline.

You are ready to go, just choose branch and yaml file and the game is done.

image

Figure 5: You can directly specify the build file in the pipeline creation wizard.

Converting an existing build pipeline to YAML it is matter of no more than 10 minutes of your time.

Now you can simply run and edit your build directly from the browser, the whole process took no more than 10 minutes, including connecting my AzDO account to GitHub

image

Figure 6: Your build is ready to run inside Azure DevOPS.

Your build is now ready to run without any problem. If you specified triggers as in Figure 6 you can just push to the repository to have the build automatically kicks in and being executed. You can also directly edit the build in the browser, and pressing the Run button (Figure 6) you can trigger a run of the build without the need to push anything.

But the coolness of actual YAML build editor starts to shine when you start editing your build in the web editor, because you have intellisense, as you can see in Figure 7.

image

Figure 7: YAML build web editor has now intellisense.

As you can see the YAML build editor allows you to edit with full intellisense support, if you want to add a task, you can simply start writing task followed by a semicolon and the editor will suggest you all the tasks-available. When it is time to edit properties, you have intellisense and help for each task parameters, as well as help for the entire task. This is really useful because it immediately spots deprecated tasks (Figure 9)

image

Figure 8: All parameters can be edited with fully intellisense and help support

image

Figure 9: Helps gives you helps for the deprecation of old task that should not be used.

With the new web editor with intellisense, maintaining a YAML build is now easy and not more difficult than the standard graphical editor.

Outdated tasks are underlined in green, so you can immediately spot where the build definition is not optimal, as an example if I have a task that have a new version, the old version is underlined in green, and the intellisense suggests me that the value is  not anymore supported. This area still need some more love, but it works quite well.

There is no more excuses to still use the old build definition based on web editor, go and start converting everything to YAML definition, your life as bulid maintainer will be better :)

Gian Maria.