AppVeyor – Continuous Integration for your .NET Core Applications

Today’s header image was created by Matthew Hamilton at Unsplash

This week we’re going to take a side step from pure development into DevOps work. We’re going to look at Continuous Integration (also known as CI).

This week’s topic isn’t necessarily .NET Core specific, but those of us who want our applications to be taken more seriously should really be looking into CI. As such, this week we’ll use a Web Service called AppVeyor to facilitate CI for our .NET Core repositories. But first, a little software Engineering theory.

psst, you can skip all the theory and jump straight to the good stuff by clicking here

Continuous Integration

In software engineering, continuous integration (CI) is the practice of merging all developer working copies to a shared mainline several times a day. Grady Booch first named and proposed CI in his 1991 method, although he did not advocate integrating several times a day. Extreme programming (XP) adopted the concept of CI and did advocate integrating more than once per day – perhaps as many as tens of times per day

Source: https://en.wikipedia.org/wiki/Continuous_integration

Before CI was codified, each developer who was working on a code base would integrate their changes (think of this as merging) with the changes from other developers on an infrequent basis.

Think back to your first group project as a young developer (maybe this was at school): Everyone came together to get an understanding of the basic design of the project and get a list of their tasks. After which, everyone would go away and write the code that they were tasked with.

After a few weeks, you’d all come together with USB memory sticks

or floppy disks, if you’re old enough to remember them

and it would be someone’s job to take the individual lumps of code and merge them together into some semblance of a rational code base.

We all did it, so there’s no point in lying to me.

I’m sure it wasn’t just me and the cohort that I studied with

This is the opposite of CI: there’s no common repository that everyone is committing code to and you cannot guarantee that the code you’re working on is in sync with the code everyone else is working on.

This is a very basic description of what CI is, for a more detailed description go read the Wikipedia article or head to the source: Grady Booch’s Object Oriented Analysis and Design with Applications 2nd Edition.

CI for .NET Core

Since the steps required to build a .NET Core project are relatively simple

dotnet build, remember?

we can make use of CI systems which give us console access to a containerised system or a virtual machine. One such system is AppVeyor.

Before we continue, AppVeyor is a paid service with a free tier specifically for Open Source projects. I’ve been using the free tier, since all of my .NET Core development is open source, but the pricing structure that they have seems pretty fair.

Looking at the documentation, it seems pretty simple:

• Sign up for an account
• Import a repo
• Add a .yml file to your repo
• Watch the magic happen

But there are a few caveats I fell prey to along the way, so let’s take a look at a worked example of how to set up AppVeyor with a .NET Core code base and how to get AppVeyor to do it’s thing.

Setting Up AppVeyor

I’m going to skip past creating an account, since that’s pretty cookie cutter stuff. Suffice to say, I OAuth’d using my GitHub account, since that’s where all my open source repos are.

Once you’ve created an account, you’ll be taken to your dashboard. It should look something similar to this:

Clicking on the “New Project” will take you to a screen where you can create a project from a repository hosted on a compatible service:

Hovering over the repository you wish you use, click on the “Add” button that will appear (shown in the above screenshot).

appveyor.yml

Now that we have the project set up, we need to add an appveyor.yml to the repository. This file informs AppVeyor of the steps required to build the project.

We could use the AppVeyor UI to set up the build pipeline, but I like to have things checked into source control. That way, if the AppVeyor project ever gets wiped (or has an issue), restoring it is extremely simple.

Caveat

There’s a caveat here, in that we can create the yml file and commit it into source control, but AppVeyor settings provided via the UI exist separate to the yml file.

By this I mean that the changes made to the settings in the AppVeyor UI will be stored as part of the AppVeyor project and not in source control. If AppVeyor detects a file in the root of your repository called appveyor.yml, it will parse the file and use that as it’s pipeline settings; otherwise, it will rely on us setting everything up in the UI.

Since we’re going to take the appveyor.yml route, if we need to change the build script, we’ll need to push a change into the repository. Which, as we’ll see in a moment, will cause a build to happen.

appveyor.yml – redux

Now that we know that the appveyor.yml file is separated from the settings supplied in the Appveyor UI, create a file in the root of your repository called appveyor.yml

you did clone a repository down, right?

Here’s a screen shot of my repository loaded with Visual Studio, with the appveyor.yml file selected:

Now paste the following code into it:

It’s pretty well commented, but let’s take a look at this file’s contents in chunks.

We’re doing some basic set up in these lines – what the version number scheme should be, which image to use

more on this in a moment – I’ve highlighted the relevant line

which branches to use, and what to do in the init stage.

image Property

The image property in the AppVeyor settings file tells AppVeyor which system image to use when building your repository. The software installed on these images is updated on a semi-regular basis, so it’s best to check the AppVeyor documentation for Build Environments and the Platform Updates pages to see what software is installed on which image.

At the time of writing, the following .NET platforms are installed across the three AppVeyor images:

As the project that I’m building (dwCheckApi) targets .NET Core SDK version 1.0.4, I could chose either the Visual Studio 2015 or Visual Studio 2017 image. I chose the Visual Studio 2017 image because I’d like to be able to compile this project when I move it to .NET Core 2.0

more on how to do that, later

We don’t need to install anything to build our repository (as .NET Core SDK 1.0.4 is installed for us)

we’ll leverage action this when we upgrade to .NET Core 2.0

but we need to do some basic checks before we build:

The first thing we do here is echo which version of the .NET Core SDK we have access to. This is for debugging purposes and can be removed if you don’t care too much about it, but it could prove helpful if we target higher a version of the SDK than is installed on the chosen image.

Then we restore all packages, but do it with the minimal output; otherwise the screen will fill up with the output of the restore action.

The real meat of the build happens here. We’re telling AppVeyor to do a publish of our application. We want a published version of our application because we may want to use the output of the build action somewhere

we could choose to upload the result of this build action to our hosting environment directly, so we’ll get the publish build

We’re (currently) not bothered about any after build actions, but we could use this to create and seed our database (or run any other commands related to our application’s initial set up) if we wanted.

Here we’re telling AppVeyor that we want to take everything from the publish directory, create a zip file (WebDeployPackage) called WebSite and upload it to AppVeyor’s cloud storage.

Here we’re telling AppVeyor’s git client to only clone the current branch and no history. We don’t need any of the history to build the application, so this will speed up the clone slightly. This, in turn, will make the whole build time faster.

Here is where we’re running our Unit Tests.

I’ve already written a quick intro to unit tests with .NET Core, but I’ve a newer post in the pipeline

We first restore all packages related to testing, then change into the tests directory (which contains the unit test code). Then we run xunit, as the unit tests I’ve written for this repository are xunit.NET tests.

you may need to change this command, if your tests are nunit tests

These are the final few steps in our pipeline, and they’re mostly empty. But we’re also telling AppVeyor not to deploy out application after successful builds.

Pipeline

We’re about ready to commit the yml file to source control – which we’ll do in a moment. But first, let’s take a look at what’s about to happen when we do.

I’m going to base this description on how the GitHub integration works, others will be similar

When we push our changes to GitHub, Git (which is what GitHub runs) will do it’s merge magic

which you can read about in the Git documentation

GitHub will then look for any web hooks set up on the repository, for each web hook it sends an announcement message. This message tells the application(s) at those endpoints that an update action has happened (a push, merge or similar).

I took advantage of web hooks in some of the code I worked on for Hack24

In the case of AppVeyor when the webhook fires, the build settings (either via an appveyor.yml file or those supplied in the UI) are then used to clone the relevant content of the repository.

in our yml earlier, we told AppVeyor to only build if the change was made to the master branch.

Once the latest version of the repository has been cloned, the before_build, build_script and after_build actions are run. The test_script stage is then run, all unit tests within this step will be performed and further steps will be cancelled if the tests fail.

After that the artifacts stage is run, this takes the output of our build script and packages it up, ready for deployment.

Commit and Build

Now that we’ve built the yml file, it’s time to commit it and for AppVeyor to perform the build. Go head and do that in which ever way that you do, and once the commit has happened you’ll see something happening in the AppVeyor UI:

The yellow bar indicates that a build or deploy action is in progress. a green bar indicates that a build or deploy action has completed; and a red bar means something in the build or deploy action failed or raised errors.

In the above screenshot, I’ve left an earlier (successful) build visible.

Clicking on the project title will take you to the build output screen. This screen shows you the output of the current build.

This information can be incredibly useful for debugging the build or deploy pipeline, so I’d recommend that you familiarise yourself with how to access it.

Artifacts

The Artifacts tab displays a list of the artifacts that were created as part of the build or deploy action.

as with most computer science things it’s spelt the American way, but I shan’t grumble.

The contents of this tab will be made up of the artifacts we outlined in the artifacts section – as you might expect.

Clicking on the name of the artifact will download the zip file, so that you can test it locally.

note: the WebDeploy artifact option creates a zip file, other artifact options create different files

Badges? We Don’t Need no Stinkin’ Badges

AppVeyor allows you to brag about your build status via badges. These are dynamically created images which you can embed into your repository’s readme file.

some of use developers love stickers, so it makes sense that we would love badges too 😛

To get the badge link for your project, you’ll need to head to the project’s settings:

Once there, you’ll need to click on the Badges option.

You can now add the badge link into your repositories readme and the users will see the current build status of your repo whenever they browse to it.

Committing

Any time that we push a change to the master branch, our AppVeyor project will clone the latest code, build a publish version of our application, run our automated unit tests on it, and package it up for deployment.

That’s pretty impressive for a half hour of work, and a free service, right?

What About Continuous Delivery?

AppVeyor will take care of Continuous Delivery (CD) for us too, but we’ll take care of that in a later blog post.

trust me, it’s super easy to do

When we do implement CD, we’ll create a separate yml file and use that to deploy to our production environment.

What About .NET Core 2.0 preview1?

To add .NET Core 2.0 preview1 support, all that’s required is that we install it during the install phase.

the following code snippet is correct at the time of writing

Adding the following code to the install phase of the appveyor.yml, will cause the .NET Core 2.0 preview1 to be downloaded, installed and added to the PATH variable:

The above code comes directly from a few of the official ASP.NET Core repositories, and is correct at the time of writing this blog post. I’ve no doubt that, once .NET Core 2.0 is out of preview and has been released as a full RTM, AppVeyor will add it to the relevant images.

Conclusion

We’ve seen how AppVeyor can be used to provide our code repositories with Continuous Integration, how it can be configured using an appveyor.yml file, and how these automated builds and tests will happen whenever a commit is made to the master branch.

in a real world application, you’d usually have separate dev and master branches. With daily commits going into dev, and those changes merged to master when ready to deploy

We’ve also seen how we can configure the AppVeyor build and test pipeline – even including .NET Core 2.0 preview1 if we want it.

Have you used AppVeyor or a similar CI tool chain? If not, why not? Let me know in the comments below.