Continuous Integration Basics: Part 2–Repeatable Builds with Build Automation

This entry is part 2 of 2 in the series Continuous Integration

Repeatable builds are central to starting on the path to a successful continuous integration strategy. What I mean by repeatable builds is that a developer should be able to get the source code from a repository and in a single step run a set of complex tasks, tasks such as code quality checking, compiling the source code, running tests, determining code test coverage, labelling versions or deploying finished software. That single step process must produce the same result for every developer, on every machine so that developers can concentrate on writing code and know confidently that the supporting processes are in place and providing the desired feedback in a centrally managed manner.

There are a number of build automation platforms, NAnt, MSBuild and Rake to name a few of the popular systems in the .NET space. There is no right or wrong answer in choosing a build automation system, although I find MSBuild to be the one I use the most, primarily do the fact that many companies are not comfortable with non-Microsoft products. It also counts in MSBuild’s favour that any machine that has a full .NET framework installed automatically has MSBuild installed. Personally I think Rake, which is ruby based, is a fantastic product that provides a very clean way to configure your builds in a programmatic manner, however I will use MSBuild for all the examples in this post.

I will explore a scenario that includes:

  • Using MSBuild to prepare the project for building
  • Using StyleCop for Code Quality Analysis
  • Using MSBuild to label the source code for a particular build version
  • Using MSBuild to build the source code
  • Using NUnit to run Tests
  • Using MSpec to run Context Specifications
  • Using OpenCover to check the code coverage of the tests

The example code can be downloaded from my bitbucket repository.

Conventions

The first step to ensure that code can be built on any machine is to make sure that all source code dependencies are part of the source code repository so that there are no prerequisites for a build to succeed on a particular machine. I follow a set of conventions for every project that makes this as easy as possible and minimises the time to setup a new project.

FolderStructure2FolderStructure1

On the left above is the root of a new project containing a folder for source code (called src) and a folder for all external dependencies (called tools). The build script is in the root folder and is named the same as the solution file in the src directory. A reports directory is created during the build process where all tool output is collected possibly to be consumed by a CI tool. The tools directory contains the binary dependencies the source code requires and external tools the build process depends on, in the past I manually downloaded these tools and placed them in this directory however with the advent of package managers such as NuGet and OpenWrap I let these tools manage as many of these as possible.

While a build process can be used to perform many different operations, there are a small number of operations that cover what most projects commonly require. A single MSBuild file can contain configurations for many of these operations and when we run the build file we are able to choose one of these operations to perform, they are termed targets by MSBuild. A target may depend on other targets for example a target responsible for running tests will be dependent on the target responsible for compiling the source code as the test runner needs a compiled assembly to test. The generic targets I begin each project with an be seen below and consist of:

  • Clean (remove any artefacts from a previous build)
  • Label (apply a version number to the projects assemblies)
  • CodeQuality (scan the source code for coding convention irregularities)
  • Compile (compile the source code projects)
  • Test (run unit tests)
  • CodeCoverage (run unit tests and analyse their coverage)
  • Specs (run context specification tests)
  • CI (target used by a CI server that pulls many of the above tasks together, excluding code coverage)
  • CICoverage (target used by a CI server that pulls many of the above tasks together, including code coverage)
<PropertyGroup>
<!-- The build target configuration (Debug versus Release) -->
	<Configuration>Debug</Configuration>

	<!-- General Paths -->
	<!-- The root directory containing the build file -->
	<RootPath>$(MSBuildProjectDirectory)</RootPath>
	<!-- The source code directory -->
	<SrcBasePath>$(RootPath)\src</SrcBasePath>
	<!-- The tools directory -->
	<ToolsBasePath>$(RootPath)\tools</ToolsBasePath>
	<!-- The reports directory -->
	<ReportsPath>$(RootPath)\reports</ReportsPath>
	<!-- The Source Code Solution Name, this is conventions based and should be named the same as the build file.  e.g. example.sln should have a matching example.msbuild file in the top level directory -->
	<BuildSolutionFile>src\$(MSBuildProjectName).sln</BuildSolutionFile>
	<!-- The tools path for the MSBuild Extension Pack -->
	<MSBuildExtensionsPath>$(ToolsBasePath)\MSBuild\MSBuildExtensionPack</MSBuildExtensionsPath>
	<!-- The tools path for the MSBuild Community Tasks Pack -->
	<MSBuildCommunityExtensionsPath>$(ToolsBasePath)\MSBuild\MSBuildCommunityTasks</MSBuildCommunityExtensionsPath>

	<!-- NUnit -->
	<!-- The tools path for NUnit -->
	<NUnitPath>$(ToolsBasePath)\NUnit.2.5.10.11092\tools</NUnitPath>
	<!-- NUnit report name and location -->
	<NUnitOuputFile>$(ReportsPath)\tests-output.xml</NUnitOuputFile>

	<!-- StyleCop -->
	<!-- The tools path for StyleCop -->
	<StyleCopPath>$(ToolsBasePath)\StyleCop.4.6.3.0</StyleCopPath>
	<!-- The StyleCop report name and location -->
	<StyleCopOutputFile>$(ReportsPath)\stylecop-output.xml</StyleCopOutputFile>
	<!-- The StyleCop max violations count -->
	<StyleCopMaxViolationCount>50</StyleCopMaxViolationCount>
	<!-- StyleCop Force Full Analysis -->
	<StyleCopForceFullAnalysis>true</StyleCopForceFullAnalysis>
	<!-- StyleCop Treat Errors As Warnings -->
	<StyleCopTreatErrorsAsWarnings>true</StyleCopTreatErrorsAsWarnings>
	<!-- StyleCop Cache Results -->
	<StyleCopCacheResults>false</StyleCopCacheResults>

	<!-- MSpec -->
	<!-- The tools path for MSpec -->
	<MSpecPath>$(ToolsBasePath)\Machine.Specifications.0.4.24.0\tools</MSpecPath>
	<MSpecExecutable>mspec-clr4.exe</MSpecExecutable>
	<MSpecPathOutputFile>$(ReportsPath)\specs-output.xml</MSpecPathOutputFile>
	<MSpecSettings></MSpecSettings>

	<!-- OpenCover -->
	<!-- The tools path for OpenCover -->
	<OpenCoverPath>$(ToolsBasePath)\OpenCover.1.0.719</OpenCoverPath>
	<OpenCoverReportGenPath>$(ToolsBasePath)\ReportGenerator.1.2.1.0</OpenCoverReportGenPath>
	<!-- OpenCover report name and location -->
	<OpenCoverOuputFile>$(ReportsPath)\coverage-output.xml</OpenCoverOuputFile>
	<OpenCoverTmpOuputFile>$(ReportsPath)\coverage-tmp-output.xml</OpenCoverTmpOuputFile>

	<!-- Assembly Versioning -->
	<!-- Major -->
	<AssemblyMajorVersion>1</AssemblyMajorVersion>
	<!-- Minor -->
	<AssemblyMinorVersion>0</AssemblyMinorVersion>
	<!-- Build -->
	<AssemblyBuildNumber>0</AssemblyBuildNumber>
	<!-- Revision -->
	<AssemblyRevision>0</AssemblyRevision>

</PropertyGroup>

For tests I follow a convention that has each test assembly named such that it ends with “.Tests”, the same goes for each context specification assembly which ends in “.Specs”. By following these conventions and using no hard coded values in the MSBuild configuration I can reuse the same build file in a new project immediately by just creating the directory structure, adding the source code to the appropriate directories and naming the build file the same as the source code solution file. As the entire build is self contained we can integrate the results into the CI server of our choice and not have to rely on a particular CI servers features. The build can be executed by a developer or a CI server by running msbuild someproject.msbuild from the command line or for example msbuild someproject.msbuild /t:Tests to run just the Tests target.

MSBuild configuration file details

I’ll give a brief breakdown of what each section of my standard MSBuild build file does. There are plenty of comments in the file so it should be most self explanatory.

MSBuild configuration global properties section –

In this section I define paths, file names, executable tool locations, tool options and version numbers that are used throughout the configuration. All paths are defined relative to the MSBuild configuration file’s location and names are based on the MSBuild file’s name.

<PropertyGroup>
	<!-- The build target configuration (Debug versus Release) -->
	<Configuration>Debug</Configuration>

	<!-- General Paths -->
	<!-- The root directory containing the build file -->
	<RootPath>$(MSBuildProjectDirectory)</RootPath>
	<!-- The source code directory -->
	<SrcBasePath>$(RootPath)\src</SrcBasePath>
	<!-- The tools directory -->
	<ToolsBasePath>$(RootPath)\tools</ToolsBasePath>
	<!-- The reports directory -->
	<ReportsPath>$(RootPath)\reports</ReportsPath>
	<!-- The Source Code Solution Name, this is conventions based and should be named the same as the build file.
			 e.g. example.sln should have a matching example.msbuild file in the top level directory -->
	<BuildSolutionFile>src\$(MSBuildProjectName).sln</BuildSolutionFile>
	<!-- The tools path for the MSBuild Extension Pack -->
	<MSBuildExtensionsPath>$(ToolsBasePath)\MSBuild\MSBuildExtensionPack</MSBuildExtensionsPath>
	<!-- The tools path for the MSBuild Community Tasks Pack -->
	<MSBuildCommunityExtensionsPath>$(ToolsBasePath)\MSBuild\MSBuildCommunityTasks</MSBuildCommunityExtensionsPath>

	<!-- NUnit -->
	<!-- The tools path for NUnit -->
	<NUnitPath>$(ToolsBasePath)\NUnit.2.5.10.11092\tools</NUnitPath>
	<!-- NUnit report name and location -->
	<NUnitOuputFile>$(ReportsPath)\tests-output.xml</NUnitOuputFile>

	<!-- StyleCop -->
	<!-- The tools path for StyleCop -->
	<StyleCopPath>$(ToolsBasePath)\StyleCop.4.6.3.0</StyleCopPath>
	<!-- The StyleCop report name and location -->
	<StyleCopOutputFile>$(ReportsPath)\stylecop-output.xml</StyleCopOutputFile>
	<!-- The StyleCop max violations count -->
	<StyleCopMaxViolationCount>50</StyleCopMaxViolationCount>
	<!-- StyleCop Force Full Analysis -->
	<StyleCopForceFullAnalysis>true</StyleCopForceFullAnalysis>
	<!-- StyleCop Treat Errors As Warnings -->
	<StyleCopTreatErrorsAsWarnings>true</StyleCopTreatErrorsAsWarnings>
	<!-- StyleCop Cache Results -->
	<StyleCopCacheResults>false</StyleCopCacheResults>

	<!-- MSpec -->
	<!-- The tools path for MSpec -->
	<MSpecPath>$(ToolsBasePath)\Machine.Specifications.0.4.24.0\tools</MSpecPath>
	<MSpecExecutable>mspec-clr4.exe</MSpecExecutable>
	<MSpecPathOutputFile>$(ReportsPath)\specs-output.xml</MSpecPathOutputFile>
	<MSpecSettings></MSpecSettings>

	<!-- OpenCover -->
	<!-- The tools path for OpenCover -->
	<OpenCoverPath>$(ToolsBasePath)\OpenCover.1.0.719</OpenCoverPath>
	<OpenCoverReportGenPath>$(ToolsBasePath)\ReportGenerator.1.2.1.0</OpenCoverReportGenPath>
	<!-- OpenCover report name and location -->
	<OpenCoverOuputFile>$(ReportsPath)\coverage-output.xml</OpenCoverOuputFile>
	<OpenCoverTmpOuputFile>$(ReportsPath)\coverage-tmp-output.xml</OpenCoverTmpOuputFile>

	<!-- Assembly Versioning -->
	<!-- Major -->
	<AssemblyMajorVersion>1</AssemblyMajorVersion>
	<!-- Minor -->
	<AssemblyMinorVersion>0</AssemblyMinorVersion>
	<!-- Build -->
	<AssemblyBuildNumber>0</AssemblyBuildNumber>
	<!-- Revision -->
	<AssemblyRevision>0</AssemblyRevision>
</PropertyGroup>

 

 

Imported Tasks Section –

The targets in MSBuild consist of a set of MSBuild Tasks, many tasks are built-in but third party sets of tasks exist such as the MSBuildExtensionPack and MSBuildCommunityTasks. Below are some examples of adding third party tasks to the configuration file, I include these tasks in the tools directory:

<!--****************-->
<!-- Imported Tasks -->
<!--****************-->

<!-- Include the MSBuild Extension Pack NUnit Task -->
 <UsingTask AssemblyFile="$(MSBuildExtensionsPath)\MSBuild.ExtensionPack.dll"
TaskName="MSBuild.ExtensionPack.CodeQuality.NUnit"/>
<!-- Include the MSBuild Extension Pack AssemblyInfo Task -->
<UsingTask AssemblyFile="$(MSBuildExtensionsPath)\MSBuild.ExtensionPack.dll"
TaskName="MSBuild.ExtensionPack.Framework.AssemblyInfo"/>
<!-- Include the StyleCop task -->
<UsingTask AssemblyFile="$(StyleCopPath)\StyleCop.dll"
TaskName="Microsoft.StyleCop.StyleCopTask"/>

Build Targets Section –

 

The Clean Target:

<!-- The Clean Target -->
 <Target Name="Clean">
	<!-- Remove the reports directory if it already exists from a previous build -->
	<RemoveDir Directories="$(ReportsPath)" Condition = "Exists('$(ReportsPath)')" />
	<!-- Create the reports directory for this builds output -->
	<MakeDir Directories = "$(ReportsPath)"  />
	<!-- Clean the source code projects -->
	<MSBuild Projects="$(BuildSolutionFile)" ContinueOnError="false" Targets="Clean"
	Properties="Configuration=$(Configuration)" />
  </Target>

The clean target firstly removes the reporting directory from a previous run if it exists using a Condition construct before recreating an empty reports directory. The MSBuild task is used to perform operations on a Visual Studio solution file, here we tell MSBuild to execute the built in “Clean” Target on the solution. This is the same as if you clicked the Build –> Clean Solution menu in VIsual Studio.

The Label Target:

 <!-- The Label Target that sets the AssemblyInfo Build Version -->
 <Target Name="Label">
	<!-- Include all assemblies that end in Tests.dll (This is convention based) ->
	<CreateItem Include="**\AssemblyInfo.cs" exclude="**\*.Tests\Properties AssemblyInfo.cs;**\*.Specs\Properties\AssemblyInfo.cs">
		<Output TaskParameter="Include" ItemName="AssemblyInfoFiles" />
	</CreateItem>
	<!-- Update the Assembly and File Version -->
	<MSBuild.ExtensionPack.Framework.AssemblyInfo AssemblyInfoFiles="@(AssemblyInfoFiles)" SkipVersioning="false" Condition="'$(CCNetLabel)' != ''"
		AssemblyMajorVersion="$(AssemblyMajorVersion)"
		AssemblyMinorVersion="$(AssemblyMinorVersion)"
		AssemblyBuildNumber="$(AssemblyBuildNumber)"
		AssemblyRevision="$(CCNetLabel)"
		AssemblyFileMajorVersion="$(AssemblyMajorVersion)"
		AssemblyFileMinorVersion="$(AssemblyMinorVersion)"
		AssemblyFileBuildNumber="$(AssemblyBuildNumber)"
		AssemblyFileRevision="$(CCNetLabel)"
		/>
 </Target>

The label target is mostly used by a CI server to version a particular build, this excerpt is from a CruisCeontrol.Net CI Server setup and uses the global variable “CCNetLabel” to set the revision number of the assembly and assemblyfile details. I use the MSBuildExtensionPack task to accomplish this. The CreateItem task is used to get a list of AssemblyInfo.cs files from the solution that exclude the Tests and Specs assemblies if they conform to the conventions of having names ending with .Tests or .Specs.

The Code Quality Target:

 <!-- The Code Quality Target, checks the source code for stylistic compliance via StyleCop -->
<Target Name="CodeQuality">
	<!-- Create a collection of files to scan -->
	<CreateItem Include="$(SrcBasePath)\**\*.cs">
		<Output TaskParameter="Include" ItemName="StyleCopFiles"/>
	</CreateItem>
	<!-- Run the StyleCop MSBuild task -->
	<Microsoft.StyleCop.StyleCopTask
		ProjectFullPath="$(RootPath)"
		SourceFiles="@(StyleCopFiles)"
		ForceFullAnalysis="$(StyleCopForceFullAnalysis)"
		TreatErrorsAsWarnings="$(StyleCopTreatErrorsAsWarnings)"
		CacheResults="$(StyleCopCacheResults)"
		OverrideSettingsFile="$(SrcBasePath)\Settings.StyleCop"
		OutputFile="$(StyleCopOutputFile)"
		MaxViolationCount="$(StyleCopMaxViolationCount)">
	</Microsoft.StyleCop.StyleCopTask>
</Target>

The Code Quality task uses the StyleCop task to scan the solution code file for stylistic compliance. An interesting parameter to take note of is “TreatErrorsAsWarings”, if you set this to false your build will fail on stylistic errors, this is desirable on new projects but may be difficult on existing projects until the code base is cleaned up with a tool such as Resharper or CodeMaid.

The Compile Target:

<!-- The Compile Target, compiles the source code for the solution -->
<Target Name="Compile" DependsOnTargets="Clean">
	<MSBuild Projects="$(BuildSolutionFile)" ContinueOnError="false" Properties="Configuration=$(Configuration)">
		<Output ItemName="BuildOutput" TaskParameter="TargetOutputs"/>
	</MSBuild>
  </Target>

The compile target is relatively simple and uses the MSBuild task to build the projects in the solution.

The Test Target:

<!-- The Test Target, runs unit tests on the compiled source code via NUnit -->
<Target Name="Test" DependsOnTargets="Clean;Compile">
	<!-- Include all assemblies that end in Tests.dll (This is convention based) -->
	<CreateItem Include="**\Bin\Debug\*Tests*.dll" >
		<Output TaskParameter="Include" ItemName="TestAssemblies" />
	</CreateItem>

	<MSBuild.ExtensionPack.CodeQuality.NUnit Assemblies="@(TestAssemblies)" ToolPath="$(NUnitPath)" OutputXmlFile="$(NUnitOuputFile)">
	</MSBuild.ExtensionPack.CodeQuality.NUnit>
</Target>

The Test target uses the MSBuildExtensionPack NUnit task to execute the NUnit console runner against assemblies that match our Test conventions, i.e. name ending in .Tests.

The CodeCoverage Target:

 <!-- The Code Coverage Target, checks code coverage using opencover and NUnit, the
			task generates both a coverage report and the test report -->
<Target Name="CodeCoverage" DependsOnTargets="Clean;Compile">
	<!-- Include all assemblies that end in Tests.dll (This is convention based) -->
	<CreateItem Include="**\Bin\Debug\*Tests*.dll" >
		<Output TaskParameter="Include" ItemName="TestAssemblies" />
	</CreateItem>

	<!-- Execute opencover -->
	<Exec Command="$(OpenCoverPath)\OpenCover.Console.exe -register:user -target:&quot;$(NUnitPath)\nunit-console.exe&quot; -targetargs:&quot;/noshadow @(TestAssemblies) /domain:single /xml:$(NUnitOuputFile)&quot; -filter:&quot;-[$(MSBuildProjectName)*.Tests]* +[$(MSBuildProjectName)*]$(MSBuildProjectName).*&quot; -output:$(OpenCoverTmpOuputFile)" />
	<!-- Use ReportGenerator Tool to build an xml Summary -->
	<Exec Command="$(OpenCoverReportGenPath)\ReportGenerator.exe &quot;$(OpenCoverTmpOuputFile)&quot; &quot;$(ReportsPath)&quot; XmlSummary" />
	<!-- Report Generator has no way to name the output file so rename it by copying and deleting the original file -->
	<Copy SourceFiles="$(ReportsPath)\Summary.xml" DestinationFiles="$(OpenCoverOuputFile)"></Copy>
	<Delete Files="$(ReportsPath)\Summary.xml"></Delete>
	<!-- Delete the original opencover output before it was transformed by ReportGenerator -->
	<Delete Files="$(OpenCoverTmpOuputFile)" />
</Target>

The Coverage task is perhaps the most complex task,mainly due to the fact that I am using an open source coverage framework called opencover that is relatively young and does not yet have a dedicated MSBuild task. If you are willing to pay for a license NCover has better MSBuild support. Points to take note of is the use of &quote; to escape quotes for command line arguments to the Exec MSBuild task and the use of the ReportGenerator tool after execution to transform the output from opencover.

The Specifications Target:

 <!-- The Specs Target, runs the context specifications on the compiled source code via MSpec -->
<Target Name="Specs" DependsOnTargets="Compile">
	<!-- Include all assemblies that end in Specs.dll (This is convention based) -->
	<CreateItem Include="**\Bin\Debug\*Specs*.dll" >
		<Output TaskParameter="Include" ItemName="SpecsAssemblies" />
	</CreateItem>

 	<PropertyGroup>
		<MSpecCommand>
			$(MSpecPath)\$(MSpecExecutable) $(MSpecSettings) @(SpecsAssemblies) --xml $(MSpecPathOutputFile)
		</MSpecCommand>
	</PropertyGroup>
	<Message Importance="high" Text="Running Specs with this command: $(MSpecCommand)"/>
	<Exec Command="$(MSpecCommand)" />
 </Target>

The specifications target uses the MSBuild Exec task to execute the Machine.Specifications console runner to run the specifications on assemblies that match our convention i.e. name ending in .Specs.

 

The Build Target:

 <!-- The default build task that pulls the other tasks together, usually executed by a developer -->
<Target Name="Build" DependsOnTargets="Clean;CodeQuality;Compile;Test;Specs">
</Target>

The build target is the default target (specified in the root Projects node at the top of the file). This target just lists the targets it depends on and these are executed in order. This target would be used by a developer to execute the build on a local machine, it does not label the project assemblies.

The CI Target:

 <!-- The CI build task that pulls the other tasks together and includes assembly labelling that pulls in the CI servers build number -->
<Target Name="CI" DependsOnTargets="Clean;CodeQuality;Label;Compile;Tests;Specs">
</Target>

The CI target is the same as the default build task just with the addition of labelling the project assemblies with the next build number. This task would be specified on the command line by a CI server.

The CICoverage Target:

 <!-- The CI build task that pulls the other tasks together, tests are run including codecoverage and includes assembly labelling that pulls in the CI servers build number -->
<Target Name="CICoverage" DependsOnTargets="Clean;CodeQuality;Label;Compile;CodeCoverage;Specs">
</Target>

The CICoverage target is the same as the CI target with addition of code coverage information.

 

These simple set of targets when used with project conventions create the ability to produce valuable repeatable builds with minimal effort, it is literally a case of create the directory structure, add the source code, name the build file to match the source code solution and you are ready to build. There is obviously a LOT more one can do and I hope to explore some complex real world problems in future posts.

Series Navigation<< Continuous Integration Basics: Part 1 – Introduction
This entry was posted in enterprise-basics and tagged , , , , , , . Series: . Bookmark the permalink. Both comments and trackbacks are currently closed.

One Comment

  1. oleksii
    Posted March 21, 2013 at 8:59 pm | Permalink

    Excellent post. Thanks for sharing.Even though it’s a bit old, this post still rocks.

    For anyone having trouble with the opencover and Nunit, saying DirectoryNotFoundException if you use a CreateItem tag to assemble all the assemblies. The problem is with Nunit console runner not being able to deal with a list of assemblies separated by semicolon. Which the default output from the CreateItem.

    This can be easily fixed by adding a separator symbol: So that the whole

    <Exec Command="$(OpenCoverPath)\OpenCover.Console.exe -register:user -target:"$(NUnitPath)\nunit-console.exe" -targetargs:"/noshadow @(TestAssemblies, ' ') ...

    Spent about a day trying to figure out this

Skip to toolbar