Running Typemock Isolator in TFS with MSBuild and Legacy Build Process

July 16, 2013

no comments

Typemock Isolator supports tests executed during an MSBuild or TFS Legacy Build Process (UpgradeTemplate.xaml) using three MSBuild tasks:

 

Task

Purpose

TypeMockRegister

Register Typemock Isolator License

TypeMockStart

Start using Typemock Isolator

TypeMockStop

Stop using Typemock Isolator
This task should always be called after TypeMockStart.

TypeMockStart does the actual loading of Typemock assemblies prior running the tests and making them available for MSTest.

The targets for executing tests (defined at C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets) provide empty BeforeTest and AfterTest targets for actions to occur before and after running the test list and BeforeTestConfiguration and AfterTestConfiguration targets for actions to occur before and after each test run.

Setting the BeforeTest and AfterTest to in TFSBuild.proj so that the Typemock tasks will be called is not enough. TFS Legacy Build Process will fail if executing Typemock tests due to MSBuild batching of the tests, which in turns runs the tests in a new call to MSBuild. As a result, BeforeTest called correctly and invokes TypeMockStart however MSTest.exe will be running in a different thread, which runs the tests without the Typemock profilers, causing the tests to fail:


Done executing task "TypeMockStart". (TaskId:65)
Done building target "BeforeTest" in project "TFSBuild.proj".: (TargetId:111)
Target "CoreTest: (TargetId:112)" in file "C:\Program Files
(x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets"
from project
"C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\BuildType\TFSBuild.proj" (target
"Test" depends on it):
Task "MSBuild" (TaskId:66) <-- this is where it fails
Task "TestToolsTask" (TaskId:71)
C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe
/nologo /usestderr
/searchpathroot:"C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\bin\Release"
/resultsfileroot:"C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\TestResults"
/testcontainer:"C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\bin\Release\\Typemock.Examples.CSharp.dll"
/publish:"http://vsalm:8080/tfs/tailspintoyscollection" /noprompt
/publishbuild:"vstfs:///Build/Build/207" /teamproject:"Sandbox" /platform:"Any
CPU" /flavor:"Release" (TaskId:71)
Loading
C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\bin\Release\\Typemock.Examples.CSharp.dll...
(TaskId:71)
Starting execution... (TaskId:71)
(TaskId:71)
Results Top Level Tests (TaskId:71)
------- --------------- (TaskId:71)
C:\Program Files
(x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1403,5,1403,5):
error : Failed
Typemock.Examples.CSharp.AssertingCallsWhereMade.VerifyingCallsWhereMade.Verify_CallWasMade_WithAnyArgument
[C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\BuildType\TFSBuild.proj]
C:\Program Files
(x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1403,5,1403,5):
error : Failed
Typemock.Examples.CSharp.AssertingCallsWhereMade.VerifyingCallsWhereMade.Verify_CallWasMade_WithExactArguments
[C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\BuildType\TFSBuild.proj]
C:\Program Files
(x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1403,5,1403,5):
error : Failed
Typemock.Examples.CSharp.AssertingCallsWhereMade.VerifyingCallsWhereMade.Verify_CallWasMade_WithMatchingArguments
[C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\BuildType\TFSBuild.proj]
C:\Program Files
(x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1403,5,1403,5):
error : Failed
Typemock.Examples.CSharp.AssertingCallsWhereMade.VerifyingCallsWhereMade.Verify_CallWasMadeTwice
[C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\BuildType\TFSBuild.proj]
C:\Program Files
(x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1403,5,1403,5):
error : Failed
Typemock.Examples.CSharp.AssertingCallsWhereMade.VerifyingCallsWhereMade.Verify_CallWasNeverMade
[C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\BuildType\TFSBuild.proj]

Inspecting the target running the tests in Microsoft.TeamFoundation.Build.targets we can see that it invokes another MSBuild task for the tests:

<Target Name="CoreTest"
DependsOnTargets="$(CoreTestDependsOn)" >

<MSBuild Condition=" '$(RunTest)' == 'true' "
Projects="$(MSBuildProjectFile)"
Properties="BuildAgentName=$(BuildAgentName);BuildAgentUri=$(BuildAgentUri);BuildDefinitionName=$(BuildDefinitionName);BuildDefinitionUri=$(BuildDefinitionUri);
BuildDirectory=$(BuildDirectory);BuildNumber=$(BuildNumber);CompilationStatus=$(CompilationStatus);CompilationSuccess=$(CompilationSuccess);
ConfigurationFolderUri=$(ConfigurationFolderUri);DropLocation=$(DropLocation);
FullLabelName=$(FullLabelName);LastChangedBy=$(LastChangedBy);LastChangedOn=$(LastChangedOn);LogLocation=$(LogLocation);
MachineName=$(MachineName);MaxProcesses=$(MaxProcesses);Port=$(Port);Quality=$(Quality);Reason=$(Reason);RequestedBy=$(RequestedBy);RequestedFor=$(RequestedFor);
SourceGetVersion=$(SourceGetVersion);StartTime=$(StartTime);Status=$(Status);TeamProject=$(TeamProject);TestStatus=$(TestStatus);
TestSuccess=$(TestSuccess);WorkspaceName=$(WorkspaceName);WorkspaceOwner=$(WorkspaceOwner);
SolutionRoot=$(SolutionRoot);BinariesRoot=$(BinariesRoot);TestResultsRoot=$(TestResultsRoot)"

Targets="RunTest"
StopOnFirstFailure="$(StopOnFirstFailure)">
<Output TaskParameter="TargetOutputs" ItemName="TestOutputs" />
</MSBuild>

<!--...-->

</Target>

Solution:

We need to make sure the tests runs after TypeMockRegister and TypeMockStart. The solution is using BeforeTestConfiguration so the tests will run the same MSBuild thread.

Apply the following changes in your TFSBuild.proj script:

1. Define a target that calls the TypeMockRegister task:

<Target Name="TypeMockRegister">
<TypeMockRegister Company ="company" License="0000-000-0000-0000-0000" AutoDeploy="True"/>
</Target>

2. Override the BeforeTest so it will call TypeMockRegister target:

<TargetName="BeforeTest"DependsOnTargets="TypeMockRegister"/>

3. Override the BeforeTestConfiguration so it will call TypeMockStart task:

<Target Name="BeforeTestConfiguration"> <TypeMockStart /> </Target>

4. Override the AfterTest so it will call TypeMockStop task:
<Target Name="AfterTest"> <TypeMockStop Undeploy="true" /> </Target>

When running a new build, it now succeeds:


Task "TypeMockStart" (TaskId:68)
Done executing task "TypeMockStart". (TaskId:68)
Done building target "BeforeTestConfiguration" in project "TFSBuild.proj".:
(TargetId:118)
Target "CoreTestConfiguration: (TargetId:120)" in file "C:\Program Files
(x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets"
from project
"C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\BuildType\TFSBuild.proj" (target
"TestConfiguration" depends on it):
Task "TestToolsTask" (TaskId:71)
Task Parameter:BuildFlavor=Release (TaskId:71)
Task Parameter:BuildUri=vstfs:///Build/Build/211 (TaskId:71)
Task Parameter:Platform=Any CPU (TaskId:71)
Task
Parameter:SearchPathRoot=C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\bin\Release\
(TaskId:71)
Task
Parameter:PathToResultsFilesRoot=C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\TestResults
(TaskId:71)
Task
Parameter:TeamFoundationServerUrl=http://vsalm:8080/tfs/tailspintoyscollection
(TaskId:71)
Task
Parameter:TestContainers=C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\bin\Release\\Typemock.Examples.CSharp.dll
(TaskId:71)
Task Parameter:TeamProject=Sandbox (TaskId:71)
Task Parameter:Publish=True (TaskId:71)
TeamFoundationServerUrl="http://vsalm:8080/tfs/tailspintoyscollection"
BuildUri="vstfs:///Build/Build/211" (TaskId:71)
C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe
/nologo /usestderr
/searchpathroot:"C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\bin\Release"
/resultsfileroot:"C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\TestResults"
/testcontainer:"C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\bin\Release\\Typemock.Examples.CSharp.dll"
/publish:"http://vsalm:8080/tfs/tailspintoyscollection" /noprompt
/publishbuild:"vstfs:///Build/Build/211" /teamproject:"Sandbox" /platform:"Any
CPU" /flavor:"Release" (TaskId:71)
Loading
C:\Builds\4\Sandbox\TypeMock.MSBuild.Logitech\bin\Release\\Typemock.Examples.CSharp.dll...
(TaskId:71)
Starting execution... (TaskId:71)
(TaskId:71)
Results Top Level Tests (TaskId:71)
------- --------------- (TaskId:71)
Passed
Typemock.Examples.CSharp.AssertingCallsWhereMade.VerifyingCallsWhereMade.Verify_CallWasMade_WithAnyArgument
(TaskId:71)
Passed
Typemock.Examples.CSharp.AssertingCallsWhereMade.VerifyingCallsWhereMade.Verify_CallWasMade_WithExactArguments
(TaskId:71)
Passed
Typemock.Examples.CSharp.AssertingCallsWhereMade.VerifyingCallsWhereMade.Verify_CallWasMade_WithMatchingArguments
(TaskId:71)
Passed
Typemock.Examples.CSharp.AssertingCallsWhereMade.VerifyingCallsWhereMade.Verify_CallWasMadeTwice
(TaskId:71)
Passed
Typemock.Examples.CSharp.AssertingCallsWhereMade.VerifyingCallsWhereMade.Verify_CallWasNeverMade
(TaskId:71)

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*