TFS Build reports partial fail even when all tests passed

by Mike Linnen 8. January 2009 09:33

I ran into a strange issue today when I was trying to figure out why my TFS Build was reporting that the build was partially successful even though every test was passing.  The normal build report really did not give any good reason why it was partially successful other than the fact that it was something related to the unit tests (I am using MS Test in this case).  So I cracked open the build log and peeled through the entries.  I noticed that when the code coverage was attempting to instrument the assemblies it reported that several of the assemblies could not be located.  Then I remembered I did some refactoring and I renamed and consolidated some assemblies. 

Well that should be an easy fix all I had to do was remove these assemblies from the test run config in the Code Coverage section.  So I opened up the LocalTestRun.testrunconfig file in Visual Studio 2008 and selected the Code Coverage section to make my changes.  As soon as I did this the config editor closed down (crashed).  Wow that was weird I never saw that before.  Hmmm I wonder what it could be.  Well here is what I did to try and locate the issue.

  1. Perhaps the Test Run Config file needs to be checked out of source control for write access.  Nope that wasn't it.
  2. Well if I cant edit it in VS 2008 then I might as well try notepad.  I removed the offending assemblies using notepad in the LocalTestRun.testrunconfig.  However once I opened up the Test Run Config editor and selected the Code Coverage the editor still crashed.
  3. Perhaps I malformed the Test Run Config xml file.  So I opened it up again in notepad and the XML looked fine.  Besides if this XML was malformed I don't think the Test Run Config editor would not open at all.
  4. Consult almighty search engine.  Wow look what I found http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=391255 and it was only reported 2 days ago.

So to be sure that I got the Test Run Config file right I removed my Database project from my solution made my edits for Code Coverage in the Test Run Config editor, then added the database project back into the solution. 

After fixing the Test Run Config file my build ran successfully.

Visual Studio 2008 Database Project problems on Vista/Server 2008 64 bit

by Mike Linnen 19. September 2008 20:31

I recently bought a new HP laptop with 64 bit Vista on it.  This new laptop is going to be my desktop development machine replacement.  I recently experienced several problems with trying to use an existing database project (from another machine) or creating a new one.  Every time the project tried to connect to the DB an error would come up saying a connection could not be made because the instance could not be found.  I did several searches on the Internet and I could not find any solution.  The only guidance I got was to be sure SQL Server 2005 Developer SP2 is installed before trying to use the database project. 

First a little background.  I wanted a dev machine that had SQL Server 2005 developer not SQL Server 2005 Express.  So I first installed SQL Server 2005 Developer and applied SP2.  I then proceeded to install Visual Studio 2008 without selecting the option to install SQL Express.  I also chose to install VS 2008 SP1.  After all this I tried to create a new SQL Server 2005 database project and I walked through the wizard.  Once I got up to the part where the wizard attempted to create the DB I got an error.  The only thing I found to work was to go into visual studio and select Tools -> Options -> Database tools -> Database Connections and I noticed the SQL Server Instance name was SQLEXPRESS.  So I cleared this out as shown below:

image

There was also an instance name of SQLEXPRESS on the Design Time Validation Database.  So I cleared that as well.

image

After that my database projects worked fine. 

Code Coverage of web applications in .Net 2.0

by Mike Linnen 2. May 2006 22:36
Code Coverage of web applications in .Net 2.0

In my day job at JDA Software I have been looking at code coverage options for determining the effectiveness of our testing.  My team uses four types of tests to test the software we write.

  • Unit Tests - Tests focused on a single component of the application.  These tests are MSTests that exercise a specific software component and typically mock out any dependant components.
  • Integration Tests -  Tests focused on multiple components of the application.  These tests are MSTests that exercise a component and it's dependencies to ensure the components work together.
  • Manual Tests - Tests that are executed by are testers manually from a GUI interface. 
  • Automated functional test - Tests that are scripted in a fashion that can be repeated build after build to ensure the build is still functional.  Sometimes referred to regression testing.

The goal of implementing a code coverage process was to determine the effectiveness of the types of tests listed above.  Visual Studio 2005 Team Edition (for testers and developers) provides some nice code coverage features we wanted to tap into.  Code coverage of unit tests and some integration tests worked fine from the visual studio IDE.  But for the integration, manual and automation tests that used web services we began to run into problems.  The web services that where hosted under IIS where not getting covered.  After some research I determined that code coverage under ISS was not going to work.  So I started looking into alternatives.  One thing I noticed is that web service projects that where not hosted under IIS had no problem getting covered.  These types of projects used the development web server that comes with ASP.NET 2.0.   So I decided to look into using the same development web server in place of IIS for code coverage purposes.

 

The generic steps for establishing code coverage for web services is as follows:

  1. Turn on instrumentation for the binaries that you want to cover
  2. Start the coverage monitor
  3. Start the development web server for a specified unused port pointing to the folder that is supposed to represent the web service.
  4. Execute your tests
  5. Stop the development web server
  6. Stop the coverage monitor
  7. Review the results in Visual Studio 2005

For the following commands use the Visual Studio Command prompt.

 

Turning on instrumentation of assemblies from the command line is done by the following command:

vsinstr -coverage myassembly.dll

 

To start the coverage monitor you use the follwoing command:

start vsperfmon -coverage -output:mytestrun.coverage

 

To start the development web server use the following command:

start WebDev.WebServer /port:8080 /path:c:\mypath

 

To stop the development web server simply right click on the task bar icon for the web server and select stop

 

To shutdown the coverage monitor use the following command:

vsperfcmd -shutdown

 

We were now able to do code coverage of all types of tests that we planned on implementing.  Some of our test runs are going to executed by the build and some will be executed by the testers.  Since VS.Net 2005 supports the ability to merge code coverage results this should not be a problem to combine all runs into a single report that now shows how effective our tests really are.

 

Some references to articles that help me come to this conclusion:

Command Line code coverage:

http://blogs.msdn.com/ms_joc/articles/406608.aspx

 

Using WebDev.WebServer from the command line:

http://www.devsource.com/article2/0,1895,1886246,00.asp

About the author

Mike Linnen

Software Engineer specializing in Microsoft Technologies

Month List