Monday, September 14, 2009

MS Test, Private Accessors, and Team Build

At my work,we use TeamBuild to run a continuous integration build on a couple of different solutions.  Recently, we had an issue where one test project (there are several in any given solution) would run its tests just fine in Visual Studio on the developers' machines, but when run under the command-line MSTest runner for TeamBuild, they would fail.  These tests were using a private accessor, and the test failures were reporting that the MSTest run-time could not properly load the class the private accessor was trying to wrap.  (Private Accessors use reflection to wrap the classes they give you access to, and it was giving a "Could not load file or assembly" error.)

I spent the better part of two days researching this issue.  I could never reproduce it on my workstation, and the particularly bizarre thing was that the tests would run fine through the Visual Studio instance installed on the TeamBuild machine.  Something was off with the way MSTest was doing things, it seemed.  I never found anything on the Internet that really described this issue, though I did find some similiar issues relating to TeamBuild and reflection.  I tried a couple of thier solutions, as well as tweaking config and rewriting the tests a bit.  Nothing worked.

Eventually, I just threw in the towel, and wrote my own private accessor class, and that finally resolved the issue.  Here's what I did:

The class were trying to test was called PageTitleModule, and a private method that needed to be tested, which depended on the state of a private variable.  Inside the test class, there was a place where we built the PageTitleModule_Accessor class and set up the private variable:

private static PageTitleModule_Accessor GetPageTitleModule(PageTitleInfo cmsContent)
{
   PageTitleModule_Accessor module = new PageTitleModule_Accessor();
   module._cmsContent = cmsContent;
   return module;
}

Then, in the actual tests, we would use the accessor class to invoke the private method:

PageTitleModule_Accessor module = GetPageTitleModule(cmsContent);
string reWrittenResponse = module.RewriteResponse(originalResponse);

Just for reference, we use this same pattern, including the built-in private accessors, many places in our test code, and none of them ever given us any grief.  So I'm sure there is some setting that was wrong or compilation option that needed to be tweaked.  But after two days, the hack-y workaround I'm about to show you just made more sense; we needed to get our tests running again so we could get back to actual development.

Before, when we had the problem, we were using the Private Accessor fuctionality built-in to .NET.  (In other words, we went to PageTitleModule, right-clicked on the class definition, told it to "Create Private Accessor" and selected the appropriate test project, thus generating the PageTitleModule_Accessor class.)  When I finally gave up on that working, I wrote a PageTitleModule_Accessor of my own that implemented this private member and private method:

internal class PageTitleModule_Accessor
{
  private readonly PrivateObject _moduleImpl;

  public PageTitleModule_Accessor()
  {
    _moduleImpl = new PrivateObject(typeof(PageTitleModule), null);
  }

  public PageTitleInfo _cmsContent
  {
    get
    {
      return _moduleImpl.GetFieldOrProperty("_cmsContent") as PageTitleInfo;
    }
    set
    {
      _moduleImpl.SetFieldOrProperty("_cmsContent", value);
    }
  }

  public string RewriteResponse(string response)
  {
    return _moduleImpl.Invoke("RewriteResponse", new object[] {response}) as string;
  }
}

This PrivateObject type that I'm using is part of the Microsoft's unit testing framework (Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll).  While researching why the default accessor generation was not working, I disassmbled the generated test assembly (using RedGate's .NET Reflector) to take a look at what was going on and where the error was coming from.  I didn't find what I was looking for, but I saw that the auto-generated private accessors employ this PrivateObject type to manage the reflection of the type being tested, so I elected to re-use it.

(The "generated test assembly" mentioned above refers to the fact that when you use the built-in private accessor functionality, when you build, the C# compiler creates a DLL named [AssemblyUnderTest]_Accessor.dll, and it is in this binary that the [ClassUnderTest]_Accessor type lives.)

So, for those of you who know a lot about .NET private accessors, I'd love to have you weigh in on this.  Those of you who might happen to run into this problem, hopefully this gives you some information on what to do with it.