Tuesday, June 16, 2015

Converting from MSTest to NUnit

Ever since I was converted to the one true gospel of Test-Driven Development in 2008, I have had to convert test projects from MSTest to NUnit and back a few different times. After my last time having to do it, I decided it was time to testify.

The Facts

MSTest and NUnit have different names for the decorations on the test classes & methods:

MSTest Attribute
NUnit Attribute
Purpose
[TestMethod]
[Test]
Identifies of an individual unit test
[TestClass]
[TestFixture]
Identifies of a group of unit tests, all Tests, and Initializations/Clean Ups must appear after this declaration
[ClassInitialize]
[TestFixtureSetUp]
Identifies a method which should be called a single time prior to executing any test in the Test Class/Test Fixture
[ClassCleanup]
[TestFixtureTearDown]
Identifies a method in to be called a single time following the execution of the last test in a TestClass/TestFixture
[TestInitialize]
[SetUp]
Identifies a method to be executed each time before a TestMethod/Test is executed
[TestCleanUp]
[TearDown]
Identifies a method to be executed each time after a TestMethod/Test has executed
[AssemblyInitialize]
 N/A
Identifies a method to be called a single time upon before running any tests in a Test Assembly
[AssemblyCleanUp]
 N/A
Identifies a method to be called a single time upon after running all tests in a Test Assembly

Source: Comparing the MSTest and Nunit Frameworks, by Naysawn Naderi, 1 Feb 2007.
This article contains some information that is no longer accurate, particularly in regards to the NUnit test runner, but this chart is still correct.

There are a few other minor syntactical differences to be aware of:
  1. Methods decorated with [ClassInitialize] in MSTest must have the signature
    public static void MethodName(TestContext context)
    while methods decorated with [TestFixtureSetUp] cannot have any arguments.
  2. MSTest's Assert and NUnit's Assert both have a method IsInstanceOfType(), but the order of parameters is reversed between frameworks.
    MSTest: void IsInstanceOfType(object actual, Type expectedType)
    NUnit:    void IsInstanceOfType(Type expected, object actual)
The Editorial

The debates over whether to use MSTest or NUnit typically revolve around three factors:
  1. Speed
  2. Integration
  3. Ease of use
People are pretty evenly split as to which one is actually faster. Personally I've found that so many factors affect this (the version of Visual Studio being used, the complexity of the tests, the framework being targeted, the version of NUnit) that we have to just call this one a tie.

When it comes to integration, if you're using Resharper, then it's a moot point. Resharper can run MSTest and NUnit tests in the very same test run seamlessly, and with a far superior interface to MSTest. And if you're not using Resharper, well then I just don't know how you do it. I had to go without Resharper for a week at a new job once, and it was like trying to run in waist-deep water. Everything took five times as long, and my confidence in my code was significantly reduced.

MSTest requires at least twice as many keystrokes to write tests than NUnit - everything is more verbose, with no additional benefit (see the difference in method signatures above for a minor example). Plus, NUnit can do this:

[TestCase("ConnectionStringA")]
[TestCase("ConnectionStringB")]
public void CreatePendingPaymentTest(string connectionName)
{
 //setup

 Repository classUnderTest = new Repository(connectionName);
 classUnderTest.CreateRecord();
 
 //assert
}

[TestCase] allows you to run the same test with different settings/inputs.
You can accomplish the same thing with MSTest by writing multiple tests that call the same method with different parameters, but again, more work to get the same result.

So, with speed being equal, integration being a non-issue, and NUnit the easier to use, the verdict is clear: use NUnit. It's available through NuGet when you need to add it into your solution, and you can learn more at NUnit.org. Enjoy!

EDIT (6/18/2015):
The syntactical differences section omitted an item that is now included.