NUnit for Visual Studio Addin

I recently stumbled upon this great addin for Visual Studio that uses the Visual Studio Test Adapter pattern to include NUnit tests within Visual Studio as MS Tests.  They appear in the Test List Editor and execute equivalent to MS Test, including those handy Run and Debug keyboard shortcuts I described in my last post.

Since they operate as MS Tests, the project requires some additional meta-data in the csproj file in order to have Visual Studio recognize this project as a Test library.  My last post has the details.

Curious to see how far the addin could act as a stand-in for NUnit, I fired up Visual Studio, a few beers, and the NUnit attribute documentation to put it through the works.  I’ve compiled my findings here in the table below.

In all fairness, there are a lot of attributes in NUnit, some of these you probably didn’t know existed.

NUnit Attribute	Supported	Comments
Category	No	Sadly, the addin does not register a new column definition for Category.  Though this feature is not tied to any functional behavior, it would be greatly welcomed to improve upon Visual Studio’s Test Lists.
Combinatorial	Yes
Culture / SetCulture	No	Tests that would normally be excluded by NUnit fail.
Datapoint / Theory	No	Test names do not match NUnit runtime.  All Datapoints produce result Not Runnable in the Test Results
Description	No	Value does not appear in the Test List Editor
Explicit	No	Explicit Tests are executed and appear as Enabled in the Test List Editor
ExpectedException	Yes
Ignore	Partial	Ignored tests are excluded from the Test List Editor, so they are ignored, but they do not appear as Enabled = False.
MaxTime / Timeout	Partial	Functions properly though supplied setting does not appear in the Timeout column in the Test List Editor
Platform	No	Tests are executed regardless of the specified platform.
Property	-	Custom properties do not appear in the output of the TRX file, which is where I’m assuming they would appear.  Not entirely sure if the schema would support custom properties however.
Random	No	Tests are generated, though the names contain only numbers.  Executing these tests produce the result Not Runnable in the Test Results.
Range	No	Tests are generated, though the names contain only numbers.  Executing these tests produce the result Not Runnable in the Test Results.
Repeat	Yes
RequiredAddin	-	Not tested.
RequiresMTA / RequiresSTA / RequiresThread	Yes
Sequential	Yes
Setup / Teardown	Yes
SetupFixture	No	Setup methods for a given namespace do not execute.
Suite	-	Not tested (requires command-line switch)
TestFixtureSetup / TestFixtureTeardown	Yes
Test	Yes	Of course!
TestCase	Yes	Tested different argument types (int, string), TestName, ExpectedException.
TestCaseSource	Yes

There’s quite a few No’s in this list, but the major players (Test, Setup/Teardown, TestFixtureSetup/Teardown) are functional.  I’m actually pleased that NUnit 2.5.2 features such as parameterized tests (TestCase, TestCaseSource) and Combinatorial / Sequential / Values are in place, as well as former addin features that were bundled into the framework (MaxTime / Repeat).

In respect to the malformed test names and non-runnable tests for the Theory / Range / Random attributes, hopefully this is a small issue that can be resolved.  The cosmetic issues with Ignore / Description / Category don’t pose any major concerns though they would be large wins in terms of full compatibility with MS Test user interface and features.

I’ve never used the SetupFixture nor the culture attributes, so I’m not losing much sleep over these.

However, for me, the main issue for me is that Explicit tests are always executed.  I’ve worked on many projects where a handful of tests either brought down the build server or couldn’t be run with other tests.  Rather than solve the problem, developers tagged the tests as Explicit – they work, but you better have a good reason to be running them.

Hats off to the NUnitForVS team.