Missing Code Coverage

The following causes and suggestions may be of use when trying to troubleshoot problems related to missing code coverage.

No Coverage Markers Showing

Build Issues

When no coverage markers are showing for code within a particular project, this most likely means that NCrunch has been unable to build the project. Check the tests window for failed builds and error information.

Instrument Output Assembly Configuration Setting

The Instrument output assembly project-level configuration setting can be used to turn off instrumentation for specific projects in the solution. With instrumentation disabled, no coverage markers will show for code within the project. Although this configuration setting is turned on by default, sometimes it can be left off by accident.

All Coverage Markers Are Always Black

Ignored Tests

A very common mistake when using NCrunch for the first time is to accidently ignore all the tests in a solution without intending to.

The first time NCrunch is used on a solution, you will be confronted with a setup wizard to adjust the default configuration settings for the solution. The last step in this setup wizard has an option that allows you to choose whether you want to ignore all tests by default (and unignore them manually as you get familiar with NCrunch), or whether you want to switch them all on. The default option is to ignore all tests.

If you have been caught out by this, it is very easy to simply unignore the tests using the tests window.

GAC Resolution Issues

If the project in question is commonly installed in the GAC on your development machine, then this can cause run-time resolution problems for NCrunch when it attempts to build test environments - resulting in loss of relevant data (including code coverage).

The CLR will always prefer to load assemblies from the GAC where ever they exist there, even when specifically instructed otherwise. There is no way to reliably override this behaviour.

If the project's output binary is installed in the GAC, one of the following may solve this problem:

• Removing the conflicting assembly from the GAC
• Changing the version number of the project under development, so it has a different strong name to the assembly in the GAC. This can be done cleanly by introducing conditional build behaviour in the AssemblyInfo.cs file to specify a different version number for NCrunch builds.
• Turn off signing for NCrunch builds of the project. This will result in a different strong name for the output assembly

Lost Project References

NCrunch makes use of <ProjectReference> information inside project files to identify the relationships between projects in your solution. A common mistake when creating references between projects is to reference project output binaries using the MSBuild <Reference> tag instead of using proper project reference. This creates a situation where NCrunch treats the project reference as being an assembly reference, and it will not record code coverage information for the 'lost' reference.

Normally NCrunch will issue a warning when it detects a possible 'lost' reference between projects in the solution.

The solution to this problem is to fix the lost reference so that the relationship works using a project reference and not an assembly reference.

Some Code Coverage Markers Are Always Black

Understanding Code Coverage

Code coverage in a conceptual sense can be interpreted in different ways. Broadly there are two ways to look at code coverage:

• Logical Code Coverage: This school of thought identifies a line of code as being covered if it would be executed at any time in the pursuit of the execution of a test within the suite. The theory is that if you were to set a debugger breakpoint on a line of code, and the debugger broke into the code when you tried to run a specific test, then this code is considered covered by the test.
• Physical Code Coverage: This approach to code coverage looks specifically at which lines of code are executed after a test has started executing. Lines are only considered to be covered if they are physically executed while the test is executing.

As an example, consider a static constructor. Code within the constructor would always need to be executed before any code in the class can run. This means that the static constructor can be logically considered covered by every test that makes use of code within its class. However, as the static constructor can only be executed once for an entire test run that may involve multiple tests, it will only ever be physically covered by the first test to make use of any of the code within its class.

NCrunch's code coverage is physical, not logical.

Dynamic Test Cases

Some test frameworks contain features that allow test cases to be constructed dynamically in user code. An example of this is NUnit's TestCaseSource attribute.

These features can cause confusion when analysing coverage results in NCrunch, as the code responsible for generating the test cases will not be reported as covered.

NCrunch's current design does not record code coverage data for user code that is responsible for creating and identifying test cases at runtime. The reasoning behind this is that the code is not associated with any specific test yet recognised by NCrunch, and therefore has no way to describe its existence in the UI or elsewhere.

Uncontrolled Multi-threaded Behaviour

A common cause of incorrect code coverage is uncontrolled behaviour of multi-threaded tests. This can cause code coverage information to overlap between tests that make up part of a single test run. Ensure you control the boundaries of multi-threaded behaviour under test very carefully.

SetUpFixture And AssemblyInitialize

NCrunch does not record code coverage for NUnit SetUpFixtures or MSTest AssemblyInitialize. The reasoning behind this is that suite-wide setup and teardown code is not associated with any individually recognised test in the suite. This may change in future.

Coverage Seems Wrong / Out Of Date

NCrunch has a sophisticated set of indexes underpinning its coverage database. There have been reported situations in which these indexes can become corrupted, making code coverage appear strange or out of date. If this happens to you, try deleting the contents of the _NCrunch_SOLUTION directory adjacent to your solution's .sln file with the engine offline. This will clear NCrunch's test coverage cache and should restore normal behaviour. If you find a way to reproduce a code coverage corruption issue, please report this via the NCrunch support forum.