Sergey Shishkin

on agile software development

StyleCop: Test Driven Rules Development

I’m playing with StyleCop now. Trying to include it into the continuous integration process in the company I work for. We have already coding style guidelines in place and they are not fully matched by the tool. For example, we use _PascalCasingWithUnderscore notation for fields and tabs for indentation. I don’t think that disabling the corresponding rules is the right thing to do in our case, because a guideline without an enforcing process worth nothing.

So we decided to implement our own custom rules to enforce our coding style. As already described, custom rules are quite simple to write. The API is very comprehensive and development is driven almost by intellisense.

The real pain is to test your custom rules. You have to restart Visual Studio (because it locks rules assemblies), open a project, start analysis, and review rules violations. All this clicking is not for cool developers of course.

Automated tests are just for that case. All we need is to build a base testing class which will setup a SourceAnalysisConsole — the class used to run the analysis:

[TestClass]

public abstract class SourceAnalysisTest

{

    private CodeProject _Project;

 

    public SourceAnalysisConsole Console { get; private set; }

    public List<string> Output { get; private set; }

    public List<Violation> Violations { get; private set; }

 

    protected SourceAnalysisTest()

    {

        var settings = Path.GetFullPath("Settings.SourceAnalysis");

        var addinPaths = new List<string>();

        Console = new SourceAnalysisConsole(settings, false, null, addinPaths, true);

 

        Console.ViolationEncountered +=

            ((sender, args) => Violations.Add(args.Violation));

        Console.OutputGenerated +=

            ((sender, args) => Output.Add(args.Output));

    }

 

    [TestInitialize]

    public void Setup()

    {

        Violations = new List<Violation>();

        Output = new List<string>();

 

        var configuration = new Configuration(new string[0]);

        _Project = new CodeProject(Guid.NewGuid().GetHashCode(), null, configuration);

    }

 

    [TestCleanup]

    public void TearDown()

    {

        _Project = null;

    }

 

    public void AddSourceCode(string fileName)

    {

        fileName = Path.GetFullPath(fileName);

        Console.Core.Environment.AddSourceCode(_Project, fileName, null);

    }

 

    public void StartAnalysis()

    {

        var projects = new[] { _Project };

        Console.Start(projects, true);

    }
    …
}

The Settings.SourceAnalysis file can be customized and must be deployed with the tests. The SourceAnalysisConsole object is reused between test runs, but the CodeProject object will be re-instantiated for each particular test method run. This gives us ability to add different code files for each test method. A concrete test class can now look something like this:

[TestClass]

public class ElementAnalyzerTest : SourceAnalysisTest

{

    [TestMethod]

    public void FieldNamesMustBeginWithUnderscore_Invalid()

    {

        AddSourceCode("Invalid.cs");

 

        StartAnalysis();

 

        AssertViolated(RuleNames.FieldNamesMustBeginWithUnderscore, new[] { 9, 10 });

    }

 

    [TestMethod]

    public void FieldNamesMustBeginWithUnderscore_Valid()

    {

        AddSourceCode("Valid.cs");

 

        StartAnalysis();

 

        AssertNotViolated(RuleNames.FieldNamesMustBeginWithUnderscore);

    }

}

Isn’t it sexy? For each test method we first set the code file to analyze. Then we trigger the analysis and verify the expectations. The AssertViolated and AssertNotViolated methods are implemented by the base class. The former method receives the name of the rule which is expected to be violated and the line numbers of the rule violating code. The later method receives only the name of the rule that should not be violated by the code file. Implementation of the assertion logic is the following:

public void AssertNotViolated(string ruleName)

{

    if (Violations.Exists(x => x.Rule.Name == ruleName))

    {

        Assert.Fail("False positive for rule {0}.", ruleName);

    }

}

 

public void AssertViolated(string ruleName, int[] lineNumbers)

{

    if (lineNumbers != null &&

        lineNumbers.Length > 0)

    {

        foreach (var lineNumber in lineNumbers)

        {

            if (!Violations.Exists(x =>

                x.Rule.Name == ruleName &&

                x.Line == lineNumber))

            {

                Assert.Fail(

                    "Failed to violate rule {0} on line {1}.",

                    ruleName,

                    lineNumber);

            }

        }

    }

}

I was worrying how much time will these tests take to run, because the tests use the file system to access the code and settings files. But the results were very promising for a file with ~100 lines of code. The first test method was about 500 ms, but all the next methods run about 15 ms, which is pretty fast (500 tests in 7,5 seconds).

I would like to contribute this code along with the custom rules that we already built to the community. But I don’t know yet what the position of Microsoft is about extensibility in StyleCop. Today we hear quite different opinions. Should we create StyleCop Contrib project at CodePlex? Stay tuned.

Technorati Tags:

Advertisements

Written by Sergey Shishkin

30.05.2008 at 00:43

Posted in Uncategorized

12 Responses

Subscribe to comments with RSS.

  1. Very cool!  Whether Microsoft likes it or not, it\’s clear the user-community\’s going to customize the rules to meet their own needs.

    Paul

    30.05.2008 at 04:48

  2. Hi. I was wondering if you have any examples of creating custom rules for Style Cop version 4.3? I have attempted to load the example you have created in version 4.2, and I replaced the SoruceAnalyzer references with the updated 4.3 Microsoft.StyleCop references, and this example does not seem to work in version 4.3. -Dave

    David McGhee

    18.09.2008 at 23:13

  3. Hello dudewhere can I find the SourceAnalysisConsole file? i mean, i had found a project named StyleCopCmd, and it fails when i tryed to build it in VS2008, told me it can not find that file.

    mic

    22.05.2009 at 17:18

  4. the location of the CodeProject class is now (4.3.1.3) mandatory!1)string location = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, <RelativePathToYour StyleCop.Settings File>); m_Project = new CodeProject(Guid.NewGuid().GetHashCode(), location, configuration);2)change SourceAnalysisConsole to StyleCopConsole … and it works like a charm

    Christoph

    07.07.2009 at 16:20

  5. Very good example. I made the minor mods mentioned and this class WORKS! Don\’t forget to mark your source code items as "Copy Always". Include them on your class as [DeploymentItem("MyRuleCode.cs")] attributes. I also did the samething for my Settings.StyleCop file for my test project. Essentially anything that says SourceAnalysis change to StyleCop and this will probably work.

    Don

    23.09.2009 at 00:55


Comments are closed.

%d bloggers like this: