Sergey Shishkin

on agile software development

Archive for May 2008

StyleCop: Test Driven Rules Development

I’m playing with StyleCop now. Trying to include it into the continuous integration process in the company I work for. We have already coding style guidelines in place and they are not fully matched by the tool. For example, we use _PascalCasingWithUnderscore notation for fields and tabs for indentation. I don’t think that disabling the corresponding rules is the right thing to do in our case, because a guideline without an enforcing process worth nothing.

So we decided to implement our own custom rules to enforce our coding style. As already described, custom rules are quite simple to write. The API is very comprehensive and development is driven almost by intellisense.

The real pain is to test your custom rules. You have to restart Visual Studio (because it locks rules assemblies), open a project, start analysis, and review rules violations. All this clicking is not for cool developers of course.

Automated tests are just for that case. All we need is to build a base testing class which will setup a SourceAnalysisConsole — the class used to run the analysis:

[TestClass]

public abstract class SourceAnalysisTest

{

    private CodeProject _Project;

 

    public SourceAnalysisConsole Console { get; private set; }

    public List<string> Output { get; private set; }

    public List<Violation> Violations { get; private set; }

 

    protected SourceAnalysisTest()

    {

        var settings = Path.GetFullPath("Settings.SourceAnalysis");

        var addinPaths = new List<string>();

        Console = new SourceAnalysisConsole(settings, false, null, addinPaths, true);

 

        Console.ViolationEncountered +=

            ((sender, args) => Violations.Add(args.Violation));

        Console.OutputGenerated +=

            ((sender, args) => Output.Add(args.Output));

    }

 

    [TestInitialize]

    public void Setup()

    {

        Violations = new List<Violation>();

        Output = new List<string>();

 

        var configuration = new Configuration(new string[0]);

        _Project = new CodeProject(Guid.NewGuid().GetHashCode(), null, configuration);

    }

 

    [TestCleanup]

    public void TearDown()

    {

        _Project = null;

    }

 

    public void AddSourceCode(string fileName)

    {

        fileName = Path.GetFullPath(fileName);

        Console.Core.Environment.AddSourceCode(_Project, fileName, null);

    }

 

    public void StartAnalysis()

    {

        var projects = new[] { _Project };

        Console.Start(projects, true);

    }
    …
}

The Settings.SourceAnalysis file can be customized and must be deployed with the tests. The SourceAnalysisConsole object is reused between test runs, but the CodeProject object will be re-instantiated for each particular test method run. This gives us ability to add different code files for each test method. A concrete test class can now look something like this:

[TestClass]

public class ElementAnalyzerTest : SourceAnalysisTest

{

    [TestMethod]

    public void FieldNamesMustBeginWithUnderscore_Invalid()

    {

        AddSourceCode("Invalid.cs");

 

        StartAnalysis();

 

        AssertViolated(RuleNames.FieldNamesMustBeginWithUnderscore, new[] { 9, 10 });

    }

 

    [TestMethod]

    public void FieldNamesMustBeginWithUnderscore_Valid()

    {

        AddSourceCode("Valid.cs");

 

        StartAnalysis();

 

        AssertNotViolated(RuleNames.FieldNamesMustBeginWithUnderscore);

    }

}

Isn’t it sexy? For each test method we first set the code file to analyze. Then we trigger the analysis and verify the expectations. The AssertViolated and AssertNotViolated methods are implemented by the base class. The former method receives the name of the rule which is expected to be violated and the line numbers of the rule violating code. The later method receives only the name of the rule that should not be violated by the code file. Implementation of the assertion logic is the following:

public void AssertNotViolated(string ruleName)

{

    if (Violations.Exists(x => x.Rule.Name == ruleName))

    {

        Assert.Fail("False positive for rule {0}.", ruleName);

    }

}

 

public void AssertViolated(string ruleName, int[] lineNumbers)

{

    if (lineNumbers != null &&

        lineNumbers.Length > 0)

    {

        foreach (var lineNumber in lineNumbers)

        {

            if (!Violations.Exists(x =>

                x.Rule.Name == ruleName &&

                x.Line == lineNumber))

            {

                Assert.Fail(

                    "Failed to violate rule {0} on line {1}.",

                    ruleName,

                    lineNumber);

            }

        }

    }

}

I was worrying how much time will these tests take to run, because the tests use the file system to access the code and settings files. But the results were very promising for a file with ~100 lines of code. The first test method was about 500 ms, but all the next methods run about 15 ms, which is pretty fast (500 tests in 7,5 seconds).

I would like to contribute this code along with the custom rules that we already built to the community. But I don’t know yet what the position of Microsoft is about extensibility in StyleCop. Today we hear quite different opinions. Should we create StyleCop Contrib project at CodePlex? Stay tuned.

Technorati Tags:

Advertisements

Written by Sergey Shishkin

30.05.2008 at 00:43

Posted in Uncategorized

StyleCop: Source Analysis Tool for C#

Recently Microsoft has finally released its internal source analysis tool for C#StyleCop. Another analysis tool—FxCop—validates compiled assemblies for conformance to the Framework Design Guidelines. StyleCop goes further and analyses source code files against many coding style rules:

 

The rules are not very flexible. You can only disable particular rules and narrow the scope of some rules (like require XML docs for public and protected elements only). But you cannot for example choose whether you like tabs or spaces for indentation—StyleCop does not allow tabs. Other debatable rules are "Field Names Must Not Contain Underscore" and "Prefix Local Calls With This".

I find quite interesting the position of MS in the rules debate. I would summarize it like "It is an internal tool, not a product. Use it as is. We don’t care if it doesn’t fit your organization":

Source Analysis originated as an internal Microsoft tool, which we have decided to release externally so that others can take advantage of it as well, if they would like to. The limited customization of the prefix rules is deliberate, in order to maintain a consistent style across a very large codebase.

At least StyleCop supports writing your own rules. Though MS does not like it:

Please keep in mind that disassembly of the tool or distribution of the source for this tool is a violation of the license that you agreed to when you installed the tool. The license appears during installation.

What?! I really don’t understand yet what MS is going to do with this tool. Their potential strategy could be to use StyleCop and its not flexible rules to force their own coding style among the developers. Organizations will either use the tool as is, or go their own way without any tool to enforce coding guidelines, or without any coding guidelines at all.

Anyway, StyleCop is a great tool helping you to enforce your coding guidelines. Because without control any guideline means nothing. And it can even be integrated into your team build.

Technorati Tags:

Written by Sergey Shishkin

28.05.2008 at 19:04

Posted in Uncategorized

Guidance Automation Extensions: Error 1001

If you get an InvalidOperationException saying that "Operation is not valid due to the current state of the object" when installing the Guidance Automation Toolkit (or any other guidance package, like the Smart Client Software Factory, for example), check whether your cmd.exe has been configured to automatically set Visual Studio environment variables. Look if the registry key "HKLM\Software\Microsoft\Command Processor\AutoRun" is set to something like "%VS90COMNTOOLS%vsvars32.bat" and remove it for a while (while you install the guidance packages).

Don’t ask why. Just try, it works. It saved me from reinstalling Visual Studio and all the other stuff.

Written by Sergey Shishkin

21.05.2008 at 21:52

Posted in Uncategorized