Alkampfer's Place» Testing

Unit testing floating point numbers

alkampfer — Thu, 05 May 2011 16:31:00 +0000

Testing for equality Floating point numbers is usually a bad idea, this because rounding occurs with floating point operations and you need to test with a tolerance. Suppose you test that some algorithm produces the expected result and you find that unit test fails with this message.

SharpTestsEx.AssertException : 2.36 Should Be Equal To 2.36.

This seems strange, but the problem is that the real value is 2.360000000003 that surely is different from 2.36. Now you have two different scenario, the first one is the test is wrong because I want to verify that the two numbers are really equals. With floating point calculation this cannot be achieved, you can use numbers with high precision, but doing operations with any floating point numbers lead to rounding, and you should never test for equality two floating point numbers.

A second and better scenario is when you the test is expressed with a tolerance, so you can write this assertion.

   1: result.Should().Be.EqualTo(expectedresult, 0.00001)

This is a better assertion, Iâ€™m asking if the result is different from the expected value with a 0.00001 tolerance. This is possible with a simple extension of SharpTestEx.

public static IComparableBeConstraints EqualTo(
   this IComparableBeConstraints constraint,
     Double expected, 
     Double tolerance)
 {
     constraint.AssertionInfo.AssertUsing(
         new Assertion(
             "Equal to", expected, a => Math.Abs(a - expected) <= tolerance));
     return constraint;
 }

Such test is much more interesting and correct, because it states the tolerance that you can suffer before stating that the algorithm is wrong.

Alk.

Writing a custom assertion for SharpTestEx

alkampfer — Tue, 22 Mar 2011 15:25:09 +0000

Actually SharpTestEx is my favorite way to make assertions in Unit testing. It has a lot of advantages, first of all it permits to write really clear assertions, then it works on the main Unit testing framework, so I can use the same assertion syntax for nunit and for MStest or MbUnit etc etc.

Another cool advantages is how simple is to extend the syntax, the IEnumerableConstraint has the Contain assertion to verify that an element is contained in a IEnumerable sequence of objects. Now I want to be able to write a ContainAny that accepts a lambda to specify a condition that must be satisfied by at least one of the elements of the collection.

public static IAndConstraints> Contain(
    this IEnumerableConstraints constraint,
    Func checkFunction)
{
    constraint.AssertionInfo.AssertUsing(
        new Assertion, Func>(
            "Contain",
            checkFunction,
            a => (a != null) && a.Any(checkFunction),
            mi =>
                string.Format("{0} {1} {2} {3}.{4}",
                            "Sequence",
                           "Should",
                            mi.AssertionPredicate,
                            Messages.FormatValue(mi.Expected),
                            mi.CustomMessage)));
    return ConstraintsHelper.AndChain(constraint);
}

The code is really simple, it is based on the Assertion class that permits to send to the assertion engine an assertion that verify something. The third parameter of the Assertion constructor accepts a predicate that will be used to validate the assertion itself. In our example I use the Any linq method to verify that at least any of the element of the collection satisfy the original lambda.

now I want to verify that a list of uri does not contain any element with mail in the address

   1: list.Should().Not.ContainAny(u => u.AbsoluteUri.Contains("mail"));

Simple and readable. In case of failure the error is.

   1: failed: SharpTestsEx.AssertException : Sequence Should Not Contain System.Func`2[System.Uri,System.Boolean].

Since this is really not so clear, I changed the assertion function a little bit.

public static IAndConstraints> ContainAny(
         this IEnumerableConstraints constraint,
         Expression> checkFunction)
       {
           constraint.AssertionInfo.AssertUsing(
               new Assertion, Expression>>(
                   "Not contain any element that satisfy predicate",
                   checkFunction,
                   a => (a != null) && a.Any(checkFunction.Compile()),
                   mi =>
                       string.Format("{0} {1} {2} {3}.{4}",
                                   "Sequence",
                                  "Should",
                                   mi.AssertionPredicate,
                                   Messages.FormatValue(mi.Expected),
                                   mi.CustomMessage)));
           return ConstraintsHelper.AndChain(constraint);
       }

The only difference is that now the function accepts an Expression tree, and uses Compile() method to create a real Function method used to search into the sequence. Now when the test fails it writes:

   1: failed: SharpTestsEx.AssertException : Sequence Should Not Not contain any element that satisfy predicate u => u.AbsoluteUri.Contains("mail").

I really love SharpTestEx.

alk.

Test helper for a single method

alkampfer — Wed, 29 Dec 2010 15:40:00 +0000

Iâ€™ve build during years a test helper infrastructure that permits me to decorate a test fixture with some custom attributes that are capable to execute code before and after fixture setup/teardown or test setup/teardown. Now my dear friend Ugo needs for Dexter a modified version of this framework to be able to apply attributes directly to tests.

Basically Ugo needs to change the principal during each test, and he want to be able to write code like this one.

[PrincipalForTest("Alkampfer", new [] {"Administrator", "User"})]
[Test]
public void VerifyPrincipalForTestSingleMethod()
{
    Assert.That(Thread.CurrentPrincipal.IsInRole("Administrator"));
    Assert.That(Thread.CurrentPrincipal.IsInRole("User"));
}
  
[PrincipalForTest("Alkampfer", new[] { "Group" })]
[Test]
public void VerifyPrincipalForTestSingleMethodSingleGroup()
{
    Assert.That(Thread.CurrentPrincipal.IsInRole("Group"));
}

Now the PrincipalForTestAttribute is a standard ITestHelperAttribute defined in this way.

public class PrincipalForTestAttribute : Attribute, ITestHelperAttribute
{
    public int Order
    {
        get { return 1; }
    }
  
    public ITestHelper Create()
    {
        return new PrincipalForTestHelper(UserName, Roles);
    }
  
    public String UserName { get; set; }
  
    public String[] Roles { get; set; }
  
    public PrincipalForTestAttribute(string userName, string[] roles)
    {
        UserName = userName;
        Roles = roles;
    }
}

All the dirty work is made by the PrincipalForTestHelper.

public class PrincipalForTestHelper : ITestHelper
{
   public String UserName { get; set; }
  
   public String[] Roles { get; set; }
  
   public void FixtureSetUp(BaseTestFixture fixture)
   {
      
   }
   public void SetUp(BaseTestFixture fixture)
   {
       IPrincipal currentPrincipal = Thread.CurrentPrincipal;
       GenericIdentity identity = new GenericIdentity(UserName);
       GenericPrincipal newPrincipal = new GenericPrincipal(identity, Roles);
       Thread.CurrentPrincipal = newPrincipal;
       fixture.ExecuteAtTheEndOfTest(() => Thread.CurrentPrincipal = currentPrincipal);
   }

Now my problem is how to be able to execute code of this helper only for the test where this attribute is applied to, and the solution is available only with Nunit 2.5.7 or above, because the concept of TestContext permits me to know test method name from the SetUp method.

First of all in the constructor of the test class I need to scan all methods looking for ITestHelperAttribute

foreach (MethodInfo methodInfo in type.GetMethods())
{
    var customMethodAttributes = methodInfo.GetCustomAttributes(true);
    IOrderedEnumerable testHelperAttributes =
        customMethodAttributes.OfType()
        .OrderBy(th => th.Order);
    if (testHelperAttributes.Count() > 0)
    {
        List  helpers = new List();
        MethodHelpers.Add(methodInfo.Name, helpers);
        foreach (ITestHelperAttribute attribute in testHelperAttributes)
        {
            helpers.Add(attribute.Create());
        }
    }
}

Basically Iâ€™m creating all the test helpers for each method, now during test setup Iâ€™m able to verify the name of the test that is going to be executed, and call the appropriate helpers if they are present.

protected override void OnSetUp ( )
{
    foreach ( ITestHelper helper in Helpers ) {
        helper.SetUp ( this );
    }
    if (MethodHelpers.ContainsKey(TestContext.CurrentContext.Test.Name))
    {
        foreach (ITestHelper helper in MethodHelpers[TestContext.CurrentContext.Test.Name])
        {
            helper.SetUp(this);
        }
    }
    base.OnSetUp ( );
}

In the first foreach Iâ€™m calling SetUp() method for every helper associated with the fixture (it should be executed for each test) and then I verify if the MethodHelpers dictionary contains some test helpers for the current test. The key here is that Iâ€™m able to know the test method name from TestContext.CurrentContext.Test.Name thanks to new Nunitâ€™s functionality.

In order for this to work with TestDriven.Net you need to copy latest nunit testrunner in the appropriate Testdriven.net Folder (C:Program Files (x86)TestDriven.NET 3NUnit2.5)

Alk.

Castle and Automock, avoid resolving properties

alkampfer — Tue, 30 Nov 2010 14:24:41 +0000

I use AutoMockingContainer extensively in my test projects, and I â€˜ve build over time an automocking container that satisfy all of my needs. Thanks to Castle Windsor, using complex logic with the AutomockingContainer is a breeze. Suppose you have this ViewModel

Figure 1: ViewModel under test

The only aspect Iâ€™m interested in is the SelectedLinkResult public property, that have a lot of logic in the set part, this is needed to react on user selection change in the UI. Now when I use AutoMockingContainer to automock this view model I have a big problem, I need to setup a lot of expectations to make the setter logic to work, this because when I try to resolve with automock, my AutoMockingContainer try to resolve each dependency, even properties. To avoid this I need to be able to tell my container

Avoid to resolve any property
Avoid to resolve dependencies with specific name

The second one is needed if I want some properties to be resolved and some other to be excluded, so I needs to extend my container with a couple of properties.

public List DependencyToIgnore { get; set; }
  
public Boolean  ResolveProperties { get; set; }
  
public Boolean CanSatisfyDependencyKey(String dependencyKey)
{
   return !DependencyToIgnore.Contains(dependencyKey);
}

Iâ€™ve added a List of property name to ignore, a property that specify if I want properties to be resolved, and finally a simple function that tells if a specific property needs to be resolved. Thanks to Castle, I can use these properties from the subresolver used by the container.

public bool CanResolve(
     CreationContext context,
     ISubDependencyResolver parentResolver,
     ComponentModel model,
     DependencyModel dependency)
{
    bool shouldResolveDependencyKey = 
        dependency.DependencyType == DependencyType.Service &&
        _relatedRepository.CanSatisfyDependencyKey(dependency.DependencyKey);
  
    Boolean resolveIfProperty = 
        !dependency.IsOptional || 
        _relatedRepository.ResolveProperties;
  
    return shouldResolveDependencyKey && resolveIfProperty;
}

First of all I check if the dependencyType is Service and calls the CAnSatisfyDependencyKey of the subresolver, then if the the dependency is optional (a property) and the ResolveProperties is false, I skip resolution. Now I can write a unit test header like this one:

[TestFixture]
[UseAutoMockingContainer(
    new[] { typeof(TreeNavigatorViewModel) }, 
    ResolveProperties = false)]
public class TreeNavigatorViewModelFixture : BaseTestFixtureWithHelper

This to avoid completely Property resolving during the test, the following one is used if I want only the SelectedLinkResult property not to be resolved with a Mock.

[TestFixture]
[UseAutoMockingContainer(
    new[] { typeof(TreeNavigatorViewModel) }, 
    IgnoreDependencies =  
    new string[]
    {
       "SelectedLinkResult"                           
    })]
public class TreeNavigatorViewModelFixture : BaseTestFixtureWithHelper

This makes my tests really simple and simple to write.

alk.

Feature Pack 2

alkampfer — Mon, 22 Nov 2010 10:44:08 +0000

Some days ago the Feature pack 2 was released. This pack will extend testing capabilities of VS2010, and one of the coolest addition is a graphical editor for the UIMap of Coded UI test.

Another killer feature is the official support to run coded ui test With Mozilla firefox 3.5 or higher and support for Coded UI testing silverlight 4 applications.

If you are using Coded UI Test you cannot miss it.

Alk.

Advanced capability in Web Performance Test of VS2010

alkampfer — Fri, 12 Nov 2010 15:49:58 +0000

Suppose you have a very stupid web page that adds two numbers together.

Figure 1: A stupid web page to sum two numbers

WOW, this is really a beautiful and useful web page , now I want to register a web performance test that is able to use extraction rules to use this page as a Fibonacci Number calculator. I start registering a simple web performance test that adds 1 and 2 together. This is a simple two steps web performance test.

Figure 2: recorded web performance test with two simple pages.

Now execute the test in Visual Studio, then go to request tab, localize the txtB parameter and press right click and then Quick Find. Now in the Quick find windows press â€œfind nextâ€ to quickly localize the position of the input textbox in the response of the second request. This permits you to immediately localize where the page render the text.

Figure 3: With quick find I can localize with easy values of the request

Now select the value â€œ2â€ in the response text, right click and then select Add Extraction Rule, repeat the same with the value 3 of the lblResult label. Now the test contains two extraction parameters called param0 and param1, just rename them to give them more meaningful names.

Figure 4: Rename extracted parameter to some more meaningful names.

Now select the second request in the test and copy with CTRL+C and CTRL+V, now edit the form post parameter setting the txtA parameter to context parameter B and txtB field value to context parameter result. Remember that for Fibonacci numbers you can grab the next number in the series just adding the last two numbers (B and result of the previous request).

Figure 5: copy the second request and use the context parameter extracted in the previous request

Now right click the request, and choose â€œInsert Loopâ€, simply select a For Loop and setup the loop to do 10 iterations.

Figure 6: A loop is added to the request, now the copied request will be executed 10 times.

Now execute the test, and with few clicks of the mouse you have created a test that calculates fibonacci numbers.

Figure 7: The fibonacci number calculator web performance test in action.

This example is really not so useful, but demonstrates how you can create a complex web automation test, with loops and requests that depends on the previous response, thanks to extraction rules.

Alk.

Extending Visual Studio 2010 Web Test–Regex extraction

alkampfer — Mon, 08 Nov 2010 08:49:48 +0000

In a previous post I showed how to create a custom loop that permits you to create a loop in a web performance test to iterate from the char â€˜aâ€™ to char â€˜zâ€™, now I want to be able to create an inner loop that

for each loop extract all the names of the customers that satisfy the search
for each name ask for detail

I need another custom loop that is able to extract strings from the body of a response and iterate for each string. Here is the full code.

public class RegexLoop : ConditionalRule
{
    private Int32 CurrentMatch { get; set; }
    private String LastUrl { get; set; }
    private MatchCollection Matches { get; set; }
  
    // Methods
    public override void CheckCondition(object sender, ConditionalEventArgs e)
    {
  
        if (CurrentMatch < Matches.Count)
        {
            e.WebTest.Context[ContextParameterName] =
                Matches[CurrentMatch].Groups[GroupName].Value;
            e.IsMet = true;
            CurrentMatch++;
            return;
  
        }
  
        e.IsMet = false;
  
    }
  
    public override void Initialize(object sender, ConditionalEventArgs e)
    {
        CurrentMatch = 0;
        Matches = System.Text.RegularExpressions.Regex.Matches(
            e.WebTest.LastResponse.BodyString,
            Regex,
            RegexOptions.IgnoreCase);
    }
  
    public override string StringRepresentation()
    {
        return "Regex condition " + Regex;
    }
  
    [Description("Name of context parameter where the current value should be set"),
    DisplayName("Context parameter"), IsContextParameterName(true)]
    public string ContextParameterName { get; set; }
  
    [Description("Regex"),
    DisplayName("Regex")]
    public String Regex { get; set; }
  
    [DisplayName("GroupName"), Description("GroupName")]
    public String GroupName { get; set; }
  
  
}

Listing 1: The full code of the RegexLoop plugin class

This plug-in has two property, the first Regex is used to define the regex to use, the other one is the name of Contextparameter used to store the result of the regex. This plugin simply iterates to each match of the regex, and permits you to create a loop based on text of the last request. This plugin is really simple, during initialization I execute the regex against the last response to grab a MatchCollection

public override void Initialize(object sender, ConditionalEventArgs e)
{
  CurrentMatch = 0;
  Matches = System.Text.RegularExpressions.Regex.Matches(
      e.WebTest.LastResponse.BodyString,
      Regex,
      RegexOptions.IgnoreCase);
}

Listing 2: The initialization function

Thanks to the e.WebTest.LastReponse.BodyString Iâ€™m able to access the body of the last response and execute the regex against it. The main loop is now straightforward to write

public override void CheckCondition(object sender, ConditionalEventArgs e)
{
  
    if (CurrentMatch < Matches.Count)
    {
        e.WebTest.Context[ContextParameterName] =
            Matches[CurrentMatch].Groups[GroupName].Value;
        e.IsMet = true;
        CurrentMatch++;
        return;
  
    }
  
    e.IsMet = false;
}

Listing 3: The main loop, iterate for each match

Now I can modify the test presented in the old post to insert an inner loop that is able to interate to all customers that satisfy the filter for each request.

Figure 1: Insert a loop based on a regular expression thanks to RegexLoop plugin

I inserted another loop inside the first one, after the search request, and insert a regular expression that is able to find all â€œselectâ€ button of the gridview, I asked the regexloop to insert the value in the CurrentCustomer context variable to use in the subsequent request

Fgirue 2: Use the CurrentCustomer context variable in the inner loop request

Now you can execute the test and verify the outcome, Iâ€™m expecting an external loop to search for all alphabet letters, then for each response an inner loop for each customer, as visible in Figure 3.

Figure 3: The result of the test, you can verify that all loops behave correctly.

Thanks to few lines of code now Iâ€™m able to execute loops based on content of a web response, that can be used to create complex web performance test in Visual Studio 2010.

alk.