Be friendly with Web Test

I’m creating some web tests with Visual Studio to test an application in an end-to-end fashion. This is absolutely not a unit test, but I want to be able to launch a series of automatic tests against a web server to verify if the whole site satisfy an initial set of core requirements.

One of this test is used to verify if some filters are passed correctly to the Service Layer, because this is a really core feature, and sometimes it happened that someone changes name of parameters, binding will fail and filters does not work anymore on various pages. One of the test populates a couple of textbox with a date range, I set range between 1/1/1900 and 1/1/1901, and I want to verify that this query returns no records. I admit that this is absolutely a bad way to write a test :), but I want to be able to “smoke test” the page as quickly as possible. If the page returns no record, the filter was surely passed because I’m sure that there is no record in that text range.

The page is quite complex, so I do not want to test the absence of records looking for the message “your query returns no record”, because if someone changes the message this test will have no meaning. When you do this kind of “end-to-end” test, it is useful to output in the page some debug information that can be used from the test.

image

I have a property called TotalResults of the page that contains the total number of records returned, and I decided to output this value with an hidden field. It makes the page bigger, but since the page contains really a lot of data, adding such a simple piece of html does not affects performances. The use of an hidden field, make possible for the web test to put that value in the context of the text, thanks to the HiddenField extraction rule.

image

You can see that in the first iteration records returned are 8, then you can scroll down the context to find value in the subsequent iteration. The second request is the one with the filter active

image

And you can easily verify that the value of the TotalResult is zero. Now I can simply insert a rule that look for the test in the response output.

image

And look for the text ‘<input name="dbTotalResult" type="hidden" value="0"/>’ in the response of the page.

Alk.

Tags:

Coded Ui Test in VS 2010

I must admit that I’m absolutely not a fan of testing through UI. A good program keeps the business logic separated from the UI, so it is testable without passing for the UI. In the real world we have application written by others, or you can simply have to test UI control interaction. In these scenario interacting with the UI can be a viable option. Let’s see how VS2010 can helps us.

You have a new type of unit test, called Coded UI test, when you create test the test will ask you how you will write the test

image

Simply choose to record action, a recording toolbar will shows up

image

Now simply press record, then press “Start without Debugging” on my project. My program opens (a simple form to divide two integers), and I begin interact with the form. Here is the result.

image

As you can see the recorded intercept the launch of my application, and the interaction with the form. When you finished recording, you can simply press the “Generate Code button” and have a new test generated.

Now I suggest you to rename the test (VS calls it CodedUITestMethod1), and when you run it, you can look at the program open again and all of your actions replayed :).

Alk.

Tags:

Rhino Mocks Arrange Act Assert and expectation on generic methods

I have a project where some actions are to be scheduled by an external library. Each action can be instantiated by a constructor or by a static factory method, and I want to verify with the test that: all action declares static factory method and inside the factory methods all objects are resolved by a call to IoC.Resolve<T>. Since all the actions are in a specific assembly into a specific namespace, I wrote this test.

Assembly asm = Assembly.Load("MyProject.Analyzer");
foreach (Type type in asm.GetTypes())
{
    if (type.Namespace == "Myproject.Analyzer.Command" && !type.IsAbstract && !type.IsNested)
    {
        MethodInfo factory = type.GetMethod("CreateCommand", BindingFlags.Public | BindingFlags.Static);
        Assert.That(factory, Is.Not.Null, "Il comando {0} non ha il metodo factory", type.FullName);
        Object res = factory.Invoke(null, null);
        Assert.That(res, Is.TypeOf(type));
    }
}

This test is simple, it load the assembly with actions, then iterate trough all types, and if the type belong to a specific namespace, it verify the presence of a static public method called “CreateCommand”, then call that method and verify that the return object is of correct type.

This test is flawed, because if you mistype the name of the namespace as I did (typed Myproject instead of MyProject) the test will succeed because no type is inspected. This is the typical situation where the test have complex code (reflection) and the test passed, not because it verifies the correct expectations, but because the test itself is bugged :). The solution is simple.

Int32 typeCount = 0;
Assembly asm = Assembly.Load("RepManagement.Analyzer");
foreach (Type type in asm.GetTypes())
{
    if (type.Namespace == "RepManagement.Analyzer.Command" && !type.IsAbstract && !type.IsNested)
    {
        typeCount++;
         //Do your assertion here
        }

    }
}
Assert.That(typeCount, Is.GreaterThan(0));

In this way we are sure that at least one type is analyzed.

But the test is still not so good, it not only tests the presence of the Factory Method, but it depends from the IoC engine, since all factories methods relay on method IoC.Resolve<ActionType>(). So I need a way to verify that each factory method call the right IoC method and return the object created by the IoC engine. A possible solution is this test

if (type.Namespace == "MyProject.Analyzer.Command" && !type.IsAbstract && !type.IsNested)
{
    IWindsorContainer mock = MockRepository.GenerateStub<IWindsorContainer>();
    using (IoC.OverrideGlobalContainer(mock))
    {
        MethodInfo factory = type.GetMethod("CreateCommand", BindingFlags.Public | BindingFlags.Static);
        Assert.That(factory, Is.Not.Null, "Il comando {0} non ha il metodo factory", type.FullName);
        Object retvalue = Activator.CreateInstance(type);
        mock.Expect(c => c.Resolve(type)).Return(retvalue);

        Object res = factory.Invoke(null, null);
        Assert.That(res, Is.EqualTo(retvalue));
    }

}

But it does not work, because in factory method my commands does not call IoC.Resolve(typeof(xxxx)) but IoC.Resolve<xxxx>(), where xxxx is the type of the command, here is the typical implementation.

public static SchedulableWebReview CreateCommand()
{
   SchedulableWebReview action = BaseServices.IoC.Resolve<SchedulableWebReview>();
   return action;
}

Now I can change all factory methods to call other method, but I really do not like to do this. The test should be write in this way.

MethodInfo factory = type.GetMethod("CreateCommand", BindingFlags.Public | BindingFlags.Static);
Assert.That(factory, Is.Not.Null, "Il comando {0} non ha il metodo factory", type.FullName);
Object retvalue = Activator.CreateInstance(type);
mock.Expect(c => c.Resolve<???>()).Return(retvalue);

But since the type is known only a run-time, it is not possible to write this expectation natively or at least I did not know it (what do you write instead of ??? in the snippet above? ). You can be tempted to use c.Resolve<Object>, but this does not work, since in this way rhino mocks expects a real call to Resolve<Object> and will not match a call such as Resolve<SchedulableWebReview>(). A possible solution is the following one

using (IoC.OverrideGlobalContainer(mock))
{
    MethodInfo factory = type.GetMethod("CreateCommand", BindingFlags.Public | BindingFlags.Static);
    Assert.That(factory, Is.Not.Null, "Il comando {0} non ha il metodo factory", type.FullName);

    ParameterExpression parameter = Expression.Parameter(typeof(IWindsorContainer), "c");
    MethodInfo resolveGeneric =
        typeof(IWindsorContainer).GetMethod(
        "Resolve",
        BindingFlags.Instance | BindingFlags.Public,
        null,
        new Type[] { },
        new ParameterModifier[] { });
    MethodInfo resolveThisType = resolveGeneric.MakeGenericMethod(type);
    Expression body = Expression.Call(parameter, resolveThisType);

    Expression<Function<IWindsorContainer, Object>> test2 =
        Expression.Lambda<Function<IWindsorContainer, Object>>(body, parameter);
    Object returnObject = Activator.CreateInstance(type);
    mock.Expect(test2.Compile()).Return(returnObject);

    Object res = factory.Invoke(null, null);
    Assert.That(res, Is.EqualTo(returnObject));
}

The solution was to recreate with ExpressionTree the real lambda that you cannot create at compile time, it is quite simple, and is composed by these steps

  1. Create the ParameterExpression of type IWindsorContainer
  2. Get the Resolve<T>() methodInfo with reflection
  3. Use the MakeGenericMethod to create a MethodInfo for Resolve<Type> where type is passed at runtime
  4. Create a MethodCallExpression that represents a call to this method.
  5. Create the lambdaExpression

Now I create an instance of the Command type with Activator.CreateInstance, and finally thanks to the Compile() method of lambda expression I’m able to set the expectation :). Finally I invoke the factory method, and set an assertion that verifies that return value of the factory method is the object set as return in the expectation.

I do not know if this method is really good but it works. It has the drawback of using the Compile() method of the LambdaExpression, that is really slow at runtime, and can slow down your unit test suite. In my computer the whole test (with 18 commands to test, thus 18 calls to Compile()) runs in 1.50 secs, but most of the time is spent initializing the test. If I run the whole test fixture class (10 tests) it runs in 1.60 secs, and if I remove this test and run the 9 remaining test execution time is 1.50 approx, and this is fine.

alk.

Tags: