How we do automated regression testing with Selenium and Hudson

When developing a piece of software that has a lifecycle that spans over several years and periodically is released you have to do regression testing, i.e. making sure that previous features doesn't break because of new stuff.

And after a while when your software grows and you add new features, the amount of regression tests increases. To avoid drowning yourself with testing you need to automate as much as possible.

The Sculptor team is a bunch of guys that are driven by interest and are developing the software on there spare time during late nights. We doesn't have the time to do deep manual testing, hence automation is very attractive for us. And since we are geographically distributed and don't have a central CI environment we have to solve some practical problems locally.

What we do (amongst other things), involves having a local Hudson server running on our developing environment (i.e. my iMac) that (of course) builds all projects and runs unit tests.

But we also use Selenium to run automated functional tests to make sure that our example application works. I though I should show some more details about how we use Selenium.

As I said, we have an example application called Library. If you look in the source code for the library-web module you will find the directory:

src/main/webapp/selenium

In there you will find a bunch of tests. The root is the suite file:

___test-suite.xhtml

Selenium test can be written in various programming languages. We have chosen to keep it simple and implement the tests in html.

Having these tests enables us to run them as soon as the code base changes thanks to Hudson and Maven. In Hudson we just creates a new job that is triggered to run as soon as the Library projects is compiled and the unit tests passes. In the pom-file for the Library web project we have a profile that we can use to start a local instance of the Jetty server. Deploy the the application. Run our selenium tests. And finally stop the server. The Maven configuration for this is:

<profile>
<id>regression</id>
<build>
<plugins>
 <plugin>
   <groupId>org.mortbay.jetty</groupId>
   <artifactId>maven-jetty-plugin</artifactId>
   <version>6.1.11</version>
   <configuration>
     <scanIntervalSeconds>10</scanIntervalSeconds>
     <stopKey>foo</stopKey>
     <stopPort>9999</stopPort>
   </configuration>
   <executions>
     <execution>
       <id>start-jetty</id>
       <phase>pre-integration-test</phase>
       <goals>
         <goal>run</goal>
       </goals>
       <configuration>
         <scanIntervalSeconds>0</scanIntervalSeconds>
         <daemon>true</daemon>
       </configuration>
     </execution>
     <execution>
       <id>stop-jetty</id>
       <phase>post-integration-test</phase>
       <goals>
         <goal>stop</goal>
       </goals>
     </execution>
   </executions>
 </plugin>
 <plugin>
   <groupId>org.codehaus.mojo</groupId>
   <artifactId>selenium-maven-plugin</artifactId>
   <version>1.0</version>
   <configuration>
     <suite>src/main/webapp/selenium/___test-suite.xhtml</suite>
     <browser>*firefox</browser>
     <results>${project.build.directory}/target/selenium.html</results>
     <startURL>http://localhost:8080/${artifactId}</startURL>
   </configuration>
   <executions>
     <execution>
       <id>run-tests</id>
       <phase>integration-test</phase>
       <goals>
         <goal>selenese</goal>
       </goals>
     </execution>
   </executions>
 </plugin>
</plugins>
</build>
</profile>

But, since the amount of tests keeps growing and the time it takes to run them all also grows, there is a need to be able to run just a single test or a few of them. For instance, it is very convenient to have that possibility when developing new features, fixing bugs, or just doing a refactoring. To enable this we use the very competent Firefox plugin called SeleniumIDE. Beside recording test case, you can also load already defined test cases (or suites) and run them.

Promote Quality with Sculptor

We have written an article that has been published in the paper magazine JayView Issue 20.

Without a vision of how to design applications within an organization the development can be compared to lawless Wild West. Development guidelines are often used, but seldom successful over the long haul. We suggest taking the architectural decisions one step further by automating them using a tool such as Sculptor.

When using a general purpose language, such as Java, and its big toolbox of APIs and frameworks there is a huge freedom of choice. This is a double-edged sword. We meet a lot of companies that have realized that they must narrow down the choices so that each new project doesn't invent its own unique system architecture and product suite. The benefits of a homogeneous architecture is obvious when looking at the big picture.

The reference architecture is often accomplished by writing guidelines and maybe a sample reference application. There are several problems with a reference architecture that is only promoted by documentation. We suggest automating some pieces of the development by using a code generator tool, such as Sculptor, to enforce consistency in the architecture.

Read more in the full article.

Decouple modules with asynchronous event dispatching using Spring and task queues in GAE

To build applications that are maintainable and robust you should strive for decoupling between modules. To build applications that scale you will always benefit from asynchronism and parallellism.
Here we will look how to accomplish the above in GoogleAppEngine and with some help from springframework.
Lets say we have an application where users can register them self. When they do, the application creates a persistence instance of a User-object. But, we will also keep track of how many users we have registered on the site. Now, being in GAE with BigTable luring in the back, doing queries and calculations (as we are used to with a traditional database) isn't a good idea. So as an alternative we choose to have a separate Counter-object that we updates when ever a new user registers. Ok, nothing strange here. But, there are a couple of flaws here:

1. The User module needs to know about the Counter module.
2. The User module has to wait for the Counter module to finish when updating the counting.

Ok, lets solve the first by using spring's mechanism for ApplicationEvent's. First, let us put some aop magic to work to intercept the call to UserService.createUser and when it returns (and we have the transaction boundaries on service methods, so no exception, all went well) fire off an event. Spring config for the aop stuff:

<bean id="userListener" class="org.fornax.sculptor.UserListener"/>
<bean id="userAdvice" class="org.fornax.sculptor.UserAdvice"/>
<aop:config>
<aop:pointcut id="userCreationPointcut" expression="execution(public * org..UserService.createUser(..))"/>
<aop:advisor pointcut-ref="userCreationPointcut" ref="userCreationPointcut"/>
</aop:config>

Next, here is the advice:

public class UserAdvice implements MethodInterceptor, ApplicationContextAware {

private ApplicationContext ctx;

public Object invoke(MethodInvocation invocation) throws Throwable {
  User user = (User) invocation.proceed();
  fireNewUserEvent(user);
  return user;
}

private void fireNewUserEvent(User user) {
  ctx.publishEvent(new UserCreatedEvent(user));
}

public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
  this.ctx = applicationContext;
}
}

The listener that is being notified:

public class UserListener implements ApplicationListener<UserCreatedEvent> {
@Autowired private CounterService counterService;
public void onApplicationEvent(UserCreatedEvent event) {
  counterService.increment();
}
}

And the event being passed:

public class UserCreatedEvent extends ApplicationEvent {
public UserCreatedEvent(User user) {
  super(user);
}
}

Ok, so now we are half way. We have the Observer pattern in place. But we still does everything synchronous.
Enter GAE's task queue's. Let us modify our UserListener:

public class UserListener implements ApplicationListener<UserCreatedEvent> {

public void onApplicationEvent(UserCreatedEvent event) {
  TaskOptions task = url("/rest/admin/counter/user").method(POST);
  Queue queue = QueueFactory.getDefaultQueue();
  queue.add(task);
}
}

And by the wonders of task queue's, we now put a task on the queue and by that we do the counting job asynchronous. And of course, we dropped the reference to the CounterService. But we miss one piece here, right? What does the url in the task point at. Well, nothing strange here, it is just a spring mvc controller:

@Controller
public class CounterCountroller {
@Autowired private CounterService counterService;
@RequestMapping(value = "/admin/counter/user", method = RequestMethod.POST)
public void incrementCounter() throws IOException {
  try {
    counterService.increment();
  } catch (Exception ignore) {
    // doesn't matter if we get an exception here, just log it
    log.error("Failed to increment counter!", ignore);
  }
}
}

And now we have a more loosely coupled system that scales better. And with a little effort, the code can be generalized so more features are easy to add with the same pattern.
Of course, the downside of this kind of design is that error handling gets more complicated and you can't always trust it to be 'right'. But that is system design, you have to decide what's best for each situation.

Even Weird Naming Conventions are Good

The good thing with naming conventions are that they are toolable. The DBAs at my department have strong opinions about database naming. They have good reasons for that, even though I don't fully understand all of them :-)

• Table names should be prefixed with application/component identifier.
• Primary key id column should be prefixed with table name (without application prefix) and followed by _GID.
• Underscore to separate words.
• Foreign key column is concatenation of role name and primary key column name of target table, except when role and table have the same name.

Does this mean that we have to specify each and every name twice, once for Java and once for the database. Argh... NO, we are using Sculptor. With a straightforward customization I implemented these conventions in the generator and we could continue with natural (java point of view) naming and please the preferences of the DBAs without additional effort.

Naming conventions are important for software quality. Supporting the conventions with a tool is the best way to make sure that they are applied in a consistent way.