Monday, November 30, 2009

Internal DSL for Criteria - part 2:3

In previous article I introduced the new internal DSL for JPA/Hibernate criteria. Let us look at how to handle the property names.

criteria().prop("lastName").eq("Svensson").build();
The problem with the above expression is that the property lastName is a String. Refactoring of Person.lastName will not be automatically detected. Another issue is that you have to remember/lookup that the property name is "lastName" when writing this expression, the IDE will not help you.

We, who are using Sculptor, define the domain object in the design model like this:

Entity Person {
String firstName
String lastName
}
From this Sculptor generates the Person class. We can also easily generate meta data of the properties of Person, which makes it possible to write expressions like this:

List<ConditionalCriteria> conditionalCriteria = ConditionalCriteriaBuilder.criteriaFor(Person.class)
.withProperty(PersonProperties.lastName()).eq("Svensson").build();
PersonProperties is generated and it contains static methods for each property, returning a Property<Person> object, which internally defines the strings. The builder is also parameterized, using the class defined in the criteriaFor factory method. This means that the withProperty method only accepts properties of correct type, i.e. Property<Person>.

Static imports can be used to make the expression more compact and readable:

import static org.fornax.cartridges.sculptor.framework.accessapi.ConditionalCriteriaBuilder.criteriaFor;
import static org.library.person.domain.PersonProperties.lastName;

List<ConditionalCriteria> conditionalCriteria = criteriaFor(Person.class)
.withProperty(lastName()).eq("Svensson").build();

Note that the eclipse keyboard shortcut for static import is <ctrl+shift+m> (mac: <cmd+shift+m>).

The expression builder looks like this:

public class ConditionalCriteriaBuilder<T> {

private final List<ConditionalCriteria> criteriaList = new ArrayList<ConditionalCriteria>();

private ConditionalCriteriaBuilder() {
}

public static <T> ConditionalCriteriaBuilder<T> criteriaFor(Class<T> clazz) {
return new ConditionalCriteriaBuilder<T>();
}

public List<ConditionalCriteria> build() {
return criteriaList;
}

private void addCriteria(ConditionalCriteria criteria) {
criteriaList.add(criteria);
}

public CondditionProperty<T> withProperty(Property<T> property) {
return new PropBuilderImpl(property.getName());
}

public interface CondditionProperty<T> {
ConditionalCriteriaBuilder<T> eq(Object value);
}

private class PropBuilderImpl implements CondditionProperty<T> {
String propertyName;

PropBuilderImpl(String name) {
this.propertyName = name;
}

public ConditionalCriteriaBuilder<T> eq(Object value) {
addCriteria(ConditionalCriteria.equal(propertyName, value));
return ConditionalCriteriaBuilder.this;
}
}
}


Alright, let us refactor the model, and introduce a BasicType for the first and last name of the Person:


Entity Person {
- @PersonName name
}

BasicType PersonName {
String first
String last
}


Generate, and you will immediately detect compilation error in the criteria expression. Fix it and it looks like this:

List<ConditionalCriteria> conditionalCriteria = criteriaFor(Person.class)
.withProperty(name().last()).eq("Svensson").build();


The generated Properties classes defines refererences also, so it easy to navigate associations with full code completion support. Great, our goals of code completion and refactoring are fulfilled.

That is not the end of this series of articles. In next article I will illustrate some more advanced operators that requires some intellectual thoughts. See you tomorrow.

Sunday, November 29, 2009

Internal DSL for Criteria - part 1:3

Hibernate has support for criteria queries and that is an awaited feature in JPA 2.0. The criteria API is rather technical and doesn't read to well with your domain terms, your DDD Ubiquitous Language. Therefore I have developed a small internal DSL in Java to be able to express conditional criteria in human readable format. It is not intended to handle everything in the criteria API.

I use this to illustrate how to implement an internal DSL. It is not hard once you have learned the tools to play with. Java is not optimal for DSLs, but let us use it as good as possible.

The goal was to create a fluent api that supported code completion and refactoring in a good way. A simple criteria may look like:

criteriaFor(Person.class).withProperty(sex()).eq(Gender.FEMALE).build();

and a more advanced criteria:

criteriaFor(Person.class)
.withProperty(sex()).eq(Gender.FEMALE).and()
.withProperty(ssn().country()).eq(SWEDEN).and()
.lbrace().withProperty(name().first()).like("A%")
.or().withProperty(name().last()).like("A%").rbrace()
.orderBy(name().last())
.build();
I will show how a few pieces of the DSL was developed. I have mostly used Expression Builder and Method Chaining, but also a few other tricks.

The expression builder doesn't create the JPA/Hibernate criteria directly. It creates an intermediate structure of objects , which can be translated to criteria API. The main reason for that is decoupling and possibility to have vendor specific implementations.

First, let me state the obvious: When developing an internal DSL it is perfect to do it the TDD way. You will add more and more features and do a lot of refactoring until you are satisfied with the language.

Enough of the introductory talking, let us go hands on to a simple equals expression:

List<ConditionalCriteria> conditionalCriteria =   
ConditionalCriteriaBuilder.criteria().prop("name").eq("Svensson").build();
This can be implemented in the builder as:

public class ConditionalCriteriaBuilder {
private final List<ConditionalCriteria> criteriaList = new ArrayList<ConditionalCriteria>();
private String propertyName;

private ConditionalCriteriaBuilder() {
}

public static ConditionalCriteriaBuilder criteria() {
return new ConditionalCriteriaBuilder();
}

public List<ConditionalCriteria> build() {
return criteriaList;
}

public ConditionalCriteriaBuilder prop(String name) {
propertyName = name;
return this;
}

public ConditionalCriteriaBuilder eq(Object value) {
addCriteria(ConditionalCriteria.equal(propertyName, value));
return this;
}

private void addCriteria(ConditionalCriteria criteria) {
criteriaList.add(criteria);
}
}

But that is not very good because it allows invalid expressions, such criteria().eq("A").prop("aaa").eq("B").

Therefore I introduce an interface for the property level:



public PropBuilder prop(String name) {
return new PropBuilderImpl(name);
}

public interface PropBuilder {
ConditionalCriteriaBuilder eq(Object value);
}

private class PropBuilderImpl implements PropBuilder {
String propertyName;

PropBuilderImpl(String name) {
this.propertyName = name;
}

public ConditionalCriteriaBuilder eq(Object value) {
addCriteria(ConditionalCriteria.equal(propertyName, value));
return ConditionalCriteriaBuilder.this;
}
}

This also facilitates code completion (ctrl+space) in a good way, which was one of our goals.

In next post, tomorrow, I will take care of the string property names that are not very refactoring friendly.

Thursday, November 19, 2009

How we do automated regression testing with Selenium and Hudson

When developing a piece of software that has a lifecycle that spans over several years and periodically is released you have to do regression testing, i.e. making sure that previous features doesn't break because of new stuff.
And after a while when your software grows and you add new features, the amount of regression tests increases. To avoid drowning yourself with testing you need to automate as much as possible.

The Sculptor team is a bunch of guys that are driven by interest and are developing the software on there spare time during late nights. We doesn't have the time to do deep manual testing, hence automation is very attractive for us. And since we are geographically distributed and don't have a central CI environment we have to solve some practical problems locally.

What we do (amongst other things), involves having a local Hudson server running on our developing environment (i.e. my iMac) that (of course) builds all projects and runs unit tests.
But we also use Selenium to run automated functional tests to make sure that our example application works. I though I should show some more details about how we use Selenium.

As I said, we have an example application called Library. If you look in the source code for the library-web module you will find the directory:

src/main/webapp/selenium

In there you will find a bunch of tests. The root is the suite file:

___test-suite.xhtml

Selenium test can be written in various programming languages. We have chosen to keep it simple and implement the tests in html.

Having these tests enables us to run them as soon as the code base changes thanks to Hudson and Maven. In Hudson we just creates a new job that is triggered to run as soon as the Library projects is compiled and the unit tests passes. In the pom-file for the Library web project we have a profile that we can use to start a local instance of the Jetty server. Deploy the the application. Run our selenium tests. And finally stop the server. The Maven configuration for this is:
<profile>
<id>regression</id>
<build>
<plugins>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
<version>6.1.11</version>
<configuration>
<scanIntervalSeconds>10</scanIntervalSeconds>
<stopKey>foo</stopKey>
<stopPort>9999</stopPort>
</configuration>
<executions>
<execution>
<id>start-jetty</id>
<phase>pre-integration-test</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<scanIntervalSeconds>0</scanIntervalSeconds>
<daemon>true</daemon>
</configuration>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>selenium-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<suite>src/main/webapp/selenium/___test-suite.xhtml</suite>
<browser>*firefox</browser>
<results>${project.build.directory}/target/selenium.html</results>
<startURL>http://localhost:8080/${artifactId}</startURL>
</configuration>
<executions>
<execution>
<id>run-tests</id>
<phase>integration-test</phase>
<goals>
<goal>selenese</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
But, since the amount of tests keeps growing and the time it takes to run them all also grows, there is a need to be able to run just a single test or a few of them. For instance, it is very convenient to have that possibility when developing new features, fixing bugs, or just doing a refactoring. To enable this we use the very competent Firefox plugin called SeleniumIDE. Beside recording test case, you can also load already defined test cases (or suites) and run them.




Sunday, November 15, 2009

Promote Quality with Sculptor

We have written an article that has been published in the paper magazine JayView Issue 20.

Without a vision of how to design applications within an organization the development can be compared to lawless Wild West. Development guidelines are often used, but seldom successful over the long haul. We suggest taking the architectural decisions one step further by automating them using a tool such as Sculptor.

When using a general purpose language, such as Java, and its big toolbox of APIs and frameworks there is a huge freedom of choice. This is a double-edged sword. We meet a lot of companies that have realized that they must narrow down the choices so that each new project doesn't invent its own unique system architecture and product suite. The benefits of a homogeneous architecture is obvious when looking at the big picture.

The reference architecture is often accomplished by writing guidelines and maybe a sample reference application. There are several problems with a reference architecture that is only promoted by documentation. We suggest automating some pieces of the development by using a code generator tool, such as Sculptor, to enforce consistency in the architecture.

Read more in the full article.

Tuesday, November 3, 2009

Mocking with App Engine and Spring

In previous article I illustrated how easy it is to get started with unit testing with the local App Engine environment. In this article I will go in to more advanced interaction based testing, i.e. mocking.

The App Engine APIs are simulated in the local environment. Some local implementations are designed with testing in mind, such as the email API. It is possible to verify the emails that were sent.

LocalMailService localMailService = AppEngineTestHelper.getLocalMailService();
List<MailMessage> sentMessages = localMailService.getSentMessages();
assertEquals(2, sentMessages.size());

Some other local implementations are not suitable for unit testing, such as the URL fetch service, which executes a real remote request. To solve this you need to encapsulate usage of external communication and make it possible to replace it when unit testing.

Since we are using Spring for dependency injection it is possible to replace any Spring bean for testing purpose. In our customer-supplier sample the InquiryRepository in the customer application sends inquiries to the customer application with a REST post.

This can be replaced when testing by defining a stub implementation that overrides the method that sends then inquiries. This is done in spring xml configuration (more-test.xml):

<bean id="inquiryRepository"
class="org.customer.inquiry.repositoryimpl.InquiryRepositoryStub"/>


public class InquiryRepositoryStub extends InquiryRepositoryImpl {
@Override
protected boolean sendInquiryToSupplier(Inquiry inquiry, Supplier supplier) {
return true;
}
}

Next step is to use a mocking framework instead. This makes it possible to verify the interaction, i.e. that the sendInquiryToSupplier method was invoked.

Then it is motivated to extract the sending to a separate class and interface. It is this interface that we want to mock.


public interface InquirySender {
boolean sendInquiryToSupplier(Inquiry inquiry, Supplier supplier);
}

The real implementation is an ordinary Spring @Component, that is @Autowired in InquiryRepositoryImpl. It is this implementation we want to replace with a mock when testing.

@Component
public class InquirySenderImpl implements InquirySender {


We use the approach described in the first part of Mocking & Spring tests. The FactoryBean is included in Sculptor so we only need to add the xml definition (more-test.xml):


<bean id="inquirySenderMockFactory"
class="org.fornax.cartridges.sculptor.framework.test.MockitoFactory"
primary="true" >
<property name="type" value="org.customer.inquiry.repositoryimpl.InquirySender"/>
</bean>


The junit test looks like this:


public class InquiryServiceTest extends AbstractAppEngineJpaTests
implements InquiryServiceTestBase {

@Autowired
private InquiryService inquiryService;
@Autowired
private InquirySender inquirySenderMock;

@Before
public void initMock() {
when(inquirySenderMock.sendInquiryToSupplier(any(Inquiry.class), any(Supplier.class)))
.thenReturn(true);
}

@Before
public void populateDatastore() {
Inquiry inquiry1 = new Inquiry();
inquiry1.setMessage("M1");
inquiry1.setOwnerEmail("foo@gmail.com2");
getEntityManager().persist(inquiry1);

Supplier supplier1 = new Supplier("S1");
supplier1.setUrl("http://localhost:8081/rest/inquiry");
getEntityManager().persist(supplier1);

Supplier supplier2 = new Supplier("S2");
supplier2.setUrl("http://localhost:8081/rest/inquiry");
getEntityManager().persist(supplier2);
}

@Test
public void testSendInquiry() throws Exception {
Key key = KeyFactory.createKey(Inquiry.class.getSimpleName(), 1L);
boolean ok = inquiryService.sendInquiry(getServiceContext(), key);
assertTrue(ok);
// there are 2 suppliers
verify(inquirySenderMock, times(2)).sendInquiryToSupplier(
any(Inquiry.class), any(Supplier.class));
}
}

Note that the mock is initialized in the @Before method and then verified last in the test method. In this case two messages should be sent, one for each supplier.

Maybe you have noticed that this approach is not at all specific for App Engine, it can be used for any Spring application. We need to learn a lot of new things when using App Engine, but some old knowledge still applies. :-)

Sunday, November 1, 2009

Unit Testing with App Engine and Spring

Sculptor makes it easy to write JUnit tests for Google App Engine. A test case looks like this:


public class SupplierServiceTest extends AbstractAppEngineJpaTests {

@Autowired
private SupplierService supplierService;

@Before
public void populateDatastore() {
Supplier supplier1 = new Supplier("S1");
getEntityManager().persist(supplier1);

Supplier supplier2 = new Supplier("S2");
getEntityManager().persist(supplier2);
}

@Test
public void testFindAll() throws Exception {
List<Supplier> all = supplierService.findAll(getServiceContext());
assertEquals(2, all.size());
}

@Test
public void testFindByName() throws Exception {
Supplier found = supplierService.findByName(getServiceContext(), "S2");
assertNotNull(found);
assertEquals("S2", found.getName());
}
}


Very natural!

It is interesting to take a look at the base class. It defines a few annotations and extends AbstractJUnit4SpringContextTests to initialize the Spring environment. This enables usage of ordinary @Autowire dependency injection directly in the test class.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = {"classpath:applicationContext-test.xml"})
public abstract class AbstractAppEngineJpaTests
extends AbstractJUnit4SpringContextTests {

The embedded App Engine environment is initialized from a method annotated with @Before, i.e. invoked before each test method.

public static void setUpAppEngine(ApiProxy.Environment testEnvironment) {
ApiProxy.setEnvironmentForCurrentThread(testEnvironment);

ApiProxy.setDelegate(new ApiProxyLocalImpl(new File(".")) {
});

ApiProxyLocalImpl proxy = (ApiProxyLocalImpl) ApiProxy.getDelegate();
proxy.setProperty(LocalDatastoreService.NO_STORAGE_PROPERTY, Boolean.TRUE.toString());
clearSentEmailMessages();
}

public static void tearDownAppEngine() {
ApiProxyLocalImpl proxy = (ApiProxyLocalImpl) ApiProxy.getDelegate();
LocalDatastoreService datastoreService = (LocalDatastoreService) proxy.getService("datastore_v3");
datastoreService.clearProfiles();
clearSentEmailMessages();
}

It is initialized with in memory data store, i.e. it is empty before each test method. You may populate it with initial data in your subclass in a @Before method, see populateDataStore in the sample above.

I learned one thing when doing junit testing in the app engine environment. When working with ordinary databases I have found the Spring transactional test support useful, i.e. Spring executes each test method in a transaction, which is rolled back after the test mehtod. That is achieved with the following annotations and usage of the annotation @BeforeTransaction instead of the ordinary @Before.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = {"classpath:applicationContext-test.xml"})
@TestExecutionListeners(TransactionalTestExecutionListener.class)
@TransactionConfiguration(transactionManager = "txManager", defaultRollback = true)
@Transactional
public abstract class AbstractAppEngineJpaTests
extends AbstractJUnit4SpringContextTests {

That was my initial approach also with app engine, but I realized that it was not good. Look at the following test. It will fail on the last assert when using the above transactional support.


@Test
public void testSave() throws Exception {
int countBefore = countRowsInTable(Supplier.class);
Supplier supplier3 = new Supplier("S3");
supplierService.save(getServiceContext(), supplier3);
int countAfter = countRowsInTable(Supplier.class);
assertEquals(countBefore + 1, countAfter);
}

The reason is that queries see a snapshot of the datastore as of the beginning of the transaction.

Data isolation between test methods is no problem, since the datastore is initialized (empty) before each test method.

That's all! Try it yourself by running the Maven Archetype for App Engine and fill in the details in the generated PlanetServiceTest.
  1. mvn archetype:generate -DarchetypeGroupId=org.fornax.cartridges -DarchetypeArtifactId=fornax-cartridges-sculptor-archetype-appengine -DarchetypeVersion=1.7.0-SNAPSHOT -DarchetypeRepository=http://www.fornax-platform.org/archiva/repository/snapshots/
  2. mvn clean eclipse:eclipse

Stay tuned, in next post I will describe how to mock.