Screencast: Introduction to Sculptor

During the summer I have published a series of articles that illustrate basic usage of Sculptor. They include screencasts so that you get a feeling of what it looks like when using Sculptor.

If you are totally new to Sculptor you might need to read What is Sculptor? before looking at the practical example.

The series illustrates the following, step-by-step:

1. Jump Start - Initial creation of maven and eclipse projects. Persistent entity and CRUD GUI are created in a few minutes.



2. The World is Changing - Adding more to the application. Quick development round trip, short feedback loop, it is not a one time shot.



3. Testing is Simple - Testability is crucial and is of course supported.



4. Adding Behaviour - Generated code is well separated from hand written code.



5. Say Hello - Entity, Repository and Service are some of the available building blocks. Yes, it is real DDD-style.



6. Introducing a Type - Developing a high quality domain model is the core of Sculptor. Small type objects are typically part of a good domain model.



7. Refactoring - How is refactoring done when having a mix of hand written and generated code?

GAE Transactions

I'm trying to understand what we should do to make Sculptor compatible with Google App Engine (GAE).

I feel a bit sad when looking back to what I have just experienced, but I guess I should be happy, since I have learned a lot. In this post I will share my mistakes and insights to GAE transactions and Entity Groups.

Together with Andreas I'm developing a little sample that consists of 3 interacting applications. Customer, Supplier and Profile apps. User stories for the initial sprint:

• As a customer I want to specify a request for consultants so that I can allocate resources to my project.


• As a salesman (supplier) I want to be notified when a customer enters a request for consultants so that I quickly can create an offer to that request.


• As a salesman I want to offer consultants to a customer so that I can sell our services.


• As a customer I want to see up to date information in the profiles so that I know that it is not obsolete.

I was developing the form enter of the inquiry in the customer app. I saved the form data in an Inquiry object and sent the request to the supplier app using RestTemplate. No problems so far.

We are using the new REST features in Spring 3.0 and have done some adjustments to Sculptor to make it generate JPA code that is compliant with GAE datastore.

Since one inquiry should be sent to many suppliers it didn't feel very scalable to send them all in the form entry request. Therefore I separated the sending to a separate job, which would be invoked by the cron service (later, better with task queue). This is not only more scalable, it is also more fault tolerant, since supplier apps may not be available all the time. By separating it we can easily retry later.

I created a Supplier entity also. In the sendToSuppliers job I got the first problem:

IllegalArgumentException: can't operate on multiple entity groups in a single transaction

Since I had two entities, Inquiry and Supplier and I was using both in the transaction I assumed that it was not allowed to query the Suppliers and update the Inquiries in the same transaction. I based that on the GAE documentation:
All datastore operations in a transaction must operate on entities in the same entity group. This includes querying for entities by ancestor, retrieving entities by key, updating entities, and deleting entities.

That assumption was a fatal mistake that got me on the wrong track. I started to separate the the retrieval of Suppliers and update of Inquiries in separate transactions.

I learned from the documentation that it was possible to disable transactions, but that it was a temporary workaround.

After removing all code except the update of the Inquiries I realized that the Inquiry instances themselves belonged to separate entity groups. I was looping over all Inquiries that had not been sent to suppliers, i.e. I was updating several instances. Of course, they belong to separate entity groups, otherwise it would not scale when the number of objects increase.

Then I redesigned the sending job so that it would only send and update one Inquiry instance. The job will have to be run many times to send all Inquiries.

On the way I learned some more things about GAE datastore:
* A transaction is necessary for some operations, such flush, otherwise; "This operation requires a transaction yet it is not active"
* Queries also require a transaction, otherwise when iterating over the result;
"Object Manager has been closed"
* Modification several times; "can't update the same entity twice in a transaction or operation"

In the end I think the defaults for transactions in Sculptor are alright. Normally we define transaction boundary at the service layer. This is ok for many cases when using GAE also, but one have to design the operations so that they only update one instance (entity group).

There is probably a need for more fine grained transaction control at the repository level. E.g. starting a new transaction for some repository operations. I think we should implement this with @Transactional annotations. Is it possible to mix txAdvice (defaults) with @Transactional (deviations from default)?

Refactoring

Sometimes I get the question "How is refactoring done when having a mix of hand written and generated code?" It is a good question, since refactoring is very important. The intention is that existing IDE refactoring tools will continue to serve you when using Sculptor.

When doing initial prototyping and you don't have any (or little) hand written code you can easily change in the model and re-generate. When you have hand written code you start with using the refactoring tools in the IDE, as you are used to. Thereafter you do corresponding change in the model and re-generate.

Eventual mistakes will normally be caught by the compiler and JUnit tests.

The following screencast illustrates how to rename Planet to Planet2.




Alternative video format (mpg)

Sculptor doesn't make it more difficult to do refactoring. Sometimes it makes refactoring easier, when the change only affects generated code.

Introducing a Type

An important building block when creating a high quality domain model is to create small type objects. In this article we will create a Length type for the diameter of the Planet of the helloworld application.



Alternative video format (mpg)

Length is a typical Quantity with a value and unit, e.g. meter, kilometer.

In the design model it looks like this:

BasicType Length {
 BigDecimal value min="0"
 -@LengthUnit unit
}

enum LengthUnit {
 cm, m, km
}

Entity Planet {
 gap
 scaffold
 String name key
 Long population min="0"
 -@Length diameter nullable
 -Set<@Moon> moons opposite planet

 Repository PlanetRepository {
     findByKey;
 }
}

We also need to convert between different units. The behaviour expressed as a JUnit test:

public class LengthTest {

  @Test
  public void shouldConvertFromMeterToKilometer() {
      Length length = new Length(new BigDecimal("31000"), m);
      Length lengthInKilometer = length.to(km);
      assertEquals(new Length(new BigDecimal("31"), km),
          lengthInKilometer);
  }

  @Test
  public void shouldConvertFromKilometerToMeter() {
      Length length = new Length(new BigDecimal("44"), km);
      Length lengthInMeter = length.to(m);
      assertEquals(new Length(new BigDecimal("44000"), m),
          lengthInMeter);
  }

  @Test
  public void shouldNotConvertSameUnit() {
      Length length = new Length(new BigDecimal("17"), km);
      Length length2 = length.to(km);
      assertSame(length, length2);
  }
}

BasicType objects may contain business logic in the same way as other domain objects. The following screencast illustrates how to implement the conversion.



Alternative video format (mpg)

BasicType is a stored in the same table as the Domain Object referencing it. It corresponds to JPA @Embeddable.

There are a lot of cases when it is a good idea to introduce types.

• Identifers, natural business keys. It is more readable to pass around an identifier type instead of a plain String or Integer
• Money
• Range
• Quantity
  

I can recommend reading When to Make a Type, Martin Fowler.

Say Hello

In previous article our Planet is capable of constructing a greeting message. This article shows how to make it possible for a client application to say hello to the Planet.



Alternative video format (mpg)

Let us create a PlanetService to expose the sayHello method to clients. We lookup the Planet from its name using the built in findByKey repository operation. In Sculptor model file this looks like this:

      Service PlanetService {
         String sayHello(String planetName) throws PlanetNotFoundException;
     }

     Entity Planet {
         gap
         scaffold
         String name key
         Long population min="0"
         Long diameter min="0" nullable
         -Set<@Moon> moons opposite planet
      
         Repository PlanetRepository {
             findByKey;
         }
     }

All hand written java code we need to add is for testing and two trivial lines in PlanetServiceImpl:

    public String sayHello(ServiceContext ctx, String planetName)
       throws PlanetNotFoundException {

       Planet planet = getPlanetRepository().findByKey(planetName);
       return planet.greeting();
   }

Adding Behaviour

In previous articles we created a simple helloworld application without any hand written code. This article shows how to add some hand written business logic.



Alternative video format (mpg)

The behaviour to implement is that the Planet should be able to construct a greeting message based on its population. Test for this behaviour looks like this:

public class PlanetTest {

  @Test
  public void shouldSayHelloWhenHasPopulation() {
      Planet earth = new Planet("Earth");
      earth.setPopulation(7000000000L);
      String message = earth.greeting();
      assertEquals("Hello from Earth", message);
  }

  @Test
  public void shouldBeQuietWhenNoPopulation() {
      Planet pluto = new Planet("Pluto");
      String message = pluto.greeting();
      assertEquals("", message);
  }
}

The video illustrates how to implement this.

As you see nothing special, you add the business logic in Java as usual.

Separation of generated and manually written code is done by a generated base class and manually written subclass, a gap class. It is in the subclass you add methods to implement the behavior of the Domain Object. The subclass is also generated, but only once, it will never be overwritten by the generator.

The gap class is not generated initially. When you need a gap class you specify that in the DSL with gap keyword.

Testing is Simple

In previous articles we created a simple helloworld application without any tests. This article shows how to do integration testing of services with Sculptor.



Alternative video format (mpg)

We turn on generation of junit test and complete the failing tests from previous articles. For each Service there is a generated JUnit test class that we are encouraged to implement. It uses Spring transactional test fixtures and DbUnit.

Spring beans are injected in the test with ordinary @Autowired annotations.

public class PlanetServiceTest extends AbstractDbUnitJpaTests
    implements PlanetServiceTestBase {
  private PlanetService planetService;

  @Autowired
  public void setPlanetService(PlanetService planetService) {
    this.planetService = planetService;
  }

  @Test
  public void testFindById() throws Exception {
    Planet found = planetService.findById(getServiceContext(), 1L);
    assertEquals("Earth", found.getName());
  }

  @Test
  public void testFindAll() throws Exception {
    List<Planet> found = planetService.findAll(getServiceContext());
    assertEquals(2, found.size());
  }

  @Test
  public void testSave() throws Exception {
    int countBefore = countRowsInTable("PLANET");
    Planet planet = new Planet("Pluto");
    planet.setPopulation(0L);
    planetService.save(getServiceContext(), planet);
    assertEquals(countBefore + 1, countRowsInTable("PLANET"));
  }

  @Test
  public void testDelete() throws Exception {
    int countBefore = countRowsInTable("PLANET");
    Planet planet = planetService.findById(getServiceContext(), 2L);
    planetService.delete(getServiceContext(), planet);
    assertEquals(countBefore - 1, countRowsInTable("PLANET"));
  }
}

The initial test data is defined in DbUnit xml file. The database is refreshed for each test method.

<?xml version="1.0" encoding="UTF-8"?>

<dataset>
  <PLANET ID="1" NAME="Earth" POPULATION="7000000000" VERSION="0"/>
  <PLANET ID="2" NAME="Jupiter" POPULATION="0" VERSION="0"/>
  <MOON/>
</dataset>

Above tests only covers the normal cases so far and we should of course do more tests for exceptional cases and validation boundaries, such as negative population.

The tests illustrated here are kind of integration tests and you should do ordinary unit tests for domain objects and other classes of importance.