Unit Testing with App Engine and Spring

Sculptor makes it easy to write JUnit tests for Google App Engine. A test case looks like this:

public class SupplierServiceTest extends AbstractAppEngineJpaTests {

    @Autowired
    private SupplierService supplierService;

    @Before
    public void populateDatastore() {
        Supplier supplier1 = new Supplier("S1");
        getEntityManager().persist(supplier1);

        Supplier supplier2 = new Supplier("S2");
        getEntityManager().persist(supplier2);
    }

    @Test
    public void testFindAll() throws Exception {
        List<Supplier> all = supplierService.findAll(getServiceContext());
        assertEquals(2, all.size());
    }

    @Test
    public void testFindByName() throws Exception {
        Supplier found = supplierService.findByName(getServiceContext(), "S2");
        assertNotNull(found);
        assertEquals("S2", found.getName());
    }
}

Very natural!

It is interesting to take a look at the base class. It defines a few annotations and extends AbstractJUnit4SpringContextTests to initialize the Spring environment. This enables usage of ordinary @Autowire dependency injection directly in the test class.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations =  {"classpath:applicationContext-test.xml"})
public abstract class AbstractAppEngineJpaTests
    extends AbstractJUnit4SpringContextTests {

The embedded App Engine environment is initialized from a method annotated with @Before, i.e. invoked before each test method.

public static void setUpAppEngine(ApiProxy.Environment testEnvironment) {
    ApiProxy.setEnvironmentForCurrentThread(testEnvironment);

    ApiProxy.setDelegate(new ApiProxyLocalImpl(new File(".")) {
    });

    ApiProxyLocalImpl proxy = (ApiProxyLocalImpl) ApiProxy.getDelegate();
    proxy.setProperty(LocalDatastoreService.NO_STORAGE_PROPERTY, Boolean.TRUE.toString());
    clearSentEmailMessages();
}

public static void tearDownAppEngine() {
    ApiProxyLocalImpl proxy = (ApiProxyLocalImpl) ApiProxy.getDelegate();
    LocalDatastoreService datastoreService = (LocalDatastoreService) proxy.getService("datastore_v3");
    datastoreService.clearProfiles();
    clearSentEmailMessages();
}

It is initialized with in memory data store, i.e. it is empty before each test method. You may populate it with initial data in your subclass in a @Before method, see populateDataStore in the sample above.

I learned one thing when doing junit testing in the app engine environment. When working with ordinary databases I have found the Spring transactional test support useful, i.e. Spring executes each test method in a transaction, which is rolled back after the test mehtod. That is achieved with the following annotations and usage of the annotation @BeforeTransaction instead of the ordinary @Before.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations =  {"classpath:applicationContext-test.xml"})
@TestExecutionListeners(TransactionalTestExecutionListener.class)
@TransactionConfiguration(transactionManager = "txManager", defaultRollback = true)
@Transactional
public abstract class AbstractAppEngineJpaTests
    extends AbstractJUnit4SpringContextTests {

That was my initial approach also with app engine, but I realized that it was not good. Look at the following test. It will fail on the last assert when using the above transactional support.

   @Test
   public void testSave() throws Exception {
       int countBefore = countRowsInTable(Supplier.class);
       Supplier supplier3 = new Supplier("S3");
       supplierService.save(getServiceContext(), supplier3);
       int countAfter = countRowsInTable(Supplier.class);
       assertEquals(countBefore + 1, countAfter);
   }

The reason is that queries see a snapshot of the datastore as of the beginning of the transaction.

Data isolation between test methods is no problem, since the datastore is initialized (empty) before each test method.

That's all! Try it yourself by running the Maven Archetype for App Engine and fill in the details in the generated PlanetServiceTest.

1. mvn archetype:generate -DarchetypeGroupId=org.fornax.cartridges -DarchetypeArtifactId=fornax-cartridges-sculptor-archetype-appengine -DarchetypeVersion=1.7.0-SNAPSHOT -DarchetypeRepository=http://www.fornax-platform.org/archiva/repository/snapshots/
2. mvn clean eclipse:eclipse

Stay tuned, in next post I will describe how to mock.

Decouple modules with asynchronous event dispatching using Spring and task queues in GAE

To build applications that are maintainable and robust you should strive for decoupling between modules. To build applications that scale you will always benefit from asynchronism and parallellism.
Here we will look how to accomplish the above in GoogleAppEngine and with some help from springframework.
Lets say we have an application where users can register them self. When they do, the application creates a persistence instance of a User-object. But, we will also keep track of how many users we have registered on the site. Now, being in GAE with BigTable luring in the back, doing queries and calculations (as we are used to with a traditional database) isn't a good idea. So as an alternative we choose to have a separate Counter-object that we updates when ever a new user registers. Ok, nothing strange here. But, there are a couple of flaws here:

1. The User module needs to know about the Counter module.
2. The User module has to wait for the Counter module to finish when updating the counting.

Ok, lets solve the first by using spring's mechanism for ApplicationEvent's. First, let us put some aop magic to work to intercept the call to UserService.createUser and when it returns (and we have the transaction boundaries on service methods, so no exception, all went well) fire off an event. Spring config for the aop stuff:

<bean id="userListener" class="org.fornax.sculptor.UserListener"/>
<bean id="userAdvice" class="org.fornax.sculptor.UserAdvice"/>
<aop:config>
<aop:pointcut id="userCreationPointcut" expression="execution(public * org..UserService.createUser(..))"/>
<aop:advisor pointcut-ref="userCreationPointcut" ref="userCreationPointcut"/>
</aop:config>

Next, here is the advice:

public class UserAdvice implements MethodInterceptor, ApplicationContextAware {

private ApplicationContext ctx;

public Object invoke(MethodInvocation invocation) throws Throwable {
  User user = (User) invocation.proceed();
  fireNewUserEvent(user);
  return user;
}

private void fireNewUserEvent(User user) {
  ctx.publishEvent(new UserCreatedEvent(user));
}

public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
  this.ctx = applicationContext;
}
}

The listener that is being notified:

public class UserListener implements ApplicationListener<UserCreatedEvent> {
@Autowired private CounterService counterService;
public void onApplicationEvent(UserCreatedEvent event) {
  counterService.increment();
}
}

And the event being passed:

public class UserCreatedEvent extends ApplicationEvent {
public UserCreatedEvent(User user) {
  super(user);
}
}

Ok, so now we are half way. We have the Observer pattern in place. But we still does everything synchronous.
Enter GAE's task queue's. Let us modify our UserListener:

public class UserListener implements ApplicationListener<UserCreatedEvent> {

public void onApplicationEvent(UserCreatedEvent event) {
  TaskOptions task = url("/rest/admin/counter/user").method(POST);
  Queue queue = QueueFactory.getDefaultQueue();
  queue.add(task);
}
}

And by the wonders of task queue's, we now put a task on the queue and by that we do the counting job asynchronous. And of course, we dropped the reference to the CounterService. But we miss one piece here, right? What does the url in the task point at. Well, nothing strange here, it is just a spring mvc controller:

@Controller
public class CounterCountroller {
@Autowired private CounterService counterService;
@RequestMapping(value = "/admin/counter/user", method = RequestMethod.POST)
public void incrementCounter() throws IOException {
  try {
    counterService.increment();
  } catch (Exception ignore) {
    // doesn't matter if we get an exception here, just log it
    log.error("Failed to increment counter!", ignore);
  }
}
}

And now we have a more loosely coupled system that scales better. And with a little effort, the code can be generalized so more features are easy to add with the same pattern.
Of course, the downside of this kind of design is that error handling gets more complicated and you can't always trust it to be 'right'. But that is system design, you have to decide what's best for each situation.

Even Weird Naming Conventions are Good

The good thing with naming conventions are that they are toolable. The DBAs at my department have strong opinions about database naming. They have good reasons for that, even though I don't fully understand all of them :-)

• Table names should be prefixed with application/component identifier.
• Primary key id column should be prefixed with table name (without application prefix) and followed by _GID.
• Underscore to separate words.
• Foreign key column is concatenation of role name and primary key column name of target table, except when role and table have the same name.

Does this mean that we have to specify each and every name twice, once for Java and once for the database. Argh... NO, we are using Sculptor. With a straightforward customization I implemented these conventions in the generator and we could continue with natural (java point of view) naming and please the preferences of the DBAs without additional effort.

Naming conventions are important for software quality. Supporting the conventions with a tool is the best way to make sure that they are applied in a consistent way.

Sculptor and Agile

For us, the agile way has always been the way of getting things done. Even before there where fancy names for it we did things in a way that enabled us to deliver the right things on the right time. This hasn't change, but now we call it scrum :-)

Sculptor will help you if you want to work agile.

Things change

They do. There is nothing anybody can do about it. And when things change, there will be delays. Doing things in an agile manner helps minimize those delays. Using sculptor helps you even more.
I'll show you by an example. Lets say you have an application that is almost done for production. Testing is done. No more bugs (yeah, right). On the final demo, suddenly, one of the stake holders realize that the customer object needs another attribute, lets call it ICE (In Case of Emergency). And, of course according to the stake holder you can't go into production without it.
So, what are the changes that needs to be done?
From the bottom up:

• Database schema
  
• Persistence layer (JPA, Hibernate, etc) :
  
• Domain object
• Service layer (if we have any logic tied to the attribute)
• Back office Client (for creating a customer object)
  
• Public Client (for viewing a customer object)
  

Most of these changes are boiler plate code. Changing declarations etc. Without Sculptor these changes takes time and are error prone. If we assume that every change is a risk and then calculate and compare all changes we need to do with and without Sculptor we end up with the following table where every point is a code change needed to do to accomplish the last minute requirement:

	Without Sculptor	With Sculptor
model:	0	1
db script:	1	0
db migration script:	1	1
property in domain object:	1	0
jpa annotation för property:	1	0
logic in service or domain object:	1	1
back office client (create/update/list/view):	7	0
public client (view):	1	1
junit test (save, find):	2	2
documentation (class diagrams):	1	0
sum:	16	6

So, being simple, lets say that each point is equally valued. By this assumption we reduce risk and time by 150%

This can be very valuable when change comes along. And it does, doesn't it...?

Sculptor in the Cloud

Now you can use Sculptor to speed up and simplify development of applications running in the Google App Engine cloud.





Let's start with a demo of how easy it is to create a new application and deploy it.



For this we are using Sculptor maven archetype for App Engine. Try it yourself:

1. mvn archetype:generate -DarchetypeGroupId=org.fornax.cartridges -DarchetypeArtifactId=fornax-cartridges-sculptor-archetype-appengine -DarchetypeVersion=1.7.0-SNAPSHOT -DarchetypeRepository=http://www.fornax-platform.org/archiva/repository/snapshots/


2. cd to the new directory


3. mvn clean


4. mvn generate-sources


5. mvn eclipse:eclipse


6. Import the project in Eclipse

Without any changes the new project is ready to run in the local development server or to be deployed at appspot.com. The sample in the demo is available here: http://sculptor-helloworld.appspot.com

The archetype creates a sample of of a RESTful Spring 3.0 Controller and JSP pages for the CRUD operations.




















The archetype also creates a simple sample model, from which Sculptor generates Entity, Repository and Service with the default CRUD operations; findById, findAll, save, and delete.

The model is defined in a textual DSL, with an intuitive syntax, from which Sculptor generates high quality Java code and configuration. It is not a one time shot. The application can be developed incrementally with an efficient round trip loop. The generator is part of the build process (maven).



Sculptor generates JPA mapping annotations for the domain objects defined in the design model. Relations are very limited in App Engine, since the datastore (BigTable) is not a relational database.

Owned and embedded associations are supported and mapped as ordinary JPA associations. They are specified with aggregate and BasicType in the Sculptor model.



Unowned associations are handled with id references and you must lookup the objects with findById when needed.



Services and Repositories are implemented as Spring components with @Autowired dependency injection. Spring AOP is used for error handling and transaction management.



Behavior is implemented with hand written code in subclass, separated from re-generated code in base class. In the above example the sayHello method is typically implemented in the Service by first using the generated findByKey method in the Repository. Note that the name attribute of the Planet is marked as key.

Sculptor also provides support for JUnit testing with the local App Engine environment. I will cover that in another article some day soon.

Maven Archetype for App Engine

I have developed a maven archetype for Google App Engine projects. The generated project supports:

• All dependency jar files are downloaded from maven repositories and copied to lib directory as required by App Engine Eclipse plugin, and local development server.


• Eclipse project is created with mvn eclipse:eclipse. The resulting Eclipse project has the necessary settings for App Engine Eclipse plugin.


• Entity classes are processed by DataNucleus enhancer in the build lifecycle.


• JUnit tests with local App Engine environment can be run from maven.

Setting up all of this is not trivial and therefore I would like to share the solution and I hope you find it useful.

Eclipse Project
The maven eclipse plugin need a lot of configuration.

<build>
<outputDirectory>war/WEB-INF/classes</outputDirectory>
<plugins>
    <plugin>
        <artifactId>maven-eclipse-plugin</artifactId>
        <version>2.5.1</version>
        <configuration>
            <!--
                buildOutputDirectory doesn't work due to
                http://jira.codehaus.org/browse/MECLIPSE-422 An workaround is the
                outputDirectory at project/build level
                <buildOutputDirectory>war/WEB-INF/classes</buildOutputDirectory>
            -->
            <testOutputDirectory>target/test-classes</testOutputDirectory>
            <classpathContainers>
                <classpathContainer>com.google.appengine.eclipse.core.GAE_CONTAINER</classpathContainer>
            </classpathContainers>
            <buildcommands>
                <buildcommand>org.eclipse.jdt.core.javabuilder</buildcommand>
                <buildcommand>com.google.gdt.eclipse.core.webAppProjectValidator</buildcommand>
                <buildcommand>com.google.appengine.eclipse.core.enhancerbuilder</buildcommand>
                <buildcommand>com.google.appengine.eclipse.core.projectValidator</buildcommand>
            </buildcommands>
            <additionalProjectnatures>
                <projectnature>org.eclipse.jdt.core.javanature</projectnature>
                <projectnature>com.google.appengine.eclipse.core.gaeNature</projectnature>
                <projectnature>com.google.gdt.eclipse.core.webAppNature</projectnature>
            </additionalProjectnatures>
            <excludes>
                <!-- Included in GAE_CONTAINER -->
                <exclude>com.google.appengine:appengine-api-1.0-sdk</exclude>
                <exclude>com.google.appengine:appengine-api-1.0-labs</exclude>
                <exclude>com.google.appengine.orm:datanucleus-appengine</exclude>
                <exclude>org.datanucleus:datanucleus-jpa</exclude>
                <exclude>org.datanucleus:datanucleus-core</exclude>
                <exclude>org.apache.geronimo.specs:geronimo-jpa_3.0_spec</exclude>
                <exclude>org.apache.geronimo.specs:geronimo-jta_1.1_spec</exclude>
                <exclude>javax.jdo:jdo2-api</exclude>
            </excludes>
        </configuration>
    </plugin>

Some dependencies must be excluded, since they are part of GAE_CONTAINER, otherwise JUnit tests will not work when running inside Eclipse. The output directory is changed to war/WEB-INF/classes. There is a bug (MECLIPSE-422) which cause the test classes to not be separated if buildOutputDirectory is used. The local development server doesn't like the test classes. The trick is to define the output at the top build level and define testOutputDirectory.

Copy Dependencies
When running the local development server and deploying to App Engine all dependent jar files must be located in war/WEB-INF/lib. I have used the maven dependency plugin to copy the jar files during the maven clean phase.

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
    <execution>
        <id>copy</id>
        <phase>clean</phase>
        <goals>
            <goal>copy</goal>
        </goals>
        <configuration>
            <artifactItems>
                <artifactItem>
                    <groupId>org.springframework</groupId>
                    <artifactId>spring-core</artifactId>
                    <version>${spring.version}</version>
                    <outputDirectory>war/WEB-INF/lib</outputDirectory>
                </artifactItem>
                <!-- more ... -->
                <artifactItem>
                    <groupId>com.google.appengine</groupId>
                    <artifactId>appengine-api-1.0-sdk</artifactId>
                    <version>${appengine.version}</version>
                    <outputDirectory>war/WEB-INF/lib</outputDirectory>
                </artifactItem>
                <artifactItem>
                    <groupId>com.google.appengine</groupId>
                    <artifactId>appengine-api-1.0-labs</artifactId>
                    <version>${appengine.version}</version>
                    <outputDirectory>war/WEB-INF/lib</outputDirectory>
                </artifactItem>
                <artifactItem>
                    <groupId>com.google.appengine.orm</groupId>
                    <artifactId>datanucleus-appengine</artifactId>
                    <version>1.0.3</version>
                    <outputDirectory>war/WEB-INF/lib</outputDirectory>
                </artifactItem>
                <artifactItem>
                    <groupId>org.datanucleus</groupId>
                    <artifactId>datanucleus-jpa</artifactId>
                    <version>1.1.5</version>
                    <outputDirectory>war/WEB-INF/lib</outputDirectory>
                </artifactItem>
                <artifactItem>
                    <groupId>org.datanucleus</groupId>
                    <artifactId>datanucleus-core</artifactId>
                    <version>1.1.5</version>
                    <outputDirectory>war/WEB-INF/lib</outputDirectory>
                </artifactItem>
                <artifactItem>
                    <groupId>org.apache.geronimo.specs</groupId>
                    <artifactId>geronimo-jpa_3.0_spec</artifactId>
                    <version>1.1.1</version>
                    <outputDirectory>war/WEB-INF/lib</outputDirectory>
                </artifactItem>
                <artifactItem>
                    <groupId>org.apache.geronimo.specs</groupId>
                    <artifactId>geronimo-jta_1.1_spec</artifactId>
                    <version>1.1.1</version>
                    <outputDirectory>war/WEB-INF/lib</outputDirectory>
                </artifactItem>
                <artifactItem>
                    <groupId>javax.jdo</groupId>
                    <artifactId>jdo2-api</artifactId>
                    <version>2.3-eb</version>
                    <outputDirectory>war/WEB-INF/lib</outputDirectory>
                </artifactItem>
            </artifactItems>
            <!-- other configurations here -->
        </configuration>
    </execution>
</executions>
</plugin>

DataNucleus Enhancer
Running the JUnit tests from maven was a primary goal as I would like to run tests from continous build server. Th JUnit tests are using local App Engine environment with in-memory datastore. Therefore the classes must be processed by DataNucleus enhancer after ordinary compilation.

<plugin>
<groupId>org.datanucleus</groupId>
<artifactId>maven-datanucleus-plugin</artifactId>
<version>1.1.4</version>
<configuration>
   <api>JPA</api>
   <mappingIncludes>**/*.class</mappingIncludes>
   <log4jConfiguration>${basedir}/src/main/resources/log4j.properties</log4jConfiguration>
   <verbose>false</verbose>
</configuration>
<executions>
   <execution>
       <phase>process-classes</phase>
       <goals>
           <goal>enhance</goal>
       </goals>
   </execution>
</executions>
</plugin>

Archetype
All of this is packaged in a maven archetype. Try it like this.

1. mvn archetype:generate -DarchetypeGroupId=org.fornax.cartridges -DarchetypeArtifactId=fornax-cartridges-sculptor-archetype-appengine -DarchetypeVersion=1.7.0-SNAPSHOT -DarchetypeRepository=http://www.fornax-platform.org/archiva/repository/snapshots/


2. cd to the new directory


3. mvn clean


4. mvn eclipse:eclipse


5. Import the project in Eclipse

As an extra bonus your new project is configured for Spring 3.0 with a sample of a RESTful controller.

Sculptor code generator tool is of course also configured and ready to be used in the new project. I will soon write another article about Sculptor's support for App Engine.

Customer Specific Addon: Deep Merge

This article illustrates the possibility to add your own features to the Sculptor code generator.

In my customer project we have a need to merge two object graphs. We have a persistent domain model and we receive messages from production systems when changes occur. There are several production systems sending the data in slightly different format and semantics.

We designed this as a first step that converts the production messages to new transient domain object instances.

Next step is to merge that object graph with present persistent objects.

This feels like a tedious and repetitive programming task. If done manually it will require some maintenance when we do changes.

At first I took a look at Dozer, but pretty soon things got complicated and required a lot of XML mapping files. So we gave up that idea.

At home, it struck me that we already have the tool we need. We are already using Sculptor, and it should be a simple addition to generate the merge methods in the domain objects.

Next morning I implemented it like this...

The final java code to be generated looks like this in each domain object. It copies attributes and new associated objects. It traverses existing associations.

  public void deepMerge(Item other) {
    Set<Object> processed = new HashSet<Object>();
    deepMerge(other, processed);
}

public void deepMerge(Item other, Set<Object> processed) {
    if (processed.contains(this)) {
        return;
    }
    processed.add(this);

    if (other.getEstimatedTimeOfArrival() != null) {
        setEstimatedTimeOfArrival(other.getEstimatedTimeOfArrival());
    }

    deepMergeShipment(other, processed);

    deepMergeEvents(other, processed);

}

public void deepMergeShipment(Item other, Set<Object> processed) {
    Shipment currentValue = getShipment();
    if (other.getShipment() != null) {
        if (currentValue == null) {
            setShipment(other.getShipment());
        } else {
            currentValue.deepMerge(other.getShipment(), processed);
        }
    }
}

public void deepMergeEvents(Item other, Set<Object> processed) {
    for (TrackingEvent each : other.getEvents()) {
        if (getEvents().contains(each)) {
            TrackingEvent currentValue = eventForKey(other.getKey());
            currentValue.deepMerge(each, processed);
        } else {
            addEvent(each);
        }
    }
}

protected TrackingEvent eventForKey(Object key) {
    for (TrackingEvent each : getEvents()) {
        if (each.getKey().equals(key)) {
            return each;
        }
    }
    return null;
}

I developed this as a project specific addon, i.e. I invoked a code generation template from SpecialCases.xpt:

«AROUND templates::DomainObject::keyGetter FOR DomainObject»
«targetDef.proceed()»

«EXPAND templates::DeepMerge::deepMerge»
«ENDAROUND»

I started with the simple attributes.

«DEFINE deepMerge FOR DomainObject»

«EXPAND deepMergeMethod»

«ENDDEFINE»

«DEFINE deepMergeMethod FOR DomainObject»
public void deepMerge(«getDomainPackage()».«name» other) {
«EXPAND deepMergeAttribute FOREACH attributes.reject(e | !e.changeable)»
«ENDDEFINE»

«DEFINE deepMergeAttribute FOR Attribute»
if (other.«getGetAccessor()»() != null) {
set«name.toFirstUpper()»(other.«getGetAccessor()»());
}
«ENDDEFINE»

I generated and looked at the result.

I noticed that the auditable fields were included. Ok, then I can use the helper function isSystemAttribute() to skip those.

«EXPAND deepMergeAttribute FOREACH attributes
.reject(e | !e.changeable || e.isSystemAttribute())»

The tricky part is the associations and I could imagine that we would have some corner cases that wouldn't be covered by the generated pattern. Therefore I created separate methods for each association so that it will be possible to override the generated methods in gap classes and handle eventual special cases manually.

I added the templates for references. Starting with the to-one references:

«DEFINE deepMergeOneReference FOR Reference»
  public void deepMerge«name.toFirstUpper()»(«from.getDomainPackage()».«from.name» other) {
      «to.getDomainPackage()».«to.name» currentValue = get«name.toFirstUpper()»();
      if (other.get«name.toFirstUpper()»() != null) {
          if (currentValue == null) {
              set«name.toFirstUpper()»(other.get«name.toFirstUpper()»());
          } else {
              currentValue.deepMerge(other.get«name.toFirstUpper()»());
          }
      }
  }
«ENDDEFINE»

Continuing with the to-many case. It is a little bit more tricky, since we need to grab existing instance for collection. Added a helper method for that.

«DEFINE deepMergeManyReference FOR Reference»
  public void deepMerge«name.toFirstUpper()»(«from.getDomainPackage()».«from.name» other) {
      for («getTypeName()» each : other.get«name.toFirstUpper()»()) {
          if (get«name.toFirstUpper()»().contains(each)) {
              «to.getDomainPackage()».«to.name» currentValue = «name.singular()»ForKey(other.getKey());
              currentValue.deepMerge(each);
          } else {
              add«name.toFirstUpper().singular()»(each);
          }
      }
  }

  protected «to.getDomainPackage()».«to.name» «name.singular()»ForKey(Object key) {
      for («to.getDomainPackage()».«to.name» each : get«name.toFirstUpper()»()) {
          if (each.getKey().equals(key)) {
              return each;
          }
      }
      return null;
  }
«ENDDEFINE»

I was testing this using the Library sample in Sculptor. I noticed problem with extended objects, such as Book, Movie that extends Media. The getKey method is not defined in Media. However, I just ignore this for now, since we don't have that kind of association in our model, and the intention is not to develop a general purpose solution.

Not completely done yet. We have the classical case with circular references. To avoid infinite recursion I added a collection that was passed as parameter to keep track of which objects that have been processed.

All this took me 2 hours to implement, probably much less than implementing it manually in all domain objects. The big benefit is that it is much less risk of manual faults and requires zero maintenance when making changes to the domain objects.

The final template file below, in case you are interested in implementing something similar:

«IMPORT sculptormetamodel»
«EXTENSION extensions::helper»
«EXTENSION extensions::dbhelper»
«EXTENSION extensions::properties»


«DEFINE deepMerge FOR DomainObject»
    «IF !isImmutable()»
        «EXPAND deepMergeMethod»

        «EXPAND deepMergeOneReference FOREACH references.select(r | !r.many).reject(e | !e.changeable)»
        «EXPAND deepMergeManyReference FOREACH references.select(r | r.many)»
    «ENDIF»
«ENDDEFINE»

«DEFINE deepMergeMethod FOR DomainObject»
    public void deepMerge(«getDomainPackage()».«name» other) {
        java.util.Set<Object> processed = new java.util.HashSet<Object>();
        deepMerge(other, processed);
    }

    public void deepMerge(«getDomainPackage()».«name» other, java.util.Set<Object> processed) {
        if (processed.contains(this)) {
            return;
        }
        processed.add(this);
        
        «EXPAND deepMergeAttribute FOREACH attributes.reject(e | !e.changeable || e.isSystemAttribute())»
        
        «FOREACH references.reject(e | !e.changeable) AS ref»
        deepMerge«ref.name.toFirstUpper()»(other, processed);
        «ENDFOREACH»
    }
«ENDDEFINE»



«DEFINE deepMergeAttribute FOR Attribute»
    «IF isPrimitive() -»
    set«name.toFirstUpper()»(other.«getGetAccessor()»());
    «ELSE-»
    if (other.«getGetAccessor()»() != null) {
        set«name.toFirstUpper()»(other.«getGetAccessor()»());
    }
    «ENDIF-»
«ENDDEFINE»


«DEFINE deepMergeOneReference FOR Reference»
    public void deepMerge«name.toFirstUpper()»(«from.getDomainPackage()».«from.name» other, java.util.Set<Object> processed) {
        if (other.get«name.toFirstUpper()»() != null) {
            «IF to.isImmutable()»
                if (!other.get«name.toFirstUpper()»().equals(get«name.toFirstUpper()»())) {
                    set«name.toFirstUpper()»(other.get«name.toFirstUpper()»());
                }
            «ELSE»
            «to.getDomainPackage()».«to.name» currentValue = get«name.toFirstUpper()»();
            if (currentValue == null) {
                set«name.toFirstUpper()»(other.get«name.toFirstUpper()»());
            } else {
                currentValue.deepMerge(other.get«name.toFirstUpper()»(), processed);
            }
            «ENDIF»
        }
    }
«ENDDEFINE»

«DEFINE deepMergeManyReference FOR Reference»
    public void deepMerge«name.toFirstUpper()»(«from.getDomainPackage()».«from.name» other, java.util.Set<Object> processed) {
        for («getTypeName()» each : other.get«name.toFirstUpper()»()) {
            if (get«name.toFirstUpper()»().contains(each)) {
            «IF to.isImmutable()»
                // replace
                remove«name.toFirstUpper().singular()»(each);
                add«name.toFirstUpper().singular()»(each);
            «ELSE»
                «to.getDomainPackage()».«to.name» currentValue = «name.singular()»ForKey(each.getKey());
                currentValue.deepMerge(each, processed);
            «ENDIF»
            } else {
                add«name.toFirstUpper().singular()»(each);
            }
        }
    }
    
    protected «to.getDomainPackage()».«to.name» «name.singular()»ForKey(Object key) {
        for («to.getDomainPackage()».«to.name» each : get«name.toFirstUpper()»()) {
            if (each.getKey().equals(key)) {
                return each;
            }
        }
        return null;
    }
«ENDDEFINE»