Customer Specific Addon: Deep Merge

This article illustrates the possibility to add your own features to the Sculptor code generator.

In my customer project we have a need to merge two object graphs. We have a persistent domain model and we receive messages from production systems when changes occur. There are several production systems sending the data in slightly different format and semantics.

We designed this as a first step that converts the production messages to new transient domain object instances.

Next step is to merge that object graph with present persistent objects.

This feels like a tedious and repetitive programming task. If done manually it will require some maintenance when we do changes.

At first I took a look at Dozer, but pretty soon things got complicated and required a lot of XML mapping files. So we gave up that idea.

At home, it struck me that we already have the tool we need. We are already using Sculptor, and it should be a simple addition to generate the merge methods in the domain objects.

Next morning I implemented it like this...

The final java code to be generated looks like this in each domain object. It copies attributes and new associated objects. It traverses existing associations.

  public void deepMerge(Item other) {
    Set<Object> processed = new HashSet<Object>();
    deepMerge(other, processed);
}

public void deepMerge(Item other, Set<Object> processed) {
    if (processed.contains(this)) {
        return;
    }
    processed.add(this);

    if (other.getEstimatedTimeOfArrival() != null) {
        setEstimatedTimeOfArrival(other.getEstimatedTimeOfArrival());
    }

    deepMergeShipment(other, processed);

    deepMergeEvents(other, processed);

}

public void deepMergeShipment(Item other, Set<Object> processed) {
    Shipment currentValue = getShipment();
    if (other.getShipment() != null) {
        if (currentValue == null) {
            setShipment(other.getShipment());
        } else {
            currentValue.deepMerge(other.getShipment(), processed);
        }
    }
}

public void deepMergeEvents(Item other, Set<Object> processed) {
    for (TrackingEvent each : other.getEvents()) {
        if (getEvents().contains(each)) {
            TrackingEvent currentValue = eventForKey(other.getKey());
            currentValue.deepMerge(each, processed);
        } else {
            addEvent(each);
        }
    }
}

protected TrackingEvent eventForKey(Object key) {
    for (TrackingEvent each : getEvents()) {
        if (each.getKey().equals(key)) {
            return each;
        }
    }
    return null;
}

I developed this as a project specific addon, i.e. I invoked a code generation template from SpecialCases.xpt:

«AROUND templates::DomainObject::keyGetter FOR DomainObject»
«targetDef.proceed()»

«EXPAND templates::DeepMerge::deepMerge»
«ENDAROUND»

I started with the simple attributes.

«DEFINE deepMerge FOR DomainObject»

«EXPAND deepMergeMethod»

«ENDDEFINE»

«DEFINE deepMergeMethod FOR DomainObject»
public void deepMerge(«getDomainPackage()».«name» other) {
«EXPAND deepMergeAttribute FOREACH attributes.reject(e | !e.changeable)»
«ENDDEFINE»

«DEFINE deepMergeAttribute FOR Attribute»
if (other.«getGetAccessor()»() != null) {
set«name.toFirstUpper()»(other.«getGetAccessor()»());
}
«ENDDEFINE»

I generated and looked at the result.

I noticed that the auditable fields were included. Ok, then I can use the helper function isSystemAttribute() to skip those.

«EXPAND deepMergeAttribute FOREACH attributes
.reject(e | !e.changeable || e.isSystemAttribute())»

The tricky part is the associations and I could imagine that we would have some corner cases that wouldn't be covered by the generated pattern. Therefore I created separate methods for each association so that it will be possible to override the generated methods in gap classes and handle eventual special cases manually.

I added the templates for references. Starting with the to-one references:

«DEFINE deepMergeOneReference FOR Reference»
  public void deepMerge«name.toFirstUpper()»(«from.getDomainPackage()».«from.name» other) {
      «to.getDomainPackage()».«to.name» currentValue = get«name.toFirstUpper()»();
      if (other.get«name.toFirstUpper()»() != null) {
          if (currentValue == null) {
              set«name.toFirstUpper()»(other.get«name.toFirstUpper()»());
          } else {
              currentValue.deepMerge(other.get«name.toFirstUpper()»());
          }
      }
  }
«ENDDEFINE»

Continuing with the to-many case. It is a little bit more tricky, since we need to grab existing instance for collection. Added a helper method for that.

«DEFINE deepMergeManyReference FOR Reference»
  public void deepMerge«name.toFirstUpper()»(«from.getDomainPackage()».«from.name» other) {
      for («getTypeName()» each : other.get«name.toFirstUpper()»()) {
          if (get«name.toFirstUpper()»().contains(each)) {
              «to.getDomainPackage()».«to.name» currentValue = «name.singular()»ForKey(other.getKey());
              currentValue.deepMerge(each);
          } else {
              add«name.toFirstUpper().singular()»(each);
          }
      }
  }

  protected «to.getDomainPackage()».«to.name» «name.singular()»ForKey(Object key) {
      for («to.getDomainPackage()».«to.name» each : get«name.toFirstUpper()»()) {
          if (each.getKey().equals(key)) {
              return each;
          }
      }
      return null;
  }
«ENDDEFINE»

I was testing this using the Library sample in Sculptor. I noticed problem with extended objects, such as Book, Movie that extends Media. The getKey method is not defined in Media. However, I just ignore this for now, since we don't have that kind of association in our model, and the intention is not to develop a general purpose solution.

Not completely done yet. We have the classical case with circular references. To avoid infinite recursion I added a collection that was passed as parameter to keep track of which objects that have been processed.

All this took me 2 hours to implement, probably much less than implementing it manually in all domain objects. The big benefit is that it is much less risk of manual faults and requires zero maintenance when making changes to the domain objects.

The final template file below, in case you are interested in implementing something similar:

«IMPORT sculptormetamodel»
«EXTENSION extensions::helper»
«EXTENSION extensions::dbhelper»
«EXTENSION extensions::properties»


«DEFINE deepMerge FOR DomainObject»
    «IF !isImmutable()»
        «EXPAND deepMergeMethod»

        «EXPAND deepMergeOneReference FOREACH references.select(r | !r.many).reject(e | !e.changeable)»
        «EXPAND deepMergeManyReference FOREACH references.select(r | r.many)»
    «ENDIF»
«ENDDEFINE»

«DEFINE deepMergeMethod FOR DomainObject»
    public void deepMerge(«getDomainPackage()».«name» other) {
        java.util.Set<Object> processed = new java.util.HashSet<Object>();
        deepMerge(other, processed);
    }

    public void deepMerge(«getDomainPackage()».«name» other, java.util.Set<Object> processed) {
        if (processed.contains(this)) {
            return;
        }
        processed.add(this);
        
        «EXPAND deepMergeAttribute FOREACH attributes.reject(e | !e.changeable || e.isSystemAttribute())»
        
        «FOREACH references.reject(e | !e.changeable) AS ref»
        deepMerge«ref.name.toFirstUpper()»(other, processed);
        «ENDFOREACH»
    }
«ENDDEFINE»



«DEFINE deepMergeAttribute FOR Attribute»
    «IF isPrimitive() -»
    set«name.toFirstUpper()»(other.«getGetAccessor()»());
    «ELSE-»
    if (other.«getGetAccessor()»() != null) {
        set«name.toFirstUpper()»(other.«getGetAccessor()»());
    }
    «ENDIF-»
«ENDDEFINE»


«DEFINE deepMergeOneReference FOR Reference»
    public void deepMerge«name.toFirstUpper()»(«from.getDomainPackage()».«from.name» other, java.util.Set<Object> processed) {
        if (other.get«name.toFirstUpper()»() != null) {
            «IF to.isImmutable()»
                if (!other.get«name.toFirstUpper()»().equals(get«name.toFirstUpper()»())) {
                    set«name.toFirstUpper()»(other.get«name.toFirstUpper()»());
                }
            «ELSE»
            «to.getDomainPackage()».«to.name» currentValue = get«name.toFirstUpper()»();
            if (currentValue == null) {
                set«name.toFirstUpper()»(other.get«name.toFirstUpper()»());
            } else {
                currentValue.deepMerge(other.get«name.toFirstUpper()»(), processed);
            }
            «ENDIF»
        }
    }
«ENDDEFINE»

«DEFINE deepMergeManyReference FOR Reference»
    public void deepMerge«name.toFirstUpper()»(«from.getDomainPackage()».«from.name» other, java.util.Set<Object> processed) {
        for («getTypeName()» each : other.get«name.toFirstUpper()»()) {
            if (get«name.toFirstUpper()»().contains(each)) {
            «IF to.isImmutable()»
                // replace
                remove«name.toFirstUpper().singular()»(each);
                add«name.toFirstUpper().singular()»(each);
            «ELSE»
                «to.getDomainPackage()».«to.name» currentValue = «name.singular()»ForKey(each.getKey());
                currentValue.deepMerge(each, processed);
            «ENDIF»
            } else {
                add«name.toFirstUpper().singular()»(each);
            }
        }
    }
    
    protected «to.getDomainPackage()».«to.name» «name.singular()»ForKey(Object key) {
        for («to.getDomainPackage()».«to.name» each : get«name.toFirstUpper()»()) {
            if (each.getKey().equals(key)) {
                return each;
            }
        }
        return null;
    }
«ENDDEFINE»