Sculptor 3.0 has a new home

Starting with the release of version 3.0 Sculptor has its new home at http://sculptorgenerator.org/.

The development is moved to GitHub https://github.com/sculptor and the new blog is hosted there as well.

So take a look at Sculptors new home!

Sculptor 2.1.0 Released

One very useful new feature is the generated finder methods. It is now possible to specify the query in the model file and let sculptor generate the code for the repository operations. In some cases the query can be derived from the method signature only.

This will generate a query returning all persons with the specified first name.

Entity Person {
  String firstName
  String lastName
  Date birthDate

  PersonRepository {
    findBy(String firstName);
  }
}

More advanced conditions can also be specified

  PersonRepository {
    findBornBefore(Date bd) condition="birthDate < :bd" orderBy="firstName";
    findByName(String name) condition="lastName i= :name or firstName i= :name";
  }

Sculptor generates findByCondition expressions and will report at generation time if there are any errors in the queries.


Another nice addition is the generated builder classes that provides a fluent interface to build domain objects in a manner that can be easier to work with and read.

    Book book = book()
        .createdBy("me")
        .createdDate(now)
        .title("Ender's Game")
        .isbn("Some-ISBN")
        .addMediaCharacter(mediaCharacter()
            .name("Ender")
            .build())
        .build();

Read more about the release.

Sculptor 2.0 is out

This is the 11th major release of Sculptor.

Three new and noteworthy features:

1. The new REST support in Sculptor makes it very easy to expose restful services by using conventions and easy delegation to Services to reduce boilerplate coding. Spring MVC is the underlaying REST framework.

2. Mixin composition is something Java developers are missing, but with Sculptor it becomes available.

3. The services can now easily be used with Spring Remoting. This is convenient for the RCP client, but can be used for other clients also.

In addition to these and several other new features we have also improved the technical quality.

We have upgraded support for latest versions of most of the underlaying tools and frameworks. Most important is Xtext 1.0, Eclipse Helios, Maven 3, Spring 3.0.

The logging API has been changed from Commons Logging with Log4j to SLF4J with Logback. SLF4J allow many backends and also have special bridges for many (legacy) logging frameworks. SLF4J with Logback is considered to be best logging framework available.

Mixin Composition

I think one of the best features in Scala is traits. Using traits it is possible to compose small pieces of behavior and state in an elegant way. I miss traits when I use Java. To mitigate that problem I have implemented support for traits in Sculptor. This article illustrates how this tool can be used for designing good, rich, domain models with traits.

Traits provide a mixin composition mechanism that is missing in Java. Similar to interfaces in Java, traits are used to define object types by specifying the signature of the supported methods. Unlike interfaces, traits can be partially implemented; i.e. it is possible to define implementations for some methods. Similar to abstract classes, but you don't have to wast your single inheritance opportunity.

How many times have you stared at java code like this:
if (p1.compareTo(p2) <= 0)

Why not spell it out, to make the code more readable:
if (p1.lessThanOrEquals(p2))

We seldom do that, because implementing 4 methods, lessThan, lessThanOrEquals, greaterThan and greaterThanOrEquals, in each and every class that needs to be compared is too much work. We stick to the cryptical compareTo.

What if I say you only have to implement them once and then you can easily mixin the methods in each and every class that needs to be compared or sorted in some way.

I hope I don't have to elaborate on how bad idea it is to try to use inheritance (base class) for these kind of reusable pieces of code.

In Sculptor's textual DSL model traits are defined like this:

        Trait Ordered {
            def boolean greaterThan(@Ordered other);
            def boolean greaterThanOrEquals(@Ordered other);
            def boolean lessThan(@Ordered other);
            def boolean lessThanOrEquals(@Ordered other);
            def abstract protected int compare(@Ordered other);
        }
        
        Entity Product with Ordered {
            String name
        }

After code generation this means that you can implement the 4 comparison methods once, in the Ordered trait, and they will be available in Product and other domain objects that are defined 'with Ordered'. The compare method must still be implemented in Product, because it is there you know what to compare with. The internal, generated, implementation is based on delegation, i.e. no magic.

Let us define another trait, which illustrates that traits also can hold state.

        Trait PriceTag {
            - protected Money normalPrice
            protected double discount
            def Money price;
            def Money price(String currency);
        }

        Entity Product with PriceTag {
            String name
        }

        BasicType Money with Ordered {
            BigDecimal amount
            String currency
        }

This means that normalPrice and discount will be mixed in to Product. You implement the price methods in the PriceTag trait. Product and all other domain objects that are defined 'with PriceTag' will have those fields and methods.

Wouldn't it be nice to be able to compare product by price? Let us do that by combining the two traits. First add the compare method in PriceTag, so that you only have to implement it at one place.

        Trait PriceTag {
            - protected Money normalPrice
            protected double discount
            def Money price;
            def Money price(String currency);
            def protected int compare(Ordered other);
        }

Then mixin both traits into Product

        Entity Product with Ordered with PriceTag {
            String name
        }

That's it. We have designed products with a rich price and compare interface.
Note that compareTo is no longer implemented in Product, only in PriceTag.

Try this new feature in latest Sculptor 2.0.0-SNAPSHOT.

EDA in Massive Web Sites

This is a comment to @h3nk3's latest blog post on Concurrency, Parallelism and Actors.

I totally agree that there is no single solution for everything. Regarding massive web sites I would like to add a few things to your conclusions, which makes asynchronous event driven solutions important for web sites also.

You should try to do as little as possible in the request thread, i.e. minimize latency. Threads in them selves are a scarce resource. You can do the essential validation and then hand off the rest of the work to a task queue (e.g. Flickr). You can precompute everything and cache it so that serving the requests becomes trivial (e.g. Reddit). Both these techniques can benefit from using event driven solutions in the backend.

We see great improvements in the area of truly asynchronous web solutions, which means that you don't have to deliver a synchronous response to every request. The result can be pushed to the clients asynchronously. Then it becomes natural to use event driven solutions in the server. The reasons for using Actors it is not only scalability and concurrency, it is also much about using a simpler concurrency model than threads and locks.

Of course it is also important to use HTTP as designed, since HTTP caching is a key ingredient in highly scalable web sites.

Event Sourcing with Sculptor - Snapshots

Yesterday I described how events can be used as storage mechanism. Instead of storing current state we store each change as a Domain Event. Current state is reconstructed by loading and replaying all historical events. For Entities with a long life cycle it can be too many events. This can be solved with an optimization that is based on periodically storing a rolling snapshot of current state.

Let us look at the code in the Sculptor port of the Simple CQRS Example to understand what this means in practice.

When loading InventoryItem we start by applying latest snapshot, if any, and thereafter replaying the events after the snapshot. This is the code in the Repository:

    @Override
    public InventoryItem findByKey(String itemId)
            throws InventoryItemNotFoundException {
        InventoryItem result = super.findByKey(itemId);

        loadFromHistory(result);

        return result;
    }

    private void loadFromHistory(InventoryItem entity) {
        InventoryItemSnapshot snapshot = getInventoryItemSnapshotRepository()
                .getLatestSnapshot(entity.getItemId());
        entity.applySnapshot(snapshot);
        long snapshotVersion = snapshot == null ? 0 : snapshot.getVersion();

        List<InventoryItemEvent> history = getInventoryItemEventRepository()
                .findAllAfter(entity.getItemId(), snapshotVersion);
        entity.loadFromHistory(history);
    }

That is how the snapshots are used when loading InventoryItem. Let us see how they are saved. We define the Snapshot Value Object for InventoryItem like this

        ValueObject InventoryItemSnapshot {
            String itemId index
            boolean activated
            Long version
            
            Repository InventoryItemSnapshotRepository {
                @InventoryItemSnapshot getLatestSnapshot(String itemId);
                protected findByCondition(PagingParameter pagingParameter);
                save;
            }
        }

Here we store the state as explicit attributes, which is simple with Sculptor, since we got the persistence (to MongoDB or JPA) for free, but it can also be stored as a blob (ecoded as xml, protobuf, or whatever you prefer). In this example the only state is the activated flag, but it can be much more in a real application.

The storage of the snapshot is triggered by a subscriber on the ordinary Domain Event flow. Simply defined as this in the model:

        Service InventoryItemSnapshotter {
            subscribe to inventoryItemTopic
            inject @InventoryItemRepository
            inject @InventoryItemSnapshotRepository
        }

The implementation calculates how many events has passed since previous snapshot by comparing version numbers. When the delta exceeds a threshold (e.g. 100 events) a snapshot is created and saved.

    public void receive(Event event) {
        if (!(event instanceof InventoryItemEvent)) {
            return;
        }

        InventoryItemEvent inventoryItemEvent = (InventoryItemEvent) event;
        String itemId = inventoryItemEvent.getItemId();

        InventoryItemSnapshot snapshot = getInventoryItemSnapshotRepository()
                .getLatestSnapshot(itemId);
        long snapshotVersion = snapshot == null ? 1 : snapshot.getVersion();
        long eventVersion = inventoryItemEvent.getAggregateVersion() == null ? 1
                : inventoryItemEvent.getAggregateVersion();
        if (eventVersion - snapshotVersion >= VERSION_DELTA) {
            takeSnapshot(itemId);
        }
    }

    private void takeSnapshot(String itemId) {
        InventoryItem item;
        try {
            item = getInventoryItemRepository().findByKey(itemId);
        } catch (InventoryItemNotFoundException e) {
            log.warn("takeSnapshot failed: " + e.getMessage());
            return;
        }

        InventoryItemSnapshot snapshot = item.createSnapshot();
        getInventoryItemSnapshotRepository().save(snapshot);
    }

By using snapshots we can dramatically improve performance for loading Entities with many historical changes. However, you can always start development without snapshotting and add it later, as a performance enhancement.

Also, note that snapshots and event are immutable and therefore we have great opportunities for using caching for improving performance.

The complete source code for this example is available here: http://github.com/patriknw/sculptor-simplecqrs/

Event Sourcing with Sculptor

A while a go Greg Young published the Super Simple CQRS Example. There are also some good documents of how to use events as storage mechanism at the CQRS Info Site.

I have implemented the same example with Java and Sculptor. In this post I will describe how the Event Sourcing is implemented. Even though my implementation is facilitated by using Sculptor and MongoDB the design can easily be done with other techniques, such as JPA or plain JDBC, and without Sculptor.

A domain model typically holds current state of the world. Event Sourcing makes it possible to see how we got to this state. Essentially it means that we have to capture all changes to an application state as a sequence of events. These events can be used to reconstruct current and past states.

The sample application is a simplified inventory of items. In line with CQRS it has a strict separation of commands and queries. Changes to the InventoryItem domain object are published as Domain Events. The InventoryItem can be modified with a few commands that I have represented as operations in a Service.

    Service InventoryFacade {
        inject @InventoryItemRepository
        createInventoryItem(String itemId, String name);
        deactivateInventoryItem(String itemId);
        renameInventoryItem(String itemId, String newName);
        checkInItemsToInventory(String itemId, int count);
        removeItemsFromInventory(String itemId, int count);
    }

All these commands are handled in the same way, except createInventoryItem, which is slightly different. The Service looks up the Entity and calls corresponding method on the domain object.

    public void deactivateInventoryItem(String itemId) {
        InventoryItem item = tryGetItem(itemId);
        item.deactivate();
        getInventoryItemRepository().save(item);
    }

The Entity performs validation and creates Domain Event for state changes.

    public void deactivate() {
        if (!isActivated())
            throw new IllegalStateException("already deactivated");
        applyChange(new InventoryItemDeactivated(new Date(), getItemId()));
    }

In this case InventoryItem has a flag (state) indicating that the item is activated or not. Note that this state is not changed directly, but it is changed later as a result of the InventoryItemDeactivated event. All changes are done with Domain Events, which is important, because we don't save current state, we only save the Domain Events. The activated flag is not stored explicitly.

The DomainEvent is applied and added to a list of changes, which later will be stored and published.

    private void applyChange(InventoryItemEvent event) {
        applyChange(event, true);
    }
    
    private void applyChange(InventoryItemEvent event, boolean isNew) {
        DynamicMethodDispatcher.dispatch(this, event, "apply");
        if (isNew) {
            changes.add(event);
        } else {
            setVersion(event.getAggregateVersion());
        }
    }

    public void apply(InventoryItemDeactivated event) {
        setActivated(false);
    }

When saving, the InventoryItem instance is saved, but only as a key and version. The version is used for detecting concurrent modifications (optimistic locking). Additionally, the changes, the Domain Events are stored. The version number of InventoryItem is stored in Each Domain Event. An additional sequence number is also used to ensure the correct order of the events.

We need to override the default save method in the Repository to handle these sequence numbers and also save the events.

    @Override
    public InventoryItem save(InventoryItem entity) {
        InventoryItem saved = super.save(entity);

        List<InventoryItemEvent> changes = entity.getUncommittedChanges();
        changes = applyVersionToChanges(changes, saved.getVersion());
        for (InventoryItemEvent each : changes) {
            getInventoryItemEventRepository().save(each);
        }
        entity.markChangesAsCommitted();

        return saved;
    }

    private List<InventoryItemEvent> applyVersionToChanges(
            List<InventoryItemEvent> changes, long version) {
        List<InventoryItemEvent> result = new ArrayList<InventoryItemEvent>();
        long sequence = version * 1000;
        for (InventoryItemEvent each : changes) {
            result.add(each.withAggregateVersion(version).withChangeSequence(
                    sequence));
            sequence++;
        }
        return result;
    }

When saving the Domain Events they are also published to a topic, which the read side subscribes on. That is handled with the publish/subscribe mechanism in Sculptor. In the model we simply need to specifiy publish on the save method.

        abstract DomainEvent InventoryItemEvent {
            persistent
            String itemId index
            Long aggregateVersion nullable
            Long changeSequence nullable
            
            Repository InventoryItemEventRepository { 
                save publish to inventoryItemTopic;
                List<@InventoryItemEvent> findAllForItem(String itemId);
                protected findByCondition;
            }
        }

        DomainEvent InventoryItemDeactivated extends @InventoryItemEvent {
        }

In the read side we add subscribers to this topic.

    Service InventoryListView {
        subscribe to inventoryItemTopic
        inject @InventoryItemListRepository
    }
    
    Service InventoryItemDetailView {
        subscribe to inventoryItemTopic
        inject @InventoryItemDetailsRepository
    }

Alright, then we are almost done. One more thing though, when retrieving a InventoryItem we must replay all events to recreate current state. We do that by overriding the default findByKey method in the Repository.

    @Override
    public InventoryItem findByKey(String itemId)
            throws InventoryItemNotFoundException {
        InventoryItem result = super.findByKey(itemId);

        loadFromHistory(result);

        return result;
    }

    private void loadFromHistory(InventoryItem entity) {
        List<InventoryItemEvent> history = getInventoryItemEventRepository()
                .findAllForItem(entity.getItemId());
        entity.loadFromHistory(history);
    }

To retrieve all events we use a simple query in the InventoryItemEventRepository.

    public List<InventoryItemEvent> findAllForItem(String itemId) {
        List<ConditionalCriteria> criteria = criteriaFor(
                InventoryItemEvent.class).withProperty(itemId()).eq(itemId)
                .orderBy(changeSequence()).build();
        return findByCondition(criteria);
    }

The loaded events are applied to the InventoryItem Domain Object.

    public void loadFromHistory(List<InventoryItemEvent> history) {
        for (InventoryItemEvent each : history) {
            applyChange(each, false);
        }
    }

    private void applyChange(InventoryItemEvent event, boolean isNew) {
        DynamicMethodDispatcher.dispatch(this, event, "apply");
        if (isNew) {
            changes.add(event);
        } else {
            setVersion(event.getAggregateVersion());
        }
    }

    public void apply(InventoryItemCreated event) {
        setActivated(true);
    }

    public void apply(InventoryItemDeactivated event) {
        setActivated(false);
    }

    public void apply(Object other) {
        // ignore
    }

In this example we have used a naive approach when loading the InventoryItem by replaying all historical events. For Entities with a long life cycle it can be too many events. Then we can use a snapshot technique, which I will describe in a separate blog post.

The complete source code for this example can be found here: http://github.com/patriknw/sculptor-simplecqrs/tree/event_sourcing_without_snapshots

Switching an existing app to MongoDB

Ron has switched project from using Hibernate/MySQL to MongoDB and wrote notes on the changes and gotchas.
Read it: Switching an existing app to MongoDB

Photo from San Francisco

Karsten saw this in San Francisco :-)

EDA CQRS Betting Sample

For todays Xtext/Sculptor seminar we have prepared a new example that illustrates some of the new event-driven features in Sculptor. It highlights a good case for using an architecture in line with Command and Query Responsibility Segregation (CQRS) pattern.

Consider an online betting system. It receives a massive amount of bets. A simple model for that could be:

        ValueObject Bet {
            String betOfferId
            String customerId
            Double amount
            
            Repository BetRepository {
                save;
            }
        }
        
        Service BettingEngine {
            inject @BetRepository
            
            placeBet(@Bet bet);
        }

The BettingEngine process the Bet and stores the information.

Questions pop up:

• How many bets have been done by customer A?


• Which customers place the higest bets, on average?


• What are the top 10 high stakes?

It would be rather easy to develop support for those kind of queries in the master betting engine domain, but problems will soon bubble up.

• Poor performance - the structure of the domain model is not optimized for all types of queries.


• Not scalable - single centralized database will become a bottleneck.


• Hard to change - too much functionality in one monolithic system.

Command-Query Responsibility Segregation (CQRS) comes to the rescue. Simplified it is about separating commands (that change the data) from the queries (that read the data). Separate subsystems take care of answering queries (reporting) and the domain for processing and storing updates can stay focused. The result of the commands are published to query subsystems, each optimized for its purpose.

Back to the betting sample. We are aiming for a design as illustrated in next drawing. Command side on the left (green) and Query side on the right (blue).



We publish domain events when bets are placed. There are several ways to do that in Sculptor, but one easy way is to mark a Service operation with the publish keyword in the model.

        DomainEvent BetPlaced {
            - Bet bet
        }

        Service BettingPublisher {
            publishEvent(@BetPlaced betEvent) publish to jms:topic:bet; 
        }

Then, in the hand-written java code simply invoke this method. In this case a one-liner in BettingEngine.

That's all for the command side, now let us take a look at the query side. We define a new module or even better a completely new business component.

   Module customer {
        Consumer BettingConsumer {
            inject @CustomerStatisticsRepository
            subscribe to jms:topic:bet
        }
        
        Service BettingQueryService {
            getHighBetters => CustomerStatisticsRepository.findHighAverageCustomers; 
        }
   
        Entity CustomerStatistics {
            gap
            String customerId key
            int numberOfBets
            double averageAmount index
            
            Repository CustomerStatisticsRepository { 
                findByKey;
                save;
                List<@CustomerStatistics> findHighAverageCustomers(double limit);
                protected findByCondition;
            }
        }
   }

We receive the events published from the betting engine by defining the subscribe keyword in the BettingConsumer.

In the query side we use domain objects that are optimized for the views and reports that are needed. We denormalize the data to minimize the number of joins needed when retrieving the data. Data is calculated ahead of time. In this case we calculate the average amount for the CustomerStatistics.

Since we are using publish/subscribe via a message bus, it might take a while until the query side is updated with the latest data, but that is not a problem. Most systems can be eventually consistent on the query side.

The full source code for the example is available here: http://github.com/patriknw/sculptor-betting

In the example we have chosen MongoDB as persistence store, Camel together with ActiveMQ as event bus. It is a matter of simple configuration to use something else, such as Oracle with JPA/Hibernate and Spring Integration for the event bus.

I was pretty impressed myself when I counted the number of lines of code that was required to implement this example. 70 lines! 40 in the model and 30 lines of hand written java code. User interface not counted.

Sculptor Presentation

Today we had a Sculptor presentation as part of the seminar 'Developing Domain-Specific Languages with Xtext'. The slides from this presentation is available at slideshare: http://www.slideshare.net/patriknw/sculptor

Free Seminar: Developing DSLs with Xtext

Next Thursday, September 09, in Stockholm, we will talk about Sculptor in the context of how to develop textual Domain-Specific Languages with Xtext. In this talk Sven Efftinge, original creator of Xtext, will present Xtext. I'm looking forward to listen to Sven's presentation.

More information and registration here: http://jwsdsl09sep.eventbrite.com/

EDA Akka as EventBus

Some time since last entry in the EDA-sequence, but here we are again. Today we are going to do something really interesting. A friend and colleague of us, Jonas Bonér, is the creator of a super interesting framework called Akka.
Akka is an event driven platform for constructing highly scalable and fault tolerant applications. It is built with Scala, but also have a rich API for java. It follows the Actor Model and together with Software Transactional Memory (STM), it raises the abstraction level and provides an easy to use tool for building highly concurrent applications.

So, today we are going to take advantage of the java API in Akka to do our own EventBus implementation.

First, update your pom with the stuff needed for Akka (repo's and dependency):

<repository>
<id>Akka</id>
<name>Akka Maven2 Repository</name>
<url>http://www.scalablesolutions.se/akka/repository/ </url>
</repository>

<repository>
<id>Multiverse</id>
<name>Multiverse Maven2 Repository</name>
<url>http://multiverse.googlecode.com/svn/maven-repository/releases/</url>
</repository>

<repository>
<id>GuiceyFruit</id>
<name>GuiceyFruit Maven2 Repository</name>
<url>http://guiceyfruit.googlecode.com/svn/repo/releases/ </url>
</repository>

<repository>
<id>JBoss</id>
<name>JBoss Maven2 Repository</name>
<url>https://repository.jboss.org/nexus/content/groups/public/ </url>
</repository>

...

<dependency>
<groupId>se.scalablesolutions.akka</groupId>
<artifactId>akka-core_2.8.0</artifactId>
<version>0.10</version>
</dependency>

Second, create your implementation of the bus, AkkaEventBus.java:

package org.foo;

import org.fornax.cartridges.sculptor.framework.event.Event;
import org.fornax.cartridges.sculptor.framework.event.EventBus;
import org.fornax.cartridges.sculptor.framework.event.EventSubscriber;

import se.scalablesolutions.akka.actor.ActorRef;
import se.scalablesolutions.akka.actor.ActorRegistry;
import se.scalablesolutions.akka.actor.UntypedActor;
import se.scalablesolutions.akka.actor.UntypedActorFactory;

public class AkkaEventBus implements EventBus {

 public boolean subscribe(final String topic, final EventSubscriber subscriber) {
  UntypedActor.actorOf(new UntypedActorFactory() {
   public UntypedActor create() {
    return new ActorListener(topic, subscriber);
   }
  }).start();
  
  return true;
 }

 public boolean unsubscribe(String topic, EventSubscriber subscriber) {
  // TODO : implement mapping between arbitrary subscriber and actor in registry
  return true;
 }

 public boolean publish(String topic, Event event) {
  ActorRef[] actorsForTopic = ActorRegistry.actorsFor(topic);
  for (int i = 0; i < actorsForTopic.length; i++) {
   actorsForTopic[i].sendOneWay(event);
  }
  return true;
 }

 @SuppressWarnings("unchecked")
 private class ActorListener extends UntypedActor {
  final String topic;
  final EventSubscriber subscriber;

  ActorListener(String topic, EventSubscriber subscriber) {
   this.topic = topic;
   this.subscriber = subscriber;
   this.getContext().setId(topic);
  }

  @Override
  public void onReceive(Object message) throws Exception {
   this.subscriber.receive((Event) message);
  }
 
 }
}

As you can see, I lack the unsubscribe implementation and I left out equals and hashCode overrides, but that I leave to you.

Third, add it as our bus implementation through spring config:

<bean id="akkaEventBus" class="org.foo.AkkaEventBus"/>
<alias name="akkaEventBus" alias="eventBus"/>

What we have done now is built an highly scalable and concurrent EventBus that dispatches event asynchronously.

Pretty easy, right? :-)

It doesn't run over the network, but Akka has some nice modules for that as well, so that is our task for next time.

EDA events over system boundaries - with camel

Last time we examined how we could make our events travel over system boundaries with the help of JMS and Spring-integration.

Today we will do exactly the same but we will use Apache Camel as event engine instead. The task is to enable application domain events over system boundaries through JMS. If you don't remember our model, here it is again:

Application Universe {
basePackage=org.helloworld

Module milkyway {
  Service PlanetService {
    @BigLandingSuccess landOnPlanet(String planetName, String astronautName)
      publish to milkywayChannel;
  }

  DomainEvent BigLandingSuccess {
    String planetName
    String astronautName
  }
}
Module houston {
  Service GroundControlService {
    subscribe to milkywayChannel
    bringOutTheChampagne(int noOfBottles);
  }
}
}

To use Apache Camel as event engine, you need to specify it in the sculptor-generator.properties file:

integration.product=camel

And add the needed dependencies to the projects pom-file:

<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-core</artifactId>
<version>2.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-jms</artifactId>
<version>2.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring</artifactId>
<version>2.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-camel</artifactId>
<version>5.3.2</version>
</dependency>
<!-- xbean is required for ActiveMQ broker configuration in the spring xml file -->
<dependency>
<groupId>org.apache.xbean</groupId>
<artifactId>xbean-spring</artifactId>
<version>3.7</version>
</dependency>
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
<version>2.1</version>
</dependency>

Regenerate and you will have a new camel.xml file in src/main/resources. This file is only generated once, and you can use it to add configuration for Camel.

To do the same as we did with spring-integration, i.e. put a domain event on a jms topic, we just have to add route rule to the camel.xml file:

<camel:route>
 <camel:from uri="direct:shippingChannel"/>
 <camel:to uri="jms:topic:shippingEvent"/>
</camel:route>

And that's it. Now we are publishing our domain event to an ActiveMQ topic. Yes, a bit strange that it works, but it gets clearer if we look in camel.xml and see whats was already there:

<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:camel="http://camel.apache.org/schema/spring" xmlns:broker="http://activemq.apache.org/schema/core" xsi:schemaLocation="         http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd         http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd         http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd         http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<bean id="camelEventBusImpl" class="org.fornax.cartridges.sculptor.framework.event.CamelEventBusImpl"/>
<alias name="camelEventBusImpl" alias="eventBus"/>

<camel:camelContext id="camel">
  <camel:package>org.sculptor.shipping</camel:package>
  <camel:template id="producerTemplate"/>
</camel:camelContext>
<!--
  Camel ActiveMQ to use the ActiveMQ broker
 -->
<bean id="jms" class="org.apache.activemq.camel.component.ActiveMQComponent">
  <property name="brokerURL" value="tcp://localhost:61616"/>
</bean>
</beans>

We have a bean that wires the jms api's to the actual ActiveMQ instance together. So you have to have an ActiveMQ instance up and running listening on port 61616.

All for today, next time...we'll see what the subject is...

EDA events over system boundaries

In the last post we explained why we have the 'event bus' notion. And earlier we have seen how to use it to publish and subscribe, both through the dsl in the model, or through plain java code.

Today we thought we should look how you can make domain events part of the public api of your business component and publish them to the rest of the world.

To do this we will use the event bus implementation that is based on spring integration. We will use its JMS outbound channel adapter to publish the 'BigLandingSuccess' domain event to a jms topic that the rest of the world can listen to.

If you don't remember, here is the model:

Application Universe {
 basePackage=org.helloworld

 Module milkyway {
   Service PlanetService {
     @BigLandingSuccess landOnPlanet(String planetName, String astronautName)
       publish to milkywayChannel;
   }

   DomainEvent BigLandingSuccess {
     String planetName
     String astronautName
   }
 }
 Module houston {
   Service GroundControlService {
     subscribe to milkywayChannel
     bringOutTheChampagne(int noOfBottles);
   }
 }
}

But first, how do we replace the default event bus implementation with the spring-integration based?

In sculptor-generator.properties, add the property:

integration.product=spring-integration

And in the project pom file, add the dependency:

<dependency>
 <groupId>org.springframework.integration</groupId>
 <artifactId>org.springframework.integration</artifactId>
 <version>1.0.4.RELEASE</version>
</dependency>

Regenerate, and you are home. So, now all event routing will use spring-integration as engine. So without any other changes, it works exactly as before.
Now you have to decide how your public api should look like, and in our case we decides that the 'BigLandingSuccess' domain event should be made available for other applications to listen to.

And now we take advantage of the power of spring-integration. When we regenerated with spring-integration enabled, we ended up with the file:
src/main/resources/spring-integration.xml

This is the place where all spring-integration definitions lands. It is generated once, so you are now free to edit it. Open the file and add:

<jms:outbound-channel-adapter id="publicMilkywayChannel"  destination="publicMilkywayTopic" channel="milkywayChannel"/>

You will need a spring jms bean named 'publicMilkywayTopic' and that bean needs to map to an actual message broker instance, for example ActiveMQ. But the spring bean and message broker setup is out scope for this blog entry, so that one we leave to you.

The only caveat now is that we are exposing our java objects in the message. This can be cured with a transformation step before the jms adapter. So add a transformer to your spring-integration configuration:

<object-to-string-transformer input-channel="milkywayChannel" output-channel="milkywayMessagesAsStringChannel"/>

And change the jms outbound channel to wire up with the new channel:

<jms:outbound-channel-adapter id="publicMilkywayChannel"  destination="publicMilkywayTopic" channel="milkywayMessagesAsStringChannel"/>

Spring-integration has a lot of transformers you can use. The above just transforms a java object to a string through its toString method. That is of course not always what you want, but it works for this example.

So that was all that had to be done to make a domain event part of your public api. Of course, you need to have proper documentation to give others a fair chance of finding your event.

If we take one step back and consider what we have done.
First, we declared the 'BigLandingSuccess' domain event and publishes it internally.

Second, we add an adapter that listens to the channel and in its turn publishes it on a jms topic, i.e. makes it public.

Third, we added a transformation step before doing a public publish to remove the dependency to our classes.

Now, this is nice, isn't it? We can have a lot of domain events internally in our application and by that take advantage of all the nice attributes of EDA. And we can by choice make a domain event public with 'just' configuration changes.

Quite long post, but we got pretty much done. Next time we will look how to do the same with Apache Camel

EDA why the event bus in sculptor?

We have come to our seventh entry about Event Driven Architecture. We are in the topic of how Sculptor supports EDA. In previous posts we have covered how to publish and subscribe, both through the dsl in the model, and through plain java code.

Today we will talk a bit more on the thin layer we call 'the event bus'.

As stated before, there are a lot of different approaches to EDA. You can use/implement it locally or apply it too the entire enterprise. But also, in its simplest form its about the observer pattern, regardless if we talk about big or small implementations. This leads us to the main motivations of our event bus abstraction.

We want to keep it simple, but at the same time still have the power of doing event handling big and small. In the Enterprise, or locally. And doing this with the same programming interfaces, i.e. keeping it simple.

So, based on the above we have the event bus api. And with the risk of repeating my self, it is a very simple one. Methods for publishing, subscribing, and un-subscribing.

It also ships with a default implementation named (you got it) simple event bus. This one is really easy (hey, I found another word instead of simple) to use and fully functional on its own. Though, if you need to integrate with another system you are going to need another implementation.

And that is one of our other motivation for the event bus, it should be easy to swap implementations.

Beside the default, we currently have two implementations based on Spring-integration and Apache Camel. Both of these are what's called "lightweight integration frameworks". Supporting these two frameworks brings a lot of power to the solution when it comes to integration.

And that brings us to the next topic of this series. Some examples of how to bring your events to life over system boundaries, i.e. integration stuff.

EDA pub/sub with plain java

In a couple of posts we have seen how to publish and subscribe to events. I showed how to do that through the dsl in the model. But you have of course also the possibility to do pub/sub by plain java code as well. The key thing is to have a handle to the event bus.

The easiest way to explain it is to show an example, so, a simple service that publish an event as part of it's business logic:

@Service
public class GroundControlService {

 @Autowired
 @Qualifier("eventBus")
 private EventBus eventBus;

 @Override
 public void receive(Event event) {
   if (event instanceof BigLandingSuccess) {
     eventBus.publish("earthChannel", new Celebration("We did it!"));
     this.bringOutTheChampagne(ALL);
     eventBus.publish("supplierChannel", new MissingResource("Champagne"));
   }
 }
}

And for the other end, the subscriber:

@Service
public class SupplierService {

@Autowired
@Qualifier("eventBus")
private EventBus eventBus;

public void init() {
  eventBus.subscribe("supplierChannel", new EventSubscriber() {
    @Override
    public void receive(Event event) {
      if (event instanceof MissingResource) {
        raceForContract(event);
      }
    }
  });
}
}

Not so hard.

Now, here and there I've used the term 'event bus', and you have also seen it in java code above. Next time we will look closer into the bus. Why is there an abstraction and what can you do with it?

EDA a simple example with Sculptor continued

Lets continue with our simple example from the last post. We declared a domain event in the model and we marked a service to publish that event to a channel.

Now, what about getting notified when such an event is published?

Continuing with our model from last time, its just a matter of editing it. Lets create another module with a service that is interested of the event:

Application Universe {
 basePackage=org.helloworld

 Module milkyway {
   Service PlanetService {
     @BigLandingSuccess landOnPlanet(String planetName, String astronautName)
       publish to milkywayChannel;
   }

   DomainEvent BigLandingSuccess {
     String planetName
     String astronautName
   }
 }
 Module houston {
   Service GroundControlService {
     subscribe to milkywayChannel
     bringOutTheChampagne(int noOfBottles);
   }
 }
}

The result is that the GroundControl service will implement the EventSubscriber interface and be marked with @Subscribe annotation. That means that the GroundControl service will automatically be added as subscriber to milkywayChannel. It will be notified, receive method called, when events are published to that channel.

You will get a stub of the receive method of the EventSubscriber interface.

@Override
public void receive(Event event) {
 // TODO Auto-generated method stub
 throw new UnsupportedOperationException("receive not implemented");
}

So that you will need to implement with your logic:

@Override
public void receive(Event event) {
 if (event instanceof BigLandingSuccess) {
   this.bringOutTheChampagne(999999);
 }
}

Ok, so now we covered some very simple notations for publishing and subscribing to events. Next time I'll show how to do the same but in plain java code.

EDA a simple example with Sculptor

Now we have gone through EDA in general, and briefly covered how we support it in Sculptor. It's time to see how it can be used.

We start with a simple example where we will show how to declare a domain event and how that event is published.

Let us use our old, well known, hello world example.

Application Universe {
basePackage=org.helloworld

Module milkyway {
 Service PlanetService {
   void landOnPlanet(String planetName, String astronautName);
 }

}
}

That is our model. So, what might the rest of the 'universe' be interested of here? Well, if NASA sent out a space ship with an astronaut on it who's mission was to land on some far far away planet, wouldn't they be interested in the event of a landing? I think so. So, when a astronaut lands on a planet, we would like the service to publish an event of this happening.

Lets start with re-defining the model with the event:

Application Universe {
basePackage=org.helloworld

Module milkyway {
 Service PlanetService {
   void landOnPlanet(String planetName, String astronautName);
 }

 DomainEvent BigLandingSuccess {
   String planetName
   String astronautName
 }
}
}

DomainEvents may contain attributes and references in the same way as ValueObjects and Entities. DomainEvents are always immutable and not persistent.

Events are about something happening at a point in time, so it's natural for events to contain time information. Sculptor automatically adds two timestamps, occurred and recorded. The time the event occurred in the world and the time the event was noticed.

Ok, so now we have the event defined, now we want it to be published. The easiest way of doing this is to mark the service in the dsl (you can of course publish events programmatically, but more about that in a later blog entry) . So, once again, lets re-define our model:

Application Universe {
basePackage=org.helloworld

Module milkyway {
 Service PlanetService {
   @BigLandingSuccess landOnPlanet(String planetName, String astronautName)
      publish to milkywayChannel;
 }

 DomainEvent BigLandingSuccess {
   String planetName
   String astronautName
 }
}
}

The operation must return a DomainEvent or take a DomainEvent as parameter. That event is published to the defined channel when the operation has been invoked.

As an alternative the DomainEvent instance can be created from the return value or parameters. The DomainEvent must have a matching constructor.

The result of the above declarative way of publishing events is a generated annotation @Publish on the method. It will trigger the Spring AOP advice PublishAdvice that is part of Sculptor framework.

Recap: We have declared an event in our model, and further marked our service to publish this event when it happens. As an API, I think this is very neat. Everyone who is interested can see that when an astronaut lands on a planet, he or she can be notified.

And that is what we will talk about next time:

How can I be notified when things happens?

EDA sculptor support

In the previous post we gave an overview of Event Driven Architecture. It is a very big area, and you can use it to a lot of things.
It is a very good complement to DDD, and is a corner stone when building scalable systems.
When constructing systems it is a very nice match to accomplish loosely coupled modules and bounded contexts, i.e. business components. And much more.

We strive for simplicity, but at the same time not restricting us.

So, how can you use with Sculptor?
In the 1.9.0 release, we have focused on support for Publish/Subscribe, Comman-Query Responisbility Segregation (CQRS) and Event Sourcing.

The most useful part is of course the pub/sub support. Event sourcing is an architectural style that has its niche. CQRS is also an architectural style that there has been some publicity around lately. CQRS uses pub/sub and can with advantage be constructed to use Event Sourcing.

To support the above, there are three central parts in our implementation:

• An event bus abstraction
• DomainEvent
• CommandEvent

"The bus"
The event bus is an extremely simple API, with three different implementations (in 1.9.0), "Simple", Spring Integration and Apache Camel. The idea is that the central parts, i.e. pub/sub, should be easy to use. The only thing you have to work with is an event bus where you publish and subscribes to and from events. And if you need some non functional behavior (asynchronism, over the wire, etc) for your events, you plug in an event bus that can handle this requirements.
And as stated above, the easiest way to accomplish that with the 1.9.0 release is to use Spring Integration or Apache Camel. But you can also choose to implement your own event bus.
You can publish and subscribe to and from the bus either declarative through the DSL, or programatically through the event bus API.

DomainEvent vs CommandEvent

CommandEvent is an instruction for something to happen. The system processes a CommandEvent and takes appropriate actions.

DomainEvent states fact - that something has happen. This fact is published to the rest of the world and the publisher just lets it go with no further interest in what happens to the event, i.e. who receives it and what they do.

If you compare it to an application API, CommandEvent could be the input API, while the DomainEvent is the output API for the application. I.e. the CommandEvent is what you can make the application perform, while the DomainEvent is what the application reports has happen.

As you probably figured out, CommandEvent is for implementing CQRS and Event Sourcing.

Finally, let us take quick look at how the notion for DomainEvents look like in the DSL.

DomainEvent ShipHasArrived {
  - ShipId ship
  - UnLocode port
}
   
DomainEvent ShipHasDepartured {
  - ShipId ship
  - UnLocode port
}

As you can see, you define a domain event in the same way as for entities or value objects.

And to declare pub/sub:

Service TrackingService {
  @ShipHasArrived recordArrival(DateTime occurred, @Ship ship, @Port port)
      publish to shippingChannel;
}
Service Statistics {
  subscribe to shippingChannel
  int getShipsInPort(@UnLocode port);
  reset;
}


That was all for today, next time will it be more concrete with examples of how to use it.