EDA pub/sub with plain java

In a couple of posts we have seen how to publish and subscribe to events. I showed how to do that through the dsl in the model. But you have of course also the possibility to do pub/sub by plain java code as well. The key thing is to have a handle to the event bus.

The easiest way to explain it is to show an example, so, a simple service that publish an event as part of it's business logic:

@Service
public class GroundControlService {

 @Autowired
 @Qualifier("eventBus")
 private EventBus eventBus;

 @Override
 public void receive(Event event) {
   if (event instanceof BigLandingSuccess) {
     eventBus.publish("earthChannel", new Celebration("We did it!"));
     this.bringOutTheChampagne(ALL);
     eventBus.publish("supplierChannel", new MissingResource("Champagne"));
   }
 }
}

And for the other end, the subscriber:

@Service
public class SupplierService {

@Autowired
@Qualifier("eventBus")
private EventBus eventBus;

public void init() {
  eventBus.subscribe("supplierChannel", new EventSubscriber() {
    @Override
    public void receive(Event event) {
      if (event instanceof MissingResource) {
        raceForContract(event);
      }
    }
  });
}
}

Not so hard.

Now, here and there I've used the term 'event bus', and you have also seen it in java code above. Next time we will look closer into the bus. Why is there an abstraction and what can you do with it?

EDA a simple example with Sculptor continued

Lets continue with our simple example from the last post. We declared a domain event in the model and we marked a service to publish that event to a channel.

Now, what about getting notified when such an event is published?

Continuing with our model from last time, its just a matter of editing it. Lets create another module with a service that is interested of the event:

Application Universe {
 basePackage=org.helloworld

 Module milkyway {
   Service PlanetService {
     @BigLandingSuccess landOnPlanet(String planetName, String astronautName)
       publish to milkywayChannel;
   }

   DomainEvent BigLandingSuccess {
     String planetName
     String astronautName
   }
 }
 Module houston {
   Service GroundControlService {
     subscribe to milkywayChannel
     bringOutTheChampagne(int noOfBottles);
   }
 }
}

The result is that the GroundControl service will implement the EventSubscriber interface and be marked with @Subscribe annotation. That means that the GroundControl service will automatically be added as subscriber to milkywayChannel. It will be notified, receive method called, when events are published to that channel.

You will get a stub of the receive method of the EventSubscriber interface.

@Override
public void receive(Event event) {
 // TODO Auto-generated method stub
 throw new UnsupportedOperationException("receive not implemented");
}

So that you will need to implement with your logic:

@Override
public void receive(Event event) {
 if (event instanceof BigLandingSuccess) {
   this.bringOutTheChampagne(999999);
 }
}

Ok, so now we covered some very simple notations for publishing and subscribing to events. Next time I'll show how to do the same but in plain java code.

EDA a simple example with Sculptor

Now we have gone through EDA in general, and briefly covered how we support it in Sculptor. It's time to see how it can be used.

We start with a simple example where we will show how to declare a domain event and how that event is published.

Let us use our old, well known, hello world example.

Application Universe {
basePackage=org.helloworld

Module milkyway {
 Service PlanetService {
   void landOnPlanet(String planetName, String astronautName);
 }

}
}

That is our model. So, what might the rest of the 'universe' be interested of here? Well, if NASA sent out a space ship with an astronaut on it who's mission was to land on some far far away planet, wouldn't they be interested in the event of a landing? I think so. So, when a astronaut lands on a planet, we would like the service to publish an event of this happening.

Lets start with re-defining the model with the event:

Application Universe {
basePackage=org.helloworld

Module milkyway {
 Service PlanetService {
   void landOnPlanet(String planetName, String astronautName);
 }

 DomainEvent BigLandingSuccess {
   String planetName
   String astronautName
 }
}
}

DomainEvents may contain attributes and references in the same way as ValueObjects and Entities. DomainEvents are always immutable and not persistent.

Events are about something happening at a point in time, so it's natural for events to contain time information. Sculptor automatically adds two timestamps, occurred and recorded. The time the event occurred in the world and the time the event was noticed.

Ok, so now we have the event defined, now we want it to be published. The easiest way of doing this is to mark the service in the dsl (you can of course publish events programmatically, but more about that in a later blog entry) . So, once again, lets re-define our model:

Application Universe {
basePackage=org.helloworld

Module milkyway {
 Service PlanetService {
   @BigLandingSuccess landOnPlanet(String planetName, String astronautName)
      publish to milkywayChannel;
 }

 DomainEvent BigLandingSuccess {
   String planetName
   String astronautName
 }
}
}

The operation must return a DomainEvent or take a DomainEvent as parameter. That event is published to the defined channel when the operation has been invoked.

As an alternative the DomainEvent instance can be created from the return value or parameters. The DomainEvent must have a matching constructor.

The result of the above declarative way of publishing events is a generated annotation @Publish on the method. It will trigger the Spring AOP advice PublishAdvice that is part of Sculptor framework.

Recap: We have declared an event in our model, and further marked our service to publish this event when it happens. As an API, I think this is very neat. Everyone who is interested can see that when an astronaut lands on a planet, he or she can be notified.

And that is what we will talk about next time:

How can I be notified when things happens?

EDA sculptor support

In the previous post we gave an overview of Event Driven Architecture. It is a very big area, and you can use it to a lot of things.
It is a very good complement to DDD, and is a corner stone when building scalable systems.
When constructing systems it is a very nice match to accomplish loosely coupled modules and bounded contexts, i.e. business components. And much more.

We strive for simplicity, but at the same time not restricting us.

So, how can you use with Sculptor?
In the 1.9.0 release, we have focused on support for Publish/Subscribe, Comman-Query Responisbility Segregation (CQRS) and Event Sourcing.

The most useful part is of course the pub/sub support. Event sourcing is an architectural style that has its niche. CQRS is also an architectural style that there has been some publicity around lately. CQRS uses pub/sub and can with advantage be constructed to use Event Sourcing.

To support the above, there are three central parts in our implementation:

• An event bus abstraction
• DomainEvent
• CommandEvent

"The bus"
The event bus is an extremely simple API, with three different implementations (in 1.9.0), "Simple", Spring Integration and Apache Camel. The idea is that the central parts, i.e. pub/sub, should be easy to use. The only thing you have to work with is an event bus where you publish and subscribes to and from events. And if you need some non functional behavior (asynchronism, over the wire, etc) for your events, you plug in an event bus that can handle this requirements.
And as stated above, the easiest way to accomplish that with the 1.9.0 release is to use Spring Integration or Apache Camel. But you can also choose to implement your own event bus.
You can publish and subscribe to and from the bus either declarative through the DSL, or programatically through the event bus API.

DomainEvent vs CommandEvent

CommandEvent is an instruction for something to happen. The system processes a CommandEvent and takes appropriate actions.

DomainEvent states fact - that something has happen. This fact is published to the rest of the world and the publisher just lets it go with no further interest in what happens to the event, i.e. who receives it and what they do.

If you compare it to an application API, CommandEvent could be the input API, while the DomainEvent is the output API for the application. I.e. the CommandEvent is what you can make the application perform, while the DomainEvent is what the application reports has happen.

As you probably figured out, CommandEvent is for implementing CQRS and Event Sourcing.

Finally, let us take quick look at how the notion for DomainEvents look like in the DSL.

DomainEvent ShipHasArrived {
  - ShipId ship
  - UnLocode port
}
   
DomainEvent ShipHasDepartured {
  - ShipId ship
  - UnLocode port
}

As you can see, you define a domain event in the same way as for entities or value objects.

And to declare pub/sub:

Service TrackingService {
  @ShipHasArrived recordArrival(DateTime occurred, @Ship ship, @Port port)
      publish to shippingChannel;
}
Service Statistics {
  subscribe to shippingChannel
  int getShipsInPort(@UnLocode port);
  reset;
}


That was all for today, next time will it be more concrete with examples of how to use it.

EDA overview

As stated in the last post, we would like to talk a bit (very shortly) about EDA in a broader sense to give an overview of it and to give you a sense for what we read into the term EDA. This is quite important as a background when we go further and explain our interpretation and implementation of it.

Event Driven Architecture is a very broad area. But in a short sentence its about:

Applications and systems that produce, consume and reacts on events.

And even if there is a lot of attention on EDA as means of implementing integration between applications and systems, EDA can be applied within applications and even in parts of applications.

The benefits of an EDA is:

• loosely coupled systems (or internals of a system)
• high performance (fire and forget)
• high scalability

Of course this comes with a trade-off. An extra abstraction is added, i.e. it becomes more complex.

In it's simplest form, EDA is about the Observer Pattern. Something happens somewhere, and 0 to n parties is interested in that happening. From this simple pattern, all interpretations, implementations and usages of EDA are spawn.

So, before we go into how we have implemented support for EDA in Sculptor, we will give you some examples of what shapes we think EDA can take. Big and small...here we go:

GUI's

Swing uses the Observer pattern. Many different GUI frameworks uses an event driven approach. Java Server Faces works with events, event handlers and event actions. Etc...

DomainEvent

A Domain Event is registration of something that has happen. It might be of interest to 0 to n consumers. The thing is; The producer doesn't care, it just tell the world that this thing has happen and happily goes on with it's life.

PubSub

Publisher and Subscribers is a central part of all event driven integrations. It is an implementation of the Observer pattern. It often comes with persistent events. Topics in the Java/JMS tech domain is an implementation of it.

SOA2.0

Now when SOA has been around for some years the next thing is SOA2.0. In the 2.0 version of SOA events plays a central role as a complement to services.

An event-driven system typically consists of event emitters (or agents) and event consumers (or sinks). Sinks have the responsibility of applying a reaction as soon as an event is presented. The reaction might or might not be completely provided by the sink itself. For instance, the sink might just have the responsibility to filter, transform and forward the event to another component or it might provide a self contained reaction to such event. The first category of sinks can be based upon traditional components such as message oriented middleware while the second category of sinks (self contained online reaction) might require a more appropriate transactional executive framework, for example an ESB.

Event servers

Event servers are servers that are specialized in processing events. They often provide filter/query capabilities for events. Weblogic Event Server is an example of this kind of product.

Event sourcing

All changes to an application is recorded as a series of events. An applications current state can be queried. But not only that, an application state can be rolled forward or backwards as you wish, giving you a lot of power. Martin Fowler has written all about it here. Also, Patrik has written a couple of blog entries of how he has played with it and now also added support for it in Sculptor, see here and here.

CQRS

Command and Query Responsibility Segregation is a very cool area. I will not try to explain it in detail, it is very well done here.

Simplified it is about separating commands (that change the data) from the queries (that read the data). Separate subsystems take care of answering queries (reporting) and the domain for processing and storing updates can stay focused. The result of the commands are published to query subsystems, each optimized for its purpose.

Ok, that was probably a very short and incomplete list of EDA related topics, but it gave you a taste for it and a ground for further reading. We will use it as a foundation when we talk about how we think about and implements EDA.

If you think we have missed an important topic, tool, concept, etc, around EDA, please add a comment about it.

Next up, how we support EDA in Sculptor.

EDA intro

Both Patrik and I have always been believers of Event Driven Architecture. EDA has been around for many years, but recently the attention towards it has increased. The reasons for that is probably many, but I guess one of the most important is that EDA will help you build more concurrent and scalable system.

Therefore, as we strive to support popular technologies, we take the opportunity to implement support for EDA in the newly released 1.9.0 version of Sculptor.

Patrik has written a couple of interesting posts (here and here) about event driven architecture in a special implementation model, event sourcing.
Event sourcing is a powerful tool in certain circumstances. Though, not always a perfect match.

Also, many people reads SOA2.0 or ESB when you say EDA. This is for sure true in many cases, but just a small set of the truth.

EDA is much more than event sourcing, SOA2.0 or ESB implementations.

EDA plays a big role in application architecture and design (or even module design) as well.

Our approach is that you as a developer should be able to choose how and where you implement EDA.

That is what we will talk about in a sequence of posts.

EDA in general.

EDA, how we see it.

EDA, how we support it.

So, stay tuned for more about EDA.

What's Next, after 1.9.0

Next release will mostly be a technical upgrade. Xtext and Xpand version 1.0 were released together with Eclipse Helios. We will upgrade to that.

Spring 3.0 and JPA 2.0 are also important upgrades.

Oliver will contribute with more cool stuff. He has some almost finished things that he will share with us (hopefully) soon.

The original design of Sculptor has worked fine and we will not change it, but 3 years of additions of different combinations of target implementations is starting to hurt some templates. Therefore we will remove some old (probably unused) target implementations. Those have been deprecated in 1.9.0 and in case you are using any of it you should start migration to some of the more modern alternatives:

• EJB/Spring combination => migrate to EJB3 or Spring


• EJB 2 => migrate to EJB3 or Spring


• Hibernate mapping with XML => migrate to JPA/Hibernate with annotations


• Spring definitions in XML => migrate to Spring with annotations


• Hibernate without JPA => migrate to JPA/Hibernate

This makes a good reason to use version number 2.0.0 for next release. Personally I get bad vibrations when Open Source project go to 2.0. That often means total redesign and no backwards compatibility. That will not be the case for Sculptor 2.0. It will be a cleanup, and some pieces are not supported any more, but most of it will stay intact and be compatible (with some migration steps, as usual).

Exploring new features in 1.9.0

This post is dedicated for those of you who are already using Sculptor and would like to have a quick overview of some selected features in the release. I will not bring up bug fixes and all improvements. Please refer to the complete list in issue tracker for that.

I have written a separate teaser about the MongoDB and EDA features of the new release.

Generated Documentation

In the model you can write documentation as a quoted string in front of almost every element (attribute, reference, entity, operation, ...). From the model Sculptor generates HTML documentation of all domain objects including their attributes and associations. Documentation of Services are also included. The generated result is located in src/generated/resources/DomainModelDoc.html

Improved Graphviz Visualization

Several diagrams with different focus and level of detail are generated. They are also included in the generated documentation. Samples here. There is a new maven plugin fornax-graphviz-m2-plugin that generates images (.png) from the .dot files.

Syntax Diagrams

We have mainly focused on sample based documentation. Some users have asked for more formal syntax description of the DSL. We have created railroad syntax diagrams.

Aggregate Syntax

Aggregates can now be defined with the new belongsTo keyword. It is more informative than the previous !aggregateRoot.

        Entity Cargo {
            - TrackingId trackingId key
            - Location origin required
            - Location destination required
            - Itinerary itinerary nullable inverse opposite cargo
        }

        ValueObject Itinerary {
            belongsTo Cargo
            not optimisticLocking
            not immutable
            - Cargo cargo nullable opposite itinerary
            - List legs inverse
        }

        "An itinerary consists of one or more legs."
        ValueObject Leg {
            belongsTo Cargo 
            - CarrierMovement carrierMovement;
            - Location from;
            - Location ^to;
        }

Nullable in key

It is now allowed to have some nullable fields as part a composite natural key.

    ValueObject HandlingEvent {
        - Type type key
        - CarrierMovement carrierMovement nullable key
        - Location location key
        DateTime completionTime key
        DateTime registrationTime
        - Cargo cargo key opposite events
    }

ToStringStyle

We are using commons-lang for toString of domain objects. It has a style parameter, which is now possible to define in sculptor-generator.properties. You can choose between several styles. I like this format:

toStringStyle=SHORT_PREFIX_STYLE

You can also override this for individual DomainObject with hint:

    ValueObject Foo { 
        hint="toStringStyle=MULTI_LINE_STYLE" 
        String aaa 
        String bbb 
    } 

databaseJoinTable

It is possible to define many-to-many join table with databaseJoinTable and its columns with databaseColumn at both sides of the bidirectional association:

- Set<@Media> existsInMedia opposite mediaCharacters
        databaseJoinTable="MED_CHR" databaseColumn="CHR"

BTW opposite doesn't have to be defined last any more.

Those keywords are also useful for unidirectional to-many associations. Additionally databaseJoinColumn is used, since there is no opposite side to define the column on.

- Set<@Person> playedBy databaseJoinTable="CHR_PERS" 
        databaseColumn="PERS" databaseJoinColumn="CHR"

Clob/Blob

Clob and Blob are now supported with default java types String and byte[]. They are mapped with JPA @Lob.

GUI with Not Persistent Value Objects

The structure of the persistent domain model might not always match the presentation, for example you might want a dialog for editing several domain objects in one single screen. Then you can create a non-persistent ValueObject with a corresponding Service. The application service handles the transformation between the presentation object and the persistent domain objects. More documentation and sample is available here.

Sculptor 1.9.0 - Support for MongoDB and Event-Driven Architecture

The Sculptor development team has a track record delivering around 3 releases per year. We have delivered 10 releases in total. That means that the core pieces are rock solid. I have personally used it successfully together with 15 other developers on daily basis for about a year now. It simply works very well.

We care about bugfixing and making small improvements. At the same time we are excited about learning new technology and using emergent design. This release contains two new features in the area of scalability. Persistence backed with MongoDB and support for Event-Driven Architecture.

MongoDB bridges the gap between key-value stores (which are fast and highly scalable) and traditional RDBMS systems (which provide rich queries and deep functionality). I think this makes MongoDB very interesting for applications that need high-performance and/or scalability, but also prefer using a rich persistent domain model with complex associations. The schema less structure is attractive from a developer productivity perspective, which is one of the two goals with Sculptor (quality is the other).

Sculptor generates data mapper classes that converts domain objects to/from MongoDB data structures, DBObjects. This makes it easy to use a domain model à la DDD with automatic mapping to MongoDB data structures.

Sculptor provides generic repository operations for use with MongoDB. This includes operations such as save, delete, findById, findByKey, findByCondition, and some more. You get CRUD operations, and GUI, for free.

Queries can be expressed with a slick fluent api that support code completion and refactoring.

Rich support for associations. Aggregates are stored as a embedded documents. Other associations are stored with referring ids. In the domain objects there are generated getters that lazily fetch associated objects from the ids. This means that you don't have to work with the ids yourself, you can follow associations as usual.

Read more about how to use MongoDB with Sculptor here.

Event-Driven Architecture (EDA) is a good complement to Domain-Driven Design. We think EDA is an important ingredient for building scalable systems. It is also an enabler for designing loosely coupled modules and bounded contexts.

For this Sculptor makes it possible to define Domain Events in the model in similar way as Entities and Value Objects. Sculptor also provide a mechanism to publish and subscribe through a simple event bus. This is done either in a declarative way in the model, or programatically with a simple API.

Implementations of the event bus that integrates with Apache Camel and Spring Integration are available out-of-the-box.

In addition to publish/subscribe Sculptor also has support for CQRS and EventSourcing.

CQRS is about separating commands (that change the data) from the queries (that read the data). Separate subsystems take care of answering queries (reporting) and the domain for processing and storing updates can stay focused.

Event Sourcing makes it possible to see how we got to the current state and query how the state looked liked in the past. Essentially it means that we have to capture all changes to an application state as a sequence of events. Sculptor provides a default implementation of EventSourcing, which can be extended and customized to fit specific needs.

Read more about how to use the new event features of Sculptor here. We will blog more about this topic later, so stay tuned.