Asynchronous Messaging with Rails

When asked how to integrate with other applications, the vast majority of Rails developers would answer "REST". And when they answer REST, they almost always mean synchronously. Now I have nothing against REST per se, but I will always favour asynchronous communication over synchronous. An asynchronous messaging solution such as JMS has well-documented advantages such as:

• The message producer can "fire and forget", sending the message and then moving on to more important work.


• The mechanisms required for reliability are relatively simple in the producer and consumer, and the more difficult tasks (such as persistence) can be handled in the message broker.


• It is easy to add redundancy into the messaging infrastructure

For further reference, a good book on the topic is Gregor Hohpe's Enterprise Integration Patterns

On a number of occasions I've started out with synchronous messaging, only to say to myself "I really want this to be reliable". So I fire off an http request and wait for a 200. If I don't receive a 200 after a certain amount of time, then I try again. I'll repeat this for a few times, then I'll give up, and log the failure somewhere.

Then I say to myself "It would be really nice to have this re-trying out-of-process". So I add another process that receives a request for a message to be sent, sends the message, and performs any re-trying that is required. And all of a sudden, I have asynchronous messaging. Now, there's nothing about REST that prevents asynchronous messaging, but I rarely see anyone from the Rails community talking about anything but synchronous messaging. So I thought I'd talk about how we've solved the problem on our project. We've used the ActiveMessaging plugin, originally written by some people from Thoughtworks. I'll demonstrate ActiveMessaging here, and then over the next week or two I'll contrast it with a JMS JRuby solution I've been playing around with.

Let's say we have two Rails applications - a Customer Management application and an Order Management application. The Customer application is responsible for managing all things to do with Customers - their personal details, any communications between our business and the customers, and the customer's user preferences. The Customer Management app has a rich view of a customer with fields such as:

Name
Address
Phone Number
Social Security Number
etc.

The Order Management application needs to associate Orders with Customers, so that even when the Customer application is down for scheduled maintenance, orders can still be taken. The Orders app has a simple view of a Customer, with just a name and an id.

We will make the Customer application responsible for creating, updating and deleting Customers. Whenever any of these actions are performed, the Customer application notifies the Orders application with a message. The message indicates the type of action performed (create, update, or delete), and the id of the customer, so that the two 'views' of the Customer object (in the two different applications) can be kept in sync. The Orders application then creates, updates, or deletes its view of the customer according to the type of message that is sent.


First, we will need to setup an ActiveMQ Server. I've set it up locally on my machine using these instructions (I used the Linux instructions for my Macbook Pro):

http://activemq.apache.org/getting-started.html

In both of our Rails applications we will need to install the ActiveMessaging plugin:

script/plugin install http://activemessaging.googlecode.com/svn/trunk/plugins/activemessaging

ActiveMessaging provides the ability to send and receive messages. Our Customer application will send messages, and our Orders application will receive them.

For the purposes of this discussion, I will only show the 'create' messages, though the process would be very similar for 'update' and 'delete'.

Customer application
Our Customer application has a Customer object that inherits from ActiveRecord::Base:

class Customer < ActiveRecord::Base
end

It has the following fields:

:name, :string
:address, :string
:telephone_number, :string
:created_at, :date_time
:updated_at, :date_time

Our message will be a YAML serialized message of the following simple data object:

class CustomerPayload
  attr_accessor :id, :name
  
  def initialize(params)
    @id = params[:id]
    @name = params[:name]
  end
end

We use a rails observer to observe the 'create' event of the Customer object, construct a CustomerPayload object, serialize it, and send it via ActiveMessaging:

require 'activemessaging/processor'
class CustomerObserver < ActiveRecord::Observer  
  include ActiveMessaging::MessageSender
  observe Customer
  
  publishes_to :customer_queue
  
  def after_create(customer)
    payload = YAML.dump(CustomerPayload.new(:id => customer.id, :name => customer.name))
    publish :customer_queue, payload
  end
end

To configure connection to the ActiveMQ server, I'll need the following configuration files in both of my rails applications:

#config/broker.yml
development:
    adapter: stomp
    login: ""
    passcode: ""
    host: localhost
    port: 61613
    reliable: false

#config/messaging.rb
ActiveMessaging::Gateway.define do |s|  
  s.queue :customer_queue, '/queue/Customer'
end

And to get the observer to work, I'll need this line in environment.rb

# Activate observers that should always be running
config.active_record.observers = :customer_observer

For consuming messages, ActiveMessaging provides a directory under app call 'processors'. In this directory of my Orders application I place my CustomerMessageProcessor:

#customer_message_processor.rb
require 'processors/application'
class CustomerMessageProcessor < ApplicationProcessor

  subscribes_to :customer_queue

  def on_message(serialized_message)
    customer_payload = YAML.load(serialized_message)
    customer = Customer.new(:name => customer_payload.name)
    customer.id = customer_payload.id
    customer.save!
  end
  
end

This class will need access to the same CustomerPayload object as the Customers application when it deserializes it.

In ActiveMessaging, a poller is used to poll the queue and trigger the on_message method shown above. We'll need to start the poller in the Orders application using the following command:

script/poller run

And that's it. If both the Orders and Customer applications are running, and have access to the ActiveMQ server, then when we create a Customer in the Customer application, it should show up in the Orders application a few moments later.