Microservices Design Patterns :Event Async Pattern

Image for post
Image for post

This is the 2nd part of the Microservice design pattern blog series. In this post we are going to discuss about the Event based communication Async design pattern.

In this blog series, I am going to give you more knowledge on the key design principles of Microservices to make it as the desired solution to have a scalable, easy to maintain enterprise application. Following are the chapters of this blog series.

  1. Microservices and Bounded Context
  2. Event communication Async pattern with CQRS — This Post
  3. Competing worker , Fanout and Async API call patterns
  4. API-based Microservices
  5. Data Consistency Across Microservices
  6. Using API Gateway to maintain centralized access
  7. Split Monolithic Databases Across Microservices
  8. Resilience in Microservices
  9. Define and Document Microservice Contracts
  10. Centralized Logging
  11. Cloud-Based Microservices Infrastructure
  12. Manage Configurations
  13. Registration and Discovery
  14. Monitor Microservices

In an Asynchronous Microservice, specific components within the architecture talk to each other in an asynchronous fashion. For an example, if we spin up a Microservice with a RESTFull API, and another Microservice calls to this endpoint, the calling Microservice doesn’t wait for the task to complete, instead the calling Microservice might receive a default response and the called service will then just carry on with the other processing that’s left.

Why do we need this kind of communication between Microservices ? One scenario might be the end point that one service is calling consists of a task which is a long running job and you can’t actually afford to wait for that task to complete because it might hinder the performance of the calling Microservice. Another reason could be that the calling Microservice do not need the result from the called Microservice straight away. One other example would be the better user experience in a front-end service, where user do not need to wait ( unresponsive user experience) until the results are returned from the called service.

We are going to discuss the event based pattern with CQRS in this post.

Event based communication

In an event based communication , the calling service or application raises an event instead of making a call directly to another API or a service.

An event consuming application(s) then picks up this event and then carries out the task. This is asynchronous because the calling application or service is not aware of the consumer and the consumer isn’t aware of the called application as well. The consumer just picks up the message and performs the required task. We can achieve this using a simple message queue protocol such as MQTT with RabbitMQ or Kafka.

Image 1 : Message queue with many consumers

At a technical level these events are raised as messages, messages that are placed onto queues via message brokers such as RabbitMQ. The consuming services of these messages are the services that are going to carry out these tasks and they actively sit there listening to the queue. There might be situations where there are many consumers who need to consume the same message and perform a specific task intended to the consumer. We can configure this using a number of different queuing patterns which controls the behavior of how the events are consumed.

A simple example for an event as a message can be demonstrated as below.

Image for post
Image for post
Image 2 : Event message

The client raises the event by sending it as a message to a message broker, and once the client application has done this, it carries on processing the other parts of the same transaction, and this is where the asynchronous nature of this communication comes from. This event can be considered as a fire and forget event because nothing gets blocked in the client application or service until the service finishes its task.The actual event itself will actually live on the message broker, possibly on an exchange or a queue, and it will sit there waiting for a consumer to pick it up and process it.

Following is the simple example in .Net Core 2.0 using RabbitMQ.

Logging is considered as an additional functionality to the application where it should not hinder the performance of the main application’s functionality. Hence, we need to design a seamless logging mechanism which has minimum impact to the main application.

Following is the detailed deign of the application.

Image for post
Image for post
Image 3 : Event based logging — Application design

We are having 3 projects in this solution and the Logger API is responsible for accepting client’s logging events. Logger service is responsible for subscribing to those logging events and perform the persistence of log events. The core feature in this solution is the common project where its doing the orchestration in between the event and command ( query ) . So this a another example of the CQRS ( Command and Query Responsibility Segregation ).

The logger API subscribes to the events that are exposed by the common projects. This is achieved by using the following lines of code when we spin up the API application.

public class Program
{
public static void Main(string[] args)
{

ServiceHost.Create<Startup>(args)
.UseRabbitMq()
.SubscribeToEvent<LogCreated>()
.Build()
.Run();
}


}

The API project’s own service host is trying to grab the EventHandler from our services collection that is defined in the ioc container in the startup class as follows.

public Startup(IConfiguration configuration)
{
Configuration = configuration;
}

public IConfiguration Configuration { get; }

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.AddMvc();
services.AddRabbitMq(Configuration);
services.AddScoped<IEventHandler<LogCreated>, ActivityCreatedHandler>();
}

In the Controller, all we need to do is publish to the event asynchronously. This is a simple fire and forget method which do not hold the post requested and the post request just returned the accepted ( 202 ) REST return type to the caller.

[Route("api/logger")]
public class LoggerController : Controller
{

private readonly IBusClient _busClient;

public LoggerController(IBusClient busClient)
{
_busClient = busClient;
}

[HttpPost("")]
public async Task<IActionResult> Post([FromBody]CreateLog @event)
{
@event.Id = Guid.NewGuid().ToString();
_busClient.PublishAsync(@event);

return Accepted($"activities/{@event.Id}");
}
}

The common project is responsible for handling the Rabbit MQ messaging with subscriptions and publications. This project implements the Events and the Command where the API and Service projects can consume. Following are the interfaces for common command handler and event handler.

public interface IEventHandler<in T> where T : IEvent
{
Task HandleAsync(T @event);
}
public interface ICommandHandler<in T> where T : ICommand
{
Task HandleAsync(T command);
}

IEvent and ICommand are just marker interfaces.

Following is the spin up code for section in the services project.

public class Program
{
public static void Main(string[] args)
{
ServiceHost.Create<Startup>(args)
.UseRabbitMq()
.SubscribeToCommand<CreateLog>()
.Build()
.Run();
}
}

As you can see the major difference between the API’s spin up class and the Service’s spinup class is , in API it starts the project by subscribing to events where the Services class is subscribing to command. Hence by this code we have achieved a clear segregation of command and query.

Following is the code in the Service’s startup class.

public Startup(IConfiguration configuration)
{
Configuration = configuration;
}

public IConfiguration Configuration { get; }

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.AddMvc();
services.AddRabbitMq(Configuration);
services.AddScoped<ICommandHandler<CreateLog>, CreateLogHandler>();
}
public class CreateLogHandler: ICommandHandler<CreateLog>
{
private readonly IBusClient _busClient;

public CreateLogHandler(IBusClient buslient)
{
_busClient = buslient;
}

public async Task HandleAsync(CreateLog command)
{
Console.WriteLine($"Creating Log:{command.Name}");
await _busClient.PublishAsync(new LogCreated(command.Id, command.Name, command.Description));

}
}

You can get the code for this demo in this git repo.

Advantages in Event based Async communication pattern

You can gain following advantages adopting this design pattern in your Microservice eco system.

Reduced information latency : Event occurrences are enabled to ripple throughout the full eco system without getting delayed in any high workload service.

Improved scalability : Individual processes block less and have reduced dependencies on remote/distributed processes. Also, intermediaries can be more stateless, reducing overall complexity.

Improved flexibility : Discrete event-driven processes tend to be more flexible in terms of the ability to react to conditions not explicitly defined in statically structured sequential business processes, as state changes can be driven in an ad hoc manner.

Conclusion

I hope you got a clear picture about how to design an event based Async communication pattern which supports the CQRS( Command and Query Responsibility Segregation). We have evaluated how to develop a logging mechanism with out hindering the performance of the main service using this event based async communication pattern.

Hoping to see you in the next chapter of this blog series (Competing worker , Fanout and Async API call patterns).

Thanks for reading the article. Try to share it with your friends on Reddit, Twitter, and LinkedIn. Contact me if you get stuck somewhere and I’ll attempt to encourage you. I’m normally very busy so it may take me awhile to get back to you.

You can follow me on Twitter and LinkedIn. What content do you need me to write about? Drop me an email on with your thought.

References

https://msdn.microsoft.com/en-us/library/dd129913.aspx

Event-Driven Architecture: How SOA Enables the Real-Time Enterprise By Hugh Taylor and Angela Yochem

Written by

CTO @ ZorroSign | Seasoned Software Architect | Expertise in AI/ML , Blockchain , Distributed Systems and IoT | Lecturer | Speaker | Blogger

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store