Skip to content
CONTACT US
SIGN IN
Otonomo Logo
Redis, Kafka or RabbitMQ: Which MicroServices Message Broker To Choose?
When using asynchronous communication for Microservices, it is common to use a message broker. A broker ensures communication between different microservices is reliable and stable, that the messages are managed and monitored within the system and that messages don’t get lost. There are a few message brokers you can choose from, varying in scale and data capabilities. This blog post will compare the three most popular brokers: RabbitMQ, Kafka and Redis.
But first, let’s learn about Microservices communication.
Microservices Communication: Synchronous and Asynchronous
There are two common ways Microservices communicate with each other: Synchronous and Asynchronous. In a Synchronous communication, the caller waits for a response before sending the next message, and it operates as a REST protocol on top of HTTP. On the contrary, in an Asynchronous communication the messages are sent without waiting for a response. This is suited for distributed systems, and usually requires a message broker to manage the messages.
The type of communication you choose should consider different parameters, such as how you structure your Microservices, what infrastructure you have in place, latency, scale, dependencies and the purpose of the communication. Asynchronous communication may be more complicated to establish and requires adding more components to stack, but the advantages of using Asynchronous communication for Microservices outweigh the cons.
Asynchronous Communication Advantages
First and foremost, asynchronous communication is non-blocking by definition. It also supports better scaling than Synchronous operations. Third, in the event Microservice crashes, Asynchronous communication mechanisms provide various recovery techniques and is generally better at handling errors pertaining to the crash. In addition, when using brokers instead of a REST protocol, the services receiving communication don’t really need to know each other. A new service can even be introduced after an old one has been running for a long time, i.e better decoupling services.
Finally, when choosing Asynchronous operations, you increase your capability of creating a central discovery, monitoring, load balancing, or even policy enforcer in the future. This will provide you with abilities for flexibility, scalability and more capabilities in your code and system building.
Choosing the Right Message Broker
Asynchronous communication is usually manages through a message broker. There are other ways as well, such as aysncio, but they’re more scarce and limited.
When choosing a broker for executing your asynchronous operations, you should consider a few things:
Broker Scale – The number of messages sent per second in the system.
Data Persistency – The ability to recover messages.
Comparing Different Message Brokers
RabbitMQ (AMQP)
Scale: based on configuration and resources, the ballpark here is around 50K msg per second.
Persistency: both persistent and transient messages are supported.
One-to-one vs one-to-many consumers: both.
RabbitMQ was released in 2007 and is one of the first common message brokers to be created. It’s an open source that delivers messages through both point-to-point and pub-sub methods by implementing Advanced Message Queuing Protocols (AMQP). It’s designed to support complex routing logic.
There are some managed services that allow you to use it as a SaaS but it’s not part of the native major cloud provider stack. RabbitMQ supports all major languages, including Python, Java, .NET, PHP, Ruby, JavaScript, Go, Swift, and more.
Expect some performance issues when in persistent mode.
Kafka
Scale: can send up to a millions messages per second.
Persistency: yes.
One-to-one vs one-to-many consumers: only one-to-many (seems strange at first glance, right?!).
Kafka was created by Linkedin in 2011 to handle high throughput, low latency processing. As a distributed streaming platform, Kafka replicates a publish-subscribe service. It provides data persistency and stores streams of records that render it capable of exchanging quality messages.
Kafka has managed SaaS on Azure, AWS, and Confluent. They are all the creators and main contributors of the Kafka project. Kafka supports all major languages, including Python, Java, C/C++, Clojure, .NET, PHP, Ruby, JavaScript, Go, Swift and more.
Redis
Scale: can send up to a million messages per second.
Persistency: basically, no – it’s an in-memory datastore.
One-to-one vs one-to-many consumers: both.
Redis is a bit different from the other message brokers. At its core, Redis is an in-memory data store that can be used as either a high-performance key-value store or as a message broker. Another difference is that Redis has no persistency but rather dumps its memory into a Disk/DB. It’s also perfect for real-time data processing.
Originally, Redis was not one-to-one and one-to-many. However, since Redis 5.0 introduced the pub-sub, capabilities boosted and one-to-many became a real option.
Reference
https://otonomo.io/redis-kafka-or-rabbitmq-which-microservices-message-broker-to-choose/
Bài viết khác
So sánh giữa gorm vs go-pg vs bun
Trong thế giới phát triển Golang, việc tương tác với cơ sở dữ liệu là một phần không thể thiếu của hầu hết các ứng dụng. Để đơn giản hóa quy trình này, các thư viện ORM (Object-Relational Mapping) ra đời, giúp các nhà phát triển thao tác với database thông qua các đối tượng […]
So sánh GORM vs go-pg vs Bun
Cộng đồng GORM Là ORM phổ biến nhất trong cộng đồng Go. Có nhiều tài liệu, ví dụ, StackOverflow câu trả lời, và nhiều package hỗ trợ mở rộng. Nhiều developer đã từng dùng Gorm. go-pg Từng rất phổ biến khi chỉ dùng PostgreSQL, nhưng đang bị Bun thay thế dần. Ít được duy trì […]
clean architecture golang
1.Clean Architecture là gì? Clean Architecture là một kiến trúc phần mềm được đề xuất bởi Robert C. Martin (Uncle Bob) nhằm mục tiêu tách biệt rõ ràng giữa các tầng trong ứng dụng, giúp mã nguồn dễ bảo trì, mở rộng, và kiểm thử. 2.Tổng quan kiến trúc Entity (Domain Model): Là tầng […]
Tìm hiểu Hexagonal Architecture
Hexagonal Architecture là gì? Hexagonal Architecture (tên gọi khác là ports and adapters architecture), là một mẫu kiến trúc được dùng trong thiết kế phần mềm. Nó hướng tới việc xây dựng ứng dụng xoay quanh business/application logic mà không ảnh hưởng hoặc phụ thuộc bởi bất kì thành phần bên ngoài, mà chỉ giao […]
Sử dụng Request/Response trong ứng dụng RESTful mô hình MVC
DTO là gì? DTO (Data Transfer Object) là một object trung gian dùng để truyền dữ liệu giữa client – server hoặc giữa các service trong ứng dụng web/API theo kiến trúc RESTful API. DTO chỉ chứa các thông tin cần thiết mà client hoặc service khác cần (ví dụ: Login Form chỉ cần thông […]
Docker
Docker là gì? Docker là một nền tảng mã nguồn mở cho phép bạn đóng gói, phân phối và chạy ứng dụng bên trong các “container” – những môi trường ảo nhẹ, cô lập nhưng vẫn chia sẻ nhân hệ điều hành của máy chủ. Khái niệm then chốt ở đây là “containerization”: thay vì […]
Khoá học lập trình game con rắn cho trẻ em 
