Scalable & real-time messaging (chat) systems with SignalR, React, .NET and Kubernetes

10 min readDec 25, 2021

You can easily build a chat application using SignalR in a couple of hours, however if you want to scale it to serve millions of your customers, more thoughts need to go into your solution to handle workload, security, and extensibility. In this post, I will cover how to build a scalable and full-stack chat application using React, Asp.NET, SignalR and Kubernetes.

The Need

There are different types of messaging; Realtime and Asynchronous. While chat could be used as an example for realtime messaging, the scope is much broader. Scenarios such as notifying a list of subscribers as soon as an event happens all fall under the same category.

On the other hand, you have asynchronous operations which also respond to events however they don’t need to be immediate; someone purchases a book online, and the shipment department is notified to process the order and the delivery the next day.

End Result

In this post, we will cover realtime messaging using SignalR in a way that our application would be able to scale and handle all kinds of chat scenarios such as

One to One (Direct Message)
One to Many (Group Chat)
Invitation Only (Private Chat Rooms)

If you’d like to read more on asynchronous messaging, and event driven architecture, please refer to Event driven architecture with Kafka & Asp.NET.

By the way, this topic belongs to the series to set up Messaging for Asp.NET and React ecosystems.

System Design

When we talk about scalable architecture, we need to cover traffic, as well as functional use cases which I covered before. For that purpose, if we design our chats around ‘who sent what message to whom’, we will have a problem with use-case scalability, as there will be a lot of edge cases involved.

Instead let’s assume every conversation happens in a room, and when someone send a message to that room, all the participants will receive that message. This not only will cover 1:1 and 1:many use cases, you could also process invitation to chat rooms, to only allow those with invitation to access the messages.

This also means when someone is invited to the room after the conversation has started, they will be able to read all the past messages, as the basis for message access is if someone belongs to a room.

Architecture

From the architecture standpoint, we have our React frontend application, talking bidirectionally to our Messaging API using SignalR, and hosted on a Kubernetes cluster supported by NGINX Ingress routing.

Additionally at various stages during the lifecycle of a chat, we would need to resolve the identity of participants, for which, we are using our Pellerex Foundation Identity Services.

Pellerex Messaging System
In case you wanted to get the entire realtime messaging system, bundled with Kafka-backed Asynchronous and Event-Driven architecture, we have built and packaged them for you, which is source-included as part of Pellerex Messaging Foundation.

Data Model

The data model, and how we store our chat records is a good place to start building our chat application. Here is the data model, and I will then explain how it works.

1. Chat Messages

Each chat message is a text communicated from a specific user Id (FromUserId), to a room( a group of people). FromUserId, is a Guid which belongs to a user in system after they have authenticated using Pellerex Foundation Identity Services as a separate micro-service, which holds all the data about that user such as name, phone number, etc.

2. Chat Groups

Chat group is space where at least two participants talk and everything they post there, will be read by all group participants. You can invite more participants in there, or you could kick people out. They also could be around specific “Contexts”, which means the same two people have a different conversations around different topics, other that 1:1 direct messages for example.

Therefore when we want to show to our users all the conversations they have had with other individuals, we also rely on this Context to group the conversations. As an example, between person A and B, there could be a 1:1 conversation, as well as a conversation around certain topics. This is indeed optional, and you could drop it if you like.

3. Chat Group Participants

Then we have Group Participants, which actually shows who is which group, and when they have joined that group.

4. Chat Message Read History

And finally we have the read history, which is used to indicate to the users they have unread messages. Every time we fetch the messages for a specific user, we mark those fetched messages as read.

Flow

Now that we have the System Design, Architecture and Data Model, lets talk about the actual step by step flow.

When person A, sees the chat window with person B, we need to load all the past conversations between them. To achieve that, we need to find all the groups A & B are part of, and fetch the conversation groups. Context comes to light here, if there is no Context set, that means this is a general 1:1 conversation between A & B, otherwise we can only fetch the group associated with that Context. Again you can ignore the concept of Context if you want to.
Once we have the GroupId, we can then easily locate all the messages associated with that GroupId, and load them in the chat window.
As mentioned, every time we fetch messages for a user, we need to mark them as read. This is useful to determine if a there are unread messages in a conversation for a user, when they log into the system.
Once that is done, our client (browser), needs to establish a bidirectional connection to SignalR hub, to be able to send, and receive messages, as other users type them in.
Once connected, every message sent by other users, will be received by all connected parties to the hub, in that room, in realtime.
When each message arrives, we record the message in the DB, so that other offline users will read them later, when they log in.

Packages

Before we jump into the code, here are the packages we need in our solution. For frontend, we need:

@microsoft/signalr

And for backend we need the below:

Now let’s cover each step.

A- React: Load Messages

When our user logs in to the system and head over to their Inbox, we need to fetch all their read and unread messages in the form of ‘conversations’. However before we are able to do so, we need to find the groups they are part of.

The same logic applies when our user wants to initiate the conversation/continue an existing conversation with another user. If there is no conversations between (i.e. there is no room these two are part of), we will need to create a new room for them.

B- React: Chat Component

We also should create a React chat component, to encapsulate all the chat logic behind the below screen, so that enabling chat would be as easy as a couple of lines:

And here is how the component could look like at the development time.

As you can see all we provide, is:

Who is involved (Parties)
Context of a conversation (optional)
Chat Group Id: fetched/generated from previous step

C- Messaging API: Setup SignalR Hub

On the Messaging API side (backend), we need to set up a SignalR hub, so clients (browsers) can connect to, to receive realtime updates.

Please consider the below points about the above snippet.

The methods in this Hub, are the actual methods which will be invoked by the clients (i.e. browsers or React app). In our case, our React application will call JoinRoom and SendMessageToGroup.
Our hub is protected, and only authenticated users can access these methods. I will show you later on, how to attach JWT tokens to SignalR requests.
By the way, if you would like to read more on Authentication in Asp.NET and how we handle claim based Auth, refer to Step by Step Guide to Authentication and Authorisation.

Pellerex Identity System
In case you wanted to get the entire Identity system, we have built and packaged it for you, which is source-included as part of Pellerex Identity Foundation.

We will also need to enable the hub and configure its Url through Startup.cs:

D- Connecting to Hub

Once we acquired/generated a chat group Id from our backend, we will need to connect to SignalR hub from React, and here is the logic:

Please consider the below points about this code:

As you can see I am supplying the access token (JWT) to this method. This access token needs to be valid at all times otherwise the chat requests will fail. If the access token expires, we need to refresh it, before we send the chat request to the server. To read more on token refresh, please refer to Token Refresh with Pellerex & Asp.NET Identity.
To access user information such as access token, user Id, etc, I am using React Higher Order Components (HOC), and that’s why you see some this.props.accessToken in that code. To read more on that, refer to Access User Details using HOC
Once connected, I immediately join the room we have determined for this current chat window, and I subscribe to the ReceiveMessage method, so every time someone posts a message, I will receive it here.
As we need the connection object later to send the message to others, I store it in the state, to be used in the next step.

E- Send Message

The last step would be sending a message to a group, which is as simple as retrieving the connect object, and invoking our Chat Hub method like below:

The rest of the operations regarding saving, and retrieving the messages on the backend side, will be only massaging the data and doing some usual CRUD operations which will be boring for this post. To read more on data manipulation, refer to Database Set up for Asp.Net 5.0 Web Api using Unit-of-Work and Repository patterns.

F- NGINX, CORS & SignalR

You are going to have some FUN with NGINX and CORS when it comes down to configuring SignalR. I am not shying away from saying I hate CORS, as from time to time I have spent a ton of time debugging CORS related issues.

The main problem in my view is that the error messaging doesn’t specify where in the cycle the CORS is causing the trouble sometimes, and you need to debug it tier by tier from your application/api to the webserver, to the gateway, and so forth. Some time ago, I wrote a piece on it, which you can find it here.

To address CORS issues here, for NGINX, you need to make sure it is passing the headers SignalR needs for communication, and hence we need to apply some configuration on the header side. I am using Helm charts to configure my Kubernetes deployments, and you probably might need to read ‘the how’ behind the next segment before proceeding, in Setup ASP.NET and React for Kubernetes in Production.

And here is the header configuration we need to apply, and in particular, please pay attention to Access-Control-Allow-Headers in which I am allowing SignalR headers to be passed through:

G- Scale Beyond One Kubernetes POD using a Redis Backplane

The above set up is not complete yer for scale, as all connections are going to one hub, hosted by an API in only one Kubernetes POD. When we need to scale (i.e. running more POD containing the Messaging API), we need to make sure all the Hubs are in sync in terms of managing the connections with the clients (browsers). To that end, we need to use a realtime cache to manage sync all the connections, and in our case I have used Redis cache.

Microsoft does have a guide on how to set this up here, which is pretty straightforward.

Pellerex Foundation: Messaging Foundation for Your Next Enterprise Software

How are you building your current software today? Build everything from scratch or use a foundation to save on development time, budget and resources? For an enterprise software MVP, which might take 8–12 months with a small team, you might indeed spend 6 months on your foundation. Things like Identity, Payment, Messaging, Infrastructure, DevOps, etc. they all take time, while contributing not much to your actual product. These features are needed, but they are not your differentiators.

Pellerex does just that. It provides a foundation that save you a lot development time and effort at a fraction of the cost. It gives you source-included Identity, Payment, Messaging, Infrastructure, and DevOps to build Web, Api and Mobile apps all-integrated and ready-to-go on day 1.

Check out Pellerex and talk to our team today to start building your next enterprise software fast.