Azure service bus is a message broker which transfers messages to another in a more secured and consistent way. Service bus employed to decouple applications and, systems. This article covers all aspects of Introduction to Azure service bus

Payload in service bus is called a message. Data in message can be of any structure like Json, XML, Apache Avro, and Plain Text.

Message Broker

It is a software that enables applications, system, and services to communicate with each other and exchange information. This allows interdependent services to talk with one another directly even they are built on top of different framework or with different languages.

Message brokers enables asynchronous communication between services so that client does not need to wait for consumers response. This improves fault tolerance in system. Since message brokers are independent of any application they can be easily scaled as per the changing number of consumers.

Two parts of service bus are queues and topic

Queue in Azure service bus

Queue is FIFO (first-in first-out) storage mechanism if used with session. Messages are sent to and are received from service bus queue and are stored in queue until receiving application is available to receive and process the messages

• Azure service bus queue maximum size can be 80 GB. If messages are sent from any service to queue after this limit is reached, then the message will be rejected, and an exception will be thrown to sender
• If your application needs to support bigger size than this, then you should consider Storage queue which can store up to 500TB of data.
• Single message size of queue can be of 256 KB for standard subscription tier and 100 MB for premium subscription tier. Here maximum header size can be of 64 KB only
• You can create service bus queues and topic in one namespace of sizes 1, 2, 3, 4 and 5 GB. When partitioning is enabled, you can create up to 16 partitions of a queue, so your queue becomes 16 times 5GB i.e., 80 GB max size for a queue

Receive Mode

When creating a queue, we can specify receive mode. The default receive mode is Peek-lock

Peek-lock

In peek-lock mode, client sends a request to receive message from service bus. After client has received the message it sends a request to complete the message. Peek-lock receive mode is required to dead-letter a message. This mode ensures At-lest once processing

Receive and Delete

In receive-delete mode both steps of peek-lock are combined in one request. Using this mode, we can increase the overall throughput of services bus in exchange of risk of losing messages and no support for transaction. This mode ensures at-most once processing

Dead-letter queue

Queues support dead lettering. Meaning if any receiving application is not able to process a message and throws any exception then the message is not lost. It is moved to dead letter queue automatically. Similar things happen when a message is expired waiting for process in queue, it is moved to dead-letter queue from where messages can be processed again or checked for failures.

How to Benefit from deadletter queue

• All the messages from queue that are not processed successfully by code moves to deadletter queue. This unsuccessful message processing could be because of timeout i.e., message was not picked up by any receiver for given time, max retry count exceeded, Receiver was down. This allows us to identify those messages and retry them after some time.
• You have an application which uses service bus queue to send messages to backend code. On Christmas eve due to heavy traffic backend service suffers downtime of 30 minutes. Message time is 10 minutes. Therefore, all the messages which were in service bus for 20 minutes will move to dealetter where they can stay for as long as the message is explicitly retrieved and completed.
• Deadletter message analysis can give us insight on root cause of failures
• We can re-send deadletter messages to queue for processing, manually or through code
• Dead-letter can used to implement fail safe mechanism

Topic in Azure service bus

Topics and subscriptions provide a one-to-many communication in a publish and subscribe pattern. It’s used for large number of recipients. Each message is made available to each subscription registered with topic.

Combination of topic and subscriptions works like queue. Pushing messages to topic is like pushing messages to queue but consumers cannot receive messages from topic directly. Instead, they receive messages from subscriptions of a topic. Each subscription acts as a virtual queue to receiver that receives the messages that are sent to topic. Consumers receive messages from subscription identical to they way message is received from queue.

The message sending functionality of a queue maps directly with topic and message receiving functionality is like subscriptions.

How to push messages to queue / topic from portal

1: In your browser type https://portal.azure.com/ and login to with your credentials

2: Search for service bus from search bar

3: Select you service bus namespace and look for queues and topics in left menu bar

4: If you select queue from menu, you will be shown list of queues in right panel. Click on queue name from right panel

5: From queue page select Service Bus Explorer from left menu

And click on Send messages from Service Bus Explorer screen

6: After clicking on Send messages a popup for sending messages will appear where you can paste you message body, custom properties and broker properties like MessageId.

7: Click send to push message to queue/topic.

Prefetching

This enables the queue or subscription client to load additional messages from service when it receives messages. The client stores these additional messages in its local cache. Each client that enables prefetching maintains its own cache which is not shared across clients.

If a client starts a receive operation to receive message from service with empty cache, the service transmits a batch of messages. The size of the batch is equal to the size of cache or 256KB, whichever is smaller. If the client starts receiving operation when it has messages in cache then message is takes from cache, saving time it would take to receive message from service directly.

When a message is prefetched the service locks the message for other receivers. If the message is not processed by received before lock expires, the message becomes available for other receivers but the prefetched copy of message remains in cache. The receiver that consumes the expired message will receive an exception when it tries to complete the message.

By default, message lock expires after 60 seconds. Lock duration can be increase to 5 minutes.

Prefetching messages increases the overall throughput of queue or subscription as it reduces the round trips. However, fetching the first message will take longer because of increase size of message for prefetching.

At-least Once processing

In peak-lock receive mode, once message processing is completed at application, the application must request service bus to complete message. If the application crashes after completing the message processing and before requesting the service bus to complete the message, Service bus redelivers the message to the application when it restarts. That is each message is processed at least once.

Duplicate Detection (Exactly Once processing)

When sending messages to service bus queue an application fails immediately after it sends a message, and when application restarts it believes that prior message delivery did not occur, hence resending the same message.

Duplicate detection takes this scenario out of picture by enabling client to send duplicate message, and the queue and topic discard the duplicate message.