Batching a Service Bus triggered Azure Function

I spent some time working with a Service Bus triggered Azure Function and found some interesting scenarios.  They are mostly intuitive, and I think when you are presented this, it makes sense.  But sometimes I also think that stating the obvious is necessary, too, because sometimes it might not be so obvious to everyone, myself included.

When you are working with messaging products, they are used to decouple clients from backends, where the backend is typically a database.  I wrote an article about that here.  And we know that there needs to be code executed on the client, which is sending the information to the server.  There is also code on the server which accepts the information and processes it.  In this scenario, the server is a messaging product named Azure Service Bus.  During my learning I found these 4 scenarios, Table 1, which you might also recognize when attempting to implement an Azure Service Bus triggered Azure Function.

Client/Sender Server/Function
single single
single batched
batched batched
batched single
Table 1, consumer producer service bus batching

The point I want to make early on here is that the only scenario out of those 4 which results in batch processes in the Azure Function is the batch – batch on both the client and server.  I’ll show you what I mean first. NOTE: This actually turned out not to be the case. When you accept an array in your Run() method declaration, some batching will happen. Still, test it out for yourself and be sure to send at least 100 messages so you give the platform the opportunity to batch.

I used the Azure.Messaging.ServiceBus NuGet to send messages to my Service Bus from my client.  Take note that the Microsoft.Azure.ServiceBus NuGet has been replaced.  You can see how I sent the messages to the Service Bus here on GitHub.  I used the following, this is psuedo code, check GitHub for the complete example.

using ServiceBusMessageBatch messageBatch = await sender.CreateMessageBatchAsync();
messageBatch.TryAddMessage(new ServiceBusMessage($"Message {i}"))
await sender.SendMessagesAsync(messageBatch);

I want to call out the name of that method, take note it is SendMessagesAsync, messages, not message…

The Function Run() method declaration looks like this, Figure 1.

image

Figure 1, batching Service Bus messages

Notice that I used the Message[] as a parameter to receive a batch of messages.  I batched and sent messages to the Service Bus, which triggered an Azure Function and it shows that both messages were processed via a single Function Invocation, see Figure 2.

image

Figure 2, batching Service Bus messages on Azure Functions

So, that’s how I would do it, if it was my objective to batch process messages from Service Bus using an Azure Function.

I think it is kind of interesting now to look and see what happens in the other 3 scenarios.  Seeing them will help you distinguish how logs can show what kind of processing is happening.

Single – Single

This is the basic one, you send a single message to the Service Bus and the bound Azure Function will process it.

You can see the complete code I used here.  The following is a brief snippet.

ServiceBusMessage message = new ServiceBusMessage($"Topic-Message-{i}-{Guid.NewGuid().ToString("N")}-{DateTime.Now.Minute}");
await sender.SendMessageAsync(message);

Figure 3 shows the Run() declaration which accepts a single message as a string.

image

Figure 3, single Service Bus message

The output of processing 2 messages is shown in Figure 4.

image

Figure 4, single Service Bus message

I interpret that logging sequence to mean that both messages are being processed in parallel.  I assume this because of the order of logging, where there are 2 Executing, followed by the code in the Run() method, followed by the Executed log.

Single – Batch

I cannot really think of a scenario in which this would be used, but it is possible and it does render an interesting scenario.

I sent 2 messages using this code snippet, to the Run() method shown in Figure 1.

ServiceBusMessage message = new ServiceBusMessage($"Topic-Message-{i}-{Guid.NewGuid().ToString("N")}-{DateTime.Now.Minute}");
await sender.SendMessageAsync(message);

The output looks like this, Figure 5.

image

Figure 5, single Service Bus message to batched Azure Function

It is interesting because the messages in this scenario seem to be processing serially and not in parallel as they were in the single-single scenario.  You can see this because you indeed have 2 Function Invocations, one for each message, but one start Executing and is Executed, before the second one begins Executing.  The pattern is serial.

Batched – Single

When I batched my messages using this code snippet.

using ServiceBusMessageBatch messageBatch = await sender.CreateMessageBatchAsync();
messageBatch.TryAddMessage(new ServiceBusMessage($"Message {i}"))
await sender.SendMessagesAsync(messageBatch);

And sent 2 messages to the Function with a Run() method declaration like Figure 3, I got the same output as Single – Single.  I.e. it was the same behavior as the one in Figure 4.  Again, this one seems to be processing the messages in parallel.  More specifically, the process which handles the invocation spawned 2 threads, 1 for each message and they ran at the same time.

This clears up a bit for me, regarding how to batch and process Service Bus messages via an Azure Function.  There are numerous changes going on here, so, check back on this topic from time to time.  Before I go, here are the versions I was targeting, Figure 6.

image

image

Figure 6, Package and Framework dependencies and framework